I am working on integrating a variety of systems, some of which will only be known sometime in the future. Some of the challenges that arise are how to deal with the complexity and variety of systems. Below are some general thoughts on how to approach this type of work. With an increasing number of software systems in all walks of life, integration work will increase exponentially.
A system is the product of interaction between parts that cannot be achieved by any single part. Russel Ackoff has a good analogy. A car moves, but a motor can’t even move itself. Physical products were fixed products, whereas software systems usually integrate using versioned APIs with versioned back-end systems.
Below is one example of going about such integration work in a methodical way, not a comprehensive guide.
Pattern matching and analysis to understand the needs. How do you allow for variety, somewhat future-proof and enable scenarios through combining available and composable functions into a solution? Extensibility is implementing interfaces in a way that allow this. Classifying scenarios and analyzing them to find patterns can help create a map of needs. Someone thinking about extensibility should think in terms of using composable parts to then fulfill the needs.
Enabling architecture to meet needs and allow for variety through synthesis. Some of the useful parts of systems thinking that might help with successfully integrating systems include thinking of how the part fits in to the whole. Using classification techniques and then using pattern matching to run over the classifications can create a map of similar parts. A good part in our system tries to absorb variety while not being useless and ambiguous. Parts that can be combined to produce an increased number of useful interactions is better than a set of parts that produce less numbers of useful interactions. In reality, you will not know every scenario enabled, but you can do thought experiments and discuss it with other people to get an idea of which parts provide this better result.
Think in terms of the system when working with the parts. One thing that makes integration more complex than it needs to be is when people creating parts, expose the part implementation to people wanting to use the extensibility layer to make new interactions. It might be impossible in some scenarios to abstract away or hide the implementation details, but in a lot of cases it is something that can remain hidden. This does not mean parts have unique properties. It means that the unique properties exposed to the integrators should be thought of in terms of the system and whole the part fits in to the system, not how the part works.
An example of this with software is that you might need to transform data from one format to another. As a software developer creating the part, you might use XSL to transform the data, but having the integrator be responsible for providing XSL to an XSL part would be the more complex solution. If the system will require 10 parts to create the system, then this approach will theoretically require the systems integrator to understand 10 unrelated sets of technical details. Because the integrator is creating a system with parts, they will obviously have to know 10 different parts, but cognitive load should be related to the system and not the intricacies of the parts.
The systems engineer creating the parts might judge the complexity by the number of parts created, thinking that fewer parts means less complication. Thus the one XSL transformer that allows any XSL to be submitted might be judged better than the more strict phone translator. When you want an email translator, the part maker will have to make another part and this might be judged as adding complexity or work.
The less complex solution for system integrator in this example, would be if the part creator (software developer) created a set of parts around specific types of translators in the context of the system. If the system is about communication, then maybe there is a US Phone Number to International Translator. It will be more valuable for the systems integrator to talk about translating phone numbers every day they make systems from these parts than to reference XSL and its intricacies. It is appropriate for the systems integrator to think about area code, but not the syntax of an XSL if statement for example.