One strategy to integrate them all (from the Internet of Things to the Enterprise)
Hello again. In this post, I am going to tell you about some of the amazing changes that most of us, and our sons for sure, will live in our day by day thanks to technology: the industry is already working on Connected Cars, or the Internet of Things (IoT), to name a few. These are some of the major exponents of the social transformation we are all living with the constant information exchange, and the integration of “everything with everything”.
And I wonder: what is the use of connecting everything at home, if as soon as I go out for a drive in my car or a walk around my city, the connectivity is over?
Actually, everything is much related. What’s more: everything is the same.
Let’s see it.
Tabla de Contenidos
- 1 THE INFORMATION ECOSYSTEMS
- 2 USE CASES
- 3 WHAT THEY HAVE IN COMMON
- 4 THE STRATEGY TO FOLLOW
- 4.1 Analyze and model the processes.
- 4.2 Analyze the messages.
- 4.3 Design the standard messages.
- 4.4 Design the Services.
- 4.5 Policies
- 5 CONCLUSIONS
THE INFORMATION ECOSYSTEMS
I invite you to watch the subject from different scales. We could include more, but in order to handle a controlled scope in this post, we will talk about three scales.
When we think of the Internet of Things, we may probably first bring to our minds the Smart Homes. Houses where all devices, appliances and every object surrounding us are able to exchange information. This information, properly exploited and controlled, can make our lives more comfortable and bring us security and new experiences.
This would be the first scale: the Smart Homes.
Now let’s leave our home. Let’s drive for a while. Today it is quite common connecting our mobile phones to the car, for example, to use the hands-free system for making and receiving calls, or to use the GPS navigator, or to listen to our music through the car’s audio system. All these things are great, and indeed there is connectivity in our car but, what else? That’s all? Yes, right, you can also receive those text messages but, to be honest, when you are driving you can’t read messages on a screen, although the screen is on your car dashboard.
Well, we will see more choices right away. At this point, let’s set here the second scale: the so-called Connected Cars.
Well, we already have two scales. The third one is rather obvious. When we leave our home, either by walk or by car, we get into our city’s streets. We walk by shops, restaurants, drugstores, healthcare centers, hospitals, banks, department stores, public offices such as the Town Hall, post offices, etc. Of course, we all like to find our streets clean, the street furniture in good conditions, and everything working. Even better if we can access useful information. Well-kept parks, lighting, etc.
With no doubt, our cities are genuine ecosystems where many business and services, both private and public, live together. And the citizens increasingly demand more information, more interaction and more automation. Avoiding queues, managing our bank or health issues from our mobile devices, planning our leisure time with quality and updated information, etc.
Well, this is our third scale: the Smart Cities.
Each of these scales, or scopes if you prefer this way, consist of a set of systems that interact with each other, by exchanging information. They are, therefore, Information Ecosystems. Accordingly, what we are talking about are actually three Information Ecosystems: our Smart Homes are an Information Ecosystem, our Connected Cars are another Information Ecosystem, and our Smart Cities are another Information Ecosystem.
And just as it happens with ecosystems in nature, the different Information Ecosystems interact with each other, as we will see right away.
Let’s see now some imaginary examples that we will find in our future (in some cases, our present), that will help us illustrate the rest of this post:
Your fridge send you two notifications, which you receive in your mobile devices. The first notification informs you that your yogurts will expire within five days. In the second notification, it informs you that the milk and the vegetables will run out in two days, and it asks you if you want to send an order to your supermarket. Upon your confirmation, you receive in less than a minute another notification which confirms that your order will be delivered at your home the next day.
Your car approaches your destination city (it may be the city where you live, or where you work, or just where you are traveling to). The onboard system informs you verbally that information about available parking has been received, as well as traffic information in the main entries of the city. It asks you for confirmation to set your GPS navigator to the best route available to the available parking around your destination, avoiding the heaviest traffic zones. Or maybe you have setup your onboard system to make this by default, without needing your confirmation. This route, of course, is updated in real time as your car receives more information from the city. Besides, the information about the parking places is received through a message exchange process between your car and the city, like this:
- Car: destination coordinates (X, Y).
- City: the available parking places, sorted by distance to your destination area, are in this list: List (X, Y). And the estimated number of vehicles with your same destination area is Z.
This way, our car will estimate the best available area probability, based on the estimated inflow of vehicles and the existing traffic in the different entry routes. It will set up an efficient route, which maximizes your choices to find a free parking, effortlessly, in the shortest time possible.
Before you go to bed, you program your waking time on your mobile device. You select a previously setup routine, called “Working Day”, which consists of:
- slowly turning on the lights in your bedroom 30 minutes before the time, until waking you up at the desired time,
- filling your bathtub with certain liters of water at a certain temperature,
- keeping your coffee warm in the kitchen,
- setting the house temperature to 23 degrees Celsius,
- playing your favorite music,
- showing your RSS feed selection on your TV screen,
- switching on your car’s air condition 10 minutes before you leave,
- when you leave home, launching the routine “Gone”, which sets all your systems to power save mode, except your security alarm.
We could imagine almost any use case, but for this entry I think this will be enough. Let’s go on.
WHAT THEY HAVE IN COMMON
All these scenarios, and any other we could think of, share many features and needs. Identifying them will help us outline the best strategic solution to implement these use cases, which looks so close to science-fiction.
All the uses cases are defined from the user’s perspective, never from the participant systems’ perspective. The processes that are of interest to the user determine the information flow that we want to model and solve. By no means the user should adapt to any restriction related to technology. As you can see, we have mentioned no specific software or device. They are all technology independent cases.
We have mentioned no participant devices’ trademark or supplier either. We have not mentioned the vehicle’s trademark, neither the bulbs, nor the air condition system, nor the TV set manufacturer, nor what RSS reader he uses… nothing at all. So, we are assuming that we need every system involved, any supplier, to be able to play its role in these information flows. The supplier independency seems to be another feature to be considered. It such independency is not guaranteed, the user would have less choices to use the devices he or she likes more, or to change them whenever he or she wants, to enhance them or just getting rid of them. This leads us directly to the next feature.
The devices should not directly exchange information with each other. That is, they need a director system, a mediator system which should orchestrate the different events, and which should ensure that each device sends and receives the messages timely and in due form. Another way to articulate this third need would be like this: each Information Ecosystem should minimize coupling as much as possible. Coupling refers to the dependency between the different systems involved in the processes. The use of a mediator system, together with an Interoperability Standard which defines the messaging that should be used, allows you to guarantee the least possible coupling in the Information Ecosystem, thus increasing user independence with respect to each manufacturer, as we mentioned above.
Finally, we will mention as well that the user needs a control panel, a dashboard with which, apart from enabling or disabling parameters and cases, the user can set up new cases, new routines, for every Information Ecosystem to adapt to his or her changing needs.
THE STRATEGY TO FOLLOW
Regarding the common features and needs that we have identified, we are going to imagine a sketch of a possible strategy to follow, step by step. Note that we are never talking about technology.
Analyze and model the processes.
For each use case, or information flows that we are handling, we should perform an analysis which leads us to identify, at least, the following:
We need to know all participant systems, including the user where applicable, of course. We need to identify which systems play a role in the case, either sending information, or receiving it, or both.
We need to know which role is played by each actor. Who produces information and who consumes it.
It is necessary to identify what relevant facts of the process determine the information flow advance. Sometimes we can find time events (as the waking time), or case-related events (as the car receiving information from the city, or the milk’s stock decreases under a setup amount, or we leave our home).
When is information sent or received? Typically, this information sending and receiving will be linked to the events. In fact, these events can be message sending and reception events, because it is often in those moments when the information flow steps forward. It is necessary to know how many messages should be sent and received by each actor, as well as identify which messages are involved. At this point it should be taken into account that, sometimes, one single message can be sent from an actor to several actors.
Once all this information has been gathered, we should model the processes to graphically represent them and understand them better. We could find similar patterns, maybe identical, in different use cases and even in different Information Ecosystems.
Analyze the messages.
In each message previously identified, what information should be included? What data are necessary? What data are of interest though not essential? What data does each actor have in order to play its role in the case and what information would it need to receive? Which actor should provide that information? Is that information specific to an actor or could it be general information, common to the whole Information Ecosystem?
This final nuance is very important, because you can find very often a self-proclaimed information provider system, but that information belongs to the Information Ecosystem instead of that system. When this happens, the system causes coupling (dependency), affecting the Ecosystem’s capability to grow and adapt to the real use needs, and the freedom to change one system for another is lost.
So, it is important to clearly identify which is the authentic origin of the data needed by each actor, considering no other actor: analyzing only the use case in the Information Ecosystem’s context.
Design the standard messages.
The key word here is “standard” and the key concept to introduce here is Interoperability. In the messages design we should use an appropriate Interoperability standard. It is desirable that each Information Ecosystem has its own Interoperability standard. Such standard should perfectly catalogue the possible events corresponding to the real functioning of that Information Ecosystem. With each event, it should specify the messages that can go with it and the associated information, mandatory and optional, including the messages’ structure specification.
This point is essential. If an Information Ecosystem does not have an Interoperability standard for all the actors involved, then the complexity for implementing the information flows shoots up, the possibility of error grows exponentially, and some actor will most likely command its own messages schemas to the rest of systems. Again, dependency and coupling appear, which endanger the scalability and the agility of the whole Information Ecosystem.
Let me insist on the importance of this subject. Information Ecosystem change because the world changes, and every city’s, every driver’s, every citizen’s circumstances do change. And in changing their circumstances, their needs change. Information Ecosystem should be able to adapt to those changes fast and with the least possible impact. That’s why coupling is the enemy that should be defeated. If we don’t have this in mind from the first minute, it often shows up when we realize that we cannot change anything without a huge economic and reengineering effort.
Design the Services.
We know the participant systems, which system sends and which system receives each message, based on which events. We know the patterns that arise in the modeled processes, and we know which of them repeat within an Information Ecosystem, or even in different Information Ecosystems.
We have identified the external events of each Information Ecosystem, typically thrown by other Ecosystems, of interest to each use case.
We have designed the standard messages, in a homogeneous way, by using the Interoperability standard of each Information Ecosystem. It should be noted here that when several Information Ecosystems are involved in a use case, each messaging standard should be understood by each Information Ecosystem. This means that we will have to be able to “understand” schemas different from ours, and “translate” their data to our messages. And vice versa.
Now, we can start designing the Services Catalog. We should bear in mind that, for every use case, it could be enough to use one service, or it could need the use of several services. This will depend on each case complexity and our own design decisions.
Another important detail when we are designing the services is the concept of granularity. This concept refers to the atomicity level of the service, or in other words, its abstraction level. And all this is directly related to reusability. It is no secret that, by promoting software reusability, we are saving costs and improving maintainability. In an Information Ecosystem, service reusability is another key factor to achieve the agility and scalability, because it directly impacts on the projects’ costs and the systems quality.
A too abstract service, that is, with low granularity, offers a high reusability. To understand well these concepts, think of an example. If we use the concept “vehicle”, we can use it to talk about any journey: we use a vehicle to fly, a vehicle to sail, a vehicle to drive with the family, a vehicle to drive on your own, a vehicle to travel far on rails, a vehicle to travel with a group… And in each use, we can reach the detail we want because the concept “vehicle” is abstract enough to use it in any case.
However, a service with high granularity, that is, with a low abstraction level, offers a lower reusability. Following our example, if we use the concept “bicycle”, we can use it in a reduced and more specific set of cases: we use a bicycle for a mountain hiking, we use a bicycle for pedaling around the city, or we use a bicycle for training. And nothing else.
For each service, there are a few elements we need to identify:
- A service provider, which sends the corresponding message,
- A standard message, which contains the information needed for the event,
- One or several service consumers, which receive the message,
- A certain business logic, which corresponds with certain validations, accesses to Ecosystem information, transformations or any other internal process required by the data received from the provider, before sending them to the consumers.
All this information make up the service’s Static Model, that is, its definition from a formal, structural and design point of view.
When we identify all this information it is very important to remember the requirements that we discussed in the section 2. WHAT THEY HAVE IN COMMON: for example, identifying the service provider is not specifying which particular system provides the service, but the “type” of system, keeping the necessary abstraction level that guarantees the manufacturer and technology independence.
For each service, we should also define its behavior, that is, its Dynamic Model.
We need to define in detail the communication sequence that will take place between the provider and the consumers involved.
In the Dynamic Model, we should involve a new actor. When we discussed the requirements shared by all the use cases, we saw that we needed a mediator system. This system, as its name suggests, mediate between the participant systems. Its use will help guarantee the independence between the different systems, and the user independence from the participant systems. Loose coupling is a key feature that we must seek in our strategy, to guarantee the future sustainability of the Information Ecosystem, its scalability and its maintainability. We are repeating this several times, but it is on purpose: it is very important, and it is not easy to achieve.
A very helpful technique to specify the communication pattern that will use the participant systems, including the mediator system, is the use of a Sequence Diagram. As you probably know, it is part of the UML standard. They allow you to graphically represent in an intuitive way what each actor does at each time while they are interacting.
The Dynamic Model should include, additionally, a description of the business logic to be resolved by the service. Some examples of business logic follow:
between the input message schema and the output message schema. That is, specifying what data from the input message should be assigned to which fields in the output message.
to be made upon the input message data, to be able to build the output message. For example, a value of the output message which does not come in the input message, but is obtained by a combination of other available data, or based on some value received in the input.
needed to take certain routing decisions, based on the input message data.
Although we are in the fifth step, this is actually a previous one. It could be more precise to say that this point encompasses the rest. In any strategy, the definition and maintenance of the rules, the policies, the procedures that should be followed by all participants involved in the implementation of the strategy, establishes the framework on which everything else should be done.
This includes, for example:
which establishes the design model of the messages and the global details to be considered in the implementation of the standard. Every standard is defined with a high abstraction level which allows its implementation in different scenarios. It is now, when implementing it in an Information Ecosystem, when we should lower that abstraction level and specify what corresponds to that Information Ecosystem, but without reaching a too low abstraction level. This fine detail will correspond to each message design, in each use case.
a basic feature of the mediator systems, which allows overcoming in an automatic and transparent way any possible communications issue, or in the event of any consumer system unavailability. When a case needs a specific retry policy, it should be included in the Dynamic Model of the services involved.
Errors Handling Policy
which establishes for all the actors how errors should be processed. Here we refer to the errors related to the information flow and the business logic involved. Therefore, the internal errors of every system are out of scope.
of any kind, such as how we will register and control which system uses which services.
which will include the encryption techniques, firewall rules, token systems, credentials, etc. that will be used to guarantee the security in messages sending and reception.
And who will define all this? Intuitively, it is clear that it should be a team with authority in the Information Ecosystem. In ICT, this is what is called Governance, another absolutely essential key factor for the implementation of the strategy.
At the beginning of this post, I told you that when it comes to systems integrations, everything is the same. And you will agree with me that the Internet of Things is about integrating systems. Whether the Smart Homes, or the Smart Cities, or the Connected Cars, three Information Ecosystems that share the Internet of Things background, in the end they are all the same thing.
But, as we have mentioned many times in this blog, every enterprise or public organization is an Information Ecosystem. We can consider this our fourth case, for this post ‘s purpose. All that we have discussed up to this point, equally applies to the Internet of Things use cases, as well as for companies and public organizations.
So, as our first conclusion, we have a strategy which applies to merely every system integration scenario where certain goals are sought:
- Technology independence
- Supplier/manufacturer independence
- Process orientation
- Loose coupling
Additionally, we have seen that, through the definition of rules, policies and procedures from Governance bodies of each Information Ecosystem, the necessary homogeneous framework is established to achieve the afore-mentioned goals.
And we have presented a simplified sketch of a System Integration Strategy which would allow achieving those goals in any use case that we could imagine, for any Information Ecosystem.
The strategy broadly shown in this post is called Service Orientation, or rather SOA Strategy.
The mediator system is known as middleware, and one of its main components is the ESB (Enterprise Service Bus).
The process analysis is performed through BPM techniques (Business Process Modeling), using the BPMN standard notation for process modeling.
The event orchestration is designed from an EDA approach (Event Driven Architecture), which is a design style which promotes the use of asynchronous messaging, triggered in real time by the events of the Information Ecosystem.
The messaging can be designed through XML and the services can be implemented through web services, either using SOAP or REST (both styles present pros and cons). Although you can use other technologies, not only web services, since middleware suites are equipped with multiple adaptors to avoid this dependency.
Well, and this is what I wanted to show you with this post. I hope you have found it of your interest. If you have any comment, please leave it below. Your opinions will always be rewarding.
Thank you for your time, and if you liked this post, your contacts may like it too, so don’t hesitate and share it on your social networks.