SOA Strategy, Information Ecosystems and Business Intelligence
If you have read other articles of this blog, you will have seen that in many occasions they appear related concepts like Information Ecosystems, Business Intelligence and the own strategy SOA.
In this article we are going to focus on that relationship, and we will pay special attention to the role of the ESB in that relationship.
Because sometimes I find myself in my professional life with situations where someone asks why they have to go through the ESB. A middleware is often seen as a nuisance, an unnecessary piece in a systems map where, at the end of the day, integrations are already working.
Let’s put ourselves in position. Two information systems integrated with the classic invocation to a web service published by one of them, and that the other invokes when it needs the information. Then comes the new interoperability team, and by the hand of a corporate governance guideline, guide that solution to another in which the ESB has to be in the middle. But not only that, but also change the pattern of synchronous communication by another asynchronous.
Tabla de Contenidos
Why does everything have to go through the ESB?
Sooner or later, someone asks.
Of course, there is no time at the moment, nor is it the right time or place to use all the advantages of a Service Orientation strategy. Decoupling, standardization, reuse, cost reduction, flexibility, scalability, and a long list of beneficial collateral effects in the medium and long term.
But here I do have time and space to tell you, and we will go a little further.
We are going to raise even more the stakes for the enormous benefits that an SOA strategy brings in an organization. To the aforementioned benefits, we can add what was also commented on previous entries: the reduction of time to market, the continuous collection and improvement of business processes, the availability of information in real time, referential integrity throughout the map Systems, semantic interoperability, etc., etc.
Does anyone give more?
You are right:
We are going to argue all this, and knowing myself surely I will use some analogy …
The human body, a reference architecture.
I really like to take as a reference very close examples, which we can all easily check. We start from the premise that our body, as a system of systems that it is, is an Information Ecosystem.
All our systems, at each of their various scales, handle a series of events. These events trigger a series of processes that in turn trigger new events, and so on. The communication of all these events occurs proactively, decoupled, and through the same semantics that allows all the systems involved to correctly interpret the events that are exchanged. We explained it here, remember?
Among all our systems, we will agree to highlight the Central Nervous System as the most important. In it lies the “control center” of our body.
Through a network of nerve communications, concentrated around a spine, our Central Nervous System coordinates and orchestrates all major systems, and responses to specific stimuli (complex events). To do this, it uses a series of rules.
Some of these rules are basic, standard (we eat when we feel hungry, we drink when we are thirsty, we complain to the pain, etc, etc). Others are learned rules, fruit of the social and cultural context of each one of us. And others are rules that are formed throughout our life experience, using past experiences, memories, lessons learned, that allow us to anticipate desired situations, or to unwanted situations, to seek or avoid them respectively. We learn them by identifying certain patterns of events that are often repeated, cause-and-effect relationships, etc.
Notice now a curious detail: some of the things that happen in our body go unnoticed to our Central Nervous System. Sometimes our body fails at a certain point, and we do not notice until some time later. Some of these failures trigger a series of effects that alert us (fever, nausea, malaise), but other times these failures go unnoticed, and come to light when it is late, causing a general failure of the body.
In my opinion, this is a clear design error. Our Central Nervous System should be aware of everything that happens in our body, to be able to minimize damages always, or even to be able to anticipate the risks in order to avoid them.
It is curious that a design as evolved and complex as our body, overlook some events.
Service-oriented Information Ecosystems.
The analogy presented in the previous point is clear. We will move it to organizations and companies.
In a service-oriented organization events are proactively sent to the ESB, which is the backbone of the Information Ecosystem. In it is the logic that allows orchestrating the events between the provider systems and the consumer systems of those events.
Each patient’s income in a hospital, every cash back at a cashier, every claim filed by a client, every insurance part created, every tax generated, every lawsuit, every resource, every meter reading, every medical appointment, every test, … all the events that compete for that ecosystem go through the ESB, which incorporates the corresponding business rules.
Millions of events are received every month. Each event includes the corresponding standard message, loaded with information about that business event.
Think for a moment how many events we receive through our senses every minute. Too many, right? And every hour?. Every day? Every month?
It is certainly impressive how much information our Central Nervous System handles. That is what I call a Big Data.
What does our Central Nervous System do? Does it store and analyze everything? Surely not. It sure filters a lot of that information it receives. It only maintains a part of the information that receives with all those events. Only the part that interests it. And how does it know which events interest it?. Well, maybe there lies part of the key of intelligence.
But let’s go back to our service-oriented company.
We have millions of events hosted at our ESB, where business services orchestrate the exchange of these events, and therefore the operation aligned with the business processes of the organization. After all, the events and services arose from the analysis of the business processes.
We have an organization that is able to function smoothly by orchestrating its business events, thereby managing information in real time, and with a superior ability to react to any need.
But can our service-oriented organization learn? Can it evolve?. How does it fit?.
The SOA Strategy, Information Ecosystems and Business Intelligence.
In that torrent of events that the ESB receives, there is an amount of information not evident but invaluable, which is practically impossible to detect manually. To be able to detect it in order to use it would be a huge qualitative leap.
For example, imagine that in a hospital the number of beds that break down increases slowly throughout the year until reaching a situation that is really dangerous to be able to guarantee the sanitary service to the citizen. How could the hospital know of this circumstance in time?. Ideally it should be able to identify the rising trend of breakdowns in beds soon to begin.
How do you detect a trend in events?
Another example: an important client of a bank begins to withdraw large sums of money from the bank in a short time. I’m sure that bank would want to detect that behavior in time, to try to offer better conditions to a potential client to escape, to try to retain it.
How is a behavioral change detected in events?
Another example: the reading of electricity consumption in a home with five tenants descends abruptly, and remains at very low levels with no apparent explanation. Surely the area of fraud control of that energy company would want to learn of that anomalous pattern of consumption to investigate.
How is an anomalous series of events detected?
Imagine similar situations in areas such as Treasury (fraud inspection), Justice (optimization of judicial processes with excessive time), Health (improvements in professional practices, identification of episodes related to insurance parts and therefore demandable collection), etc., etc etc.
Well, to all this comes to answer the Business Intelligence (BI). Concepts like Big Data, Analytics, Predictive Models, etc., are related to BI. Even CEP is perfect for assisting in identifying patterns and alerts right at the entrance of the entire stream of business events being transmitted, as we saw in a recent article.
Do you realize the enormous ground we have gained in our service-oriented organization precisely because everything goes through the ESB?
Indeed, while we improved the organization’s system map, we made it much more flexible and scalable, and the organization reduced its time to market, and put all information systems at the service of business processes, reducing drastically the costs in ICT projects, and ensuring that the standard information was perfectly interoperable across the entire system map, in addition … we were building a knowledge engine of enormous potential for the BI of the organization.
Identifying complex patterns to measure, setting alert thresholds, and setting up scorecards to visualize business indicators that are considered key indicators, our information ecosystem will have acquired nothing more than its own “business intelligence“.
Now you can anticipate likely situations based on past events that have been filtered because they are of interest to your business. Just what our Central Nervous System does to learn, react and anticipate.
Why does everything have to go through the ESB? …for no particular reason…