Synchronous or asynchronous, that’s the question
Throughout this blog we have commented several times that the Service Orientation supposes an important change of focus in the way of designing the information systems in an Information Ecosystem. The impact is even greater when we rely on an event-driven architecture (EDA), because it introduces one of the most striking changes to the traditional way of integrating systems: asynchrony; And with it, decoupling. So in our work as SOA consultants and SOA architects, we will often see ourselves in situations where the doubt arises: synchronous or asynchronous integration pattern?.
In this entry we are going to deal in depth with this question. First we will analyze the pattern of synchronous integration, and then we will analyze the corresponding asynchronous pattern.
Tabla de Contenidos
- 1 Typical synchronous integration pattern.
- 2 Asynchronous integration pattern (Event Driven Architecture).
- 3 Conclusion: Synchronous or asynchronous integration pattern?
Typical synchronous integration pattern.
This graph shows a very simplified outline of the usual scenario in an organization, regarding its systems integrations. We will assume throughout the example that web services are used as technology implementation of services.
The synchronous service provider.
In the lower part, a certain system publishes a web service to make some information available to the rest of the information systems, through the request-response pattern.
In the functional area that affects this information system, events of different type occur (human actions, automatic processes, or scheduled tasks, among others). Some of these events alter the information contained in its database, so that the information corresponding to the entity that maintains this system, evolves according to these events. Suppose, for example, that this system is in charge of maintaining the information of the customers of the company.
At a certain point in time a customer can be registered, modified or deleted. These basic operations take place over time. In the graph we have represented that certain moment in time (Time 1), in which one of these events happens (Event 1). Do not forget that before that moment, other events have happened on the entity “Customers”, and after that other events will continue to happen.
But our system simply processes them locally. Facing the rest of the organization, it simply has a published web service to be invoked by other systems that may need its information.
The synchronous service consumer.
Now let’s look at the system on the left. This system has its own functional scope, and its users use it to, for example, maintain the billing and collection status of the organization.
At any given time, in some of its use cases, it is integrated with the customer maintenance system through a call to the web service that system has published. At this other time (Time 2) this system invokes the web service to request information from a client.
Depending on the quality of the communications and their status at the time of the invocation, our user could find an unreasonable waiting time. Moreover, it could happen that at the time of that invocation, the web service was not available (for various reasons, from a specific communications problem, or a temporary problem of the operating system, or a problem of the application itself at a time of locked tables in its database, etc, etc).
In this case our user would receive an error message and would have to re-query the information.
Suppose the web service responds, and returns the client information. Now our user can continue his work with information that will not be updated until a use case that includes the invocation to the customer service web service is returned. Until that moment our user is working with obsolete information, only updated when the remote web service is invoked and it responds.
Meanwhile, in our first system, the events that are happening may be affecting the same client: a change of address, a change of bank data, a change of contact telephone, or simply a client cancellation. Of all these events our system 2, billing and collection, will not find out until it re-invokes the synchronous service.
Let us now turn to the system on the right. This system could be for example the customer complaints manager.
When this system needs to consult the data of a client, it uses the same web service of consultation of clients that publishes our first system. But this usually happens at another time (Time 3), possibly when a customer files a claim, or when that claim is going to be resolved.
It is quite probable that the information that refers to this system at this time is different from the one that was consulted by the billing and collection system. And besides, it could happen perfectly that after consulting it, an event occurs on the entity “Clients” of some relevance for the whole business, such as the client being discharged from the company.
Typically, the systems that implement this type of integration solution have other deferred processes that try to reconcile massively all the information of clients in the different systems concerned. But for a considerable portion of time, customer information throughout this small systems map has completely lacked referential integrity, a basic and crucial requirement for the information quality, and therefore the functioning of a organization.
The Dashboard under a synchronous integration pattern.
Before finishing with this synchronous model, let’s now look at the system that appears at the top of the graph. It is a managerial profile, which uses scorecards and information exploitation systems, to obtain aggregate views and statistics, which help in making corporate decisions that affect the operation of the company and its strategic decisions.
With all this panorama of outdated information and events that are not known, what is this user viewing?
At any given moment, random, are you sure you have a real, complete, accurate and homogenous view of what is happening in your systems map? Perhaps just after the data is reconciled across the systems map, you can have a consistent view.
But is it real? Is it complete? Is it accurate? How long does it maintain the required quality?
At this point it seems clear that the doubt between using a synchronous or asynchronous integration pattern is clearer. But let’s continue.
Asynchronous integration pattern (Event Driven Architecture).
In this other graph we see the same systems map, but this time under an SOA strategy that uses the event driven architecture (EDA) as a design principle. Let’s look at the situation we described above with this new approach.
For a start, we have a new actor: the SOA infrastructure. For the purpose of this entry, assume only a very light ESB with some of the usual capabilities on the market: configurable retry policies, delivery guarantee, persistence, auditing, adapters for different technologies (web services and others), etc.
The asynchronous service provider.
Let’s look at the customer maintenance system, at the bottom.
Now this system does not publish a web service. Now, every event that occurs over the entity “Clients” is communicated to the SOA infrastructure through a web service published in it. This communication is synchronous between these two actors: our system receives a response from the ESB in the same thread, validating the delivery and correction of the message that it sent with the registration, modification or elimination of the client (or event).
To whom does it send this information? It does not need to know, nor does it care. Let’s say it’s sendt to the Information Ecosystem. And for our customer maintenance system, with the confirmation of delivery to the Ecosystem, it is sufficient to end its transaction successfully.
So far what we have is the assurance that all events that occur to the entity “Client” reach the SOA infrastructure of our organization in real time.
The asynchronous service consumer.
Now let’s look at billing and claims systems for a moment.
Both systems now publish a web service, which is invoked by the SOA infrastructure. When? As soon as it receives an event to which these systems are subscribed. The SOA infrastructure maintains subscription lists that allow it to know, before the arrival of each event, which systems need to know that information.
In this way, the information flows in real time to all the systems that need it.
In our example, both the billing system and the claims system receive, through their respective web services, the event that has been sent by the customer maintenance system, and they receive it at the same time (or at most a few seconds later).
Each of these invocations that makes the SOA infrastructure is synchronous between both partners, and as soon as it receives the confirmation of delivery in each destination, the delivery is given for correct.
What happens if there is no such confirmation? In that case, the retry policies defined in the ESB are executed automatically until the destination is reached and it answers back. If the answer is correct, the process ends. If the response is not correct, the error is logged and the delivery attempts are interrupted.
On an overwhelming majority of occasions, events reach all of their destinations virtually in real time (or after a few retries) and with correct result. To such an extent it is overwhelming most successful deliveries that the ESB boast of the delivery guarantee as a factory feature.
Thanks to this, the information in the entire system map is correct, homogeneous and up to date.
Each of the communications between each end is synchronous (logically), but the integration between the three systems is asynchronous. None of the systems depend on any of the others for the end and success of their use cases. This is called decoupling and is crucial for scalable and flexible information ecosystems.
Finally, let’s think about the managerial profile that wants to see access to the aggregated and statistical information of the company. In this second scenario, you will always have real, updated and complete information.
Conclusion: Synchronous or asynchronous integration pattern?
In some circumstances the synchronous integration pattern may be indicated. But asynchrony and decoupling bring significant additional benefits to an organization’s system map. So important benefits that we must consider the asynchronous pattern as the first option in all cases.
In a synchronous map, when the customer information system (or any other) needs to evolve in some way, side effects (impacts) can be identified in the other systems. Projects become unnecessarily expensive, and may even be unviable over time, due to constraints on available budget, or sufficiently qualified equipment.
In an asynchronous map, however, when the organization needs to modify one of the information systems, collateral impacts are not present. Each information system can evolve independently, and if the change reaches more than one system, both projects can be undertaken independently over time, resolving any temporary eventualities in the SOA infrastructure (transformations, mapping, etc).
In this way, the ability of the entire organization to adapt its systems to the needs of the business is much greater.
So, for me, is it better to use a synchronous or asynchronous integration pattern? No doubt, I choose the asynchronous pattern.