The 10 most frequent errors implementing an SOA strategy
Achieving success in implementing an SOA strategy is difficult. Although we have all the clear concepts, we are experts in BPM, we have high level professionals and experience, and everything is ready in our mind to get to it, it is extremely easy to derail the strategy. You have to align many planets to achieve it, and as you know the planets do not move quickly.
The experience is always a good adviser, so let’s take advantage of it and dedicate this entry to list some errors that I consider more likely during the implementation of an SOA strategy in an organization.
Surely you also have some experience from which we can all learn, so I invite you to use the comments and share what you think appropriate with us.
Tabla de Contenidos
- 1 Consider it a technology.
- 2 Disregarding technology
- 3 Do not involve the appropriate stakeholders
- 4 Seek global results too soon
- 5 Proliferation of services (too much granularity): low reusability
- 6 Excessive abstraction (insufficient granularity): Too much reusability
- 7 Misuse of standards
- 8 Synchronicity abuse
- 9 Stockholm Syndrome
- 10 Absence of Governance or weak Governance
Consider it a technology.
If you are thinking something like “oh no! You will say it again!”, it is a sign that you are following my previous entries with attention (of which I am glad). It’s true, I never get tired of repeating that SOA is not a technology. It is a common misconception, which leads to frustration, to disappointment by not seeing anywhere all the wonders that SOA promises. And it spreads the idea that SOA is smoke … and it does not finish taking off … and that’s how it goes …
When it is strategically and organizationally focused, SOA starts well. If not, it starts hurt, and the closer to technology the focus is, the more deadly that wound will be.
Come on! What? Make up your mind…
Calm down … one thing is that SOA is not a technology, and another thing is that it stays on a theoretical plane. SOA startup obviously needs at some point a technological implementation to bring all its benefits to reality.
Here, the risk hides in thinking that the implementation of SOA services is easy, or thinking that permanent auditing of compliance with interoperability regulations or any other policy dictated by SOA Governance can be relaxed.
In this sense it is necessary to keep the muscle of abstraction well aware, even in the construction of services. The components of the orchestrations themselves should be encapsulated and reusable, the pursuit of a kind of “box of lego” should be always in mind, with the pieces to be able to assemble many constructions, lots of services.
If we do not do so, we will risk having the coupling at home, within the Services Catalog itself. Certainly a terrifying idea. And all the agility we want to win out of the catalog, thanks to the catalog, we lose it within it … Which in the long run will be a growing burden for the whole strategy and, finally, its degradation.
Do not involve the appropriate stakeholders
As a strategy in ICT, SOA has to involve many people, from different profiles, with different interests and degrees of responsibility within the organization. It is imperative that all these stakeholders are properly identified and involved in the strategy.
It is useless for development teams and their project managers to do the best work, if their managers do not set goals aligned with the strategy. Likewise, if those responsible for the organization have assimilated the strategy, but those responsible for implementing it on a project-by-project basis do not follow the guidelines and philosophy of SOA Governance, it is of little use.
Diplomacy, sensitivity to the different profiles and trajectories of those responsible at each level, should be highlighted in order to work towards aligning all of them with the new strategy, without being compelled, but participating in it, while maintaining their control and supervision scopes. It is necessary to know how to show each profile the vision of SOA that best fits with its interests. And of course this work corresponds to SOA Governance.
Seek global results too soon
Implementing an SOA strategy in an organization involves a certain investment. And as with any investment, it is legitimate to want to see a return on that investment. And this is one of the darkest and elusive points of SOA.
The return on investment of an SOA strategy is neither immediate nor totally economic. It always translates into savings of collateral costs, important savings like:
- shorter projects, thanks to the reuse of services and the elimination of redundancy of requirements,
- cost-effectve maintenance, having a map of systems decoupled and therefore reducing the impact of any change,
- improvement on efficiency of the communications network, by eliminating mass information traffic that required all network capacity.
But the return on investment will also come from:
- agility to react quickly to competition or new business demands and challenges by reducing time-to-market,
- scalability, to grow without impacting the rest of the organization,
- flexibility, being able to change one system for another without impacting the others, thanks to the decoupling of the systems map: all you need is that the new system publishes and consumes the same services,
- ability to exchange information through standard services, improving strategic business moves (mergers, agreements, etc.),
- optimization of business processes, thanks to the analysis of existing processes and the detection of patterns, events, rules, etc.
And all these benefits are (as well as extremely complex to estimate and calculate) slow to appear, as slow is the implementation of an SOA strategy. Any other approach is not realistic.
Proliferation of services (too much granularity): low reusability
This problem is most likely when we are extracting bottom-up services. It is easy in this case to identify the software components and integration solutions already made in spaghetti mode in the existing systems map, and it is very easy to incorporate the same use cases as SOA services in the same way into the catalog, or simply with a standard layer.
While this may be a valid first step to minimizing impacts on existing systems in their shift towards SOA, they should not be left behind. It is necessary to prepare and follow a roadmap to analyze the entire process affected by these low-level services to obtain SOA services with the correct level of abstraction, to actually change the orientation of those systems and provide the organization with true standard reusable SOA services.
In a top-down approach, analyzing business processes, it is important to look for, not only more or less atomic processes that are repeated to include them in the Services Catalog, but above all patterns of events, flow and processes that are repeated. These composite services often provide much more power, agility and capacity to the organization’s SOA Services Catalog.
In both approaches, a level of granularity that is too high implies a very low reuse rate, losing saving capacity, homogeneity in the systems map and greatly complicating the maintenance of the catalog.
Excessive abstraction (insufficient granularity): Too much reusability
Most likely in top-down approximations, this is the opposite of the previous case. When we stay at a level of abstraction so high that practically is always reused. You have to be careful with these cases, which can seem like a success (for such reuse rate).
A problem arises around the possible evolutionary needs that arise. I think it will be better understood by an example. Imagine three business processes, processA, processB and processC. All three include in their flow an initial event called equal, say notifyEvent. Given this similarity we can think of creating a single business service, notifyEventProcess, which would be consumed by all systems that need to know that this event has occurred in any of the three processes (A, B and C).
So far we seem to have done something very good. With a single service we are solving a present need in several information systems.
But with a deeper analysis we find out that, actually, processes A, B and C are different processes. Although all three include an event called NotifyEvent, they are different processes. What happens if one of these processes changes, evolves or is replaced by another? Since our service notifyEventProcess must be modified, the information systems using that service would be affected by that change, not only those affected by the business process that has changed.
Usually in these cases, the temptation to create a service for all processes that match events with the same name must be overcome. The important thing in an event is not its name, but the event itself. And if they are different events, even if they have the same name, they must trigger different services. In our example it would be better to create three services: notifyProcessAEvent, notifyProcessBEvent and notifyProcessBEvent.
Another additional problem is located in the monitoring of the business. The monitoring tools audit and count a number of indicators that are distributed throughout the Services Catalog, to measure the quantity or frequency of certain events, duration of certain business processes, etc. If different flows, events and processes are “solved” with the same abstract service, what do we monitor there?. It would be necessary to get into the logic of the service to make monitoring dependent on that logic, or dependent on certain data transmitted in the processes.
Misuse of standards
Saying “I use standards” does not mean that they are used correctly. To say that using standards we are complying with this SOA premise is not entirely true: we must use them well.
In this I would like to comment on two issues.
On the one hand, the definition of a standard is like a mold, like the definition of a class if you prefer the analogy with object orientation. A standard defines by default everything necessary to solve a particular scenario where that standard is applicable. But it is important to realize that its definition is by default, that is, it is used as a starting point to reach an implementation of the standard, making it fully operational in the organization and the specific case in which it will be used.
Therefore, it requires an important effort of analysis of the business in general and of the organization in particular, to know where the standard can be used as proposed (by default), and where it is necessary to substitute what the standard proposes with concrete values of the organization itself. In this way a kind of instance of the standard is created, an implementation of that mold is extracted, which is fully functional and operative in the reality of this organization that is implementing it. This exercise often entails other important advantages for the information management by the organization, since it allows to collect and unify all the business semantics of the organization, thus ensuring semantic interoperability inside and outside the organization.
On the other hand, it is necessary to be very precise in the implementation of the standard. Ambiguity is a very elusive enemy, and it is very easy to fall into it even though you may be reasonably sure that you’re not being at all ambiguous. The most typical example I always mention when I speak of this subject is based on a real case that I lived at work. Defining a node of a schema of a service, we published that it should be informed with the “professional identifier”. This seemed to be quite accurate. But then we discovered that for some teams the professional identified with a local id, for others he identified with a national code, for others he identified with his corporate user, for others with his email address … Of course we had to modify the definition of that Service to indicate expressly and with absolute precision that in that node should go the identifier of the professional that consists of the national code of identification.
The spaghetti scenario usually includes a good fist of proprietary web services published by different information systems to receive request-response type invocations. These services are naturally synchronous.
As we have discussed in previous entries, we have two possible approaches to identify SOA services: top-down, starting at business processes), and bottom-up, starting from the existing systems map and integration solutions in use. This last approach means analyzing the spaghetti map and discovering SOA services.
The abuse of synchrony usually comes from translating too much identity to those Web services that query the SOA services catalog, thus dragging a synchrony between the systems that need that information and the system that provides that information. This is the fastest route, the least traumatic, and the least expensive … short term. Because in the end, the systems remain interlinked, and therefore we have made little progress.
In these cases, this inclusion of synchronous services can be supported if we use SOA patterns especially designed for these bottom-up approaches, such as the façade pattern. This pattern suggests that the proprietary service is maintained as it is, but that no system invokes it directly. Instead, they must invoke a standard service on the middleware layer that hides the original service, which is the only point that invokes it and thus disconnect the information systems involved. These types of patterns are often useful as an initial step in a roadmap that leads to a complete change of approach towards an orientation to events, where the system publishing the service waiting for queries ends invoking an asynchronous service to notify a Event, and the systems invoking the initial query service end up subscribed to that service to receive the associated information. The asynchrony of an event orientation introduces an ideal decoupling between the participating systems, and this is the scenario to look for.
During the implementation of an SOA strategy you must develop an important skill to adapt your speech to the audience. This feature, which could be recommended in many situations, is essential here. You have to tune in properly with your interlocutor to guide him or her properly in the change of mind needed, in order to understand, assimilate and apply the philosophy of an SOA strategy.
The same is true with the Head of the IT Department of the organization, or with the functional experts of each functional area, or with a manager of an external supplier, or the project managers of the organization, or the person in charge of Change Management, etc.
Don’t miss that each and every one of these actors will resist change. Some more, others less, but all will resist. Even those who claim to be totally convinced and sincerely bet on the strategy, will drag practices, customs, ways of doing things, that have been used for many years and that, in principle, tend to maintain (understandably or in their own interest).
For SOA architects (or SOA consultants, SOA analysts, SOA developers, etc.), it will often be difficult to stay straight on the correct path marked by the strategy. We can easily find ourselves accepting proposals that are not aligned with the strategy, or consenting to practices that jeopardize the execution of a certain roadmap to achieve a correct service orientation, etc., etc.
To these situations we refer here when we talk about the Stockholm syndrome of the SOA consultant. It is essential not to fall into interests outside the strategy, neither give in to proposals that are not aligned with the strategy, which are usually accompanied by a certain pressure from the teams that raise them. To do this you have to go to the SOA governance office whenever you have doubts or a risk is identified in this regard. This group should have sufficient decision-making capacity or sufficient reach to the organization’s ICT direction to adequately position these situations, or elevate them so that they are tracked from the direction.
Absence of Governance or weak Governance
I think that along other entries and in this same post, we have already seen very clear examples of problems and risks whose solution passes through a strong and constant SOA governance office.
Governance is the main driver of the SOA strategy, so that its absence would make the strategy unfeasible, and its weakness would make it extremely difficult to resolve potential conflicts and risks. Without such solutions, proper progress of the strategy would be impossible.
When this happens, some information systems do not end up evolving to SOA. Those systems do not finish publishing clearly identified services, or do not finish consuming them. Full functional areas appear that seem alien to the strategy, and continue to launch projects and new information systems as if the SOA strategy was not with them. This is a double problem because on the one hand, the expansion of the strategy and the possibility of publishing more business services in these areas are delayed and, on the other hand, while the strategy against the spaghetti-type scenario is being fought, the wrong architecture keeps growing where the strategy fails to reach, deepening the problems it seeks to correct, delaying the benefits of the SOA strategy and threatening its credibility in the absence or delay in promised results.
This is why it is absolutely vital to understand SOA governance and its decisive role. The higher the governance in the pyramid of the organization, the better chance of success will have the implementation of an SOA strategy.
I am finishing this long entry with this link. This is a Gartner report (there are others out there) about common SOA errors. Adding our experience and what the people at Gartner says, I think you can have a pretty complete idea of where the main threats are in the way of implementing an SOA strategy in an organization.
Again I encourage you to use the comments below to share your experiences, opinions, suggestions, etc.