Say no to integration requirements

Do you prefer pasta Bolognaise or integration spaghetti for lunch? The latter is a common dish served in IT-departments worldwide.

The usual approach to develop integrations is to define a set of requirements to solve a given use-case and the make an integration between two different systems, either direct or via an Enterprise Service Bus. It works well, until the number of integrations become unmanageable and you stop to reuse because it's complicated.

So should we just throw all requirements out of the door? My answer is sort of yes, but, so lets explain in more detail.

Image from http:/www.wikimedia.org

Image from http:/www.wikimedia.org

The challenge is to define generic interfaces, but not to generic. I was discussing sales process in a previous blog and the conclusion was if a process was to generic, it was was not usable.

If we look at generic interface, then we all can agree that one generic interface to handle all integrations between all systems is probably not a good idea. It would be to complex to maintain and the performance of such an interface would not be so good. At the other extreme, one interface per attribute will be very good performance wise, unless you need several attributes at one time. The number integrations would be staggering and effort to implement front-end logic would be massive. Our solution must be somewhere in the middle to be able to handle non-functional requirements as performance, data integrity and maintainability in a good way.

Let us begin with a hypothetical example. We assume that we have a legacy back-end system and we would like to develop an number of interfaces to this system. How should we approach this problem?

A good starting point is often the customer or the product. Both very vital types of information entities for the business. 

The information about the customer is normally stored in a relational database. We should therefore have interfaces for creating, read, update and delete information about a customer.  We now have four different interfaces to manage a customer. Add a fifth interface for searching for customer if you don't have a customer id and couldn't use indexed tables. If we combine all of these five interfaces to one service called ManageCustomer, with methods for CRUD+S, then we are home free. 

What if we would like to keep track of which order a customer have? Think about creating an interface of type ManageCustomerOrder, would that work?

I would say yes based on examples from industry standards.  If we look at the SID model from TMForum can we identify two components called Customer and CustomerOrder as part of the Customer area.

My suggestion would therefore be to create generic interfaces for each of the level 2 ABE's. Instead of documenting requirements will your task be to define an information model and the derive interfaces from this model. If you doing work for a telco, then use the SID model instead if inventing your own model.

Is this a way of working that would fit in an agile world or is it just usable when having a old school waterfall development approach? I would say it's very agile.

To develop one service for a level 2 ABE would be a task possible to deliver within a two week sprint for one or two developers. When we plan the sprints doesn't we need to know all the fields needed for that particular service. This will be resolved in each sprint. What we need from start are the non-functional requirements on a high-level and the structure of generic interfaces. Not the detailed requirements for each interface.

If we later need additional information from the back-end systems, we should add these fields to the XML-schema, instead of developing a new interface. We also need to add some governance for services and version handling. 

What about when we need more aggregated information and would like to handle complex queries? The solution would either be to put them as aggregations defined by business rules in on top of the integration layer or in the business logic in the front-end systems.

Like to cheat?

You can have a look at OSS/J developed by TMForum members and used by COTS suppliers instead of develop each interface on your own. Based on the SID-model and aligned with eTOM process models can they simplify development of interfaces.