Enterprise Application Integration Transformation modelling using Neo4j and Archimate

Outline

Many organizations document IT enterprise level assets in spreadsheets, and partly implemented portfolio applications; these don’t support what-if integration analysis at the enterprise level. This is a candidate use-case for graphical database technologies.

In addition, to program planning, large system integration projects are subject to continual sub-project delay and reprioritization; the ability to assess these impacts efficiently is desirable. Although EA tools are available, there is limited support for wholistic multi-stage what-if enterprise transformation analysis. Consequently, transformation projects are hard to understand, and high risk to attempt.

This note discusses multi-stage enterprise transformation planning focusing the interactions between independent applications. Modelling uses Neo4J’s graphical database.

?The? TOGAF and Archimate frameworks use the term Plateau to describe intermediate architectural landscapes. A Plateau represents?a relatively stable architecture that exists for a limited period of time.

Modeling transformation with a graphical database represents application components, and integration interactions using the node-edge model. Graph databases suit expressing and generating relationships between applications. Neo4J’s ability to manipulate whole graphs with simple verbs, makes it suitable to modelling this domain.

Plateaus are target milestones in an enterprise transformation. Here Plateaus are created by collections of Rules applied to groups of Applications; Rules may cause changes to interactions between applications:

Neo4J’s Cypher and APOC scripts load the data from flat data sources. The high level Rules are stored in Excel, and executed with on a Neo4J database, through Python. At the end of a Stage a Neo4j database is available, and a flat output. Graphical analysis uses yFiles data explorer.

The example below discusses a simple replacement of a? single highly connected application; real use-cases may involve multiple plateaus,? hundreds of applications and thousands of interactions:

As-Is Baseline State

The blue application will be replaced. Three states are considered, As-Is, Coexistence, and Final.

Coexistence Plateau 1

Coexistence with all legacy and new integration interactions in place. The integration styles are appropriate to the new technology, e.g. direct calls converted to API Gateway or some form of integration SOA, depending on rules and ?allowed integration transitions.

Final State Plateau 2

The blue application is decommissioned; the red application has replaced the blue.

The example represents the Plateau-Rules below; intermediate inventories are created with statistics and effort estimates.

Plateau-Rules

Below are additional modelling rules, these can be extended to support other transformation ideas,? e.g. point Non-Financial enquiries to a read-only cache for listed applications.

Transitive Relationships

?The ability to handle transitive relationships is necessary, usually lack of ?integration detail indicates poor documentation, e.g. Application 1 interacts with Application 2 – without documented hops between the two; graph databases naturally handle this issue.

Multi-hop interactions can reduce to single edges, with encoded parameters, thereby allowing work with mixed complete and incomplete documentation.

The above representation can always be made transitive; below the multi-hop relationship is reduced to a pattern label.

Integration Transition

New applications cause integration methods to change, e.g. new application online integrations use REST style https/json, legacy is usually a buffet of approaches.

The allowed and transition ?effort table is based on standard patterns, e.g. could be: Legacy-01 = Web Service=>EAI_HUB=>CICS transaction, New-01 = API Gateway, using OpenBanking standards. Below shows allowed transitions, and estimated direct effort:

?

Planning and Statistics overview

Below are fictitious statistics using seven-plateau transformation rule set starting with 437 applications and 6,286 interactions, plateau naming is arbitrary. ?Effort is allocated to legacy to-be kept, and to-be replaced at each plateau. The effort model is viable for integration testing and some parts of SIT.

Plateau and rules combinations can be iterated, re-running the model to understand the consequences of different approaches.

Effort estimates require a Cost Function,? ideally, this includes effort specifics for different applications and integration styles,? e.g. enhancing undocumented AS400 applications requires more effort than modern better documented platforms.

A single cost function is not enough to select? an approach, as there are conflicting metrics and constraints: duration, effort, level of impact at each stage, number of times an application is touched, etc.

Manipulation of the rules and plateaus combinations, will create different plateau states and effort profiles, these should be iterated:

Enhancements

Two main extensions are discussed:

·????? Guided-Modelling

·????? Integration with architectural tools

Adding Guided Modelling

In the above user application knowledge and organization constraints ?are manually encoded in rule combinations. The model mechanically creates plateau states. This can be improved.

Meta constraints that generate and run different Plateau-Rules combinations repeatedly to select “good” rule sets would guide modelling. Examples of meta constraints: ‘must be more than 3 plateaus’, ‘each plateau, no more than 10,000 days’ effort, ‘Application X must be replaced first’. ?Rules and plateaus state contents are auto generated, and then the metrics evaluated.

A practical enhancement would combine hand-written plateau rules, and meta rules. The Archimate language may help to build a basic meta rule editor, with the Motivation Aspect, and Implementation & Migration Layer elements and relationships.

Integration with architectural tools

The Archimate? Enterprise Architecture modeling language, is supported by most EA vendors to some degree.? These tools have import and export functionality. So the transitive connections discussed in the example can be represented using the ‘Flow’ relationships below:

The ‘application-interaction’ structure is also a candidate to store this information.

Using the Open Source Archi tool, the follow process can be used; more automated versions are possible. Using Archi, each Plateau stage could become its own model. So the final loading process would be done, after a viable transition strategy has been modelled. A plugin could automate this process.

?