Processes as Microservices

With microservices architecture becoming the de facto standard for building distributed applications, the drive to bring in the advantages of the 12-factor app methodology across all layers of enterprise solutions is more intense now. A few weeks back, there was a customer that said, “we’ve got mission-critical BPM applications that were built over years on traditional BPM suites and have reached a point where we could no longer easily enhance, modify and maintain the process models, without impacting business services”. Having embarked on cloud journey, the customer felt the need for making BPM applications more modular, nimble and cloud-ready.

Similar concerns are on the rise wherever complex business processes are deployed on BPM behemoths. With disruptive technologies and new business models forcing IT to rethink the value that business automation initiatives deliver, the need for lean and high performant process engines is very evident.

So, what are the options?

Many FOSS BPM projects like jBPM, Activiti, Camunda provide a fresh approach to these challenges and present choices to run BPM engines in Contained, Containerized, and Remote modes. Well, how different are they?

Application managed process engines

In a Contained mode, the process engine can be directly embedded into an application/microservice (say into a Spring Boot app) as a library, thereby allowing each application to manage the life-cycle of the process models contained in it. This is very helpful in architectures that require microservices to handle its state flow, ability to retry a specific aspect of a process flow logic and to provide compensation mechanisms & timeouts for exceptions. Also, it simplifies the communication between application and process engine through simple APIs and provides data consistency / transactional integrity across the contained engine and the embedding microservice.

Moreover, this approach cascades the 12-factor app methodology benefits (especially isolation, scalability & security) to the process engines contained in each of the microservice.

As a simple example, a back office credit card application processing solution could potentially embed process engines for each of the bounded contexts (to effectively manage localized process flows) which in turn would make the deployment model look like this.

Each microservice / bounded context uses its own process engine

Taking a step further, a process model like the one shown below, when embedded within the Background Check microservice would simplify the orchestration of the context-specific stateless services & serverless functions, handle decisions and wait/notify touch points in an elegant manner.

Preprocessor flow embedded in a Background Check microservice to manage process complexities

What more?

More interesting usages of the Contained option would be in edge-points where tiny applications run inside intelligent hubs or smart gateways that have local computing power.  The intelligent correlation and state handling capabilities of BPM can be brought into the edge computing scenarios around autonomous cars, home automation, smart buildings, fault detection, smart cities, green monitoring and so on.

For instance, the combination of a lightweight event processing and a state engine (Drools + jBPM for example) when embedded into a specialty appliance of a railway track monitoring system would help to automate the fault detection and remediation in real time.

Based on the short-lived event streams coming in, the process engine can correlate the events to identify potential track faults, alert track engineers, divert rail traffic to other tracks, initiate detailed track inspections and replacement, activate drones to inspect potential issues and so on.

Sample embedded process flow in an edge-point application

While the embedded approach extends the reach of BPM capabilities across multiple domains, it becomes challenging when:

• Multiple applications/microservices need to use the same process engine
• There are human tasks involved in each of the contained process flows and users need to have a unified view of their tasks in a universal inbox
• Monitoring process states of a large business process that is distributed and contained within several microservices.

A potential solution to this challenge would be to push the process audit data/log from each of the embedded process engines to a centralized data store or to ELG(/K) stack and perform the search, KPI analysis and business activity monitoring across the distributed process models in real time.

Micro process flows

Another different stream of a problem that is often encountered with large and complex business process models is that the BPM applications are often deployed as monoliths. This results in a tightly bound context where making a change to one part of the process model would either affect the other parts of the process models or requires a re-deployment of the application as a whole. Though this could be easily circumvented through careful decomposition of large process models into separate logical units, the ability to run/execute the decomposed process models in a distributed manner would still be a challenge.

Once again, lightweight BPM projects like jBPM provide capabilities to run process engines in Container mode, wherein a BPM engine with a modular process flow(s) can run inside a lightweight servlet container or an application server. Application of Domain Driven Design on large and complex business processes models would help to break down monoliths into microservices/micro bpm process flows and each micro process flow could in turn potentially run in Containerized mode in different process engines. With this model, the CI efficiency of the BPM applications would reach higher levels through non-intrusive BPM application deployments & upgrades (that touches only those micro process flows that undergo enhancements) without any impact to the other micro process flows of a larger business process.

Besides that, when Containerized process engines are deployed into application container management platforms (like OpenShift, GKE etc.), the BPM applications naturally become cloud ready with very high availability and scalability.  A potential deployment architecture with process models modeled as microservices (in the Containerized mode) and put into a container management platform would look like this.

Decomposed micro process flows deployed across multiple process engines in Container mode

With this, each of the micro process flows can auto-scale as per the needs (indicated by groups in the above image). Also, the inter-process and external communications (typically through REST APIs) mediated by a Router Service would hide the underlying complexities in determining the right process engine group to which the API calls need to be routed. jBPM provides a Smart Router that supports this model. The Smart Router exposes a single point of access and routes requests to the right service/process engine group. Besides that, it also aggregates data across services which is very useful for

• Rendering a single task list to a user whose user tasks are distributed across process engines of different groups
• Unified monitoring of different process instances across all process engines

While Contained and Containerized modes open up new ways of building modular BPM applications, a BPM engine in remote mode provides centralized orchestration capabilities. Running a process engine in this mode is much similar to the ways in which traditional BPMS have been used all along.

Well…, then, what’s the right mode to be used for building BPM applications?. There is no one size fits all approach and the choice has to be based on what works best for a given requirement with its underlying constraints.