RPA vs. Native Integration

For years, I have told our integration partners and customers; “Don’t let RPA become the hammer that makes every integration project look like a nail. If you can natively integrate applications through APIs, data exports and imports, or direct writes to backend databases, that is the way you should integrate.” While I still believe this in theory, hundreds of real world projects have educated me on the delicate balancing act most organizations engage in when integrating systems. More often than not, realities on the ground lead us down the path of IT practicality rather than IT purity.

So what are the factors we should consider when trying to decide whether to go with an RPA or native integration solution? Let’s start with availability. Well this seems kind of obvious. In order to integrate applications natively, native interfaces need to exist. However, the definition of the term “exist”, can vary based on the eye of the beholder. It is one thing for a vendor to provide product native integration interfaces, but another thing entirely as to whether those interfaces can be used in a given situation. Here are the factors I consider when trying to decide which way to go:

1)   Is there a cost associated with using the interfaces? While many vendors license its APIs as part of its product license, many do not. In fact, in some cases, those additional licensing fees may be quite high. Further, there may also be additional runtime fees associated with any solutions you create using a given vendor’s API. Make sure you read fine print in the EULA before you decide.  

2)   Does my organization have the requisite skills to use the interfaces? While web services has diminished some of the issues associated with language dependent interfaces, the using organization still needs to have access to a developer who knows, or can learn the interfaces, and write to them given his skillsets and tooling. Can my organization scale to support multiple integrations using multiple vendor interfaces or does it make more fiscal sense to learn one RPA framework and use that for each integration, thus creating a scalable integration framework? This is even more important for system integrators who, by design, must support multiple applications for many customers. For system integrators, employing a scalable integration framework is critical success factor in preserving margins.  

3)   Do the native interfaces support all the functions required? Native interfaces come in all shapes and sizes, some good, some not so good. When evaluating native interfaces, it is critical that the API or import/export facility accommodates all the transactions you want integrated. Don’t assume a product’s native interfaces will support every feature you see in the application’s user interface. It is my experience that they rarely do.

What about writing directly to the backend database? Though a valid option, it is the option most fraught with peril. I say this because writing to backend databases requires an intimate understanding of the database model. If you miss one entity relationship or are unaware of a specific trigger or stored procedure, your integration could be doomed. This becomes even more complicated if you are trying to write to commercial application databases which tend to be larger and significantly more intricate than home grown application databases.

One other thing to consider regarding direct database writes has to do with data validations. Well-designed applications should relegate all data manipulations and validations to the data tier. However, many applications do not follow this rule often including validations in the presentation tier. If this is the case and you attempt direct database writes, it is quite possible you will bypass important application validations and write data the application was not designed to handle. These kinds of insidious validation sidesteps can cause significant application problems and may be difficult to detect. This does not happen when you integrate through the user interface. You are sure to catch every downstream application process regardless of which tier performs the process.

The last consideration when making the RPA vs. native integration decision has to do with monitoring and transaction rollback. Once an integration is moved into production, the majority of the user’s time spent on the process is monitoring transactions, determining causes when exceptions are thrown, and correcting data and/or rolling back transactions. I contend that the easiest way for an end user to deal with these issues is by having them presented in a way that is familiar to the user. In most cases, the most familiar way is through RPA because RPA mimics the process the user already understands. For example, in our Ratchet-X platform, a user can monitor a given bot’s processing in any, or all of the following ways:

1)   View interactive log data in the form of steps where they user can see what specific action is being performed and the input and output data states. The user can drill down into these steps and view the natural language translation of the step defined in the automation’s specification document. In other word’s the user can view the step in the “user’s voice”.

2)   If the transaction is still in process, the user can view screen shots of the bot desktop at any step along the automation processing route. Here, the user is seeing the integration in a “language” he already understands – the language of screen shots.

3)   If the automation is complete, the user can view a recording of the automation transaction. Even a techno neophyte can detect where an errant transaction went wrong by viewing the playback of a familiar transaction.

In the case of many API integrations and imports/exports facilities, the user’s only recourse when things go awry is to pour over cryptic log data in order to decipher which transaction abended and why. Worst yet, the process is even less opaque in the case of direct database writes. However, if your platform can show you where the problem occurred, display the state of the data, tie it back to the specification document and allow the user to see a playback of the actual transaction, I believe the user will have a much easier time managing the integration in production.

Suffice it to say, when it comes to selecting the right integration solution, it is often not our training and instincts that guide us towards the answer. Rather, the right solution is the one that best fits the task at hand. While we should not let RPA become the nail that makes every project look like a hammer, when the project is nail, we should pick the most appropriate hammer.