When AI meets Document Processing

A constant for many organizations, and in our personal lives, is the relentless increase in speed at which information is growing. One type of information that is part of this is what is referred to as unstructured information; things that don't easily fit inside the rows and tables of a database. Examples include documents, images and videos. Not only the volume of this information increases, but the importance to the enterprise too as a source of data.

In our daily lives, we are confronted by vast amount of data that is completely unstructured and in so many various formats that we cannot just handle through the traditional systems. One example of this we can all relate to is how we store our data at home such as our photos. We have stopped tagging or categorizing the pictures in our drives as the technology readily available today can already tell us which photos we are with our family or search our photos with the objects in it.

We used to be able to handle capturing and understanding this information for enterprises with capture technologies like optical character recognition (OCR) and zonal recognition often with hand-crafted rules for where to read data. With the increased volume and need to read and understand more complex documents, AI-based capture is now a requirement to scale out in terms of volume but also the types of information we can automatically process.

These new approaches also need to be applied to unstructured content being captured and stored in the enterprise. Technology has largely been commoditized to address challenges such as moving data between systems, even at scale, and we are seeing wide adoption of technologies such as Robotic Process Automation (RPA) to automate many tasks previously left as manual tasks even with the presence of an ERP or a BPM platform. One challenge that has remained, however, is the handling of the unstructured documents and how they can be handled at scale.

It has been costly and time-intensive to automate the extraction of the data from unstructured information at times and often the resulting solutions have been inflexible. Common approaches included developing templates to identify where information was physically on a document or via a set of hand-crafted rules to identify in a slightly more flexible way the type of data we were looking for. In the agile, fast-paced world we live in, these techniques are now too slow for many projects and cannot be maintained at a speed to keep up with the changes that are thrown at them. These types of issues can cause an organization’s automation initiatives to be bottlenecked by something seemingly as simple as reading somebody’s name and address from a set of documents.

The solution to overcoming this is through the use of Artificial Intelligence (AI) technologies. There are plenty of solutions that exist today that target specific information types such as personal identification (e.g. passports, driver’s license, national identification cards) or only addressing the one capability such as only classifying or extracting data that consider themselves as next-generation capture powered by AI. At IBM, we are aiming to bring this next level of AI-reimagined capture as a more generic set of capabilities to tackle a wide variety of unstructured documents.

IBM Automatic Document Processing (ADP) capability, embedded into IBM Cloud Pak for Business Automation, provides a holistic approach to handling all use cases and addressing all the capability requirements. Not only that, but it also provides a logical framework for handling unstructured documents that are one of the critical business assets that IBM Cloud Pak for Business Automation is responsible for assisting organizations to manage and extract value from.

Some of the key capabilities provided by IBM ADP are:

No-Code Development

A guided no-code development studio is included which means that a business user can build an application for classifying and extracting information from unstructured documents. It is not just about developing the classification and extraction mechanisms using AI, but it also enables the person to build a complete application that can ingest, classify and store data in a content management system (such as IBM FileNet Content Manager).

Pre-Trained Modules

IBM ADP comes with pre-trained models for specific documents. It also provides a no-code experience for building models step-by-step with the help of AI and deep learning. This makes building classifications and extraction much easier and more accessible. Also by leveraging deep learning it eliminates the need for a large volume of samples data up-front to develop the model. With the no-code guided approach, enterprises do not need a data analyst to build the application.

Data Validation

Often critical information needs to be validated as part of a recognition and capture solution. Another capability inside IBM Cloud Pak for Business Automation, Business Studio allows the user to build verification interfaces in a no-code way to validate the critical data. This validated set of data can then be leveraged via traditional integration techniques or via RPA to feed other systems.

Integration with IBM FileNet Content Manager

Another very important feature is storing the documents in IBM FileNet Content Manager solution out of the box during the processing of the documents in Automation Document Processing. With most traditional capture solutions, documents only exported to a secure content management system after process. By utilizing IBM FileNet up-front, it means that the documents are stored securely throughout the process. This helps organizations comply with the many regulations that exist for the secure management of information, including during the ingestion/processing part of the information lifecycle inside an organization.

IBM Automation Document Processing, part of IBM Cloud Pak for Business Automation, is a critical set of capabilities to help organizations address the challenge of classification and extraction of data from unstructured documents. This will continue to be a key requirement for many automation projects; organizations that can enhance this capability will be able to implement automation in a more agile manner compared to those that rely on manual processing or the more traditional template/rules-based capture solutions.

If you would like to see ADP in action, a great place to start is this demonstration video. (https://www.youtube.com/watch?v=Y3dyHRjFvxU)