Is ChatGPT Super AI?

Last month we wrote to you about ChatGPT, specifically whether OpenAI’s most recent Large Language Model (LLM) is revolutionary, or merely propelling us into another artificial intelligence (AI) hype cycle. This new model, powered by GPT-3.5 and optimized for conversational dialogue, is objectively better (and more intuitive) than what came before it. But will it replace?Google Search ,?marketers ,?therapists ,?software engineers , and everything else? Probably not. Like many things in life, reality lies somewhere in the middle. Let's pick up where we left off last month—with a question:

Is ChatGPT super AI?

This fun play on words asks two things: Is ChatCPT as amazing as everyone thinks, but also is it something we would incorporate into our product offerings at super.AI? The short answer is: yes, we will. This month’s newsletter details the potential for LLMs like ChatGPT to revolutionize Intelligent Document Processing (IDP) and move the industry toward 100% processing of complex documents. Let’s jump in!

The Intelligent Document Processing Gap

Document automation technology has matured considerably since the early days of optical character recognition (OCR). While second-generation IDP solutions are incredibly capable, achieving near-perfect accuracy at high automation rates still requires some amount of human-in-the-loop (HITL) oversight. At super.AI, we view this as a feature, not a bug, and it’s in that spirit that we created the best version of HITL possible with our?Data Processing Crowd .

But AI's rapid pace of advancement means a future where IDP is capable of 100% processing, with limited to no human involvement, is within reach. Top-performing IDP platforms can automatically process anywhere from 60-80% of complex document data, meaning 20-40% requires extra attention. There also remain certain components of complex documents that IDP solutions consistently struggle with, including signatures, handwriting, and tables. Of course, moving from a low automation rate or an entirely manual process to automating well over half of document data extraction is no small feat. Yet the question of how things can get?even better?remains.

ChatGPT and the future of IDP

It is difficult to say with certainty exactly how the IDP gap will be closed. However, renewed public interest in AI sparked by ChatGPT got us thinking about how LLMs might help. We see LLMs helping IDP/UDP solutions in the following areas:

Improving data extraction accuracy:?A primary challenge for IDP solutions is handling semi-structured and unstructured data in various document types, including invoices where labels can vary. Despite advanced fuzzy matching and ML models, automated document processing has difficulty extracting data accurately from 20-40% of invoices. Although research is lacking, using LLMs could improve automation and make it closer to human-level processing for such data. This involves using OCR-extracted text to train a Large Language Model, then querying it for desired key/value pairs.

Responding to natural language queries and commands about critical business information:?LLMs offer the possibility of asking questions or issuing commands in natural language about document archives and getting near real-time results. The process starts with converting each document to machine-readable text through OCR, then training a LLM with the extracted information. This provides a conversational interface for querying and analyzing business documents, enabling users to ask questions or issue commands such as:

• Give a one-paragraph summary of the document
• Translate the document into Spanish and Mandarin
• What are the payment terms?
• What is the check or payment number?
• When does the contract expire?

Simplifying the creation of Unstructured Data Processing (UDP) and AI applications:?Unified AI platforms like super.AI simplify processing of unstructured data, like documents, images, and videos. The platform allows users to leverage existing AI applications built by our team, or create their own "data programs" quickly using our SDK. These programs require defining inputs/outputs, creating a processing workflow, setting task routing, combining results, and training models with human assistance. While easier than programming an AI app from scratch, some degree of script writing is still involved. LLMs may simplify "data programming" even further, allowing users to create their own AI applications with text prompts like those used with ChatGPT for generating content or code.

This is just the beginning

From improving extraction accuracy to making extracted data more useful, Large Language Models like ChatGPT have the potential to revolutionize Intelligent Document Processing. While closing the IDP gap will likely involve a confluence of evolving technologies, the future is promising. Super.AI will be incorporating LLMs into our IDP solution to help our customers make the most of their unstructured data and stay ahead of the curve in the AI industry. This is just the beginning. AI progress will continue to accelerate, and LLMs will be an important piece of the?Unstructured Data Processing ?puzzle.