How AI OCR Data Capture Can Greatly Reduce Data Entry
Larry Phelps
I help organizations like yours go paperless, automate business process and improve staff productivity.
Introduction
You probably have heard about CHAT GPT. It is everywhere. The media has gone crazy over Chat GPT and AI or machine learning.
Many people are leery of this technology. They think of all the things that could go wrong. But, we have decided like all new technology advancements, there are some down-side but for the most part, there is a huge upside.
We have been working on data capture technology for a couple of years and helped a couple of customers capture data off of invoices and POs to reduce data entry. But, it was up to our programmers to do the heavy lifting when it comes to figuring out what on a page is important and needs to be captured. So, we decided to dive into the AI world.
This year we kicked it into high gear, and with the help of Khalan and a very smart AI engineer, we developed our own AI OCR data capture solution.
What is AI OCR Data Capture
If you are not sure what OCR is, you will find an explanation further down in the article.
But, AI OCR is a technology that utilizes machine learning algorithms to process the text that was extracted from a document using OCR. It can understand, organize, and categorize the text to make meaningful guesses about the content.
That may be a little hard for you to understand, let me explain it in another way. Let's use an invoice for example. The AI looks at the text extracted from an invoice and determines what text is most likely the invoice number and what text is most likely the vendor name. And, as more invoices are captured and validated, the AI gets smarter and therefore more accurate.
Comparison of Manual Data Entry and AI Optical Character Recognition
We all know too well what manual data entry is. We do too much of it. In a study I read recently, it said that accounts payable staff spend on average 60-70% of their time doing data entry.
Manual data entry is:
AI OCR data entry:
Traditional Data Capture vs. AI OCR software
Traditional capture requires templates and rules for document processing, which can be time-consuming and less adaptable to diverse document types. Machine learning-powered OCR does not require templates and rules, offering greater adaptability and efficiency in document processing, leading to reduced time and costs, increased accuracy, and improved productivity.
Traditional capture may struggle with extracting data from various types of documents, especially those with poor image quality or complex layouts. . AI-powered OCR can significantly enhance data extraction accuracy across a broader set of circumstances, making it an indispensable tool for numerous industries. It can extract data from various document types, including invoices, purchase orders, receipts, bank statements, and service orders, even from low-quality documents.
Traditional capture technology may struggle with poor image quality, complex layouts, and handwritten or cursive text, which can impact accuracy and reliability. AI-powered OCR offers improved accuracy, even with distorted or low-quality text, and can handle complex layouts, handwritten text, and multilingual documents more effectively. I think the handwritten accuracy is amazing. From my experience, this is a huge benefit over traditional data capture technology.
How Does AI-Powered OCR Data Capture Accuracy Compare to Manual
The accuracy of machine-learning OCR is dependent upon several factors:
1.????? Document Condition: Issues like stains, tears, fading, and artifacts reduce accuracy. Clean, crisp hard copies are ideal.
2.????? Image Resolution: Higher resolution scans improve precision. We recommend documents to be scanned at 300 dpi. Higher resolution results in too large of file size. Lower resolution can increase the chance of errors.
3.????? Image Quality: Skew, rotation, uneven lighting, and noise degrade OCR reliability.
4.????? Text Legibility: Small fonts, artistic fonts, handwriting, and poor contrast can hurt accuracy.
5. Complexity of the document: too many fields to capture, too much extra text that does not need to be captured, and variety of formatting from one document to the next.
But, if you have a relatively good set of documents, you can achieve 80-95% accuracy.
I decided to test this out on a project we were doing for a public utility company. I grabbed a sample of 25 as-builts, a document that is often a hand sketch of where utilities are brought into a property. The AI OCR was trained to capture about 10 fields of information from handwritten documents. I processed all 25 documents with our AI solution. Then manually read each document and entered the information into a spreadsheet.
When I compared the results, I found a 15% difference in what I captured versus what AI captured. I thought to myself, well AI is almost as accurate as I am. But on further investigation. I found out I was the least accurate one. The AI was able to read the handwriting of various workers more accurately than I was able to.
I guess a career in data entry is not in the cards for me!
How much do AI OCR data capture systems cost?
AI OCR data capture systems typically charge per page. And in some cases, per document. The price varies quite a bit from one vendor to the next. I have seen prices as high as $5 or more per invoice, for example. But I have also seen prices as low as $1.
领英推荐
According to a study by Adobe, it costs anywhere from $15 to $40 to process a single invoice. So, if you find a solution that will do it for $1, that will amount to huge savings in one year.
Our solution is in that price range, you can find more info here. Hemingway AI OCR Data Capture
Considerations For Evaluating AI OCR Solutions
A year or so ago, I was evaluating AI capture solutions for invoice capture. What I did was put together a sample of 25 invoices from various vendors and various forms. I picked vendors that were frequently used. I wanted to make sure that high-volume vendors had a high score - very accurate.
Then I scanned each one and compared the data captured. I was particularly interested in vendor name, vendor ID, vendor address, invoice date, due date, PO number, and invoice total.
And I created a spreadsheet and entered each value retrieved for each capture vendor. Then I was able to compare the accuracy of each.
Then I went one more level deep. I included the cost of the vendor. And assigned a cost to fix each invoice based on the number of errors. I did this because just because one vendor is more accurate, it doesn't make it the most cost-effective.
For this, it was easy to pick a solution that was the most effective. It was a lot of work, but picking the correct solution will save a lot of time and money in the long run.
One warning: Some vendors don't make it easy for you to try your documents. They prefer to demo their documents that they know will perform well. Test all AI OCR data capture products with your documents. That will give you the best understanding of the capability of the tool.
And make sure that it captures all the fields you need. Some applications do not capture the vendor. And so if you use that application you would have to select the vendor for each invoice. I think that kind of defeats the purpose.
In summary, verify:
·???????? Captures all fields you need
·???????? The accuracy using your documents
·???????? The price including your estimated cost to fix capture errors.
If you look at these factors carefully, you will be able to pick the best solution for your needs.
How accurate is AI OCR?
We are seeing better than 90% accuracy using machine learning on invoices. And to get more accuracy, we can use workflow tools to validate and normalize captured data. For example, we can pull the vendor’s name from the invoice, do a lookup in our vendor list, and pick the best match, this will greatly help capture the correct data.
A small error in OCR happens when it looks at a character and tries to determine, for example, if a character is a "0" or an "O". These errors often are resolved when we validate or normalize the data.
And when your data capture software has automation software that does advanced data validation. Such as looking up values in a PO and comparing them to values in the invoice. These features found in invoice automation software verify that invoice data was captured accurately.
What is the difference between OCR and AI OCR?
OCR captures data one character at a time. AI OCR can look at the context and better determine what the whole word is and what category it belongs to.? For example, if recognizing characters that are part of the context of a vendor name. Then a circle character is more likely the letter O and not the number 0. This helps make the AI OCR much more accurate than the plain OCR.
Can OCR software read handwriting?
Most traditional OCR programs do not read handwriting. That is understandable. Sometimes I have difficulty reading my own handwritten notes.
But many AI OCRs do a great job. I remember doing one of my 1st tests of this technology on 20 handwritten service orders. I was capturing the service order number which was near the top of each of the orders. And, on the 1st 20, I did not find one artificial intelligence capture error. That amazed me because they were written by numerous different people with different handwriting. And they were in different parts of the page. I was shocked.? I know if I did it manually, I would have made at least one error.
How Does Artificial Intelligence Transform OCR Data Capture?
Traditional optical character recognition cannot consider the context when determining what the next character is. But artificial intelligence can determine the context and therefore make better guesses as to what the next character is.? For example, if artificial intelligence has determined that the context of the characters it is working on is probably a date. It knows the various date formats such as 01/01/2021 or 1/1/21, and it knows to look for a "/" after one or two characters. So, if it reads a '[" or "|", it probably is a "/" instead.
This helps artificial intelligence be much more accurate than traditional OCR.
What Optical Character Recognition Is?
Optical character recognition is the technology that converts an image to text.? It reads one character at a time. The process of OCR involves reading an image file and converting it to binary data, analyzing the scanned image for light and dark areas to identify characters that need to be recognized, and then using software to recognize characters using algorithms such as pattern recognition. This process is often called text recognition. The image above may help you understand the process.
This technology has been used for the past 30 years. However, the advent of machine learning has taken optical recognition to a new level.
Conclusion
Artificial intelligence has greatly improved the accuracy of data capture tools. If you are looking for one of the best AI OCR solutions for capture and data extraction from invoices. I suggest you look at Hemingway Solutions AI OCR Invoice Automation Solution.
If you would like to discuss this, please feel free to book a discovery call with me.