Extract rich insights from documents and build simple but impacful experiences with Generative AI

From financial statements in finance to product manuals in manufacturing we are swamped with thousands of pages of documents, reports… in our daily lives. They are not easy to understand, nor can be consumed in a timely manner. In this short article, I’ll cover sample use cases and tech blueprint on how we can extract rich insights from thousands documents and provide summarize insights and experiences for users. This Generative AI use case can be easily implemented across different industries.

Manufacturing company completely getting rid of product manuals…and training a generative AI model to provide experiences where users of their products can simply ask what they need using natural language. A light came up in your car console, simply ask what it is, and model will tell you why that light is on, what you should do and where is the nearest service point…Your refrigerator smells, learn why and how to fix it. (Imagine a world with no printed product manuals and its Sustainability Impact getting rid of millions of printed product manuals, energy and resources spent on them!)

Your tax returns, your mortgage documents … Again, pages of complex documents you sign without reading and deeply understanding fully. Imagine a tax broker company or a bank simply providing experiences for you to have digestible insights to users before and after submitting taxes or applying loans.

Or you are an investment banker, you just received last 20 quarters of financial statement of a company you are interested in, but you have a meeting in an hour. How can you get key insights. Or maybe you are not a finance expert, but still want a smart services that can use to get the key insights of a stock you are thinking of buying from these financial statements.

Generative AI also helps people to do research. The developed model can be used for looking through literature to generate summaries and extract key information from various resources: articles, papers and magazines…

Let’s see how we can extract rich insights from documents and build simple experiences with Generative AI using Azure OpenAI Services:

In this blueprint, using Azure Form Recognizer, we read different types of documents such as PDFs, word docs and images. Form Recognizer can apply machine-learning-based optical character recognition (OCR) and document understanding technologies to extract text, tables, structure, and key-value pairs from provided documents.

Then, Azure Cognitive Search indexes the documents to make it searchable. After this step, your documents become the knowledge base for Azure OpenAI queries.

With the help of serverless structure of Azure Function Apps, you can forward the output coming from Cognitive Search to your services in the next steps.

According to your business scenarios, Azure OpenAI Service can extract key information from your knowledge base or summarize the content for you using natural language in your own language.

You need a dynamically scalable and globally reliable database such as CosmosDB to store documents, document text, and outputs of OpenAI and Cognitive Search.

For reporting and analysis needs, can use PowerBI. For front end, you can build a Web Application or ChatBot or Integrate it into your LOB app for end users to access generated insights and summaries.

It is now extremely easy to build such experiences, and most likely you can automate the whole process using GitHub Co-Pilot??