What is Qlik Answers?

One of the biggest announcements from Qlik Connect 2024 was the introduction of a new product: Qlik Answers. Back then, it was promised that Qlik Answers would launch in July of this year. That promise has been fulfilled, as Qlik Answers became available to the public on July 31. In this blog, we’ll explain what Qlik Answers is, what you can do with it, and what its current limitations are.

Retrieval Augmented Generation (RAG)

Qlik Answers is an application of a technique known as Retrieval Augmented Generation (RAG). You might be wondering: what is Retrieval Augmented Generation? To better understand this, let’s start with generative AI techniques like ChatGPT.

By now, you've likely used ChatGPT. It works like this: you ask a question in plain language, and you get a response in plain language. However, there are two important caveats with the answers you get from ChatGPT:

1. Hallucinations: The first issue is the risk of so-called “hallucinations.” The answer you receive might sound convincing, but it can be partly or completely fabricated.
2. Not up-to-date: Another major limitation of ChatGPT is that it is restricted to a specific point in time. The model is trained on publicly available information up to a certain date. The problem with the internet is that millions of new pages and articles are added every day. To prevent the model training from taking forever, developers set a cutoff point. For example, the latest model, GPT-4, is trained on data up to April 2023. If you ask ChatGPT about Apple’s stock prices this month, it won’t be able to provide an answer.

Retrieval Augmented Generation enhances the language model with search capabilities. When you ask a question, the language model doesn’t generate an answer itself. Instead, a function is triggered to search for the correct answer in a predefined source. Once the answer is found, the language model is used to relay it back to the user in plain language. This approach solves the aforementioned problems: the triggered function can search for new information that might not have existed when the language model was created. Additionally, RAG can include references to the sources consulted by the function, allowing users to verify the accuracy of the answer.

How Does It Work?

To get started with Qlik Answers, you first need to define the sources that can be used to answer user questions. Qlik Answers makes this easy; you can manually upload documents or connect to storage services like OneDrive, Amazon S3, or Google Drive where these documents are stored.

The Knowledge Base

The set of documents you upload becomes part of a "knowledge base." Once the documents are uploaded, you click the "index all sources" button. This button essentially instructs Qlik Answers to read through the documents and try to understand them as much as possible so that the content can later be used to answer questions.

Once you’ve created one or more knowledge bases, you can create an "assistant." This assistant is essentially the chatbot you can interact with. You then link one or more knowledge bases to an assistant. The major advantage of this approach is that you have substantial control over which information can and cannot be used to generate answers for different user groups.

Interaction with the Assistant

Once you’ve created the assistant, you're ready to go, and your users can start asking questions. When an answer is generated, you’ll see four buttons in the answer bubble. First, there are the thumbs-up and thumbs-down buttons. Users can click these if they’re satisfied with the answer or if they didn’t get the answer they were looking for. This feedback is available to you later, so you can identify gaps in your knowledge base. There’s also a button to easily copy the text of the answer. The last button says "View x sources." When pressed, a secondary panel opens, displaying the text segments on which the answer is based. This panel also includes a download button, allowing users to download the original document.

Applications and Limitations

With Qlik Answers, Qlik now offers a product to extract value from unstructured data—a relatively new area for many Qlik users. A logical question is: how can I apply this in my organization? The possibilities seem endless, but to get you started, here are a few ideas:

• Simplify onboarding new colleagues by using Qlik Answers as an assistant. You can add all departmental manuals and work instructions as sources.
• Compile all product specifications, frequently asked questions, and previous support inquiries to enable your support staff to find answers even faster.
• Combine applicable laws and regulations with internal policy documents and procedures, allowing your employees to easily answer legal questions.

There are likely many more applications to consider. The longer an organization has been around, the more documents it accumulates. With the first release of Qlik Answers, a wealth of new possibilities is immediately available. However, there are a few limitations to be aware of:

1. No Image Processing: Qlik Answers cannot yet read images and can only work with text files. For example, a scan of a handwritten document cannot be processed effectively.
2. Table Data: Tables can be read but not yet used to generate answers. So, if your document contains a table with quarterly revenue per department, you cannot query this data based on that document.
3. No Integration with Structured Data: It’s currently not possible to combine unstructured data with structured data, such as that found in Qlik dashboards. It’s expected that Qlik is working hard behind the scenes to make this possible, especially since Qlik Talend Cloud already allows for the creation of “AI-ready” data products.

Finally, it’s important to realize that Qlik Answers taps into a new type of data source. While we generally have established measures for data quality and governance with structured data, this is often not the case for unstructured data. To truly succeed with Qlik Answers, you’ll need to consider measures in this area.

In summary, Qlik Answers makes valuable information in documents far more accessible to all your employees. This can be incredibly valuable in advancing data-driven operations. Do you have a great use case for Qlik Answers or want to explore it further? Let me know!