Using Ollama, LLaVA LLM and Gravio to Build a Local Visual Question and Answer AI

Overview

Since the rising of OpenAI, Generative AI has been the talk of the town. However, there are still many issues to be addressed with such emerging technologies in terms of reliability, stability and usability, especially when applied to a real business. There is one other aspect that sometimes helps finalize a decision when using a Generative AI web/cloud service like OpenAI. That is, privacy. Any files that you will need to upload for inference and training will be stored with the service provider and that poses as a risk, especially if the files or images are highly sensitive and bespoke to the business.

?

With Gravio and Ollama, you can build a completely Local AI Assistant that analyzes images with your prompts to give you a generative answer that you can use in your solution. By using Gravio and Ollama, you can ensure that data privacy is at the core of the solution that you build, all without coding. We have created a step by step tutorial, just for you.

??Tutorial??

We have a complete step by step tutorial on YouTube and on our Blog that shows you how to build this entire solution locally. Here is the flow of the solution:

1. Image gets uploaded into a Folder on your local machine.
2. Gravio picks up this image and brings it into Gravio Platform as a form of “Data”
3. Whenever an image upload is detected, Gravio will create a HTTP Post to LLaVA LLM via Ollama API that will send the image and prompt as part of the body.
4. LLaVA LLM will generate a response and return to Gravio.
5. Gravio will use the AI response as part of the solution and then send the response to Line Message Application.
6. Finally, the response is also logged in a text file.

The solution should look something like this ??

You can show us what you've built or if you want to find out more about unique solutions like these, you can reach out to us on Slack. See you guys next month!