The Future of Web 3.0: Vector DBs and ChatGPT Plugins

As technology continues to evolve, so does our ability to find and access information. The latest development in this realm is ChatGPT's new plugins which search 3rd party resources to add context to the prompt given to OpenAI before answering the user's question. This breakthrough technology promises to transform the way we interact with the web, creating a more contextual and personalized experience for users.

But ChatGPT plugins are just the beginning. Thinking ahead, the vector DB is the new website. These plugins are merely proxies for Vector DBs, and there is nothing stopping a Vector DB from offering a query endpoint that can be exposed as a ChatGPT plugin. As a result, everyone will want to have a vector DB for their content. We are seeing the rise of Web 3.0 driven by LLM and vector DB, where the website is a corpus of vectorized data that is queried with NLP.

In the near future, the website will be a place where data is queried using natural language processing. ChatGPT plugins have already replaced llama_index, which used a similar framework to add contextual information to prompts. Now, ChatGPT plugins serve as API servers with some type of domain-specific vector DB behind them supporting query by string.

For individuals who prefer to keep their work local and limit their use of OpenAI APIs, ChatGPT plugins can also be used as readers (servers) in a local model. With some additional glue and flexibility, these plugins can be chained to retrieve additional context, craft a prompt, and send it to OpenAI to answer. By using Google Universal Sentence Encoder and Whisper python code with models, the need for OpenAI calls for embeddings and whisper can be eliminated.

One potential downside of using OpenAI for adding context is that they control how the plugin context is incorporated into the prompt, using hidden messages that the user doesn't have control over. By using ChatGPT plugins, individuals can gain more control over how context is added to the prompt. This provides greater flexibility and the ability to customize prompts to better meet individual needs.

Overall, ChatGPT plugins represent an exciting new development in the world of information retrieval. With the rise of vector DBs and a more contextually-driven web, the possibilities for how we find and access information are endless. Whether used as readers or as part of a larger framework, ChatGPT plugins offer a gateway to a more personalized and dynamic web experience.