Indexing Methods for Vector Retrieval

Good morning everyone!

ChatGPT is pretty useful, but you can’t build a product out of it.

I have been talking about RAG for many videos now, and it’s not for nothing. As we’ve discussed in our book and course , retrieving documents or useful text is necessary for all LLM-based products.

You need to give additional information to your LLM if you want it to be really useful. Which means being more than just a Google assistant. It needs to have up-to-date context and relevant information to share with your user. Basically, you want your own LLM but without the cost of training one. This is why RAG exists.

But what if your database is immense, or if there’s a massive chunk of information to compare? How do we ensure the “retrieval” part of the pipeline doesn’t slow us down?

Learn more in the video (or article here ):

This is the first video of 10 for the "From Beginner to Advanced LLM Developer " course by Towards AI.

We did our best to create the most practical and in-depth LLM Developer course out there (~90 lessons) for software developers, machine learning engineers, data scientists, aspiring founders or AI/Computer Science students.

We’ve gathered everything we worked on building products and AI systems and put them into one super practical industry-focused course. Right now, this means working with Python, OpenAI, Llama 3, Gemini, Perplexity, LlamaIndex, Gradio, and many other amazing tools (we are unaffiliated and will introduce all the best LLM tool options). It also means learning many new non-technical skills and habits unique to the world of LLMs.

Check it out here or look at our book Building LLMs for Production to learn more!

And that's it for this iteration! I'm incredibly grateful that?the What's AI newsletter ?is now read by over 20,000 incredible human beings. Click here to share this iteration with a friend if you learned something new!

Looking for more cool AI stuff? ??

• Looking for AI news, code, learning resources, papers, memes, and more? Follow our weekly newsletter at Towards AI !
• Looking to connect with other AI enthusiasts? Join the Discord community: Learn AI Together !

Want to share a product, event or course with my AI community? Reply directly to this email, or visit my Passionfroot profile to see my offers.

Thank you for reading, and I wish you a fantastic week! Be sure to have?enough sleep and physical activities next week!

Louis-Fran?ois Bouchard