EncodeAgent AI Digest #4

In every issue of our digest, we carefully curate a trio of articles that we believe hold considerable importance for professionals involved in product development or business management, especially those leveraging the revolutionary potential of artificial intelligence, with an emphasis on generative AI technologies.

OpenAI RAG vs. Your Customized RAG: Which One Is Better?

https://thenewstack.io/openai-rag-vs-your-customized-rag-which-one-is-better/

The article discusses the comparison between OpenAI’s built-in Retrieval Augmented Generation (RAG) feature and a customized RAG system using a vector database like Milvus. RAG is an AI framework that enhances large language models (LLMs) by retrieving facts from an external knowledge base to provide accurate and up-to-date information. The evaluation of RAG systems is done using Ragas, an open-source framework that offers various scoring metrics. The Financial Opinion Mining and Question Answering (FiQA) dataset was chosen for its specialized financial knowledge and well-annotated snippets. Two RAG systems were set up for comparison: one using OpenAI Assistants and another using Milvus with the BAAI/bge-base-en embedding model and LangChain components. The evaluation showed that while OpenAI’s RAG performed slightly better in answer similarity, the customized RAG system outperformed in context precision, faithfulness, answer relevancy, and correctness. The customized RAG’s superiority is attributed to its better use of external knowledge, document segmentation, data retrieval, and the flexibility to adjust parameters. OpenAI Assistants rely more on pretraining knowledge and have file storage limitations, whereas the Milvus-powered system can scale without limits. In conclusion, for developers seeking effective RAG applications, a customized RAG system based on a vector database is preferable for achieving better results.

OpenAI’s gen AI updates threaten the survival of many open source firms

https://www.computerworld.com/article/3710328/openais-gen-ai-updates-threaten-the-survival-of-many-open-source-firms.html

OpenAI’s first developer conference introduced updates that could challenge the open source software community, with new offerings like the Assistants API, custom GPTs, a model store, and revised pricing. These updates aim to replicate functionalities of open source frameworks and libraries, potentially threatening the survival of some open source software providers. The Assistants API offers advanced features like a Code Interpreter and Retrieval Augmented Generation (RAG), simplifying the development of sophisticated AI applications. This could lead to revenue losses for companies like LangChain, LLamaIndex, and vector database firms such as ChromaDB and Pinecone. However, some believe the updates could also drive market innovation and create new revenue streams for enterprises. OpenAI also launched GPT-4 Turbo, a faster, more efficient, and cheaper model with multimodal capabilities, which poses a significant challenge to smaller generative AI firms and startups. The updates are expected to lower the entry barrier for developers and make OpenAI’s offerings more attractive to large businesses, while potentially impacting the market share of other LLM providers like Cohere and Anthropic. Despite the competitive threat to some, the new features could enable enterprises to develop new applications across various sectors, from advanced chatbots to AI-powered games.

OpenAI prompt engineering — six strategies for getting better results

https://platform.openai.com/docs/guides/prompt-engineering/strategy-test-changes-systematically

The OpenAI guide on prompt engineering emphasizes the importance of clear instructions to obtain desired outputs from language models. It suggests tactics like including details, adopting personas, using delimiters, specifying steps, providing examples, and setting output length. For reference texts, it advises using the text to answer questions and citing from it to reduce fabrications. Complex tasks should be split into subtasks, using intent classification, summarizing long dialogues, or summarizing documents piecewise. Models benefit from “thinking time,” which can be facilitated by asking for a chain of thought or using inner monologue. External tools like embeddings-based search, code execution, and specific functions can enhance model performance. Systematic testing with representative samples and model-based evaluations against gold-standard answers ensures improvements. The guide also includes strategies for writing clear instructions, providing reference text, splitting complex tasks, giving models time to think, using external tools, and testing changes systematically. Each strategy is supported by specific tactics, and the guide encourages creative solutions beyond the provided examples.