RAG (Retrieval Augmented Generation) vs. Fine-Tuning

Aashish Mohan . Aashish Mohan .

Aashish Mohan .

Futurist, AI and Digital Technologies

发布日期: 2024年9月5日
+ 关注

RAG involves retrieving relevant information from a knowledge base and then using a language model to generate a response based on the retrieved information. This approach allows for more flexible and adaptable models that can handle a wider range of tasks.

Fine-tuning involves adjusting the parameters of a pre-trained language model to improve its performance on a specific task. This approach can be more efficient for tasks that are closely related to the original pre-training task.

Mathematical Equation:

  • RAG:

Retrieval: Similarity measures like cosine similarity or dot product are used to find the most relevant documents in the knowledge base.        
Generation: A language model (e.g., GPT-3) is used to generate a response based on the retrieved documents and the original query.

  • Fine-tuning:

Gradient descent: The model's parameters are updated using gradient descent to minimize a loss function.        
Loss function: A loss function measures the difference between the model's predicted output and the desired output.

Key differences:

  • Knowledge base: RAG relies on a knowledge base, while fine-tuning typically doesn't require one.
  • Flexibility: RAG is more flexible as it can be adapted to different tasks by changing the knowledge base.
  • Efficiency: Fine-tuning can be more efficient for tasks that are closely related to the original pre-training task.
  • Control: RAG provides more control over the generated responses through the choice of retrieved documents.
  • Task-specific: Fine-tuning is often better suited for task-specific applications.

要查看或添加评论，请登录

Aashish Mohan .的更多文章

  • The Solace of Stillness
    2024年10月25日

    The Solace of Stillness

    Less words, more peace. I've found solace in silence, where my thoughts untangle and my soul breathes deep.

  • The crisp takeaway of the day-
    2024年10月6日

    The crisp takeaway of the day-

    "True freedom begins when you stop seeking approval from others and start living authentically for yourself." & "When…

  • RAG Cosine Similarity
    2024年9月5日

    RAG Cosine Similarity

    Retrieval Augmented Generation (RAG) models primarily use cosine similarity to measure the similarity between a query…

  • Be-sincere-but-not-too-serious
    2024年8月24日

    Be-sincere-but-not-too-serious

  • Why you should visit Cemetries?
    2024年8月23日

    Why you should visit Cemetries?

    Wherever Andy sees, he sees a rockstar. There are lots of them.

  • To make u smile ....
    2024年8月21日

    To make u smile ....

    "I told my AI assistant to write a joke about cybersecurity. It came back with, 'What do you call a hacker who loves…

  • AI one liner Gyan -47. Embeddings in transformer architecture
    2024年5月11日

    AI one liner Gyan -47. Embeddings in transformer architecture

    It is nothing but a map'ing of words or tokens to high-dimensional vectors, facilitating the model's understanding of…

  • AI one liner Gyan-46
    2024年5月10日

    AI one liner Gyan-46

    Fine-tuning tweaks existing models, while RAG mixes generation with fetching extra info to improve answers.

  • A few things which keep me happy and motivated each day-
    2024年5月8日

    A few things which keep me happy and motivated each day-

    -Mantra chanting -Reading Book -Deep breathing -feeding the needy souls

  • Kindly opine -whose article would you read first :
    2024年4月12日

    Kindly opine -whose article would you read first :

    (1)Rajan singh (2)Akshat Srivastava 3) Sandeep kochar 4)I will not read any article- its too much Gyan 5)Ankur Warikoo…

社区洞察

其他会员也浏览了