Top LLM Papers of the Week (August Week 3, 2024)

[1] SelectLLM

This paper introduces SELECTLLM, a novel algorithm developed to overcome limitations of individual LLM by choosing appropriate LLMs for a given query. SelectLLM utilizes the predictions and confidence scores of a multilabel classifier for selecting the appropriate LLMs. It outperforms individual LLMs and achieves competitive results compared to top-performing LLM subsets.?[Tweet] and [Paper]

[2] GraphRAG (Survey)

Retrieval-Augmented Generation (RAG) addresses LLM challenges, but struggle to handle complex entity relationships in databases. GraphRAG addresses this by leveraging structural information for more precise retrieval and context-aware responses. This paper presents the first comprehensive overview of GraphRAG methodologies and also explores applications, evaluation methods, and future research directions. [Tweet] and [Paper]

[3] LLMs for Finance Applications

FinLLaMA is pre-trained on a 52 billion token financial corpus, including text, tables, and time-series data. FinLLaMA-instruct is developed by fine-tuning FinLLaMA? with 573K financial instructions. FinLLaMA-instruct achieves SOTA results by outperforming GPT4 and other Financial LLMs on a number of datasets. [Tweet] and [Paper]

[4] CommunityKG-RAG

This paper introduces CommunityKG-RAG which integrates community structures within Knowledge Graphs with Retrieval-Augmented Generation systems to enhance fact-checking. CommunityKG-RAG can adapt to new domains and queries without additional training which makes it highly versatile and applicable across various contexts.?[Tweet] and [Paper]

[5] LLM Pruning and Distillation

The report focuses on compressing popular open-source LLMs like? Llama 3.1 8B and Mistral NeMo 12B models to 4B and 8B parameters, respectively using? pruning and distillation techniques. This process results in a notable 4B model from Llama 3.1 8B and a state-of-the-art MN-Minitron-8B model from Mistral NeMo 12B. The model weights are open-sourced on Hugging Face with a permissive license. [Tweet] and [Paper]

[6] W-RAG

Training of dense retrieval in RAG systems is challenging due to the scarcity of ground-truth evidence. This paper introduces W-RAG, which utilizes LLMs' ranking capabilities to create weakly labeled data for training dense retrievers. W-RAG enhances both retrieval and OpenQA performance compared to baseline models.?[Tweet] and [Paper]

[7] RAGLab

RAGLab is a modular, research-oriented open-source library that includes the implementation of? 6 existing RAG algorithms. It provides a comprehensive ecosystem for investigating RAG algorithms, addressing the constraints in RAG development. [Tweet] and [Paper]

[8] Combining PLMs and LLMs for Text Classification

Open LLMs moderately outperform or match pretrained language models only when fine-tuned, raising questions about their cost-effectiveness. This paper introduces a confidence-based approach to combine PLMs with open LLMs for text classification. The proposed solution outperforms PLMs, zero-shot, and few-shot LLMs, while competing closely with fine-tuned LLMs at a significantly lower cost. [Tweet] and [Paper]

[9] Flexora

LoRA is one of the most popular parameter efficient fine-tuning techniques. However, LoRA can underperform on certain tasks due to potential overfitting. Flexora overcome LoRA's limitations by automatically selecting the most important layers for fine-tuning.? Flexoa outperforms LoRA on various downstream tasks. [Tweet] and [Paper]

[10] JSON Response Formatting with LLMs

StructuredRAG is a new benchmark introduced to assess LLMs' ability? in generating structured outputs like JSON. Across 24 experiments, an average success rate of 82.55% was observed.? Llama 3 8B-instruct often performed competitively with Gemini 1.5 Pro, despite being a smaller model.? The findings highlight the need for further research to improve the reliability and consistency of structured output generation in LLMs.? [Tweet] and [Paper]

If you like this, do subscribe to the newsletter so that you won't miss any of the interesting LLM and RAG-related papers.