Decoding Orca 2 by Microsoft Research: Insights from Cazton

Introduction

The rapid advancements in the field of artificial intelligence have led to the development of increasingly powerful language models, capable of understanding and generating human-like text. These large language models (LLMs) have demonstrated remarkable abilities in various applications, such as coding, web search, chatbots, customer service, and content creation. However, as these models grow in size and complexity, they also demand more computational resources, making them less accessible and efficient for many applications. This raises the question: can smaller language models be trained to exhibit advanced reasoning capabilities similar to their larger counterparts?

In this blog post, we will explore the research paper "Orca 2: Teaching Small Language Models How to Reason" by Arindam Mitra et al., which addresses this question by developing a method to enhance the reasoning abilities of smaller language models. We will discuss the key insights, techniques, and results of the study, as well as its limitations and potential future directions.

Orca 2: A Cautious Reasoner

The primary goal of the Orca 2 project is to teach smaller language models how to reason effectively by employing a variety of reasoning techniques and determining the most effective solution strategy for each task. The researchers built upon the Orca 1 model, which utilized explanation tuning to train student models on richer and more expressive reasoning signals. In Orca 2, the authors focus on two main objectives:

1. Teach smaller models to use a suite of reasoning techniques, such as step-by-step processing, recall-then-generate, recall-reason-generate, extract-generate, and direct-answer methods.

2. Help these models decide when to use the most effective reasoning strategy for the task at hand, allowing them to perform at their best, irrespective of their size.

To achieve these objectives, the researchers used more capable LLMs to demonstrate various reasoning strategies across different tasks. They then trained the smaller models on this synthetic data, carefully tailoring the reasoning strategies to the task at hand and considering the capacity of the student model. This approach, called Prompt Erasing, encourages the student model to learn not only how to execute specific reasoning steps but also to strategize at a higher level on how to approach a particular task.

Experimental Setup and Benchmarks

The researchers evaluated the performance of Orca 2 using a comprehensive set of 15 diverse benchmarks, corresponding to approximately 100 tasks and over 36,000 unique prompts. These benchmarks cover various aspects, including language understanding, common sense reasoning, multi-step reasoning, math problem solving, reading comprehension, summarization, groundedness, truthfulness, and toxic content generation and identification.

The performance of Orca 2 was compared with several state-of-the-art models, including LLaMA-2, WizardLM, and GPT models. All baseline models were instruction-tuned models, as they have been shown to improve the model's ability to follow instructions, enhance the overall quality of the generations, and give models enhanced zero-shot and reasoning abilities.

Results and Comparisons

The results of the study demonstrate that Orca 2 significantly surpasses models of a similar size, even matching or exceeding those 5 to 10 times larger, especially on tasks that require reasoning. Some key observations from the results include:

1. Surpassing models of the same size: Orca-2-13B outperforms models of the same size on zero-shot reasoning tasks, providing a relative improvement of 47.54% over LLaMA-2-Chat-13B and 28.15% over WizardLM-13B.

2. Competitive with models 5-10x larger: Orca-2-13B matches or surpasses all other models, including models 5-10x larger, on a variety of benchmarks in a 0-shot setting.

3. Cautious system message adds a small boost: Using the cautious system message with both the 7B and 13B models provides small gains over the empty system message.

Limitations

Despite the promising results, Orca 2 has several limitations, including:

1. Data biases: The model may carry biases present in the source data.

2. Lack of transparency: The complex nature of LLMs makes it difficult to comprehend the rationale behind specific outputs or decisions.

3. Content harms: The model may generate outputs that could be potentially biased, unfair, or harmful.

4. Hallucination: The model may generate content that is not grounded in the provided context.

5. Small model capacity: While Orca 2 can enhance the small model's ability to reason, it does not expand its ability as a knowledge store.

Conclusion

The Orca 2 project has demonstrated the potential of smaller language models in achieving advanced reasoning capabilities similar to their larger counterparts. By employing a variety of reasoning techniques and determining the most effective solution strategy for each task, Orca 2 models have shown remarkable performance in various benchmarks, surpassing models of the same size and even competing with models 5-10 times larger.

While there are still limitations and challenges to overcome, the study represents a significant step forward in the development of more capable and efficient smaller language models. The use of tailored synthetic data and the focus on teaching smaller models to reason open up new possibilities for future research and applications that require different deployment scenarios and trade-offs between efficiency and capability.