Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models

Today's paper introduces a new zero-shot prompting method called Plan-and-Solve (PS) Prompting to improve the reasoning capabilities of large language models. The method guides models to first devise a plan to break down complex tasks into subtasks, then carry out those subtasks step-by-step. This approach aims to address common pitfalls in existing zero-shot methods like calculation errors and missing reasoning steps.

Method Overview

The Plan-and-Solve (PS) Prompting method works by providing more detailed instructions to large language models in a zero-shot setting. The key idea is to prompt the model to first devise a plan for solving the problem by breaking it down into subtasks, and then carry out those subtasks systematically.

The method has two main components:

1. A planning stage where the model is asked to understand the problem and create a step-by-step plan to solve it.
2. An execution stage where the model carries out the plan and solves the problem step-by-step.

One possible prompt would be: “Q: [X]. A: Let’s first understand the problem and devise a plan to solve the problem. Then, let’s carry out the plan and solve the problem step by step”

To further improve performance, they introduce an enhanced version called PS+ prompting. This adds more detailed instructions like "extract relevant variables and their corresponding numerals" and "calculate intermediate results". These additional prompts aim to reduce calculation errors and ensure important reasoning steps are not missed.

The prompts are designed to be general enough to work across different types of reasoning tasks without requiring task-specific examples or fine-tuning. This allows the method to leverage the embedded knowledge and reasoning capabilities of large language models in a zero-shot manner.

Results

The PS and PS+ prompting methods consistently outperformed existing zero-shot baselines across 10 datasets covering arithmetic, commonsense, and symbolic reasoning tasks. On arithmetic reasoning, PS+ prompting achieved an average accuracy of 76.7%, compared to 70.4% for the standard zero-shot chain-of-thought method.

Notably, PS+ prompting performed comparably to or even exceeded some few-shot methods that use manually crafted examples, despite being fully zero-shot. It also reduced calculation errors and missing-step errors compared to existing zero-shot methods.

Conclusion

This paper introduces an effective zero-shot prompting strategy that improves the reasoning capabilities of large language models across diverse tasks. By guiding models to plan and systematically solve problems, it addresses key limitations of existing methods and achieves strong performance without requiring examples or fine-tuning. For more information please consult the?full paper.

Congrats to the authors for their work!

Wang, Lei, et al. "Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models." arXiv preprint arXiv:2305.04091 (2023).