Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models

Today's paper investigates how format restrictions impact the performance of large language models (LLMs) across various tasks. It examines whether constraining LLMs to produce structured outputs (like JSON or XML) affects their reasoning and knowledge comprehension abilities. The study reveals surprising declines in LLM performance under strict format constraints, especially for reasoning tasks.

Overview

The study uses three main approaches to structured generation, each with progressively relaxed constraints:

1. Constrained Decoding (JSON-mode): This is the strictest method, enforcing a predefined token space during generation to ensure valid JSON output. It's commonly used in industrial settings.
2. Format-Restricting Instructions (FRI): This approach instructs the LLM to generate responses in standardized formats like JSON, XML, or YAML, adhering to specified schemas. It's more relaxed than JSON-mode but still provides structure.
3. NL-to-Format: This two-step process first has the LLM answer in natural language, then convert that response into the target format. It's the most relaxed method, aiming to maintain natural language performance while providing structured output.

They evaluate these methods across various datasets that test different skills, including mathematical reasoning (GSM8K), symbolic manipulation (Last Letter Concatenation), and classification tasks (DDXPlus, MultiFin, etc.). The study uses multiple LLMs, including GPT-3.5-turbo, Claude-3-haiku, and open-source models like LLaMA-3 and Gemma-2.

To account for prompt sensitivity, they test multiple prompt variations for each task and format. They also use an LLM-based "perfect parser" to extract final answers, ensuring fair comparison across different output formats.

Results

The key results are:

1. Stricter format constraints generally led to greater performance degradation in reasoning tasks. JSON-mode often performed worst, while natural language responses performed best.
2. For classification tasks, JSON-mode sometimes improved performance by constraining possible answers.
3. Removing schema restrictions from prompts (e.g., just saying "Reply in JSON" without specifying the exact structure) improved performance for some models.
4. No single format (JSON, XML, YAML) consistently outperformed others across all models and tasks.
5. Parsing errors were not the primary cause of performance differences between formats. When present, these errors could be mitigated with a simple corrective prompting step.

Conclusion

The paper demonstrates that format restrictions can significantly impact LLM performance, with effects varying by task type. While structured outputs can benefit downstream processing, overly restrictive schemas may hinder LLMs' reasoning abilities. For more information please consult the?full paper.

Congrats to the authors for their work!

Tam, Zhi Rui, et al. "Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models." arXiv preprint arXiv:2408.02442 (2024).