Chain-of-thought prompting: working through intermediate steps
Chain-of-thought prompting asks a language model to produce a series of intermediate reasoning steps before its final answer. Research shows that this method improves performance on arithmetic, commonsense, and symbolic reasoning tasks, and that it works even without example prompts when the model is told to think step by step. Extensions such as self-consistency and tree of thoughts further improve reliability.
What is chain-of-thought prompting?
Chain-of-thought prompting is a technique for large language models that asks the model to generate a series of intermediate reasoning steps before giving the final answer to a multi-step problem. The arXiv paper on chain-of-thought prompting describes experiments on arithmetic, commonsense, and symbolic reasoning tasks in which providing a few chain-of-thought examples significantly improved performance.
The method was developed because even very large language models can struggle with reasoning tasks that require multiple steps, such as math word problems. The Google Research blog about chain of thought explains that chain-of-thought prompting enables models of sufficient scale, roughly 100 billion parameters or more, to solve such problems.
Research suggests that the reasoning abilities elicited by chain-of-thought prompting emerge as model scale increases, and that they do not require fine-tuning or a large training dataset.
How chain-of-thought prompting works
Standard prompting typically provides the model with input-output examples and asks for a direct answer. Chain-of-thought prompting instead includes examples that show the intermediate steps, or it adds an instruction such as "explain your reasoning step-by-step." The IBM explainer on chain of thought describes how this guides the model to break down a problem into a coherent series of logical steps.
For example, solving a quadratic equation with chain-of-thought prompting would involve steps such as identifying coefficients, applying the quadratic formula, and simplifying the result, rather than simply giving the solutions. The model articulates each step in natural language, making the reasoning process transparent and traceable.
This method is useful for observability and debugging because the model output reveals how the answer was derived. It differs from prompt chaining, where separate prompts are used sequentially; chain-of-thought keeps the reasoning inside a single prompt.
Zero-shot chain-of-thought
A striking finding is that chain-of-thought reasoning can be elicited without any examples at all. The arXiv paper on zero-shot reasoners shows that simply adding the phrase "Let's think step by step" before each answer significantly improves performance on diverse reasoning tasks.
The same paper reports that on the MultiArith benchmark, accuracy improved from 17.7% with standard zero-shot prompting to 78.7% with zero-shot chain-of-thought. On GSM8K, accuracy rose from 10.4% to 40.7% for a large InstructGPT model.
This suggests that large language models have untapped reasoning capabilities that can be elicited with a single, simple prompt template.
Extensions: self-consistency and tree of thoughts
Self-consistency is an extension of chain-of-thought that replaces greedy decoding with sampling multiple diverse reasoning paths and then selecting the most consistent answer by majority vote. The arXiv paper on self-consistency reports gains on several arithmetic and commonsense benchmarks, including a 17.9% improvement on GSM8K and an 11.0% improvement on SVAMP.
Tree of Thoughts generalizes chain-of-thought by allowing the model to explore multiple reasoning paths, evaluate intermediate thoughts, and look ahead or backtrack when needed. The arXiv paper on Tree of Thoughts shows significant gains on tasks that require planning or search, such as Game of 24, where chain-of-thought achieved only 4% success while Tree of Thoughts reached 74%.
These approaches address some of the limitations of a single linear chain, particularly for problems that require strategic exploration.
Prompting best practices from model documentation
Model providers offer practical guidance for using chain-of-thought and related techniques. The Anthropic prompting best practices recommend being clear and direct, providing context about why a behavior matters, and structuring instructions as sequential steps.
They also advise that a prompt should be sufficiently detailed that a person without task-specific context could follow it. This principle applies to writing chain-of-thought examples, which should show the exact reasoning steps the model is expected to reproduce.
The documentation notes that techniques such as chain-of-thought are most effective when the model is given explicit instructions about the desired output format and constraints.
| Benchmark | Improvement |
|---|---|
| GSM8K | +17.9% |
| SVAMP | +11.0% |
| AQuA | +12.2% |
| StrategyQA | +6.4% |
| ARC-challenge | +3.9% |
Frequently asked questions
What is chain-of-thought prompting?
It is a prompting method that asks a language model to generate intermediate reasoning steps before the final answer, improving performance on multi-step reasoning tasks.
How does zero-shot chain-of-thought work?
It uses a single phrase like "Let's think step by step" to elicit reasoning without providing any example solutions.
What is self-consistency?
Self-consistency samples multiple reasoning paths and takes the majority vote to choose the most consistent answer.
How does Tree of Thoughts differ from chain-of-thought?
Tree of Thoughts explores multiple reasoning paths, evaluates intermediate thoughts, and allows backtracking, which helps on problems requiring planning.
Are chain-of-thought prompts always better?
They help with complex reasoning tasks, but standard prompting can be sufficient for simpler tasks; the benefit is typically seen with large models.