What Is Chain of Thought? – Definition and How It Works
Learn what chain-of-thought prompting is, why intermediate steps help, where it fails, and how reasoning models changed the practice.
Chain of thought is producing intermediate reasoning steps before a final answer. Generating those steps measurably improves accuracy on problems that require several dependent inferences.
1. What It Is
Asking for the working, not only the result.
- Key idea: each generated token conditions the next, so writing the steps gives the model more to condition on.
- Mechanism: the prompt requests reasoning, or the model is trained to produce it.
- Effect: better performance on arithmetic, logic, and multi-constraint problems.
Example
Prompt: A crawler fetches 3 pages per second for 4 minutes, then stops for
1 minute, then repeats twice more. How many pages in total?
Think step by step, then give the total.
2. Why It Helps
Without intermediate steps the model must produce the answer in a single forward pass. With them, it decomposes the problem, and each step is a smaller inference conditioned on the last.
The gain is largest where an answer depends on several intermediate results, and negligible where the answer is a direct recall or a single transformation.
3. Where It Does Not Help
- Simple lookups. Reasoning about a fact the model either knows or does not adds nothing.
- Factual accuracy. Steps do not verify anything; a wrong premise is elaborated confidently.
- Latency and cost. More tokens means slower and more expensive responses.
- Short outputs. Where a single label is wanted, reasoning is overhead unless it improves the label.
4. Reasoning Is Not Explanation
The generated steps are text produced by the same process as the answer. They may not correspond to whatever computation produced the conclusion, and a model can produce plausible steps leading to a wrong result — or correct results with faulty stated reasoning.
Treat the chain as a technique that improves accuracy, not as an audit trail.
5. Variants
| Variant | Idea |
|---|---|
| Zero-shot | An instruction to work step by step |
| Few-shot | Examples that include worked reasoning |
| Self-consistency | Sample several chains, take the majority answer |
| Least-to-most | Decompose explicitly, solve sub-problems in order |
Self-consistency reliably improves accuracy at a multiplied cost, which suits high-value decisions rather than routine calls.
6. Reasoning Models
Newer models perform extended reasoning internally, so an explicit instruction to think step by step is often unnecessary and occasionally counterproductive.
The practical guidance has shifted: state the problem and its constraints clearly, and let the model allocate reasoning, rather than prescribing the technique.
7. Keeping Reasoning Out of Output
When steps are useful for accuracy but not for the reader, separate them — request the reasoning and the final answer in distinct fields, and return only the answer downstream.
That keeps the accuracy benefit while preventing intermediate text from leaking into a user-facing surface or a parsed field.
8. Summary
Chain of thought produces intermediate steps before the answer, improving accuracy on multi-step problems by giving the model more to condition on.
It does not verify facts, it costs tokens and latency, and its stated reasoning is not a reliable explanation. With current reasoning models, clear problem statements usually beat explicit instructions about how to think.