Get the 2026 ML Training Cookbook | 52 recipes — GRPO, Flow Matching, World Models, and everything in between Download Now →
Chain-of-Thought (CoT)
A prompting and training technique where models generate intermediate reasoning steps before producing a final answer.
Chain-of-Thought (CoT) is a reasoning technique in which a language model generates intermediate reasoning steps before producing its final answer, rather than outputting the answer directly. By decomposing a complex problem into a sequence of simpler steps, the model can solve problems that require multi-step reasoning — such as math word problems, logical deduction, and code generation — more accurately than with direct answering.
CoT can be elicited through prompting (e.g., appending 'Let's think step by step' to the prompt) or instilled through training (by fine-tuning on data that includes reasoning traces). Prompting-based CoT requires no model modification but depends on the model's existing capability. Training-based CoT — where the model is fine-tuned or reinforced on data containing step-by-step reasoning — produces more reliable and consistent reasoning, as demonstrated by models like DeepSeek-R1 and OpenAI's o1.
The effectiveness of CoT stems from giving the model more 'compute' in the form of additional tokens to process each step of reasoning. Each generated token provides the model with additional context for subsequent tokens, effectively allowing it to perform intermediate computation in the output sequence. This is why CoT is especially beneficial for arithmetic and logical tasks where the model must manipulate information that doesn't fit in a single forward pass.
- ■Model generates intermediate reasoning steps before the final answer
- ■Can be elicited via prompting ('Let's think step by step') or instilled via training
- ■Particularly effective for math, logic, and multi-step reasoning tasks
- ■Additional tokens provide implicit additional compute per problem
- ■Training-based CoT produces more reliable reasoning (e.g., DeepSeek-R1, o1)
Prompting a model to solve a multi-step math problem by first writing out each calculation step before stating the final numerical answer.