Module 6 of 11 · Advanced · 10 min read

Chain-of-thought and reasoning models

When step-by-step reasoning helps, when it doesn't, and how to prompt models that already think before they answer.

Key takeaways

1Chain-of-thought (Wei et al., 2022) improves multi-step reasoning in models that don't reason by default.
2Reasoning models think internally; don't script their steps. Describe the goal and use effort settings.
3Ask for a short justification or a verification step, not a transcript of thinking.
4Written reasoning is not guaranteed to reflect how the model actually reached its answer.

01What chain-of-thought is

Chain-of-thought (CoT) prompting asks the model to work through intermediate steps before giving a final answer. Wei et al. (2022) showed that including worked, step-by-step examples substantially improved large models on arithmetic, commonsense and symbolic reasoning tasks. Kojima et al. (2022) found that simply adding 'Let's think step by step' produced a similar effect without examples.

The intuition: a model produces each token based on what came before, so writing out intermediate steps gives it more room to work before committing to an answer.

02When it helps and when it doesn't

Use it where there is genuinely something to work out.

Helps: multi-step maths, logic puzzles, planning, comparing several options against criteria, debugging.
Little or no benefit: simple lookups, classification with clear labels, short rewrites, and creative writing where deliberation adds nothing.
Costs: more tokens, more latency, and longer outputs you may need to strip before showing users.

03Prompting models without built-in reasoning

For models that don't reason by default, ask for the reasoning in a separate, tagged section followed by the answer. Your code can then show only the answer, and you can inspect the reasoning when debugging.

Example

Decide which of the three suppliers in <quotes> we should choose, given the criteria in <criteria>. First, inside <thinking> tags, compare each supplier against each criterion and note any missing information. Then give your recommendation inside <answer> tags in three sentences or fewer, including the main trade-off.

04Prompting reasoning models

Reasoning models (sometimes called thinking models) are trained to deliberate internally before they reply, and many APIs let you set a reasoning effort or thinking budget. With these models, rigid scripts like 'Step 1: list assumptions. Step 2: ...' can override a better plan the model would have found, and asking for visible chain-of-thought may be redundant or unsupported.

Instead, give them what a senior expert would want: the goal, the context, the constraints, what a great answer looks like, and how you will check it. Raise the effort setting for hard problems instead of adding 'think harder' to the prompt.

Example

Goal: find the root cause of the intermittent timeout described below and propose the smallest safe fix. Context: <logs>{{logs}}</logs> <code>{{relevant files}}</code> Constraints: no new dependencies; the public API must not change. A great answer names the root cause with the evidence that proves it, gives a patch, and lists how to verify the fix. If the evidence is not conclusive, say what extra logging would settle it.

05Ask for verification, not a transcript

Often the most useful reasoning request is a check: 'Before answering, verify each figure against the table' or 'After writing the code, check it against each requirement and fix anything that fails'. This targets the errors you care about without flooding the output.

If you show reasoning to users, ask for a brief justification written for them, rather than the model's working.

06A caution about faithfulness

Research has found that the reasoning a model writes out does not always reflect what actually drove its answer. Treat visible reasoning as a debugging aid and a prompt for the model to check itself, not as proof that the answer is right. Verify conclusions independently when they matter.

Watch out

A long, confident chain of reasoning can make a wrong answer more persuasive. Check the conclusion, not the eloquence.