Chain of thought prompting: what it is and how thinking models change it

Chain of thought prompting is a technique where you ask a language model to work through intermediate reasoning steps before it gives its final answer. It was introduced by Google researchers in 2022 for math and logic problems. On today's reasoning models from Anthropic, OpenAI and Google, that step-by-step reasoning happens internally by default, so the main lever has moved from prompt wording to a setting such as effort or thinking level.

This guide covers where the technique came from, what the prompts look like, how built-in thinking changes the advice, and when a manual chain of thought prompt is still the right tool. Vendor details were checked against official docs on 26 October 2026.

Where did chain of thought prompting come from?

From a 2022 Google Research paper by Jason Wei and colleagues. The accompanying Google Research blog post (May 2022) summarizes it: "Called chain of thought prompting, this method enables models to decompose multi-step problems into intermediate steps."

Three findings from that post shaped how people used the technique for years:

• Examples were enough. Thought processes "can be elicited by including a few examples of chain of thought via prompting only, which does not require a large training dataset or modifying the language model's weights."
• Scale mattered. The authors found that "the benefits of chain of thought prompting only materialize with a sufficient number of model parameters (around 100B)."
• It targeted multi-step problems. Their tests used arithmetic word problems, including the GSM8K benchmark, and commonsense reasoning tasks such as CommonsenseQA and StrategyQA.

What does a chain of thought prompt look like?

There are two classic forms.

Few-shot chain of thought shows worked examples, each with its reasoning, then poses the new question:

Q: A team has 3 open bugs. They fix 2 and 5 new ones are reported. How many are open?
A: Start with 3. Fixing 2 leaves 1. Adding 5 new ones gives 6. The answer is 6.

Q: A build takes 4 minutes. Caching cuts it by a quarter, then a new test suite adds 2 minutes. How long is it?
A:

Zero-shot chain of thought gives no examples and asks for the reasoning directly:

A build takes 4 minutes. Caching cuts it by a quarter, then a new test suite adds 2 minutes.
How long is the build now? Work through it step by step before giving the answer.

When you need to parse the result, separate the reasoning from the answer. Anthropic's fallback pattern asks the model to think the problem through and "put the final answer in <answer> tags so you can extract it."

How do thinking and reasoning models change chain of thought?

They do the reasoning themselves, in a separate channel, before they write the answer. Anthropic's thinking docs describe it: "When thinking is active, Claude works through the problem in its own words before answering: it restates what is being asked, tries approaches, checks intermediate results, and abandons paths that do not hold up." OpenAI's reasoning guide says reasoning models "use internal reasoning tokens before producing a response," and Google's Gemini thinking docs say "Gemini models engage in dynamic thinking by default, automatically adjusting the amount of reasoning effort based on the complexity of the request."

Each vendor now exposes a dial instead of asking you to write "think step by step":

On the Claude side, Anthropic's prompting best practices say Claude 4.6 and later models use adaptive thinking, "where Claude dynamically determines when and how much to think," and that on Claude Fable 5.1, Claude Mythos 5.1, Claude Fable 5, Claude Mythos 5 and Claude Opus 5.5 "thinking is always on and adaptive thinking is the only mode." The effort docs add that most Claude models default to high effort, while Claude Opus 5.5 and Claude Haiku 5.5 default to medium, and that "Effort is a behavioral signal, not a strict token budget."

Older Claude models used manual extended thinking with a budget_tokens cap. Anthropic says setting budget_tokens on Claude 4.7 and later models returns a 400 error, and recommends effort instead.

Should you still write "think step by step"?

On a reasoning model, usually not. All three vendors now say so in their own words:

• OpenAI: "Avoid chain-of-thought prompts: Since these models perform reasoning internally, prompting them to "think step by step" or "explain your reasoning" is unnecessary." Its reasoning best practices add that such instructions "may not enhance performance (and can sometimes hinder it)."
• Google: For Gemini 2.5 and 3 models, "it's generally not necessary to have the model outline, plan, or detail reasoning steps in the returned response itself." It does note that "simple requests like "Think very hard before answering" can improve performance, though at the cost of extra thinking tokens."
• Anthropic: "Prefer general instructions over prescriptive steps." A prompt such as 'think thoroughly' "often produces better reasoning than a hand-written step-by-step plan. Claude's reasoning frequently exceeds what a human would prescribe."

There is a sharper reason on the newest Claude models. Anthropic says that on Claude Fable 5.1, Claude Fable 5, Claude Opus 5.5, Claude Opus 5 and Claude Sonnet 5.5 you should rely on thinking instead, because "a prompt that asks the model to write out its reasoning, for example in <thinking> tags, may be declined." Its refusals docs list this as the reasoning_extraction category and name common triggers: a <thinking> or scratchpad section, or a reasoning field in JSON output. The fix is to ask for "a short explanation, the evidence behind a result, or a summary of the actions it took" instead.

What still helps on reasoning models is everything around the reasoning: a clear goal, constraints, context, and a definition of done. OpenAI's advice for reasoning-capable GPT-5 models is to give "a clear goal, strong constraints, and an explicit output contract without prescribing every intermediate step."

When is manual chain of thought still useful?

When the model is not reasoning on its own. That covers:

• Models with thinking off. Anthropic notes that on Claude Opus 4.6 through 4.8 and Claude Sonnet 4.6, thinking is off when you omit the thinking parameter, and Claude Opus 5 and Claude Haiku 5.5 can disable it at high effort or lower.
• Non-reasoning or low-effort settings on any vendor, where you have chosen speed over reasoning.
• Pipelines where you need the steps as output, such as a tutoring app that shows a worked solution. Here the steps are part of the product, so ask for them as a user-facing explanation, not as the model's private reasoning.

For these cases Anthropic describes manual chain of thought "as a fallback": "When thinking is off, you can still encourage step-by-step reasoning by asking Claude to think through the problem before it answers." One wording note from the same page: when extended thinking is disabled, "Claude Opus 4.5 is particularly sensitive to the word "think" and its variants," so it suggests words like "consider" or "evaluate."

Examples still pair well with thinking. Anthropic says "Multishot examples work with thinking" and that worked examples "shape how Claude approaches similar problems in its own thinking blocks." See few-shot prompting.

Can you see the model's reasoning?

You can see a summary. Anthropic says "what you see is never the raw chain of thought: the text in a thinking block is a summary of Claude's reasoning," and the display field controls whether it is returned: "summarized" returns it, while "omitted", "the default on many models," returns an empty thinking field. OpenAI says "While we don't expose the raw reasoning tokens emitted by the model, you can view a summary of the model's reasoning using the summary parameter." Gemini returns thought summaries when you enable thinking_summaries.

Summaries are useful for debugging prompts. Google's thinking docs recommend you "Analyze thought summaries to understand failures and improve prompts."

What does reasoning cost?

Reasoning is generated text, so you pay for it. Anthropic: "the tokens Claude spends reasoning are billed as output tokens, even when the thinking text isn't returned to you, and they count toward max_tokens alongside the response text." Google: "When thinking is turned on, response pricing is the sum of output tokens and thinking tokens." OpenAI counts reasoning tokens in addition to input and output tokens.

To control spend, lower effort or thinking level first. Anthropic's docs say effort is soft guidance and max_tokens is the strict limit; Google warns that a low max_output_tokens can truncate a response mid-reasoning and suggests lowering thinking_level instead. In Claude Code, the same ideas show up as effort levels and the ultrathink keyword. For how tokens add up, see LLM tokens explained.

Where should you keep reasoning prompts that work?

Prompts that steer reasoning well, such as a "define done and verify" instruction for an agent, are worth saving and reusing. Kipt keeps prompts as Markdown in a private GitHub repo you own, with {{name:default}} blanks for the task or constraints, so you can find the tested version later instead of rewriting it.

What should you read next?

The prompt engineering guide covers the techniques around reasoning, such as structure and context placement. Few-shot prompting explains how examples interact with thinking. If you need the final answer in a fixed shape, LLM structured output covers JSON Schema output, including why a "reasoning" field can backfire on Claude.