Chain-of-Thought Prompting

The idea in one paragraph

Chain-of-thought prompting asks the model to write out intermediate reasoning steps before giving its final answer. Each step becomes context for the next, so the model works through the problem instead of jumping straight to an answer. The technique was formalized by Wei et al. in 2022 ("Chain-of-Thought Prompting Elicits Reasoning in Large Language Models", arXiv:2201.11903, NeurIPS 2022): with just eight worked examples containing reasoning steps, a 540B-parameter model reached state of the art on the GSM8K math word problem benchmark, beating even a fine-tuned GPT-3 with a verifier.

Zero-shot: ask for steps directly

You do not always need examples. Kojima et al. showed in 2022 ("Large Language Models are Zero-Shot Reasoners", arXiv:2205.11916, NeurIPS 2022) that appending "Let's think step by step" before the answer makes large models reason without any demonstrations. On text-davinci-002, that single phrase lifted accuracy on the MultiArith benchmark from 17.7% to 78.7% and on GSM8K from 10.4% to 40.7%.

```

Q: A bakery made 84 loaves in the morning and 36 in the afternoon.

They sold 95 loaves during the day. How many are left?

A: Let's think step by step.

```

The model then writes the arithmetic out and lands on 25. This works reliably on current large models. One caveat: dedicated reasoning models already generate an internal chain of thought, so an explicit "think step by step" adds little there; it helps most on standard chat models.

Few-shot chain of thought for consistent reasoning

When you need the reasoning to follow a particular shape (a diagnosis checklist, a pricing calculation, a debugging procedure), show the model worked examples with the steps written out:

```

Q: A server handles 120 requests per minute. Each request takes

0.4 seconds of CPU. How many CPU cores are needed to keep up?

A: Total CPU per minute: 120 x 0.4 = 48 core-seconds.

One core provides 60 core-seconds per minute.

48 / 60 = 0.8, so 1 core suffices with headroom.

Answer: 1 core.

Q: A queue receives 300 jobs per hour. Each job takes 20 seconds

of GPU time. How many GPUs are needed?

A:

```

The model continues in the same style: compute the totals, convert units, then answer. Few-shot chain of thought controls both the reasoning method and the output format, which matters when you parse the answer programmatically.

Use it for multi-step problems only

Chain of thought helps when the answer requires combining several facts: math word problems, logic puzzles, debugging, multi-hop questions. It adds nothing to single-step tasks like classification, translation, or factual lookup. Worse, on small models it can backfire: Wei et al. found the gains concentrate in large models, and smaller ones can produce confident-sounding but wrong reasoning chains. If the task is one lookup, skip the reasoning and ask for the answer directly.

Vote across several reasoning runs

A single chain of thought can take one wrong turn and never recover. Self-consistency, introduced by Wang et al. in 2022 ("Self-Consistency Improves Chain of Thought Reasoning in Language Models", arXiv:2203.11171), runs the same prompt several times at a non-zero temperature and takes the majority answer. Different reasoning paths that converge on the same answer are unlikely to share the same mistake. The reported gains were large: +17.9% on GSM8K, +11.0% on SVAMP, and +12.2% on AQuA over single-run chain of thought.

```python

from collections import Counter

from openai import OpenAI

client = OpenAI()

answers = []

for _ in range(5):

response = client.chat.completions.create(

model="gpt-4o-mini",

temperature=0.7,

messages=[

{"role": "user", "content": "Let's think step by step.\n" + question + "\nAnswer:"},

],

)

answers.append(response.choices[0].message.content.strip().splitlines()[-1])

final = Counter(answers).most_common(1)[0][0]

```

This multiplies cost by the number of runs, so reserve it for high-stakes answers where one wrong result costs more than five calls.

Reasoning text is for the model; your code needs the answer. End the prompt with a format instruction so the answer lands in a parseable spot:

```

Show your reasoning step by step. On the last line, write exactly:

Answer: <value>

```

Then take the last line and strip the prefix. For strict schemas, prefer Structured Outputs (a JSON schema the model must conform to) over string parsing, which breaks the moment the model adds an extra sentence.

Treat the chain as a draft, not a proof

A readable chain of thought does not guarantee a correct chain of thought. Models sometimes produce sound-looking steps that contain a slipped number or a skipped condition, and the final answer inherits the error. Verify reasoning-heavy outputs the same way you verify anything else: an evaluation set with known answers, re-run on every prompt or model change. Chain of thought also costs tokens; long chains on every request add up. Reserve the technique for the calls where reasoning quality decides the outcome. Log the chains you sample during development; they are the fastest way to see where the model's reasoning actually breaks.