Module 2 of 11 · Expert · 11 min read

Refinement loops: generate, critique, revise

Improve outputs by looping through feedback, as Self-Refine and Reflexion do, without drifting into blandness.

Key takeaways

1Self-Refine (Madaan et al., 2023): the model drafts, critiques its draft, then revises, repeatedly.
2Reflexion (Shinn et al., 2023): after a failed attempt, the model writes a lesson it uses next time.
3Critique against explicit criteria, not 'make it better'.
4Stop when criteria pass or improvement stalls; more rounds can make things worse.

01The basic loop

A refinement loop alternates between producing and judging. Self-Refine (Madaan et al., 2023) showed that the same model can generate an output, give itself specific feedback, and revise using that feedback, improving results across tasks such as dialogue responses and code optimisation without any extra training.

The loop is simple to run in a chat or build in code: generate, critique against criteria, revise, and check whether to stop.

Example

draft = generate(task, context) # first pass repeat up to 3 times: critique = critique(draft, rubric) # criterion by criterion if critique says every criterion passes: stop draft = revise(draft, critique) # fix only what was flagged return draft, critique

02Writing the critique prompt

The critique step is where most loops succeed or fail. A vague critic ('any improvements?') always finds something to change, which leads to endless tinkering. A strict critic with explicit criteria finds real problems and can say pass.

Example

Review the draft in <draft> against each criterion in <rubric>. For each criterion, quote the part of the draft that fails it, or write PASS. Only flag real problems a demanding editor would fix. Do not suggest changes that are a matter of taste. End with VERDICT: PASS if every criterion passes, otherwise VERDICT: REVISE and a numbered list of the specific fixes needed.

03Writing the revise prompt

Ask the reviser to address only the listed issues and keep everything else the same. Without that constraint, each revision rewrites the whole piece, and the good parts get lost along with the bad.

Example

Revise the draft in <draft> to fix each numbered issue in <critique>. Change only what is needed to fix those issues; keep the rest of the wording, structure and facts the same. After the revised draft, list each issue number and how you fixed it.

04Reflexion: learning across attempts

Reflexion (Shinn et al., 2023) adds memory to the loop. When an attempt fails a test or gets negative feedback, the model writes a short reflection on what went wrong and what to do differently. That reflection is stored and included in the next attempt. In the paper this improved results on coding, decision-making and reasoning tasks.

You can use the same idea by hand: keep a lessons list in the conversation or a notes file and add to it after each failed attempt.

Example

The last attempt failed. Attempt: <attempt>{{attempt}}</attempt> Result: <result>{{failure}}</result> Lessons so far: <lessons>{{lessons}}</lessons> Write a two-sentence reflection: what specifically went wrong, and what you will do differently. Then make a new attempt that applies the reflection and every earlier lesson.

05Same model or a separate critic?

Wherever you can replace a model's opinion with a real check, such as a test, a schema validator or a fact lookup, do.

Same model, same conversation: simplest, but it may defend its own work.
Same model, fresh context: give the critic only the draft and rubric, so it isn't anchored by the drafting process.
Different model or a person: most independent, useful for high-stakes outputs.
External signal: tests, validators, search results and compilers are the most trustworthy critics of all.

06Pitfalls

Most of the gain usually comes in the first round or two of revision. Stop when it stops improving.

Rubber-stamping: the critic approves everything. Use binary criteria and require quotes as evidence.
Nitpicking forever: the critic always finds something. Restrict it to rubric criteria and cap iterations.
Drift to blandness: repeated editing sands off voice and specifics. Keep the original as a reference and protect what works.
Regression: a revision fixes one issue and breaks another. Re-check every criterion each round, not just the flagged ones.