Module 4 of 11 · Expert · 10 min read

Coding loops: keep going until the tests pass

Use tests as the stopping condition for coding agents, keep them honest, and stay in control of the result.

Key takeaways

1Tests turn 'looks right' into an objective stop condition for a coding loop.
2Write or approve the tests first, and never let the agent weaken tests to make them pass.
3Set iteration and time budgets, and commit working checkpoints.
4Passing tests is necessary, not sufficient: review the diff.

01Why tests make great loop conditions

Coding is where loops shine, because code can be checked objectively. A coding agent can write a change, run the test suite, read the failures, fix them and run again, without you in the loop for each step. The tests are the critic, and unlike a model's opinion, they can't be talked round.

Example

loop (max 8 iterations or 30 minutes): run tests → all pass? → run linter and type-check → clean? → stop and report otherwise → read the first failure → form a hypothesis → make the smallest change → repeat if the same failure appears 3 times in a row → stop and report what was tried

02Tests first

The loop is only as good as its tests. For a new feature, write the tests first, or ask the agent to propose them and review them before it writes the implementation. Include edge cases and at least one test for behaviour that must not change.

Example

We are adding {{feature}}. Before writing any implementation, propose the test cases you would add, as a list of test names with one line each on what they check. Include edge cases and one test proving {{existing behaviour}} still works. Wait for my approval before writing code.

03The brief for the loop

Once the tests are agreed, give the agent a brief with a clear exit and clear limits.

Example

Implement {{feature}} so that all tests in {{test file}} pass, along with the existing suite. Work in a loop: run the tests, fix the first failure, repeat. Rules: - Do not modify, skip or delete tests. If you believe a test is wrong, stop and explain why. - Keep changes inside {{directories}}. No new dependencies. - Commit after each green run with a short message. - Stop after 8 iterations without progress and summarise what you tried. Finish with the diff summary, the test output, and anything you are unsure about.

04Keeping the agent honest

Optimising hard for a metric invites gaming it. These rules close the most common loopholes.

Forbid editing tests, or review any test changes separately.
Watch for special-casing: code that detects the test inputs and returns hard-coded answers.
Check that tests actually ran: a command can report success after silently running zero tests.
Run the suite yourself, or in CI, before merging.

05Budgets, checkpoints and flaky tests

Give every coding loop an iteration cap and a time or cost budget. Commit or checkpoint whenever the suite goes green, so a later misstep can be rolled back. If a test fails intermittently, the loop can thrash forever; tell the agent to re-run a failing test once before changing code, and to report suspected flakiness rather than 'fixing' it.

06Green is not the same as good

Passing tests means the code meets the tests, not that it is well designed, secure or what you meant. Read the diff. Ask the agent, or a second model in a fresh context, to review the change for security, performance and readability. For anything touching money, personal data or authentication, a human review is non-negotiable.

Watch out

An agent that can run commands can also delete files or push code. Run coding loops in a sandbox or on a branch, with limited credentials.

Finished reading?

Mark it complete to track your progress, then test yourself with the Loops, Graphs & Agents quiz.

Next: Loop control: stopping conditions, budgets and drift →