The Failing Test Is the Review

Encoding the release, part 2 of 8.

Open the commit history of ekohacks/cli and you will see the same shape over and over:

feat: cut stops when the release branch already exists
test: cut stops when the release branch already exists
feat: the nulled gh answers waiting run rounds in order
test: the nulled gh answers waiting run rounds in order

Every behaviour is two commits. The test: commit lands first and it is red — a failing test, committed to main on purpose. The feat: commit follows with the production code that makes it green. Our agent notes state the contract bluntly:

A failing test on main is not a bug. It is the committed specification for the next piece of work. Make it green by writing production code; never edit a committed red test to make it pass, and never commit production code in the same commit as the test that specifies it.

Why work this way? Because the red test is the review artefact. A reviewer looking at a test: commit is reviewing pure intent: here is the behaviour we claim to want, expressed as running code, with no implementation to be seduced by. Arguments about whether the behaviour is right happen at the cheapest possible moment — before the implementation exists. By the time the feat: commit arrives, the only question left is "does this satisfy the committed spec?", and the suite answers that mechanically.

It also keeps the work honest in the other direction. "Never edit a committed red test to make it pass" means the implementation must rise to the specification, not the other way round. And the refactor step keeps its classical meaning: a refactor: commit changes no observable behaviour and therefore needs no new test — the moment a change would alter what a test observes, it is not a refactor, it is a red test waiting to be written.

Honesty requires admitting what this costs. A committed red test is live ammunition. Ours run directly on Node's type stripping — types are removed, not checked, at run time — and we once committed a red test that called an option that did not exist yet: NpmWrapper.create({ cwd }) before cwd was implemented. The option was silently ignored, and the red test ran npm version against our own repository instead of its temporary directory, cheerfully bumping the CLI's own version mid-test. We caught it, reverted it, and wrote the lesson into our agent notes: typecheck the red half before committing it, and point any red test that reaches real infrastructure at throwaway state. Red-first is a discipline, and disciplines have safety rules.

What we get in exchange is a repository where the suite is the specification — not in the aspirational sense that phrase usually carries, but literally: every behaviour the tool has was a reviewed, committed, failing test before it was code. When part 6 of this series adds a brand-new preflight check in five commits, you will be able to watch the specification arrive before the behaviour, in public.

Next: Testing a Release Without Releasing.