We spent one evening giving every wrapper in the EkoHacks CLI a contract list: one list of behaviour that the unit file runs on the wrapper's createNull() and the integration file runs on its create(). If the null and the real ever answer differently, one of those runs fails. The lists found four places where a null had drifted from the thing it stands in for, which is what they are for. They also found one place where a null cannot follow the real thing at all, and that one is worth a post, because the reason is not a bug.
Everything below can be replayed from two commits in github.com/ekohacks/cli; the last section has the commands.
The contract in two lines
A contract list is a function that takes a make and registers tests against whatever make returns. The unit file hands it the null; the integration file hands it the real wrapper over a throwaway repository with a bare origin:
// tests/infrastructure/git.test.ts
gitContract((state) => Promise.resolve(GitWrapper.createNull(state)));
// tests/infrastructure/git.integration.test.ts
gitContract((state) => Promise.resolve(GitWrapper.create({ cwd: throwawayRepo(state).dir })));
state is the repository the test wants: which branch is current, whether the tree is dirty, whether main is behind origin, which branches exist, what version origin's manifest carries. The null is handed the state; the real make builds a repository in that state, with a tracked file so that commit --all has something to stage when the state says dirty.
What went red
Real git refuses to commit when there is nothing to commit. git commit --all on a clean tree exits 1. So the git contract got this item:
it('refuses to commit a clean tree and records nothing', async () => {
const git = await make({});
const actions = git.trackActions();
await expect(git.commitAll('chore: nothing')).rejects.toThrow(/nothing to commit/);
expect(actions.data).toEqual([]);
});
The null, as it stood, answered every git command from a fixed map of outputs and never rejected anything. Run at the commit that introduced the contracts, the unit suite says so:
FAIL tests/infrastructure/git.test.ts > creates a branch, which is then the current one and exists, and records it
FAIL tests/infrastructure/git.test.ts > commits everything on an edited tree, which is then clean, and records it
FAIL tests/infrastructure/git.test.ts > refuses to commit a clean tree and records nothing
FAIL tests/infrastructure/npm.test.ts > refuses a bump that is not a version, naming it, and records nothing
Tests 4 failed | 142 passed (146)
The first two were plain drift, and the fix was to give the null a state it keeps: a created branch becomes the current one, a commit cleans the tree. The fourth was the npm null accepting anything as a version. The third is this post. I taught the null to refuse from its own state, the contract went green on the null and on a real repository, and nine other tests went red, all on the release rail: the command that ships a package.
Two wrappers, one world
The rail begins by demanding a clean tree. Then cut creates a branch, asks npm to bump the version, writes the changelog, and commits everything. On a laptop that commit has plenty to commit: npm just rewrote package.json and the lock file, and the changelog has a new entry. Git sees those files because git and npm share a filesystem. Each one's writes are the other's reads.
In the rail's tests, the git null is created clean, as preflight requires. The npm null bumps nothing on any disk; it answers the version and records the bump on its tracker. The changelog is written by the shell, which the tests do not run. When cut reaches the commit, the git null looks at the only tree it has, its own state, finds it clean, and refuses. The real rail works. The test rail does not. The two nulls share nothing, because there is nothing for them to share.
The rule
A null can model what passes through it. The same evening, the git null learned to keep a branch it had just created as the current one, because createBranch goes through the wrapper. Earlier the same day the observatory's database null learned to remember what it saves, for the same reason. Those were drifts, and the contract list caught them.
A file written by npm, or by the shell, does not pass through the git wrapper. The null has no filesystem on which that file could appear. So the rule for what belongs in a contract list is this: behaviour whose inputs the wrapper itself can observe. Behaviour that depends on the state of the world outside the wrapper belongs to the real side's narrow test, where the world is real.
The limit of a null is the limit of what it can observe.
Three ways out
| Option | What it needs | What it costs |
|---|---|---|
| Configure the null dirty from the start | nothing new | preflight demands a clean tree first; one null cannot be clean at preflight and dirty at the commit |
| Give the nulls a shared world | an in-memory filesystem that npm's bump dirties and git's status reads | a second filesystem to keep faithful across every wrapper that touches disk; the nulls become a simulation |
| Make the null's commit permissive, keep the refusal on the real side, write down why | three lines fewer in the null, one test in the integration file | a test on nulls can no longer catch "you committed before writing anything" |
We chose the third. The null's commit now reads, with its reason beside it:
// The null is a repository in a given state, and it keeps that state the way a repository
// would: a created branch becomes the current one and exists, a commit cleans the tree. It
// cannot see a file npm or the shell wrote, so it never refuses a commit for having nothing
// to commit; that is the real side's own test.
if (command === 'commit') {
state.dirty = false;
return Promise.resolve('');
}
Shore's nulls are embedded stubs at the lowest level of each wrapper. A world they all share is a different design, and we did not want one for a single assertion.
Where the lost check lives now
The refusal did not vanish; it moved to where the world is real. The integration file keeps it, against a throwaway repository, and the contract carries a comment saying what the null cannot see:
it('refuses to commit a clean tree, as git does', async () => {
const { dir } = throwawayRepo({});
const git = GitWrapper.create({ cwd: dir });
const actions = git.trackActions();
await expect(git.commitAll('chore: nothing')).rejects.toThrow(/git commit/);
expect(actions.data).toEqual([]);
});
And the CLI releases itself: every version of ekohacks ships by running ekohacks release, so the rail runs for real against a real tree before anyone else depends on it. The suite proves the tool does what we said; the release proves what we said matches the world. That split was the point of the whole series on encoding the release, and this is one more thing that landed on the release's side of it.
Replay it
The red commit and the green commit are both on main. The moment in between, the rail going red, lived in a working tree and was never committed, so the last step recreates it with a three-line change.
git clone git@github.com:ekohacks/cli.git && cd cli
npm ci
git checkout 39943ba # the contracts, before the nulls were fixed
npx vitest run # 4 failed | 142 passed, the four lines quoted above
git checkout fece0c2 # the nulls fixed, the refusal moved to the real side
npx vitest run # 145 passed
npm run test:integration # the real side: throwaway repositories, a real shell, the registry
To watch the rail break, give the null back its refusal. In src/infrastructure/git.ts, inside createNull:
if (command === 'commit') {
+ if (!state.dirty) {
+ return Promise.reject(new Error('nothing to commit, working tree clean'));
+ }
state.dirty = false;
return Promise.resolve('');
}
Then npx vitest run tests/logic: nine failures, four in cut.test.ts, four in release.test.ts, and one in docs.test.ts, because docs draft also commits files the shell wrote. Not one of those tests mentions git's refusal. They all go red because the null now knows something that is not true in any of them.
What this says about contract lists
Contract lists find drift. Eleven drifts across the observatory and EkoLite on the morning we wrote the first ones, four more in the CLI that night. They also find the edges of the pattern, which no amount of drift-hunting would show you: the exact point where a null stops being able to stand in for the real thing, and why. When a contract item will not go green on the null, ask first what the null would have to see to honour it. If the answer is "something another wrapper wrote", the item is not a contract. It is a narrow test, and it was always going to be.
The receipts are two commits in github.com/ekohacks/cli: 39943ba, the contracts, red, and fece0c2, the nulls fixed, green, both from 2 November 2026.
