Unknown Is an Answer

A stub is not a small copy of the thing it stands for. It is a statement of what your code is allowed to know. When a stub cannot answer a question, the tempting move is to make it cleverer. The better move is to ask whether the real thing can always answer it either.

This is the story of one fix in EkoLite, our small real time backend, where a stub's refusal to answer turned out to be the most accurate thing in the room.

The leak was the easy part

A publication in EkoLite is a name, a collection and a query. A client subscribes to tasks.mine for one owner and is sent that owner's tasks. While building an app with rooms on top of it, we found that the query held for the first read and for nothing after. Every later insert and update on the collection went to every subscriber, whatever they had asked for. A tab in one room was being sent the other room's documents as they changed. The page showed only its own room, so nothing looked wrong. The data travelled all the same.

The fix is easy to say. When a change arrives, test the changed document against each subscription's query, and send it only where it belongs. A document that stops matching leaves. One that starts matching arrives.

With the real database that is nearly all there is to it. The change stream hands over the whole document as it now stands, so the test has everything it needs.

Then we wrote the first failing test, and the stub would not play.

What the stub could not say

Our tests run against a Nulled Mongo, a stand in that answers at the edge of the system while all of our own code runs for real. It holds no documents. That is a rule we wrote down long before this fix:

Simulate state only when a faithful model needs no reimplementation of the dependency's own logic.

Mongo's reads are queries. A stub that kept documents and answered find(query) faithfully would have to rebuild how Mongo matches, which is rebuilding the dependency. So ours stays dumb, and when a test updates a document, the change it emits carries only what the update wrote.

await mongo.update('tasks', { _id: 'task-a' }, { $set: { title: 'renamed' } });
// the change carries { title: 'renamed' } and nothing else

Now hold that against a subscription to { owner: 'ada' }. The change says nothing about owner. Is the field missing, so the document no longer belongs? Or is it untouched, so the document belongs exactly as it did a moment ago? The change cannot say.

Two ways out, and both were lies

Give the stub a memory. Let it keep every document it is handed, so an update can be answered with the whole document. It sounds harmless, and it is the first step of writing a second database. The moment the stub holds documents, a test will ask it to find some, and it has to match queries to answer. We had already refused that once, in writing.

Read absence as missing. Treat a field the change does not carry as a field the document does not have. Then every rename of a task would take it off its owner's screen, because the rename says nothing of owner. The tests would fail for a reason that has nothing to do with the code under test, and the next move would be to bend the production code until the stub was satisfied.

The two share a fault. Each makes the stub claim to know something it does not.

Unknown is an answer

The third way was to let the change say what it is. An update that carries only what was written is marked as such, and the matcher answers in three values where it used to answer in two.

export type Verdict = 'match' | 'no-match' | 'unknown';

unknown means the change says nothing about a field the query reads. The engine reads it with care, and its rule fits in two sentences. A document the client holds stays until a change says it has left. A document the client lacks comes in only when a change says it belongs.

const verdict = verdictFor(doc, query, { partial: change.partial === true });

if (documentIds.has(change.id)) {
  if (verdict === 'no-match') {
    // it has left: tell the client, and forget it
  } else {
    // it belongs, or nothing says otherwise: send the change
  }
} else if (verdict === 'match') {
  // it belongs now: send it whole
}

The three values carry through the logical operators the way you would hope. One failed condition sinks an $and, whatever else is unknown. One met condition carries an $or. Anything less stays unknown.

The stub was describing production

Here is the part that changed how we saw the whole fix. We went to mark the real database's changes as whole, and found this line, which had been there all along:

fields: fullDocument ? extractFields(fullDocument) : (updateDescription?.updatedFields ?? {}),

The real change stream looks the document up after the update. If the document is gone by then, deleted in the moment between, there is no whole document to hand over, and the code fell back to the fields the update wrote. It had always been able to hand us part of a document. It had never said so.

So the stub had not been failing to imitate production. It had been describing a state production could already be in, one we had never named. The mark we added for the sake of the tests, partial, is set by the real database in exactly that case. The test double and the real thing now tell the same truth in the same words.

What we refused

The matcher follows the part of Mongo's query language that publications are written in: equality, paths into embedded documents, values held in arrays, the comparisons, and $and, $or and $nor. It is not Mongo, and it does not pretend to be.

A publication whose query uses anything else, $regex for instance, is refused when a client subscribes, with an error that names the operator. We weighed the other two choices. Forward every change and the leak is back. Forward none and the client goes quietly out of date. A refusal at subscribe is the only one of the three that a developer sees on the first run.

Checked from both sides

The unit tests drive the engine with the Nulled Mongo, so they walk the partial path. An integration test drives it with a real replica set, so it walks the other, where a change carries the whole document and a missing field is missing. There, unsetting the field a query reads takes the document away from the client, as it should.

Each failing test was run against the code as it stood before the fix: 42 failed for the matcher, 2 for the mark and 6 for the engine. Then we opened two rooms of the app in a browser and read what each tab had been sent. Before, a tab held the other room's session and its people. After, each held its own room and nothing else.

The move to take home

When a stub cannot answer, stop before you teach it to. Ask three questions.

• What would the stub have to know to answer, and would knowing it mean rebuilding the thing it stands for?
• Can the real thing always answer, or is there a moment, a race or a failure where it cannot either?
• If there is such a moment, does your code have a name for it?

Ours did not. The honest answer was unknown, and once it had a name, the code that reads it could be written in two sentences.

The fix is in the open, beside the rest of what we have built. This way of working is how we teach everything, and the nearest course in this stack is API Design in Node.js with TypeScript: four weeks, one real API built test first, with no mock anywhere.