Back to Ideas 6 min read

The Long Way Round

E
EkoHacks Team
·
The Long Way Round

A method can run a script now, but a script that runs and hands its answer back to one caller is an ordinary thing that any framework gives you. This is the post where the number a script computes finds its way to the screen, and it gets there by a route you might not expect.

Here is the shape. Upload a file with a sequence in it. Ask the server to count the C's. Watch the number appear next to the file. That is the whole feature, and it is smaller than it sounds, because every piece it needs was already standing. EkoLite, our small real time backend, could already take an upload. It could already run a method over a socket. It could already stream a document to a subscribed client as Mongo changed. The only new thing is the joining, and the joining hides one decision that is the whole reason EkoLite exists.

The short way and the long way

The method takes the id of an uploaded file. It looks the file up, runs the count script against it, reads the number the script printed, and hands that number back to whoever called. So far, so ordinary. A client calls a function on the server, the function returns a value, the value comes back. If that were the end of it, the count would land in the caller's promise and nowhere else.

But that is not how the count reaches the screen. Look at what the method does with the number before it returns it: it writes it onto the file's own document in Mongo. And the client is not watching for a return value at all. It is subscribed to the files collection, the same subscription that drew the list in the first place. When the document gains a countC field, the change streams down that subscription, the local store updates, and the row shows the number. The count arrived, but not through the door the client knocked on.

So why write it to the database when the method already returned it? Because a return value is the short way, and the short way is a dead end for everything that matters here.

What the long way buys

Return the count and exactly one caller sees it, once. Write it to the document and everyone looking at that file sees it, because the count is now a fact about the file rather than a private reply to one request. Open the page in a second window and the number is there too, with no second call.

Return the count and it is gone the moment you stop holding the promise. Write it to the document and it persists. Reload the page and the count is still on the row, streamed back on reconnect through the same subscription that fills the list. Nobody runs the script again.

Return the count and whoever triggered the analysis has to be the one to show it. Write it to the document and the trigger and the display come apart. A button clicks it today. A batch job could click it tomorrow. The screen does not care who asked, it just watches the collection.

That is what reactivity is in EkoLite, said plainly: a result becomes reactive by being written into a collection somebody is subscribed to. There is no other channel. If the method only returned the count, we would have built an ordinary request and response, the kind any framework hands you. Writing it back is the difference between that and the thing EkoLite is actually for. The short way works. The long way is the point.

Proving a loop with no database and no python

Here is the part that would make you nervous if you had not met Nullables before. A test for this feature seems to need the world. A real upload. A real Mongo, and not a plain one either, a replica set, because the change stream that carries the count back only fires on a replica set. A real python on the box to run the script. A real socket. That is a lot of world to stand up just to prove that a number goes round a loop.

We stand up almost none of it. The acceptance test boots the real server, opens a real socket, and drives the real client. But the Mongo underneath is a nulled Mongo, an off switch built into the wrapper, primed to answer with the file and to carry its own writes through as change events. The script runner is a nulled runner, primed to answer as if python had printed a three. No database, no replica set, no python. The test subscribes, calls the method over the wire, and waits for the count to appear in the store, exactly as a browser would.

It passes. And because the socket, the server, and the client in that test are the real ones, the same code that runs in the test is the code that ships. The only pieces switched off are the two that reach the outside world, and they are switched off in a way the code above them cannot tell. That is the whole trick, and it is why the loop could be built with confidence before a single real byte moved.

One honest note on the counting itself, since it is the sort of thing that hides a bug. A C count does not care about case, so a c and a C both count, and the script prints a bare number so the method can read it straight. The nulled test rehearses the wiring with a fixed answer. The real script, running for real against a real file, is proven in a separate test that does shell out, so the two never drift: the wiring is honest about the command it sends, and the script is honest about the number it prints.

The move you can reproduce

You do not have to take our word for the loop. The write back is the whole mechanism, so watch it happen with no database and no python. Build a null app, prime the runner, and define the method the way an app would, then call it:

import { App } from 'ekolite';

const app = App.createNull({ scriptResponses: { python3: '3' } });

app.methods.define('runCountC', async (id) => {
  const { stdout } = await app.scriptRunner.exec('python3', ['scripts/countC.py']);
  const count = Number(stdout);
  await app.files.recordCountC(String(id), count); // the write that makes it reactive
  return count;
});

await app.methods.call('runCountC', ['file-1']); // 3

The return value is the short way, the one line the caller sees. The recordCountC write is the long way, and it is the write a subscribed client watches. Nothing shelled out and nothing touched a database, yet both halves of the loop ran: the method read the runner's answer, and it wrote that answer onto the document where a subscription would find it. Swap the null app for a real one, point it at a replica set and a real script, and the same two lines send the count the long way round, down the subscription that was already open, which is the only way anything ever comes back here.

E

Written by

EkoHacks Team

More from Ideas

·6 min read

What the Nullable Gave Back

One file, seven behaviours held fixed, the database swapped for a Nullable: about 180 times less time inside the tests, and coverage flat to two decimals.

E
EkoHacks Team
·6 min read

Twenty Six More Tests, Four Fewer Behaviours

Removing the mocks grew the suite from 44 tests to 70 and quietly deleted four behaviours, every one of them a failure path. Test count is not coverage.

E
EkoHacks Team
·6 min read

The Best Coverage Number in the Room

Same commit, same spec, same test count. The mocked suite ran 5.6 times faster, covered 3.5 fewer points of real code, and posted the best branch coverage.

E
EkoHacks Team