Back to Ideas 6 min read

The Framework in the Description Field

E
EkoHacks Team
·
The Framework in the Description Field

Our package.json had been calling EkoLite "a lightweight, real time backend framework" for months. It is a fair description of the ambition. It was also, until recently, untrue in one precise sense: a framework is a thing you install, and ours could not be installed.

Two commands measure the distance between a sentence and the truth.

npm view ekolite      # 404, the name is free
npm pack --dry-run    # a couple of hundred files

The first is fine, the name was ours to take. The second is the tell. With no files list, the tarball is a photograph of the whole repository: the tests, the docs, the demo, the continuous integration config, every source file. Nobody had ever consumed EkoLite as a dependency, and the tarball showed it. A framework packs the thing it built. A repository packs itself.

The build that built nothing

It got quieter and worse when we looked at what a consumer would actually load. package.json pointed main and types at dist/server/index.js and its declarations. Reasonable. Except the build never produced them.

The source imports its own modules with explicit extensions, import { App } from './app.ts'. That is a deliberate choice that makes the development runner happy, and it comes with a condition. TypeScript will only accept a .ts in an import specifier when it is forbidden to emit:

{
  "allowImportingTsExtensions": true, // the source says './app.ts'
  "noEmit": true                      // so the compiler emits nothing
}

So the build script, tsc && vite build, did two honest things and one absent one. It type checked the whole project. It built the browser demo. And it emitted no server at all, because the compiler was under orders not to. npm run build exited zero and produced nothing you could ship. The description field said framework. The build shipped a repository with a dead pointer at its centre.

Why the whole suite stayed green

Here is the part that matters, because it is the same shape we keep meeting. None of this failed a single test.

Inside the repository every file is present. The .ts sources resolve because they are right there. main points at a file that does not exist yet, but nothing inside the repo ever asks main to resolve, because inside the repo you import the source, not the package. The development server runs. The type check passes. The demo loads. Every green tick is telling you the truth about the code as seen from where the author is standing.

A packaging bug is invisible from inside the package. The author has all the pieces, so the author cannot feel the gap. The only person who can feel it is the one who receives the tarball and nothing else.

The decision: leave the repo

At this point there is a tempting move, and it is the one we did not make. Reach for a bundler. Point a tool at the entry, let it trace and inline and rewrite, and get a dist out the other end. It would have worked, and it would have buried the real question under a dependency we would then have to keep alive.

The decision was smaller and ruder. Do not trust the build. Prove the package the only way that means anything, by becoming the stranger. Pack the tarball, install it into a folder that has never seen this repository, and import it as someone who paid for it and read the README.

// in an empty directory, well outside the repo
import { App } from 'ekolite';

const app = App.createNull();
app.methods.define('sum', (a, b) => a + b);
console.log(await app.methods.call('sum', [2, 3])); // expect 5

That script is the whole test. If it prints 5 on a machine that has never seen our source, EkoLite is a framework in fact. If it throws, the description field was writing cheques the build could not cash. We let that script, and not the green suite, decide.

What the stranger caught

Run from outside, the install caught two things the repo had been hiding in plain sight.

The first was a dependency in the wrong drawer. The websocket server is imported as a runtime value, import { WebSocketServer } from 'ws', but ws was listed under devDependencies. Inside the repo that distinction is invisible, because development dependencies are installed, so the wrapper loads and every test runs. A consumer's install does not carry development dependencies. The first time their code reached the socket, it would throw on a module that was never delivered. The repo resolved it; the tarball dropped it; only the stranger could tell.

The second was subtler and went to the types. Once the build actually emitted, we rewrote the .ts specifiers to .js on the way out, which the compiler does for the runtime files. It does not do it for the declaration files. So the shipped .d.ts still imported ./app.ts, ./infrastructure/websocket.ts, files that live in our source and were never packed. The package ran and its types resolved to nothing, which is the worst kind of working: a consumer's editor would light up green and know nothing. The fix was a short pass over the declarations that finished the job the compiler started. The rule that fell out of it is one we will keep: a shipped declaration may only import a file the tarball actually carries.

The build we kept

The mechanism underneath is deliberately small. The compiler emits. A flag rewrites ./x.ts to ./x.js in the runtime output. A dozen lines rewrite the same in the declarations. The demo moved out of the folder the compiled client library needed, so the two stopped fighting over one name. A files list cut the tarball from the entire repository down to the build, the README and the licence.

No bundler. The stance under the decision is that a build's one job is to tell the truth about what ships, and the smaller the build, the fewer places it can lie. Every tool you add between the source and the tarball is another thing that can be right about your repo and wrong about your package. We would rather have less machinery and one honest test than more machinery and a green tick we cannot cross examine.

The move to take home

If you publish anything, do this before you believe your own build.

Run npm pack --dry-run and read the list. Not the count, the list. If it contains your tests or your CI config or a lockfile, your tarball is a repository wearing a package's name.

Then, in a directory that has never met your source, run npm install ./your-package-0.0.0.tgz and import the thing as the person who will pay for it. Call one real function. Compile one real type. The repository lies to you by being complete, because it hands you every file whether or not you shipped it. The tarball is the only artefact that tells the truth, and the only way to read it is to stop being the author for five minutes and be the stranger.

Why we work this way

The thread runs through everything we do. A green suite proves the code understands itself. It does not prove the package does, or the deploy does, or the person on the other end can do the thing the description promised. Those are different vantages, and a test is only ever as honest as the place you run it from.

So we move the vantage. We test the shutdown from outside the process that is shutting down, the sync from the client that receives the data, and the framework from the folder that installs it. The word "framework" was in our description field for months. It moved into the world the day a stranger's empty directory could import it and get back a five.

E

Written by

EkoHacks Team

More from Ideas

·6 min read

What the Nullable Gave Back

One file, seven behaviours held fixed, the database swapped for a Nullable: about 180 times less time inside the tests, and coverage flat to two decimals.

E
EkoHacks Team
·6 min read

Twenty Six More Tests, Four Fewer Behaviours

Removing the mocks grew the suite from 44 tests to 70 and quietly deleted four behaviours, every one of them a failure path. Test count is not coverage.

E
EkoHacks Team
·6 min read

The Best Coverage Number in the Room

Same commit, same spec, same test count. The mocked suite ran 5.6 times faster, covered 3.5 fewer points of real code, and posted the best branch coverage.

E
EkoHacks Team