Webstack: three small libraries, tested the way people use them · Gor Berlian · diadem.fun
← Gor Berlian
ReportNov 18, 20266 min read

Webstack: three small libraries, tested the way people use them

Contents

Webstack is the base that our other projects are built on. It has three open-source libraries for Bun:

  • TradJS (tradjs on npm): a web framework. It has file-based routes, pages rendered on the server, small browser scripts where a page needs them, API routes, middleware, static pages and a starter app.
  • sqlite-zod-orm (sqlite-zod-orm on npm): a SQLite database layer. You describe tables with Zod schemas and get typed reads and writes, relations, soft deletes, timestamps, hooks, transactions and live change events.
  • measure-fn (measure-fn on npm): timing and logging for functions. Nested spans show where time goes.

This report covers one day of work on them. The question was simple: do they work when used for real? Unit tests alone weren't enough to say.

Why unit tests were not enough#

The libraries already had thousands of unit tests, all passing. We still found five real bugs in one day. Each one only showed up when the code ran the way an app runs it:

  • a real server talking to a real browser;
  • a real database file written by an older version;
  • the default options instead of the test-friendly ones.

So we built tools that do exactly that, and fixed what they found.

Tool 1: the starter app in a real browser#

A script creates a new app with tradjs init and installs it from npm like a new user would. It starts the server and drives a headless Chrome through 30 steps:

  • register, log in, log out, wrong password, duplicate account;
  • add, edit and delete notes, 20 per page;
  • a second browser tab that updates live;
  • a second user who must not see the first user's notes;
  • typing hostile text;
  • forged requests from another website;
  • keyboard-only use, plus checks that every control has a label.

Each step saves a screenshot, and we look at them.

The starter app's home page

What it found:

  • Text responses were downloaded instead of shown. In every TradJS app, a route that returned plain text reached the browser as an unnamed download. An error message like "Invalid credentials" showed up as "This site can't be reached". The cause: Bun picks the text type only when it sends the original response, and TradJS copied every response to add a request id. Fixed in tradjs 4.6.4.
  • A failed login was a dead end. The starter app showed a bare text page with no way back. Now the form returns with a clear message:

A wrong password brings the form back with a message

  • Hostile text stays text. A note containing <script> and an onerror image shows as plain text and runs nothing:

Hostile note text is shown as text

  • Forged posts are refused. We sent posts from another site carrying the real session cookie, and all were refused (403). After logout, the notes API answers 401.
  • The screenshots changed the design. The first version opened an edit box for every note, so a page of 20 notes was a wall of text boxes. Editing now opens on demand:

Notes with Edit and Delete toggles

Result: 30 of 30 steps pass against the published packages.

Tool 2: SVG in a real browser#

We rendered the same SVG drawing on the server and in the browser, then compared both inside Chrome. In tradjs 4.6.4, an icon written as <use xlinkHref="#icon"> and drawn in the browser pointed nowhere, so sprite icons came out blank. In 4.6.5, the server and the browser produce identical SVG and MathML: 17 elements, 0 differences. React-style names like strokeWidth now work everywhere, and javascript: links are refused in any spelling.

Tool 3: upgrading a real database file#

A script installs an old published version of sqlite-zod-orm and builds a database with it the way an app would. It then opens the same file with the new version and compares every field, writes to it, and runs live change events between two processes.

  • Files built by versions 3.26.1, 3.34.0 and 4.0.3 all open with no stored row changed and every field identical (12 of 12 checks each).
  • Found: bulk update().where() and delete().where() reported twice the number of rows they changed. Bun's row count also includes rows written by the library's own change log, which is on by default. The right rows were always changed; only the reported number was wrong. Fixed in 4.0.11.

Tool 4: type-checking real apps#

sqlite-zod-orm promises type safety but had no type tests. We added a matrix that pairs each type claim with a run-time check of the same fact. On the old version, 63 of its claims failed. For example:

  • .with('user') was still typed as a function, though the code returns the related row;
  • timestamp fields were missing from row types.

A second tool type-checks real apps against the new types without touching them. It caught one proposed change that would have broken an app's generic helper, so we dropped that change. The final result: 9 real apps, 0 new type errors (4.1.0).

Tool 5: what every write returns#

A matrix ran every write function under every combination of options. The options were:

  • change events on or off;
  • soft deletes on or off;
  • timestamps on or off;
  • hooks on or off;
  • inside a transaction, outside one, or inside one that was rolled back.

Each return value was compared with a direct SQL count. On the old version, 66 of 71 cases failed. In 4.2.0:

  • After-hooks inside a transaction now run after the commit, and never for work that was rolled back.
  • Bulk deletes follow cascade rules like single deletes do.
  • Native upserts run their hooks.
  • Bulk updates set updatedAt.

Tool 6: speed against plain SQL#

This was the first measurement against hand-written bun:sqlite, and it wasn't flattering. Single-row operations ran at 2 to 12% of plain SQL's speed. After caching Zod parsers and per-table plans, 4.2.1 is faster with the same behaviour:

Operation Faster by
update(id) 5.6x
changing a field on a row 5.3x
upsert 3.9x
insert 3.8x
reads and .with() relations 2.2 to 2.5x
bulk update 2.4x

The other apps on the machine made each run's speed swing by about 35%. So we compared each run's library speed to plain SQL's speed in the same run, which cancels that out. The benchmark is now in the repository with a saved baseline, and it fails if any operation gets more than 15% slower.

Releases from this work#

Package Version What changed
tradjs 4.6.4 plain-text responses shown, not downloaded
tradjs 4.6.5 SVG/MathML attributes identical on server and browser
sqlite-zod-orm 4.0.11 correct bulk row counts
sqlite-zod-orm 4.1.0 types match what the code returns
sqlite-zod-orm 4.2.0 hooks after commit, cascading bulk deletes, upsert hooks
sqlite-zod-orm 4.2.1 2 to 6 times faster

Install: bun add tradjs sqlite-zod-orm measure-fn.

What is next#

  • A second speed round for sqlite-zod-orm, guided by the benchmark. Even after 4.2.1, a single update runs at about 15% of plain SQL's speed.
  • A production build check for TradJS is in progress. It builds an app twice and checks three things:
    • both builds are byte-identical;
    • no server secret reaches browser code;
    • assets are cached and compressed correctly. It then serves the result to the same browser tests.
  • The starter app (tradjs init) and the next measure-fn version are finished and waiting for release review.
Gor Berlian

One person, many companies, one coin. Every project I build sends a share of its revenue back to buy $BERLIAN on the open market, and every buy is published with its transaction.

See the page

More from Gor Berlian