# 🤖 TodoMVC bakeoff on Carloku — design

Status: draft for Paul's approval, 2026-10-23. Reviewed by Astra (builder check).

## Why

CARLOS apps don't have to be written in Go, and the CARLOS Platform runs
any binary that keeps its process contract. This bakeoff shows that with
working apps. It's driven by curiosity and acts as a showcase. It isn't
meant to pick a winner or feed a decision.

We build the same app, TodoMVC, in nine language setups. Each one renders
HTML on the server and layers JavaScript on top, and all of them are
deployed to the Carloku **bakeoff** account (`bfj`). A report page on the
same account compares them.

## Goals

- Nine working TodoMVC apps, live on Carloku, one per setup:
  1. Go, without Rastrillo
  2. Go, with Rastrillo
  3. Rust
  4. Zig
  5. OCaml
  6. Haskell
  7. Swift
  8. Nim (replaced Mojo, Paul's call 2026-10-24: Mojo can't make a
     single-file binary; see `docs/toolchains.md`)
  9. Spinel (a Ruby compiler that produces native binaries)
- Every app passes the same behaviour tests with JavaScript on and with
  it off.
- A report page that puts honest numbers side by side: what each build
  cost in tokens, how readable the code is, how fast it responds, how it
  holds up under load, and the other measures listed below.

## Non-goals

- Choosing a "best" language. The report presents numbers and doesn't
  crown anyone.
- Tuning one app beyond the others after the build. Every app gets the
  same build process and the same review-and-fix round, and nothing more.
- Rich features beyond TodoMVC: no sign-in, no sharing, no sync between
  devices.

## Anti-goals (failure even if everything technically works)

- **A report that looks rigged.** One app gets a different UI, a different
  database, a bigger instance, more tuning, extra review rounds, or quietly
  drops part of the spec.
- **An app missing from the report without comment.** If a setup can't be
  built or can't run on Carloku, the report says so plainly and says why.
- **Working around Carloku.** If the platform lacks something an app needs
  (for example, a binary that needs files beside it), we file a request
  against the platform and report the app as blocked on it. We don't
  build workarounds. (Paul's standing rule.)

## Decided with Paul (2026-10-23)

- **One shared UI.** Every app uses TodoMVC's standard markup and CSS
  (`tastejs/todomvc-app-template`, `todomvc-app-css`). For the bakeoff
  this overrides the CARLOS skill's default that Rastrillo apps use
  Rastrillo's own UI components. The Go + Rastrillo app uses the TodoMVC
  CSS too.
- **Server-rendered with progressive enhancement.** No client-side app
  framework. The page is HTML from the server and still works with
  JavaScript turned off.
- **Each agent chooses its own stack.** Web server, router, templating,
  SQLite binding and so on are each agent's own best recommendation for
  its language.
- **The builder is Opus 5.5 for every app.** Astra reviews every app
  through the codex CLI, and the same Opus agent fixes what the review
  raises.
- **Spinel is already supported on Carloku.** The Spinel builder starts
  from lpay's working setup (`paulca/lpay`, directory `spinel/`): tep
  (a Sinatra-style layer that drives Spinel), the `tep-carlos.patch` for
  the socket contract, and the zig-based cross-build to a static
  aarch64-musl binary in `BUILD-arm64.md`. Mojo is expected to work if it
  compiles to a single binary.
- **Arm64 builds:** try docs-t1 first (cross-compiling or an emulated
  arm64 container), and fall back to tito-studio over Tailscale.
- **After the one fix round:** remaining correctness gaps ship as they
  are, and the report shows them. Security findings get a second fix
  round, counted separately, because the apps are public.
- **Downloads approved:** GHC (via ghcup) and Mojo (via pixi) may be
  installed under `~` on docs-t1.
- **Everything lives on the bakeoff account** (`bfj`): nine apps and the
  report page.

## What every app must do (the shared brief)

Each builder gets the same written brief. It contains:

### Behaviour

- The TodoMVC app spec (`tastejs/todomvc/app-spec.md`), with these changes
  because the server holds the data:
  - **Persistence:** in SQLite on the server, not in `localStorage`.
  - **Routes:** real paths `/`, `/active` and `/completed`, not `#/`
    fragments. The selected filter survives a reload because it's in the
    URL.
  - Editing state isn't persisted (same as the spec).
- **With JavaScript off,** everything works through plain forms and links:
  add, toggle, toggle all, edit, delete, clear completed, and the three
  filters. Each action is a POST that redirects back to the current
  filter (post/redirect/get). Editing without JS uses a link to an edit
  view of that item, such as `/?edit=<id>`, rendered with the edit field
  open.
- **With JavaScript on,** the page behaves like the reference TodoMVC:
  double-click to edit, Enter saves, Escape cancels, blur saves, an empty
  title deletes, toggles happen without a full page reload, and the
  counter and the "Clear completed" button update in place. The JS sends
  the same POSTs and swaps in server-rendered HTML or applies a small
  JSON reply (the builder chooses). There's no build step and no
  framework. Plain ES modules are allowed, and so is vendored VanJS (the
  one reactive library CARLOS sanctions).
- **Who owns a list:** the whole app is one instance. Each visitor gets
  their own list through an anonymous, random, `HttpOnly`, `SameSite=Lax`
  cookie set on the first visit. There's no sign-in. Forms are protected
  against cross-site posts (by SameSite plus an Origin check, or a CSRF
  token; the builder chooses).

### Platform contract

- One binary for `linux-arm64` (the platform default; `carlos deploy
  --target`), started as `<bin> --socket <path> --db <path>`. It listens
  on the unix socket, has no `$PORT`, and serves `GET /healthz` and
  `GET /api/version`.
- **Linked statically, or with only libc.** Instances run sandboxed and
  see only their own build, so the builder must not assume any shared
  library exists on the box. `carlos vet --boot` (a local check that the
  binary starts and answers `/healthz`) must pass before the first deploy.
- Reads `CARLOS_HOST` when it needs its own hostname. Never parses the
  socket path.
- Uses SQLite in WAL mode. Migrations only ever add.
- The app starts quickly after hibernation (sleeping when idle) and shuts
  down cleanly on SIGTERM.
- Configures the build probe (`carlos apps create --build-probe
  /api/version`) so `carlos deploy` confirms that the new build is what's
  actually serving.

### Repo shape for each app

- `apps/<slug>/` in this bakeoff repo. The slugs are `go`, `go-rastrillo`,
  `rust`, `zig`, `ocaml`, `haskell`, `swift`, `mojo` and `spinel`.
- `AGENTS.md` first. It holds the one rule and the tenancy declaration:
  one instance, anonymous lists by cookie, trust class C (plaintext todo
  text, chosen and written down).
- A `README.md` with how to build, run locally, test and deploy. One
  command for each (a `Makefile` or a script).
- The app's own tests, whatever is normal for its language.
- The Carloku app name is `todo-<slug>`, e.g. `todo-rust`.

### What the builder may not do

- Change the markup or the CSS beyond adding hooks the JS needs, such as
  `data-` attributes or wrapping each toggle in a `<form>`. The rendered
  page must look identical to the reference.
- Add caching layers, a CDN or edge tricks that the other apps don't
  have. Each app may set ordinary HTTP cache headers on its static
  assets.
- Spend effort on performance tuning beyond what an experienced developer
  would write the first time. "Write it the way you'd normally write it
  well" is the instruction.

## How the apps get built

1. **Shared tooling first, built by the orchestrator (this session)**
   before any builder starts, so every app is judged by the same tools:
   - The brief, as one file every builder reads.
   - A **behaviour test suite** adapted from TodoMVC's Cypress tests
     (`tastejs/todomvc/cypress`). It runs against any URL, once with JS
     on and once with JS off. The JS-off run uses forms only and skips
     the parts that are JS-only, such as double-click editing (replaced
     by the edit link). Builders run it locally against their own app.
   - A **load test script** and a **metrics collector** (see "Metrics").
2. **Toolchains installed** per user, under `~` (`rustup`, Zig tarball,
   `opam`, `ghcup`, Swift's static Linux SDK, Mojo through `pixi`,
   Spinel built from source). GHC and Mojo each take several GB, so we
   ask Paul before downloading them (machine rule).
3. **Nine Opus 5.5 builders run in parallel**, each in its own headless
   `claude` session with JSON output so token use is recorded. Each
   starts from a fresh context with the same brief and the CARLOS skills
   (`carlosframework/skills`), plus Rastrillo's own `SKILL.md` for
   Go + Rastrillo. Each builds, tests locally, then deploys to `bfj` with
   `carlos deploy` and checks it on the live URL.
4. **Astra reviews each app** through `codex exec` against the brief, the
   spec and the code. The review is limited to correctness, spec
   compliance and security. It doesn't ask for performance tuning.
5. **One fix round.** The same Opus builder session fixes what Astra
   raised, then redeploys. Only one round, for every app. The one
   exception is security: if Astra still finds a security problem after
   that round, there's a second round for security fixes only, with its
   tokens reported separately.
6. **Blind readability scoring by Fable.** Each app's code is shown with
   the language named but nothing else identifying, and scored against a
   fixed rubric. Astra doesn't score readability because it already
   reviewed the code.
7. **Measurement and report** (below).

All local servers bind to `127.0.0.1`. Scratch space goes in `$TMPDIR`
and is removed at the end.

## Metrics

All of them are collected by scripts in this repo and can be re-run.

| Measure | How |
|---|---|
| Tokens to build | From each builder's session log: input, output, cache read and cache write, plus turns and wall-clock time. The first build and the fix round are reported separately, and so is the Astra review. |
| Correctness | Behaviour tests passed, JS on and JS off, against the live URL. Spec gaps found by review. |
| Readability | Fable's rubric score (naming, structure, how easy it is to follow one request end to end, error handling, comments), with a short written justification. |
| Size of the code | Lines of code split three ways: server code, templates, and JS. Also the number of third-party dependencies. |
| Build | Three build times for the arm64 binary that ships: **cold** (the app and every third-party library compiled from already-downloaded sources; compiler and standard library as shipped), **warm** (libraries already compiled, app rebuilt from clean), and **one-line change** (rebuild after a trivial edit in the request-handling code). Plus binary size. Paul asked for cold and warm, 2026-10-23. Measured on docs-t1 where the app cross-compiles there; otherwise on the machine that built it, with the machine named beside the number and kept out of the ranking. |
| Response time | Time to the first byte for `GET /` with an empty list and with 50 items, from docs-t1, median of repeated runs. |
| Wake time | Time to the first byte on the first request after the instance has gone to sleep. |
| Under load | A ramp for reads (`GET /` on a 50-item list) and for writes (toggling an item): 1, 2, 4 … 1,024 concurrent connections, 10 s per step. Reports peak requests per second, latency at every step, and **where it slows down**: the most connections at which doubling the load still raised throughput by 10% or more. A ramp stops early after two flat doublings or more than 1% errors. Paul asked for this, 2026-10-23, replacing fixed loads at 1, 16 and 64. |
| Memory | Peak and idle memory from the platform, if it exposes them. If it doesn't, we file a request against the platform rather than measure inside the app. |
| Page weight | Bytes of HTML and JS sent for `GET /`. |

Load tests run from docs-t1 over the public internet, so the numbers
include network time. That's the same for every app, and the report says
so. Every app runs on the same box class and instance size, and we run
the apps one after another, never at the same time.

## The report page

- A static site on `bfj` (`carlos deploy --kind static`), named
  `bakeoff`.
- One page with a summary table of every measure for every app, and a
  chart per performance measure. Each app gets a short section covering
  its stack, notable choices, Astra's main findings, Fable's readability
  notes, a link to the live app and a link to its source.
- A "How we measured" section and a "What this does and doesn't show"
  section, including the network caveat, Spinel's single-threaded server
  and any app that's blocked.
- Built from the metrics files by a script, so a re-run regenerates it.
- Report copy goes through the copy-review gate before it ships.

## Known risks

- **Everything targets `linux-arm64`, and docs-t1 is x86-64.** Rust, Zig,
  Go and Swift (static Linux SDK) cross-compile well, and so does Spinel
  (lpay's zig recipe). OCaml, Haskell and Mojo are harder. Each tries
  docs-t1 first, by cross-compiling or in an emulated arm64 container,
  and falls back to tito-studio. If neither works within the effort
  limit, that app is reported as blocked.
- **Mojo** may need runtime files beside the binary. The binary kind takes
  a single file. If Mojo can't produce one, we file a platform request and
  report Mojo as blocked.
- **Spinel** has no threads. lpay's notes flag its aarch64 fiber context
  switch as untested at runtime. Whatever concurrency model the Spinel
  server ends up with, the report states it next to the load numbers.
- **Load testing shared boxes** can affect other people's apps on the same
  box. We keep load runs short, one app at a time.

## Effort limit

Every builder gets the same limit. If an app still doesn't have a local
build passing the behaviour tests after **4 hours of wall-clock time**,
the builder stops and writes down where it got stuck, and the app is
reported as blocked. All tokens a builder spends are reported, including
for blocked apps and time spent getting toolchains working.

## Left to the builders

- Each app's libraries, framework, server, templating and SQLite binding.
- How the JS talks to the server (HTML fragments or small JSON).
- Internal code layout, within the repo shape above.

## Calls made by the orchestrator, not by Paul

- Real paths instead of `#/` routes, and per-visitor lists by anonymous
  cookie.
- Fable for blind readability scoring, separate from Astra's review.
- One review-and-fix round per app.
- All apps in one repo under `apps/<slug>/`.
- App names `todo-<slug>`, and the report app named `bakeoff`.
- The shared test suite and load test are written once by the
  orchestrator, not by each builder.
- The effort limit: 4 hours of wall-clock time per builder.
- Build time for apps that can't cross-compile on docs-t1 is measured
  where they're built, with the machine named, and isn't ranked against
  the others.