# 🤖 Planet scale — the running system

This is the distillation of issue **#550** ("use an object store as the only
database — planet-scale plan", twelve parts across two working sessions).
Most of what that thread designed has since shipped; this doc describes the
system as it **runs**, with the still-unbuilt parts clearly marked as roadmap.
Where the issue and the code differ, the code wins, and the difference is
noted. #550 stays open only as the historical record of how we got here and
closes when this merges.

## The one idea

The E2EE rule did the hard part before any of this started. Home's server-side
state is opaque client-encrypted ciphertext plus a few kilobytes of public-key
metadata, with zero cross-user queries — so home scales by **removing the
database**: an S3 bucket, used raw, is a strictly better fit than SQLite ever
was. An instance's SQLite file is opaque blobs plus routing metadata the server
already sees — so replicating it to a bucket is threat-model-free, and
instances scale by **removing the idle process**: state parks in S3 as
ciphertext, a process exists only while someone's talking, and creating a
space becomes (nearly) a metadata write. Launch-day load lands on S3's
capacity, not ours.

## What runs today

### Home: the store is a seam, and S3 is implementation #2

`home/store.go` (PR #611) extracts **`vaultStore`**, home's whole persistence
surface, shaped like an object store on purpose: point reads by key,
conditional writes (CAS on an *opaque version string*, with the one universal
sentinel `"0"` = "no blob yet"), keep-last-N blob versions, prefix deletes,
and write-once pointers. SQLite is implementation #1 and stays the zero-config
self-hoster default forever — dual backend is a feature, not migration debt.

`home/s3store.go` (PR #627) is implementation #2: **one bucket, no database**,
laid out exactly as the issue drew it —

```
blobs/<uid>        the vault; S3-native versioning, version = the ETag
users/<uid>        one metadata JSON per user (credentials + devices),
                   every mutation an If-Match CAS loop
creds/<credId>     write-once pointer, body = uid
tokens/<sha256>    write-once pointer, body = uid
```

CAS rides S3 conditional writes (`If-Match` to replace; the `"0"` sentinel
maps to `If-None-Match: *` for create-only). Pointers are never authoritative:
resolving one re-loads `users/<uid>` and verifies the credential/token is
still listed, so the stale pointer a revocation leaves behind denies by
construction. Two open questions from the issue were settled at PR time,
both worth recording:

- **The client is a hand-rolled SigV4 signer** (`home/sigv4.go`), not the AWS
  SDK — the surface home needs is six calls, and home keeps its tiny dependency
  footprint. (The *activator* side, below, does carry `aws-sdk-go-v2`, brought
  in by litestream — the two made opposite calls for good local reasons.)
- **Keep-last-5 is enforced at the interface, not by the bucket**:
  `listBlobVersions` caps at five; the bucket's lifecycle rule prunes
  noncurrent versions asynchronously, days later, as a backstop. And S3 cannot
  distinguish "a version string that was never ours" from a plain mismatch, so
  the SQLite backend's `errBadVersion` (→ 400) is deliberately backend-specific;
  on S3 both read as conflicts (→ 409).

Selection is by env (`home/main.go`): `LCHAT_HOME_STORE` unset or `sqlite` is
the default; `s3` requires `LCHAT_HOME_S3_BUCKET` + `LCHAT_HOME_S3_REGION`
plus AWS credentials in the standard env vars, with optional
`LCHAT_HOME_S3_PREFIX` (test isolation) and `LCHAT_HOME_S3_ENDPOINT`
(path-style, for S3-compatible stores). Misconfiguration fails at boot, never
limps.

The wire contract was already ETag-opaque: PR #612 made the blob `version` a
JSON string on the wire (clients tolerated both) and coarsened link-token
`last_seen` to daily, so an object-store backend never pays a write per
authenticated read.

### Home is stateless

`home/stateless.go` (PR #613) removed the last two things a home process
remembered between requests. Ceremony proofs became **HMAC-signed tokens**
(uid + expiry, 5-minute window) the client carries; in-flight WebAuthn
sessions are **AES-GCM-sealed into the `?session=` query param**. Both keys
derive from one server secret, `LCHAT_HOME_SESSION_KEY` — auth plumbing like
the admin token, it cannot open a byte of any vault. Unset, an ephemeral key
is minted at boot (with a logged warning): single-process self-hosters keep
working with zero config, and ceremonies just don't survive a restart, which
is exactly what the old in-memory map gave them. With the bucket backend and
the stateless ceremonies together, home is a pure function from
(request, bucket) → response — any replica behind a load balancer can finish
what another began.

### Fleet backup: replication as a property of normal operation

`deploy/fleet-backup.sh` (PRs #637, #643) closed the actual gap the plan
started from — instance DBs living on one box with no off-box copy. It runs on
the router box, idempotent, regenerating `/etc/litestream.yml` from what's in
`/data`: a CLI **Litestream** (0.5.14) streams every `/data/*.db` — instances
plus home/push/cloud/registry — to the fleet-backup bucket
(`eleven-prod-fleet-backup-…`, one prefix per database), under the
`lchat-litestream` unit and a scoped IAM user that can touch objects but never
versioning, lifecycle, or the bucket itself. Versioning + a 14-day noncurrent
window is the ransomware posture: an overwrite or delete with the box's own
credentials leaves recoverable versions behind. Continuous replication, not
snapshots — seconds of RPO, point-in-time recovery, and it supersedes the
parked R2-snapshot (`r2d2.sh`) plan entirely. The restore drill is one
command, documented in the script's header; run it against a scratch path,
never over a live file.

One exclusion, added when hibernation went live (PR #658): **hibernating hosts
are skipped** — the activator's embedded Litestream owns their replication (to
the instances bucket, under the lease), and two replicators on one WAL would
fight over checkpointing.

### Instance hibernation, end to end

This is Part 4 v2 of the issue, running. Four layers:

**The lease** (`internal/activator/replica.go`, PR #630). Litestream is
embedded **as a Go library** — pure `modernc.org/sqlite`, no cgo, so the
single-binary property survives. Its S3 lease (`lock.json` acquired via
conditional writes) is the distributed single-writer lock: a second claimant
fails loudly with a typed `*litestream.LeaseExistsError`. This is a
correction the plan earned by testing (recorded on #550): the issue assumed
the lease was CLI config, but **the lease exists only in the library API — the
CLI silently absorbs a second replicator**, orphaning the loser's writes with
no error signal. Embedding it turned that from a footgun into the mechanism.
TTL is 15 m, renewed every 5 m (each renewal is one S3 Tier-1 PUT — at the
original 30 s/10 s cadence the lease heartbeat alone was ~70% of the fleet's
S3 request bill, ~$16/yr per always-awake instance against €5/y plans); a
failed renewal means leadership is lost and the holder stands down. The
failover window follows the TTL: a wedged or hard-killed process is claimable
by a healthy peer within ~15 m of its last renewal — every graceful path
(sleep, deploy, shutdown) releases explicitly and hands over in seconds, so
the window prices only real crashes. Restores run
*under* the held lease (restoring outside it could resurrect a stale copy that
shadows the true holder's newer state).

**The activator** (`activator.go`, PR #641). With `LCHAT_HIBERNATE_BUCKET`
set, the router stops relying on systemd to keep flagged instances alive and
runs them as its **own children, on demand** — "units are for the router,
instances are its children", exactly as the issue put it. The per-host state
machine is asleep → waking → awake → stopping:

- **Wake** (a request for a host with no live process): claim the per-host
  lease at `<prefix>/<host>/lock.json`; restore the DB from S3 only if the
  local copy is gone (a present local DB is the warm cache — on one box, under
  the lease, it can only be our own last checkpoint); spawn the instance
  (exec of the same versioned `live/<host>` binary its systemd unit would run,
  same env file); start replication under the lease. Concurrent requests
  coalesce on one wake (singleflight); lease contention and wake timeouts are
  503 + `Retry-After` — "sleeping spaces are slow to wake", never a 5xx
  cascade.
- **Sleep** (idle past `LCHAT_HIBERNATE_IDLE`, default 15 m — idleness is
  in-flight-count *and* time, so one open WebSocket holds an instance awake):
  probe the next due time, SIGTERM the child (its own graceful drain is the
  final write), final checkpoint to S3, release the lease, forget the entry.
  The local DB stays as the warm cache.
- **Failure is the wake path**: a dead child or a lost lease stands the entry
  down, and the next request re-wakes it. S3 blips leave warm instances
  serving (local SQLite is authoritative; replication queues and drains) —
  only cold wakes stall.

**Due-time wakes** (PR #653). Scheduled deliveries (future messages,
reminders) fire on time while their instance sleeps: at sleep the stop path
asks the instance for its earliest pending delivery (`GET /api/next-due`,
admin-gated, over the socket) and records it in the registry
(`routes.next_due` — persisted, so it survives router restarts); the sweeper
wakes any sleeping host whose time has arrived, no client request needed.
Worst-case lateness ≈ one sweep interval (≤30 s) + spawn + one scheduler tick
(≤5 s). One honest gap the issue glossed: **member-bot charter schedules run
in the sidecar** (`docs/member-bots.md`), so they didn't wake a sleeping
instance — the index covered server-side schedules only. Closed by
**charter-wake registration**: after every schedule recompute the charter host
reports its earliest next run to the instance (`POST /api/charter-next-due`,
bot-token-gated — one bare timestamp; the charters stay ciphertext), the
instance folds it into `GET /api/next-due`, and the same index wakes for it —
zero activator changes. A stale report from a dead sidecar is ignored once
past (it changes only when the sidecar re-reports, so honoring it would re-wake
after every sleep): at most one wasted wake, never a loop — and a sidecar that
can't come up couldn't run the charter anyway. The due-wake's other half lives
in the sidecar: a co-spawned host that loses the delivery race (the instance's
scheduler fires the tick before the host's cursor baseline) treats a live
charter with no pending alarm at connect as owing a run now, so the woken
charter actually runs instead of skipping to its next occurrence.

**Sidecar co-spawn** (PR #670). A host with a bot sidecar staged
(`/data/<host>-bot.env` — the same file `lchat-bot@.service` reads) gets its
`bot run` child spawned right after the instance turns healthy and SIGTERMed
first at sleep: the sidecar exists exactly while its instance is awake, so its
lifecycle solves itself — the issue's "co-located over loopback" default,
built. The subtlety that makes it work: the sidecar dials its instance's
public URL back through this very router, so it stamps every request
(`X-Lchat-Sidecar`, `internal/bot/sidecarmark.go`) and the activator makes
that traffic **activity-invisible** — no idle-clock credit while awake, a 503
(never a wake) while asleep. Without that, the pair could never sleep and an
orphaned sidecar's reconnect loop would wake its instance right back, forever.
Spoofing the marker only downgrades the sender's own requests. Supervision is
sweep-driven Restart=always with capped exponential backoff; a sidecar that
won't start never fails the wake — members before bots.

### The flip, per host, and creates that are born asleep

Opt-in is **per host** (`routes.hibernate` in the registry, PR #659), which is
what makes a production canary safe: `LCHAT_HIBERNATE_BUCKET` ships fleet-wide
as a no-op, then

```sh
lchat hibernate <host> --on     # stop+disable its unit; the router wakes it per request
lchat hibernate <host> --off    # activator releases (checkpoint + lease back), unit restarts
lchat instances                 # HIBERNATE column shows who's activator-managed
```

migrates exactly one instance, and back. The flip's ordering is deliberate
(flag → stop unit → reload router → warm request) so the host is never left
unserveable, and `--off` waits for the child to actually release the socket
before the unit starts (an instance refuses to clobber a live listener). A
staged bot sidecar hands over both ways in the same flip.

One lesson found live, now baked in: **env-file ownership.** Systemd reads a
root-written `EnvironmentFile` as root; the activator reads it as the service
user — the canary's first wake died on permission denied. `add-instance` and
both flips now chown the env (and bot session dirs) to match the data dir.

Creation splits into **register** (the row: host, minted-or-imported token,
placement fields — pure metadata, no capacity) and **materialize** (env file,
staged binary, unit, health check, router reload, setup invite) — PR #625,
exactly the seam the issue asked for. A `--hibernate` create (and, behind
`LCHAT_CONTROL_HIBERNATE_CREATES=1` on the control daemon, **every
provisioner-created space** — PR #663) skips only the systemd half: no unit,
no resident process, no DB — the first wake finds no replica in S3 and boots
fresh. Capacity for hibernating rows is a row ceiling
(`LCHAT_MAX_REGISTERED`, default 10× `LCHAT_MAX_INSTANCES`), not a process
cap. Honest delta from the issue's "create = a registry row + an S3 prefix,
microseconds, no box contacted": today's hibernating create still writes the
env file, stages the binary, reloads the router — and its readiness probe *is*
the first wake, because minting the setup invite needs a running instance to
pre-seed the admin user. Cheap, but not yet free; the fully-deferred
materialization remains roadmap.

The **reaper** got the property Part 5 promised — reclaiming never-claimed
spaces **without waking anything**. A hibernating space's stats
(`setup_complete` / members / `has_content`) are read straight off its local
DB file, read-only — wake-free because the warm cache is the newest truth on
one box. The case the code had to add: a hibernating space that has *never
woken* has no DB at all, and before the fix that read as a stats error — and
a stats error is a skip-never-reap, so register-only spaces would have been
immortal. The stats handler now answers those **definitionally from the
registry row** (0 members, never set up, no content); a missing DB on a
*non*-hibernating row stays a 404, meaning a failed create, skipped as before.

### The mesh: multiple boxes, one owner per instance

The multi-box mechanism from Part 4 v2 shipped as PR #678, gated on
`LCHAT_MESH_SELF` (a peer-reachable base URL); absent means zero mesh
codepaths, and the current single-box deployment runs exactly as before.

- **Membership is S3 heartbeats**, not gossip: each box PUTs one object per
  60 seconds under the hibernation prefix's `mesh/members/`, and the object
  *key* is the record (base64url of the peer URL) so a single LIST yields the
  whole membership. Liveness reads S3's own `LastModified`, so a box with a
  wrong clock can't poison the staleness judgment. Stale after 3 m (three
  missed beats — well inside the lease expiry that actually gates takeover).
- **Rendezvous (HRW) hashing is the placement hint** — `owner(host)` is a
  pure function of the live member set; adding a box moves only the hosts
  that land on it. **The lease stays the truth**: a box that wrongly believes
  it owns a host loses the CAS race loudly and proxies instead, so transient
  membership disagreement structurally cannot split-brain.
- **Wrong-box requests reverse-proxy to the owner**, TLS verified against the
  instance hostname with the connection pinned to the peer's address (no
  `InsecureSkipVerify` anywhere); a one-hop marker turns disagreement loops
  into a wrong-box wake the lease resolves.
- **Drain deletes the heartbeat first**, then checkpoints and releases as
  usual — so deploys, scale-down, and box death are the same code path. The
  kill-a-box property is a live test, both ways: a drained owner's instances
  are serving on the survivor ~2 seconds later; a SIGKILLed owner's once its
  leases expire (the TTL — proven live at the original 30 s TTL, since raised
  to 15 m for S3 request cost), data intact in both.

Hardening that same week (PR #681, both lessons from the first real
production flip): flipping a host refuses while the box-wide backup
replicator still tracks it — two litestream processes on one DB share a
derived meta directory and corrupt each other — and the lease release runs
under its own budget so a wedged final checkpoint can't strand the lease
until TTL.

### The seams cut for what's next

Small PRs that exist so later phases are config, not surgery:

- **Attachments** sit behind a `blobStore` interface (`blobstore.go`, PR
  #636): write-once immutable bytes, no CAS or versioning needed — sized and
  thread-tagged in SQLite, ciphertext in the store. Disk is implementation #1;
  the object-store implementation (R2 for zero-egress media, per the issue) is
  roadmap.
- **The cert cache** is behind `autocert.Cache` (`certcache.go`, PR #661):
  `LCHAT_CERT_CACHE` unset/`dir` is today's `/data/acme` layout; `s3` is
  reserved for the pooled-router future and **refuses to start** until it
  exists — failing loudly beats minting certs into a local dir the rest of a
  pool can't see.
- **Placement fields**: the registry row carries `home_region` and `provider`
  (PR #625), single-valued and inert today — the geo (Part 7) and co-op
  (Part 10) seams, as schema.
- **Billing rails** (PR #628): the cloud ledger's spaces table holds exactly
  Part 11's shape — `stripe_customer_id` (an opaque pointer into Stripe) and
  an `entitlement` flag — no email, name, or card detail anywhere in our
  stores (identity lives in Stripe, where the operator can see it). Dormant
  until the Stripe env is set.
- **The release-feed mirror** (PR #662, `deploy/release-server.sh` +
  `updater.go`): the fleet self-updates from GitHub, whose CDN has served
  stale manifests — the Part 9 SPOF. Publishing now mirrors the signed feed
  and binaries to S3 (`LCHAT_RELEASE_MIRROR_BUCKET`), and updaters fall back
  (`LCHAT_RELEASE_MIRROR_URL`) when the primary fails *or verifies but is
  stale* against the newest feed the box ever accepted. Signature verification
  is identical on both paths: the mirror adds availability, never trust.
- **The secrets inventory** (`docs/secrets.md`, PR #639): every server-side
  secret, blast radius, and rotation story — Part 12's operability item 11.
- **Infrastructure as code** (`infra/`, PRs #609, #637, #638, #658): OpenTofu,
  managing the static skeleton only — dev and prod as separate states. Dev
  holds the proof-build buckets and a `t4g.small` box; prod holds the
  fleet-backup and hibernation buckets (versioning, lifecycle, Block Public
  Access, scoped IAM users whose keys never enter tofu state) plus CloudTrail.
  The boundary test from the issue holds and is written into `infra/README.md`:
  **if the IaC needs to know an instance hostname, the design is wrong.**

## Decision log

Part 12's table, one line per settled call, updated with what shipped. "Held"
means the decision stands with nothing built against it yet.

| # | decision | chosen | status |
|---|---|---|---|
| 1 | home store | S3 as the only database (over R2-sole / DynamoDB) | **Shipped** #611/#627 — dual backend, SQLite stays the self-hoster default; minimal SigV4 client, not the SDK |
| 2 | home runtime | stateless Go binary, Lambda-able later (over a Workers rewrite) | **Shipped** #613 — statelessness done; Lambda step not taken |
| 3 | multi-region writes | single write region + CRR warm standby (over active-active, which breaks per-region CAS) | Held — single region today, no CRR yet |
| 4 | metadata at rest | sealed bucket: HMAC(pepper) keys + pepper-encrypted metadata | **Not built** — object keys are plain uids; bucket stays private (Block Public Access) |
| 5 | public bucket | design as if public; the flip is its own reviewed decision (takedown process prerequisite) | Held — posture unchanged |
| 6 | instance runtime | hibernating processes over S3: local SQLite + Litestream, lease = single-writer lock (over Lambda / S3-as-only-DB / cells / renting DO-Fly) | **Shipped** #630/#641 — with the correction: the lease is Go-library-only; the CLI silently absorbs a second writer, so the activator embeds litestream |
| 7 | placement | rendezvous hash as hint, S3 lease as truth (over a routing table) | **Shipped** (#678) — S3-heartbeat membership + HRW + proxy-to-owner; kill-a-box passes live both ways. **Live in production 2026-07-13** across two boxes: domain-scoped placement (#820), automatic route replication + `mesh-seed` (#821), and fleet-wide status aggregation partitioned by ownership (#822/#849/#851/#852) |
| 8 | instance data | one SQLite per space, forever (shared DB tier is the anti-goal) | Held, load-bearing |
| 9 | attachments | blob interface → R2 from day one | Seam **shipped** #636; disk impl only, R2 roadmap |
| 10 | provisioning | create = register only; materialize at first wake; controld dissolves | Split **shipped** #625, hibernate-by-default creates #663; full deferral + controld dissolution roadmap |
| 11 | bootstrap | thin golden AMI + `lchat bootstrap`; release channel owns versions | `bootstrap` + channels predate #550; AMI roadmap |
| 12 | migration posture | dual plane — Hetzner as-a-self-hoster + Eleven Cloud on AWS; origin-preserving moves | Held — production is the Hetzner plane; AWS holds buckets + a dev box |
| 13 | custom domains | CNAME → `<sqid>.11ch.at`, per-host autocert, cert cache → S3 | Cache seam **shipped** #661 (`s3` reserved, refuses); rest roadmap |
| 14 | IaC | OpenTofu, static skeleton only (over Terraform/Pulumi/CFN) | **Shipped** #609/#637/#638/#658 |
| 15 | capacity | ASG target-tracking on warm-set RAM; one shared drain path; Spot slice | Roadmap |
| 16 | geography | per-space `home_region`; relocate while hibernated | Field **shipped** #625, single-valued and inert; behavior roadmap |
| 17 | push relay | small pool of long-lived processes; self-serve relay tokens | Relay predates #550; self-serve registration roadmap |
| 18 | co-op | commons/provider seam; delegation unit = the space; ciphertext escrow | `provider` field **shipped** #625, inert; rest roadmap |
| 19 | billing | Stripe holds all customer data; our record = opaque pointer + entitlement flag; public ledger | Rails **shipped** #628 (dormant until env set); public ledger roadmap (`docs/tier-economics.md` is the cost record) |

Corrections the build taught us, beyond the table: the lease-is-library-only
finding (#6 above); env-file ownership must follow the reader (root-written
files, service-user activator — a wake died on it live); a never-woken
hibernating space has no DB, so reaper stats answer from the registry row or
register-only spaces become immortal; keep-last-N on S3 is an interface cap
with the lifecycle as backstop, not a bucket guarantee; `errBadVersion` is a
SQLite-only nicety S3 cannot express; and the first live mesh flip (#7,
2026-07-13) taught that a replicated route is a registry row only — it never
stages a binary or creates a `live` symlink, and the sandboxed router
(`ProtectSystem=strict`) cannot do that itself, so a replicated route's first
wake 503ed outright until the wake path was rerouted through the unsandboxed
control daemon (#844) instead of writing `/opt/lchat` directly.

## The invariants — the review checklist

Part 12's twelve lines, refined where building sharpened them. Every future
PR that touches this territory gets checked against these:

1. The server — and any future *provider* — only ever holds ciphertext plus
   auth plumbing; never a usable key. (The root rule; everything below is it,
   applied.)
2. No cross-user state in home — the zero-join property *is* the scalability.
3. Never consolidate spaces into a shared DB tier — per-space isolation is the
   sharding, security, and migration story at once.
4. DB migrations are additive-only — this is what makes wake-onto-new-binary
   and channel-following safe.
5. The IaC layer never learns an instance hostname.
6. Wire contracts stay backend-opaque — clients never learn (or depend on) the
   storage backend; versions are opaque strings, `"0"` the one sentinel.
7. The wildcard TLS private key never crosses a trust domain.
8. A space's hostname survives every migration — re-keying is never the cost
   of moving (region, plane, or provider).
9. The bot sidecar remains a pure outbound client and bot code never runs in
   the server process — co-spawn makes it a router *child*, but still a
   separate process whose traffic is marked, activity-invisible, and never a
   wake source.
10. No cross-provider protocol, ever — spaces are islands; if a feature seems
    to need one, the design is wrong.
11. The 18 KiB vault pad/cap and the no-per-user-analytics / no-email absences
    move only by deliberate decision.
12. No PII in our stores, ever — customer identity lives in Stripe; we hold
    opaque pointers and entitlement flags. Income publishes aggregated;
    expenses and disbursements publish itemized.

## Roadmap — designed, not built

Everything below is settled design from #550 with no code behind it (or only
a seam). Build triggers are events, not dates.

- **Multi-region / `home_region`** — one write region per space, wakes
  deterministic, relocation *while hibernated* (copy the prefix, flip the
  field, next wake claims the lease elsewhere). The field exists; nothing
  keys on it. Trigger: member-latency data from a real non-EU/US cohort. The
  full design that builds on this — region as a *mutable property* (not encoded
  in the address), stable-address relocation, and multi-edge replication, with
  the append-only/CRDT reasoning that makes it tractable — is
  **`docs/multi-edge.md`** (agreed direction, 2026-07-18).
- **The co-op / provider seam** — `provider` as a real interface with
  ourselves as provider #1, the Hetzner box formalized as #2, escrow replicas
  in storage the provider doesn't control. Federation without a federation
  protocol.
- **Lambda-able home / autoscaling runtime** — statelessness (the
  prerequisite) is done; the same binary on an autoscaling platform is the
  next rung, an edge rewrite deliberately skipped.
- **Sealed bucket, then the public-bucket decision** — pepper-keyed object
  names and pepper-encrypted metadata first (hardens the private design too);
  flipping public-read stays its own reviewed decision with a written
  takedown process as prerequisite.
- **Attachments to an object store** — the `blobStore` seam is cut; R2 for
  zero-egress media is the intended implementation #2.
- **S3 cert cache** — `LCHAT_CERT_CACHE=s3` is reserved and refusing; needed
  the day two boxes share issuance.
- **CRR + second-provider mirror** — the region checkbox and the Ente-style
  read-only mirror; everything is ciphertext, so copies are threat-model-free.
- **The AWS front plane** — NLB (TCP passthrough), ASG on warm-set RAM, Spot
  slice, golden AMI: Part 7's ~$35/mo shape. The account, state buckets, and
  IAM posture exist; the compute plane doesn't. Trigger: warm-RAM pressure or
  a launch date.
- **Observability gauges** — wake latency, lease conflicts, replication lag
  are loud log lines today, not metrics; Part 9's E2EE-safe metrics discipline
  is still owed.
- **The public ledger** — Part 11's append-only books repo. Partially
  shipped: the public *status dashboards* exist (fleet state + aggregated
  Stripe income, published from the box every 5 minutes — `docs/status.md`);
  the itemized expense/disbursement ledger repo is still to come.

## The historical record

The full twelve-part design conversation — including the superseded "cells"
Part 4, the cost math, the rejected alternatives in their original words, and
the lease experiment writeup — is issue
[#550](https://github.com/elevenmessenger/messenger/issues/550). Related
docs: `docs/instance-router.md` (the router this grew inside),
`docs/cloud-provisioner.md` (create-a-space), `docs/server-releases.md` (the
signed feed + mirror), `docs/secrets.md` (the key inventory),
`infra/README.md` (the IaC skeleton), `docs/bots.md` / `docs/member-bots.md`
(the sidecar model co-spawn inherits).

One sentence, carried over intact because it survived contact with the build:
**the E2EE rule did the hard part everywhere — home scales by removing the
database, instances scale by removing the idle process, the fleet scales by
removing the operator, the co-op scales by removing us — and the books
balance in public.**