# 🤖 Managed AI on provisioned spaces — design

Status: **design agreed 2026-07-14** (decisions below are Paul's). Built so
far: phase 0 (#806 fix + Go invite mint, PRs #864/#865), phase 1 (the usage
pipe, PR #867), phase 2 (attach-bot + the onboarding opt-in, #858). Phases
3–5 are unbuilt. This is the plan for making the
onboarding "enable AI" answer *do* something on self-serve spaces: provision a
sidecar, meter what it spends, bill it honestly, and eventually move the whole
bot trust plane off the serving box.

Related: `docs/bots.md` (trust model), `docs/member-bots.md`,
`docs/cloud-provisioner.md` (the attach-bot gap this closes),
`docs/planet-scale.md` (co-spawn + the sidecar invariant),
`docs/tier-economics.md` ("AI is never bundled"), issues #562 (pricing),
#668 (usage metering pipe), #685 (evict the trust plane), #806 (sidecar
self-provisioning bug).

## The gap today

The onboarding toggles are real gates — `bots_enabled` 403s new-agent creation
(`main.go`) and member-bot charters (`memberbots.go`); `ai_enabled` is a
genuine kill-switch the sidecar honors on every reconnect
(`internal/bot/brain.go`, via `GET /api/bot-config`). But a freshly
provisioned `<sqid>.11ch.at` space has **no sidecar and no model key**, so
answering "yes, enable AI" produces nothing: Eleven never appears. The
provisioner chain (`cloud.go` → `controld.go` → `addInstance`) contains zero
bot hooks.

Meanwhile most of the machinery already exists:

- `lchat add-bot` (`bot.go`) automates invite minting, the `-bot.env` file,
  ownership, and the unit — or defers to co-spawn on hibernating hosts.
- The activator **co-spawns** sidecars with their instance and makes sidecar
  traffic activity-invisible (`internal/bot/sidecarmark.go`), so a managed
  sidecar can't hold a space awake or wake it.
- The storage meter (`meter.go`) is a proven per-space measure → record →
  enforce loop keyed on the cloud ledger.
- Per-bot model choice already ships (`bot_configs.model`, admin UI menu,
  re-read on reconnect); charters carry a per-task override.

## Decisions (Paul, 2026-07-14)

1. **Eleven only** is provisioned for cloud spaces. Admins can add further
   agents through the existing admin "Add an agent" path; on managed spaces
   that path must feed the managed sidecar automatically (see
   *Roster follows the admin* below).
2. **Bedrock is phased**, not first: phases 0–3 ship on the existing direct
   Anthropic key; Bedrock lands with the off-box VPC tier where IAM roles and
   VPC endpoints actually pay off.
3. **Small included allowance funded from a capped pool** — free AI is a
   bounded trial, never an unbounded subsidy.
4. **On-box first, then iterate** — sidecars stay co-located with instances
   (today's trust posture) until the bot-host tier exists.
5. **Bring-your-own-key is an advanced admin option** — paste-a-key first,
   "click to connect" via an OAuth-style flow when a provider supports one.

## Trust model and disclosure

Managed AI means *we* run a machine that holds the bot's private key and can
decrypt exactly the threads the bot is in. That is already the disclosed
exception-by-membership in `docs/bots.md` and already true for the
hand-provisioned fleet — but self-serve opt-in turns it into a consumer-facing
promise, so:

- **AI is opt-out by default.** The setup flow writes `ai_enabled=false`
  unless the admin takes a specific opt-in step. (Deterministic bots — the
  command floor, no model, nothing sent anywhere — can stay default-on.)
- **The opt-in copy discloses plainly**, in the lock-and-key voice: the
  helper's key lives on Eleven's machines, and it can read the chats it's
  invited to — that's what makes it able to help. Membership stays visible in
  the roster.
- The invariants hold throughout: bot code never runs in the server process,
  the sidecar stays a pure outbound client, and **the instance server never
  stores a usable model key** (load-bearing for BYO key, below).

## Phase 0 — zero-touch provisioning that works

Prerequisites for everything else; both PR-sized.

- **Fix #806**: `bot run` must redeem the invite when a session path is set,
  the file is absent, and the matching invite env var is present (option 1 in
  the issue — it also covers the co-spawn path, and it is what makes keys
  *born on the host that runs the bot*, which phase 4 relies on).
- **Mint agent invites in Go.** `add-bot` shells out to
  `agent/mint-invite.mjs`, dragging Node onto any box that provisions. Port
  the mint to the control binary.

## Phase 1 — meter before money (#668)

Measure first, price from data. No billing in this phase.

- Parse the `usage` block in `internal/bot/anthropic.go` (today
  `anthResponse` doesn't even deserialize it). Record all four counters —
  input, output, cache-write, cache-read — they're priced differently.
- Aggregate in the sidecar per (day, model, dimension); flush to a new cloud
  `POST /api/usage` authed by a **per-instance meter token** minted at create
  time (mint/list/revoke pattern copied from the push relay). Ledger:
  `usage(space_id, day, dimension, quantity)` beside `storage_bytes`.
- Add bot-activity dimensions (replies, charter runs) on the same pipe.
- Run silently on the existing sidecar fleet for a couple of weeks; the
  allowance and add-on price in phase 3 are set from this data.

The meter flush doubles as the **budget heartbeat**: the response to
`POST /api/usage` carries the space's current AI budget state, so no new
cloud→instance channel is needed for enforcement (phase 3).

## Phase 2 — the opt-in that provisions (on-box)

- **Onboarding**: the extras step's AI question becomes an explicit opt-in
  chip with the disclosure copy; declining (or skipping) writes
  `ai_enabled=false`. Opting in triggers provisioning *during* setup —
  "Eleven is joining your space…" — and Eleven's welcome DM lands seconds
  later. The same opt-in surfaces in the Admin app for existing spaces (note:
  the Admin mini-app's `setFeatures` bridge is currently dead code — wire it
  up here).
- **Signal path**: instance → cloud over the meter-token channel ("this space
  wants AI") → cloud calls a new controld `POST /instances/{host}/bot`, which
  runs the `add-bot` logic: mint the Eleven invite, stage
  `/data/<host>-bot.env` with the fleet `ANTHROPIC_API_KEY`, chown. On a
  hibernating host that's all it does — co-spawn picks the sidecar up (the
  activator needs one small hook: notice a newly staged `-bot.env` while the
  child is already awake, so opt-in doesn't wait for the next sleep/wake
  cycle).
- **Roster follows the admin**: when an admin adds another agent on a managed
  space, the same signal path appends that identity's invite to the bot env
  and restarts the sidecar; post-#806 it self-provisions. The sidecar's fixed
  two-identity env shape (`LCHAT_INVITE` / `LCHAT_ELEVEN_INVITE`) generalizes
  to a numbered or manifest form as part of this.
- Latency budget: chip-tap to welcome DM in seconds, not sweeps — this is why
  the signal is a push over the meter channel, not a stats-poll.

## Phase 3 — usage accrues to the subscription

- **Stripe Billing Meters** on the existing per-space subscription
  (`billing.go` — the provisioner's Stripe integration, not the in-space
  Connect one). Metered price = provider cost × a published markup, recorded
  in `docs/tier-economics.md` like every other rate. AI becomes the first
  real marginal-cost line in the public books.
- **Included allowance from a capped pool**: quantities under the per-space
  monthly allowance never forward to Stripe; a **fleet-wide monthly AI
  budget** (mirror of the `creates_log` creation-budget circuit breaker)
  bounds aggregate subsidized spend no matter how many free spaces opt in.
  Allowance and pool sizes come from phase 1 data. Cost reality check at
  today's list prices: a typical cached Sonnet reply runs ~1–1.5¢, so ~100
  replies/month ≈ €1.2/month against Family's €4.68/*year* net — the
  allowance is trial-sized by necessity, and "AI is never bundled" stands.
- **Exhaustion behavior**: the meter-flush response flips the space to
  budget-exhausted; the sidecar's existing `AIEnabled` gate grows that state —
  the bot drops to the deterministic command floor and says so once, honestly.
  Per the conduct rule (knowledge.md §20), Eleven describes the upgrade path
  and never posts a payment link.
- **Abuse guards land with billing, not after**: a per-space daily reply
  ceiling independent of budget (`minEvery` floors charter schedules but
  nothing bounds conversational replies), and loop protection stays audited.

## Phase 4 — the bot-host tier: egress-only VPC (#685)

The "hardened off-box sidecar tier" from #550 Part 4 v2; sidecars follow the
commons, not the provider.

- **Sandbox: systemd, not Docker.** The bot unit already carries near-maximal
  hardening (empty capability set, `ProtectSystem=strict`,
  `MemoryDenyWriteExecute` — possible because the sidecar is pure Go — syscall
  filter, `MemoryMax`), and the sidecar runs no untrusted code: member-app
  code executes in the *client's* sandboxed iframe, charters are bounded tool
  loops. Docker would add a daemon and an image pipeline — against the
  one-boring-binary rule — for less isolation than the unit provides. The
  real gap is that **co-spawned** sidecars inherit the router's sandbox;
  moving them to a dedicated host puts each space back under its own hardened
  unit (optionally per-space `DynamicUser`).
- **Host shape**: EC2 (Graviton, instance-role pattern from
  `infra/dev/compute.tf`) in a **private subnet — no public IP, no inbound
  anything**; egress via NAT restricted to 443. The sidecar is already a pure
  outbound client; the `X-Lchat-Sidecar` marker means off-box sidecars can't
  wake or hold instances awake, and charter due-time wakes already work from
  off-box (`POST /api/charter-next-due`). The IaC boundary test holds: tofu
  builds the box and role; the `lchat` binary (a bot-host mode) learns which
  spaces it hosts by outbound-polling the control plane.
- **Keys are born on the bot host.** Provisioning hands the host one-time
  invites; the sidecar self-provisions (#806 fix), so bot private keys never
  transit anywhere. Combined with #685 this evicts bot keys *and* the model
  key from the serving box.
- Lifecycle v1: off-box sidecars run always-on (tens of MB each; one small
  host carries dozens; scale horizontally before getting clever).
- Ops debt that must close before this scales: **#811** — awake bot-sidecar
  instances never pick up auto-updates.

## Phase 5 — the provider seam: Bedrock, local models, catalog

One small interface in `internal/bot` (today's client becomes implementation
#1):

1. **Anthropic direct** — the existing hand-rolled Messages client; stays the
   default for BYO key and self-hosters (`ANTHROPIC_BASE_URL` seam exists).
2. **Bedrock** — for the managed VPC tier. The draw is not model choice:
   it's **IAM-role auth** (no `ANTHROPIC_API_KEY` exists anywhere for managed
   spaces — the shared-key blast radius disappears) and **VPC interface
   endpoints** (model traffic never touches the public internet). Same
   Messages body under SigV4 — `home/sigv4.go` proves the no-dependency
   signer pattern; model ids gain the `anthropic.` prefix. Check eu-west-1 /
   EU cross-region profile availability — EU-resident inference is a real
   selling point given bots see plaintext.
3. **OpenAI-compatible local** — the SLM companion direction: a small model
   in the same private network, called by the sidecar. First workload to
   move: the addressing classifier (hardcoded Haiku today, `brain.go`) —
   highest volume, lowest difficulty; moving it in-network cuts both spend
   and plaintext egress.

**Model management**: per-bot choice already exists; the improvement is a
fleet-served **model catalog** (menu entries + ids) so `admin.js` stops
hardcoding model ids and fleet-wide swaps/deprecations need no client
redeploy. Keep the menu small: default, better, cheaper.

## Bring your own key (advanced admin option)

- **The invariant shapes the design**: compromising the server must not yield
  the model key (CLAUDE.md), so a BYO key is **never persisted in the
  instance DB**. Set flow: admin UI → instance (held in memory only, never
  stored, never logged) → cloud over the meter channel → controld writes it
  into `/data/<host>-bot.env` → sidecar restart. Same at-rest posture as the
  fleet key today; treated like the admin token in transit.
- **BYO spaces don't meter and aren't billed** for AI (#668's rule: their
  key, their cost). The allowance/budget machinery disarms; `ai_enabled`
  still works as the kill switch.
- **Click to connect**: Anthropic offers no third-party OAuth key
  provisioning today, so v1 is paste-a-key (Anthropic key, Anthropic wire
  format). The **connect broker** (`lchat-connect`, which already holds OAuth
  client pairs for GitHub/DNSimple) is the natural seam when a provider
  supports key issuance by consent — OpenRouter's PKCE flow exists today if
  we ever want it, with its own disclosure (a third party would see the
  bot's plaintext); an Anthropic OAuth flow is a watch item.
- Advanced means advanced: buried in the admin console, not in onboarding.
- v2 candidates: per-space Bedrock via cross-account role (Business tier),
  arbitrary OpenAI-compatible base URL + key for self-managed local models.

## Kill switches and guards (summary)

- Per-space: `ai_enabled` (exists, enforced in the sidecar), budget
  exhaustion (phase 3), daily reply ceiling (phase 3).
- Fleet: the subsidy-pool circuit breaker (phase 3); env kill on bot hosts.
- Egress: charter/research URL fetches from our infrastructure need an
  egress policy on the bot host (a prompt-injected charter fetching internal
  endpoints is SSRF from our VPC) — the serving box's unfurl SSRF guard has
  no sidecar equivalent today.
- Terms: one pass over Anthropic/Bedrock commercial terms (resale with
  markup, data-retention posture we advertise) before the price goes public.

## Sequencing

Phase 0 → 1 → 2 ship independently and in order; 3 needs 1's data and 2's
opt-in; 4 and 5 ride the #685 arc and don't block 2–3. Native parity is
mostly free (onboarding/admin render through the webview) but the opt-in flow
gets an explicit iOS verify. `docs/tier-economics.md` gains the AI rate line
when phase 3 prices are set, and its stale "a bot-sidecar space never sleeps"
note gets reconciled with co-spawn (#670) and the observed #811 behavior.