# Fleet architecture diagram — `/fleet/architecture`

**Status:** approved in conversation 2026-08-19 (Paul).
**Scope:** one new operator page on the console, one new provider-neutral
infrastructure-inventory layer behind it, and the IAM grant that lets a
deployment see its own cloud estate.

## 1. Why

The console can already answer every question about the platform's own
model of itself — which boxes report, what they run, what they cost under
the active scenario — and cannot answer a single question about the
machine the model sits on. Nothing in CARLOS knows that a box is an EC2
instance, that a hostname resolves through an Elastic IP, that six
buckets exist, or that an auto-recover alarm is armed. That knowledge
lives in an OpenTofu checkout in another repo and in the AWS console,
which is exactly where CLAUDE.md says day-to-day operation must NOT go.

The result is that no page anywhere answers "what is actually stood up,
and how does a request get from DNS to a process?". This spec builds that
page: DNS at the entry, routers, boxes with their size, cost, apps and
instance counts, and every other resource the deployment's cloud account
is carrying — including resources CARLOS itself knows nothing about,
which are precisely the ones worth seeing.

Four rulings shape it (Paul, 2026-08-19):

- **Live inventory plus CARLOS overlay.** Not a hand-drawn diagram, not a
  picture assembled only from what CARLOS already models. The cloud
  estate is discovered, and the platform's own knowledge is layered onto
  it.
- **Whichever account the console runs in.** No cross-account roles. The
  flagship console draws the flagship estate; tito is a separate deploy
  and gets its own version of this page by running the same code, as any
  third-party hoster would.
- **Rate card prices the boxes.** The discovered instance shape is shown
  beside the price and used to flag drift, never to invent a second cost
  model that can disagree with `/fleet/economics`.
- **Do not couple the page to AWS.** AWS is where we are, not what the
  page is about. One provider interface, one implementation, no plugin
  system built speculatively.

## 2. Shape of the answer

```
GET /fleet/architecture      operator-only page: graph + resource cards
POST /fleet/architecture/refresh   run the inventory pass once, now

infra/inventory.json         bucket object, written hourly by the pass
internal/infraview           provider-neutral inventory types + Provider
internal/awsinv              the AWS implementation (only AWS SDK importer)
internal/console/archgraph.go   join + layout, emits SVG
```

## 3. The provider seam

`internal/infraview` defines what an inventory IS, in terms no cloud owns:

```go
type Resource struct {
	Provider   string            // "aws" | "hetzner" | "self-hosted"
	Kind       string            // see the kind vocabulary below
	NativeType string            // "ec2:instance", "hetzner:server" — display only
	ID         string            // provider-stable identity (instance id, ARN, bucket name)
	Name       string            // human name (Name tag, bucket name, zone name)
	Region     string            // "" for a global resource
	Attrs      map[string]string // size, state, ip, bytes, … stringly and additive
	Tags       map[string]string
}

type Inventory struct {
	Provider  string
	Account   string    // account id / project id, for display
	TakenAt   time.Time
	Resources []Resource
	Partial   []string  // human-readable list of reads that failed
}

type Provider interface {
	Inventory(ctx context.Context) (Inventory, error)
}
```

**Kind vocabulary** — deliberately small and neutral: `machine`, `ip`,
`firewall`, `network`, `disk`, `bucket`, `dns_zone`, `dns_record`,
`identity`, `alarm`, `certificate`, `other`. A provider maps its own
types into these; `NativeType` carries the truth for display. `other` is
not a failure — it is the honest home for anything discovered that the
graph has no opinion about, and it must render, not be dropped.

**`Attrs` is stringly on purpose.** A typed struct per kind would need a
migration every time a provider exposes a new field, and this object is
written to a bucket that must stay parseable by older readers (the
repo's wire-shape rule). Keys the renderer knows about are documented
where it reads them; keys it does not know are shown verbatim in the
resource card. Attr keys used by the joins and the cards:

| Kind | Attrs |
|---|---|
| `machine` | `type`, `state`, `az`, `public_ip`, `private_ip`, `launched` |
| `ip` | `address`, `attached_to` |
| `disk` | `size_gb`, `volume_type`, `attached_to` |
| `bucket` | `versioning`, `encryption`, `public_access_block` |
| `dns_zone` | `record_count`, `private` |
| `dns_record` | `zone`, `type`, `value` |
| `firewall` | `ingress`, `egress` (rendered summaries, never raw JSON) |
| `alarm` | `state`, `metric`, `target` |
| `identity` | `kind` (`role`/`user`), `attached_policies` (count) |
| `certificate` | `domain`, `status`, `expires` |

**`internal/awsinv` is the only package in the tree that imports an AWS
SDK.** `archgraph.go`, the handler and the template consume `infraview`
types and never see an AWS type. That is the whole seam: a second
provider is a new file implementing `Provider`, not a rewrite of the
page.

**No abstraction beyond that.** No provider registry, no config-driven
selection, no speculative second implementation. The console constructs
`awsinv.New()` directly. The cost of adding a provider later is one
package; the cost of a plugin system now is paid every day it is unused.

## 4. AWS discovery (`internal/awsinv`)

Two tiers, both strictly read-only. **No call in this package mutates
anything**, and the IAM grant in §8 contains no write action, so that
property is enforced by the credential and not only by the code.

**Tier 1 — catch-all.** `resourcegroupstaggingapi:GetResources` per
region returns every taggable resource in the account. This is what makes
"any and all resources" tractable instead of a service-by-service
checklist that silently misses whatever was stood up last week. Regions
come from `ec2:DescribeRegions` (opted-in regions only). Everything it
returns that no typed handler claims becomes a `Kind: "other"` Resource,
grouped by ARN service in the page's "other resources" section.

**Tier 2 — typed detail** for what the graph draws:

| Call | Produces |
|---|---|
| `ec2:DescribeInstances` | `machine` (+ type, state, AZ, IPs, tags) |
| `ec2:DescribeVolumes` | `disk` |
| `ec2:DescribeAddresses` | `ip` |
| `ec2:DescribeSecurityGroups` | `firewall` |
| `ec2:DescribeVpcs`, `DescribeSubnets` | `network` |
| `s3:ListBuckets` + `GetBucketVersioning`/`GetBucketEncryption`/`GetPublicAccessBlock`/`GetBucketLocation` | `bucket` |
| `route53:ListHostedZones` + `ListResourceRecordSets` | `dns_zone`, `dns_record` |
| `iam:ListRoles`, `ListUsers`, `ListAttachedRolePolicies` | `identity` |
| `cloudwatch:DescribeAlarms` | `alarm` |
| `acm:ListCertificates` | `certificate` |

**What is never read and never rendered:** access key material, IAM
policy documents, bucket contents, instance user-data, SSM parameters,
secrets of any kind. Identities appear as name, kind and a count of
attached policies. An operator who wants a policy document goes to AWS;
this page is a map, not a credential viewer.

**Per-call soft failure.** Each read that fails appends one line to
`Inventory.Partial` and the rest continue. A page that can show eleven
of twelve resource classes shows eleven and says which one is missing —
it never renders "no security groups" because a single call was
throttled. This is `accountfleet.go`'s known-versus-none rule applied to
a dozen reads instead of three.

**Record-set volume.** `ListResourceRecordSets` is paginated per zone and
a zone can be large. `dns_record` Resources are materialised only for
records that participate in a join (A/AAAA/ALIAS and the apex/wildcard
records); every other record is counted into the zone's `record_count`
and not carried individually. The inventory object stays small enough to
read on every page load.

## 5. The pass

`StartInfraPass(ctx)` in `internal/console/infrapass.go`, modelled line
for line on `StartEconomicsPass` (`economicspass.go:38`): run once at
start, then hourly, log and continue on error. Started from
`cmd/console/main.go` beside the economics pass.

Each tick calls the provider, marshals the `Inventory`, and CAS-writes
`infra/inventory.json` to `s.cfg.Releases` using the same
read-etag-then-put helper `upsertBill` uses. A console with no bucket
handle configured starts no pass.

**Failure posture is `cepull.go`'s, verbatim and for the same reasons.**
Construction-class failures — AWS config never resolves, no usable
credential in the chain, an `AccessDenied`-shaped error from the first
call — latch the pass off for the process after a single log line
(`isAuthShapedError` is reused, not reimplemented). There is no "the IAM
grant might appear later" retry worth making, and a deployment with no
grant must show "not granted", never a recurring error. Everything else
— throttling, a 5xx, a network blip — logs and the next tick tries
again.

`POST /fleet/architecture/refresh` runs one pass synchronously (bounded
by a context timeout), then redirects back to the page with a flash. It
is CSRF-protected, operator-gated, and single-flighted so a leaned-on
button cannot stack passes. It deliberately does NOT clear the latch: a
deployment without the grant gets the same honest answer however many
times the button is pressed.

## 6. The graph (`internal/console/archgraph.go`)

### 6.1 Nodes and edges

Typed nodes — `zone`, `record`, `ip`, `router`, `box`, `app`,
`instances`, `bucket`, `alarm`, `identity`, `other` — and typed edges
(`resolves`, `routes`, `runs`, `stores`, `watches`). The builder takes
three inputs and owns every join:

1. `infraview.Inventory` (the cloud estate),
2. `FleetView` + `HostRows` (`view.go` — boxes, staleness, per-box
   `Metrics.RouteVersions` and `RouteCounts`),
3. the registry's routes and the active economics scenario.

### 6.2 The joins

- **DNS → machine**: `dns_record` value → `ip.address` →
  `machine.public_ip`. This needs no tags and no naming convention, and
  it is the join that makes the picture true rather than decorative.
- **machine ↔ CARLOS box**: `machine.Name` (the `Name` tag, which
  `infra/modules/region-host/main.tf:188` sets to `var.name`) matched
  against the box label, which is the box's hostname
  (`internal/ops/ops.go:319`). Case-insensitive, exact.
- **box → apps and instances**: the box's own
  `Metrics.RouteVersions` keys are its hostnames; each is looked up in
  the registry for app, channel, version and status. Counts come from the
  box's `RouteCounts` (`static`/`resident`/`hibernating`, plus
  `awake`/`asleep` which are `*int` and render "unknown" when null).
- **box → cost and size**: the economics rate card's `BoxSpec` for that
  box label (falling back to the `""` default spec, exactly as
  `AllocateBox` does), priced `HourUSD × 730 + EBSGB × EBSGBMonthUSD`.
  The discovered `machine.type` and summed `disk.size_gb` are shown
  beside it. A box with **no cloud machine behind it** — a customer's
  fleet box, a Pi on a desk — is not priced at all (amended 2026-08-27):
  the rate card prices what this platform pays for, so a monthly figure
  there would be a claim about someone else's bill, and a "priced by the
  default spec" chip would warn about a pricing that should not be
  happening. The card still says plainly that there is no cloud machine.
- **box → bucket**: every box that reports edges to the deployment bucket
  the console itself is configured with (`s.cfg.Releases`) — the one
  bucket relationship the console can state as fact. Per-box replication
  targets are not drawn: the console cannot read a box's litestream
  config, and a guessed edge on this page would be worse than no edge.
- **alarm → machine**: alarm dimensions naming an instance id.

### 6.3 Unjoined is shown, never hidden

This is the rule that makes the page worth building:

- A `machine` no box reports is **drift** — an instance nobody is running
  CARLOS on, or a box that has gone dark. Drawn, marked "no box
  reporting".
- A box with no `machine` is a **customer or BYO box** (the Pi, a fleet
  box on someone else's hardware). Drawn outside the provider estate, and
  correct — not an error. This is also why the non-cloud path is
  exercised on flagship from day one and cannot rot: it is on the page
  today.
- A `dns_record` pointing at an address the estate does not own is drawn
  as leaving the diagram, not dropped.
- Anything in `other` is listed with its native type and ARN.

### 6.4 Layout

A deterministic layered pass in Go, no JavaScript library, no CDN
(the vendored-icons rule):

1. **Layer by kind** — left to right: `zone`, `record`, `ip`,
   `router`/`external`, `box`/`machine`, `apps`, `bucket`. `bucket` is the
   one satellite drawn as a node; `identity`, `firewall`, `network`,
   `disk` and `certificate` are listed in the resource tables below the
   diagram instead, because a graph that draws every IAM role is a
   hairball.

   **Alarms are drawn ON the node they watch, not as nodes of their own**
   (amended 2026-08-27, after seeing the page rendered). They were first
   given the column immediately left of the boxes — the only gap in the
   flow — and that column is the one every IP-to-box edge crosses. Nodes
   paint after edges, so each alarm covered the line running beneath it
   and the picture read as a request reaching its box *through* an alarm.
   The deeper problem was the modelling: an alarm is not a hop a request
   takes, it is a property of the thing it watches. Each alarm is now a
   mark on its subject's node, coloured when firing, with its name and
   state in the node's hover title — which is what "anchored to its owner"
   should have meant all along, and which no layer can put in the way of a
   line. The Alarms table still lists every one in full.

   The invariant this cost us is now enforced directly: **no node may sit
   on an edge's path** unless it is one of that edge's endpoints.
2. **Order within a layer** to reduce crossings: one barycentre sweep
   down and one up, then a stable sort by name so the same fleet always
   produces the same picture. A diagram that reshuffles between refreshes
   cannot be read for change.
3. **Fixed node sizes**, orthogonal edges, emitted as inline SVG with a
   `viewBox` so it scales without script.

Legibility as the fleet grows: instances are drawn as one `instances`
node per box carrying a count, never one node per instance.

**Not delivered (follow-up, 2026-08-19):** nodes were to link by anchor to
their detail cards below the SVG. They do not; the page needs no JS
regardless, because everything on it is server-rendered, but clicking a
node does nothing and the reader scrolls to find the card. Recorded here
rather than left as a promise the page does not keep — see §10.

## 7. The page

`GET /fleet/architecture`, gated by `handleFleet`'s exact shape
(`server.go:1720`): session → `IsOperator` → bare `http.NotFound` on
every failure including the store error, so the route's existence stays
unprobeable. Operator-only, not member-visible: it names hostnames,
instance ids and the estate's shape. Nav entry beside Status, Metrics and
Economics.

Body, in order:

1. **The graph**, full width.
2. **Boxes**, one card each: label, region, instance type, cores/memory,
   disk, monthly cost, apps, instance counts, release, load, staleness,
   and a **drift chip** when the discovered shape contradicts the rate
   card entry (or when no rate card entry exists for a reporting box) —
   which makes this page a rate-card drift detector as a side effect.
3. **Resource groups** — compute, network, storage, DNS, identity,
   alarms, certificates, other — as tables of `Name / NativeType /
   Region / key attrs`.
4. **Footer**: the snapshot's `TakenAt` as an age, the account id, every
   line in `Partial`, and the refresh button.

**Every section soft-fails independently.** A missing inventory renders
the CARLOS half of the page with an "inventory not available" panel,
naming which of the three states applies: never taken, not granted
(latched), or unreadable — the snapshot object exists but could not be
fetched or parsed. ("Unreadable" rather than "stale", which an earlier
draft of this section said: the snapshot's age is already on the page
beside the account id, so staleness is a number the operator reads, not a
fourth state, and a snapshot that will not parse is the failure actually
worth naming.) A failed registry read costs the overlay, not the page.
Nothing renders a confident "none" for a read that did not happen.

## 8. The IAM grant (precondition)

The inventory needs read-only grants the box role does not have:
`ec2:Describe*`, `tag:GetResources`, `s3:ListAllMyBuckets` and the
per-bucket `Get*Configuration`/`GetBucketLocation` reads,
`route53:List*`, `iam:List*`, `cloudwatch:DescribeAlarms`,
`acm:ListCertificates`. No write action, no `iam:Get*Policy*` (policy
documents are out of scope by §4), no `s3:GetObject`.

Two places, because the module in this repo is not what provisioned the
flagship:

1. `infra/modules/region-host/main.tf` — the module's own
   `aws_iam_role_policy`, so any future box gets it.
2. A matching pinfra change and apply on the flagship box, which is
   Paul's to land.

Until (2) lands, the page renders the CARLOS half and the panel says the
inventory is not granted. **The page must be built and merged without
depending on the grant existing** — the ungranted state is a first-class
rendering, not a broken one.

**Status (2026-08-19):** the module statement is in this repo as of this
branch. The flagship box runs a role provisioned from pinfra, so the grant
reaches production only when the matching pinfra change is applied — until
then `/fleet/architecture` renders the CARLOS half and reports the estate
as ungranted, which is the designed state and not a fault.

## 9. Testing

- `internal/infraview`: shape round-trip, and the additive rule — an
  inventory object written before a field existed still parses, and an
  unknown `Kind` renders as `other` rather than being dropped.
- `internal/awsinv`: the AWS calls sit behind narrow interfaces so the
  mapping is tested against fakes — instance/EIP/record fixtures in,
  `Resource`s out. One test per typed mapping, one for the catch-all
  producing `other`, one asserting a failing call lands in `Partial` and
  costs only its own resources. No test touches a real AWS endpoint.
- `archgraph`: table-driven join tests — the DNS→IP→machine chain, the
  name↔label match, the unmatched-machine and box-without-machine cases,
  and the cost/drift computation against a fixture rate card. Layout is
  tested for determinism: the same input twice yields byte-identical SVG.
- Handler: the four-way gate (signed out, signed in non-operator,
  store error, operator) with the first three byte-identical to a missing
  route — the `economicspage_test.go` pattern. Refresh door: CSRF
  required, operator required, single-flight.
- Pass: an auth-shaped failure latches after one attempt; a transient
  failure does not.

## 10. Out of scope

- Cross-account inventory (tito, or any future estate). Each deploy sees
  its own account; that was ruled on.
- A second provider implementation.
- Any write, tag, or remediation action from this page.
- A `carlos` CLI view of the inventory. `infra/inventory.json` is
  deliberately a durable bucket object rather than a page-local cache so
  that a CLI door is a small follow-up, but it is not this spec.
- Cost Explorer actuals on this page. `/fleet/economics` owns money;
  this page owns shape, and shows the modelled per-box figure only.
- **Deferred, not dropped:** anchor-linking each diagram node to its detail
  card below the SVG (§6.4). The page ships without it; every fact the
  anchors would have jumped to is on the page already.