# Isolated Exec, Plan 4: amendments before execution (2026-09-25)

Plan 4 (`2026-09-12-isolated-exec-4-layout.md`) was written against main at c1d40ee and spec revision 14. Before building it on branch `isolated-exec-layout` (cut from 52f71143, spec revision 15), an audit checked it against main. These rulings amend the plan and **take precedence over its text** where they differ. Each names the task it changes. LB means load-bearing: it must be applied when that task runs.

The audit's evidence (the drift table, semantic findings S1 to S15, and the conflict table C1 to C13) is summarised in each ruling. Line references are to main at 52f71143.


**R1 (LB, T3/T4; spec §8.2 step 2, §6.3, §6.7). How the pass proves a host is stopped.**
> Add `func (a *Service) Fenced(host string) bool` and `func (a *Service) StopUncertain(host string) bool` (true when `stopHistory[host]` exists and is not complete, or when an entry exists) to `internal/activator/maintenance.go`. In the pass, treat `FenceAndStop` returning `ErrStopUnproven` as quiesced **only** when `svc.Quiesced(host) && !svc.StopUncertain(host)` and the agent's startup unit sweep succeeded (pass a `SweptAtStart bool` into `Pass`). Otherwise, lift the fence at this pass's generation, return `Result{Outcome: "deferred", Reason: "stop unproven"}`, and write no journal. Never leave a layout fence installed after a refused quiesce. Add a test that runs `Converge` against a real `testActivator` service for an entry-less host and asserts it migrates and that the host can wake afterwards.

**R2 (LB, T1/T3; #175 instance-moves §6.1 G12, spec §8.2 step 2). The layout fence and the move fence.**
> Do not add `Unfence`. Use the existing `activator.(*Service).LiftFence` everywhere (the `Fence` interface is `FenceAndStop` + `LiftFence`). Drop `TestUnfenceReleasesOnlyTheMatchingGeneration`. In `convergeHost`, skip with `Outcome: "deferred"` any row whose `Status` is neither `""` nor `"ready"` (this covers the cut's `"moving"`), and any host where `svc.Fenced(host)` is already true before this pass fences. After `FenceAndStop`, re-read the row and also refuse if its `Status` is no longer ready or it is no longer `ActivatorManaged`. The pass lifts only its own generation. Add a test with a pre-installed fence at a lower generation: the pass defers the host and the original fence is still in place afterwards.

**R3 (LB, T3/T4; spec §8.5, §13 R15). The probe gate.**
> `Pass` gains `NestedOK func() error`. The agent passes the same `nestedGate(instancefs.MountNoSymfollow, eb.Control.Ready)` value it gives `eb.NestedOK` (agent.go:330). `Converge(ModeNested)` calls it once per pass before touching any host. On error it returns a single `Result{Host: "", Outcome: "refused", Reason: "isolation not ready: <err>"}` and moves nothing. `Converge(ModeFlat)` and `Recover` are never gated.

**R4 (LB, T3/T4; spec §3.3, §8.2 "Recovery is the journal"). Recovery converges before it releases.**
> The signature becomes `Recover(ctx, intent Mode)`, where `""` means no intent. `recoverHost` completes the journal under its fence. If `intent != "" && intent != j.Mode`, it then runs the migrate steps for that host **under the same fence generation**, and releases only once the host is at the intent. `Converge(ctx, mode)` calls `Recover(ctx, mode)`. At agent start, run `Recover` **after** `eb.Control.Sweep` succeeds and before `svc.Start()` (agent.go:367-372), with the intent loaded by one `LoadLayoutIntent` from the deploy store. If the intent cannot be read, complete journals but keep any host whose completed endpoint is `flat` and whose pre-journal layout was `nested` fenced (journal kept at `done`) until an intent is read. X5's test must record fence lifts and assert none happens between recovery and convergence.

**R5 (LB, T2/T3; spec §8.2 steps 3 and 6, §3.1). The two missing artifacts.**
> Append two optional artifacts to `artifactNames`, after `socket`: `litestream-shadow` (`<dir>/.activator-<host>.db-litestream`, a directory, derived as lease.go:287 does) and `corrupt` (`<db>.corrupt`). The destination check treats an existing destination shadow dir as a refusal (two replication states), like `db`, and moves an existing destination `.corrupt` aside, like `env`. `Validate`'s length check becomes 9.

**R6 (LB, T3; spec §3.2, §3.3). Renames through directory fds.**
> Add `internal/instancefs` helpers, `MkdirBeneath(dir, name string, perm os.FileMode) error` and `RenameBetween(srcDir, srcName, dstDir, dstName string) error`. Build them on the existing `openDir` (`openat2` `RESOLVE_NO_SYMLINKS|RESOLVE_BENEATH`) plus `unix.Mkdirat` / `unix.Renameat`, with `!linux` stubs returning `errors.ErrUnsupported`. `EnsureDirs` creates `<data>/.instances/<iid>` and `<run>/.instances/<iid>` through `MkdirBeneath` from fds on `<data>/.instances` and `<run>/.instances`, then refuses unless an `Lstat` says it is a directory. Every artifact move uses `RenameBetween`. Add a test that plants `<data>/.instances/<iid>` as a symlink and asserts the pass refuses and moves nothing.

**R7 (LB, T1; spec §8.2 first paragraph). The intent's wire shape.**
> The shape is `release.LayoutIntents{Schema int; Requests []LayoutIntent}`, oldest first, at `control/layout.json`. `LoadLayoutIntent(ctx, s, boxLabel)` returns the **newest request whose scope applies to boxLabel**. `AppendLayoutIntent(ctx, s, in)` is the CAS-append writer, mirroring `AppendRestartScope` (restartscope.go:259) including the retry. A schema above 1 is an error, as planned.

**R8 (LB, T2/T3; plan's own tests). One socket directory.**
> `PlanFor` derives both socket endpoints from its `socketDir` argument: flat `filepath.Join(socketDir, host+".sock")`, nested `filepath.Join(socketDir, "instances", InstanceID(host), "app.sock")`, which is exactly `LayoutInputs.sockets` (layout.go:50-56). `Pass.layoutInputs` passes `p.socketDir()`, not the raw field. In `box()` and every T3 fixture the row's `Socket` is `filepath.Join(rd, host+".sock")`. Production is unchanged, because both default to `provision.SocketDir`.

**R9 (LB, T6; spec §8.2 "That order is enforced"). Where the downgrade guard goes.**
> Put the guard in `systemApplier.install` right after `newPlan, newUnits, err := a.stagePlan(ctx, staged, newDir)` (systemapply.go:339). The candidate predates the layout when `!slices.Contains(newUnits, "data-instances.mount")`. Delete `candidatePredatesLayout(stagedRoot)`. Refuse with `a.record(req, fmt.Errorf("%w: …", errSystemRefused, …))` before the journal is written. The tests cover a pre-layout unit set with a nested row (refused), the same set with every row flat (installs), and a post-layout set with nested rows (installs).

**R10 (LB, T5; spec §8.4). The instance page's layout fact.**
> Do not add `RouteView.Layout` or touch `Assemble`. Add `Layout string` to `instanceDetail` (instancepage.go:13-18). Fill it in `handleAppInstance` from one `status.LoadBoxes` read: the first box whose report names the host, otherwise `unknown (box predates layout reporting)`. The template line is `<dt>Layout</dt><dd>{{.Layout}}</dd>` inside `{{with .InstanceDetail}}`. Members see the word only, never the reason (reasons carry paths). The strings go through copy review before merge.

**R11 (LB, T3; spec §3.2 "a CI test greps the tree"). The grep test's pattern.**
> The grep test matches the path literal, not the substring: pattern `["/]\.instances\b` over `--include=*.go`, across `internal/` **and** the repository root's `*.go`. The allowlist is `internal/layout/` and `internal/host/bootstrap.go`. Before committing, confirm the test passes on current main plus this task (the substring pattern fails on console/server.go:2932 and instances_cli.go:109 today).

**R12 (minor, T6; spec §8.2 downgrade rules, gap). Guard `carlos add` too.**
> In `runAdd`, before `Upsert`, if a row for the host exists and `provision.NestedPaths(dataDir(), existing)` is true, `log.Fatalf("add %s: its paths are nested; layout revert first", host)`. Record this in the task report as a spec gap, so the spec owner can add it to §8.2's list.

**R13 (minor, T4; spec §8.2 step 6). Serialize with config sync.**
> `migrateHost` and `recoverHost` hold the agent's `configSyncMu` (agent.go:913) from journal write to journal removal. The pass takes a `Serialize func(func())` seam; the agent wires it to `configSyncMu.Lock/Unlock`, and it is nil in tests. Keep the `fenced` skip in SyncExec as a second net. Add it as a variadic option (`...SyncOption`) so the 62 existing call sites stay as they are.

**R14 (minor, T4; spec §8.2 "when it is asleep"). Keep the serial loop moving.**
> `layoutLeg` runs `Converge` in its own goroutine behind a `TryLock` single-flight, with a dirty flag that re-runs once if a serial move arrives mid-pass. On the serial-move leg, `convergeHost` defers a host that is awake (`!svc.Quiesced(host)`) with `Outcome: "deferred", Reason: "awake"`. Insert the leg after `scheduleLeg` and before `systemLeg.observe` (agent.go:1657-1667).

**R15 (minor, T5; snapshot.go:295-305, contract_test.go:26-80). Status placement.**
> Put the report under the existing private exception as `BoxMetrics.Layout *LayoutReport`, not at the Box root. Add `"layout"`, `"metrics.layout": {"mode","rows"}` and `"metrics.layout.rows": {wildcardAnyKey}` to `allowedBoxKeys`, and assert `allowedFleetKeys` has no layout. Report only instance-kind rows: `external` for unflagged socket instances, and skip `Kind "sidecar"`. `Mode` is `"none"` when the last-seen intent is `""`. The console door also reads `fleets/<name>/` boxes the way apisystemupdate.go:233 does.

**R16 (minor, T1/T4 tests; store.go:37-41).**
> The newer-schema fixture is written by `s.Delete(ctx, key)` then `s.Put(ctx, key, body, "")`. The T4 test switches intents by `Get` for the etag, then `Put` with that etag (or through `AppendLayoutIntent` once R7 lands).

**R17 (minor, T3; go vet lostcancel).**
> `deadline(ctx)` returns `(context.Context, context.CancelFunc)` and each caller `defer cancel()`s. No `//nolint`.

**R18 (minor, T2/T3; review hygiene).**
> Delete `errNoSuchArtifact` and `var _ = filepath.Join`. List `journal_linux.go`/`journal_other.go` under Files. Rename the test to `TestReadJournalRefusesASymlink`, or add a FIFO case (`unix.Mkfifo`, linux-only file). Define `openLayoutRegistry(t, dir)` in `pass_linux_test.go` as `registry.Open(filepath.Join(dir, "registry.db"))` plus `t.Cleanup(Close)`.

**R19 (minor, T3; spec §3.1).**
> `Preflight` refuses when `len(plan.To.Socket)+len(".control") > provision.MaxUnixPath` (for mode nested), naming the length.

**R20 (minor, T4; agent.go:510-520).**
> Publish the edge's reload the way `setUpstreamSource` publishes probes: `var layoutReload atomic.Pointer[func()]`, set in the `RunWithActivatorReady` hook to `func() { _ = e.Reload() }`. `Pass.Reload` loads it and is a no-op before the edge is ready, when the 2 s poll covers.

**R21 (minor, all; current file locations).**
> - `carlos status --layout` goes in `RunStatus` (cli.go:3342-3375) and `statusUsage`. The fetch and render go in status_instances.go beside `statusInstances`. The CLI table test is required.
> - The unflag refusal goes in `runRouteHibernate` (cli.go:1386, before :1443).
> - Reconcile edits go at reconcile.go:967-978 (flag) and :1011-1021 (drift).
> - The door is registered as `s.cliRoute("GET /api/cli/status/layout", svcrole.Set{}, s.handleAPIStatusLayout)`, gated by `cliUser` + `s.store.IsOperator` → 404, as in apistatusinstances.go:91-104.
> - One predicate, `layout.JournalPresent(dd, host)` (Lstat not-ErrNotExist), is used by the pass, SyncExec's option, the litestream `fenced` hook in ops.go:117 (via `provision.LayoutJournalPath`, no layout import needed) and `layoutReversalIncomplete`.

**R22 (minor, T5/T6 docs; spec §8.4, house copy rules).**
> T5 documents `carlos status --layout` only. It does not add a `#### Instance layout` section for a verb that does not exist yet; plan 5 writes that section. T6's sentence under `### carlos system` states the refusal and names "revert the layout first" without a command line. All new user-facing strings (the CLI help, the refusal messages, the page fact) go through copy review.

**R23 (minor, plan header; D49, D50).**
> Cite spec revision 15. The branch is `isolated-exec-layout`. Gate env is `GOCACHE=$TMPDIR/gocache GOFLAGS=-mod=mod GOPROXY=off TMPDIR=/tmp/g`. Use the commit trailer from the controller's current attribution.

**R24 (minor, T3 tests; S5).**
> Add a `Replica: true` flat managed row to `box()` and assert it migrates and reverts like any other. There is no `replica_of` column; the replica is its own host.

**Load-bearing rulings: 11 (R1-R11).** Minor: 13 (R12-R24).

**R25 (minor, T5/T6; copy-review rule).**
> The user-facing strings T5 and T6 introduce (CLI help and table headings for `carlos status --layout`, the instance page's Layout fact and its unknown text, the refusal messages from `system apply`, `carlos route --hibernate=false`, `carlos add` and the reconciler, and the docs/site/cli.md sentences) go through copy review with Paul before they are written into files. No em dashes.

## Findings the rulings answer

**S1. Quiesce is unprovable for every asleep host after an agent restart, and the plan then never releases the fence.** For an entry-less host, `FenceAndStop` returns `ErrStopUnproven` unless `stopHistory[host]` is complete (maintenance.go:90-99). `stopHistory` is in-process memory (activator.go:626). After any restart or roll, no asleep host has history. The plan's global constraint and `migrateHost` keep the fence installed on `ErrStopUnproven` and never release it. The first `Converge` under a nested intent would therefore fence every asleep managed flat host on the box. `doWake` then refuses them all with `ErrFenced` (activator.go:1060) until the next agent restart: a box-wide outage. `Recover` at start (before `svc.Start`, so there are no entries) hits the same answer. The journaled host stays `fenced`, never wakes, never gains history, and never recovers. The plan's `fakeFence` returns success, which hides all of this. The #175 worker meets the same fact and handles it: it ignores `ErrStopUnproven` and asks `svc.Quiesced(host)` instead (movespass.go:104-114, 395).

**S2. The #175 move fence and this plan's layout fence share one slot per host, and they collide.** `a.fences` is `map[host]uint64` (activator.go:624). `FenceAndStop` overwrites any prior fence at a lower-or-equal generation (maintenance.go:80-85). `LiftFence` deletes only on an exact match (:133-145). The move's generation is `Move.RequestedAt.UnixNano()` (movespass.go:240). The layout pass's default is `time.Now().UnixNano()`, which is later. Consequences:
- A layout pass on a host whose cut is in progress (`Status "moving"`, fence installed) overwrites the move's fence, then lifts its own generation. That removes the move's G12 fence until the next moves round re-installs it. The cut itself triggers this: `nudgeSerial` (movespass.go:363-366, 432) bumps the serial, and the serial move runs `fullPass` and then the layout leg in the same tick.
- A layout fence left installed after an unproven stop (S1) is always newer than any move requested earlier. Every later `cfg.Fence` then fails with `ErrFenceGeneration`, which is not `ErrStopUnproven`, so the cut errors every pass and is stuck until restart (movespass.go:110-113, 364-367).
- `runMovesPass` also runs from `instancesLoop` (agent.go:2334), a different goroutine from `fastTick`. The layout pass and a moves round can therefore interleave. The move's journal check (movespass.go:381) covers files mid-move, but only if the layout pass never lifts a fence it did not install.
- A moving row is about to leave this box. Migrating it is wasted work that races the purge (`PurgeLocalDBArtifacts(dbPath)`, movespass.go:425).

**S3. SyncExec and litestream race the pass.** SyncExec runs from `configSyncLoop` and `configGenLoop` through `syncExecSerialized` (agent.go:918-922), on goroutines separate from `fastTick`. The flat writer uses a fixed temp name `<host>.config.env.tmp` and then a rename (syncexec.go:1176-1180). A per-route `fenced(host)` check before rendering is a check-then-act. SyncExec can pass the check, the pass can then move `config.env`, and SyncExec then lands a flat `config.env` for a row that is now nested. That leaves an orphan, or two bundles. The litestream generator does not matter: hibernating rows are excluded already (litestream.go:344-346, 436-438), and only hibernating rows migrate.

**S4. The probe gate (R15, §8.5) is not consulted before migrating to nested.** `eb.NestedOK = nestedGate(instancefs.MountNoSymfollow, eb.Control.Ready)` (agent.go:330, 700-715) refuses every nested wake unless the three mounts carry `nosymfollow` and `/ready` answers ready. On dev, the probe answered not-ready for 13 days (R15 a/b). The plan's `Converge(nested)` would still move rows. Each moved flat host that woke fine would become a nested host that refuses to wake. §8.5: "A box that cannot establish the namespace **does not start nested tenants**."

**S5. Replica rows (#162) need no special case, and there is no `replica_of`.** A replica is its own host with its own record (provision/replica.go:44-66). It is minted flat by `routeFor` with `Replica: true` (reconcile.go:364-380), and only on the box its placement names (reconcile.go:202-211). #175 notes that a replica never gets a route on a platform box (provision.go MoveRequest, "G14") and refuses to move one. Its DB, env files and socket are named after its own host, so `PlanFor` derives them correctly and it migrates like any activator-managed row. Nothing reads a replica's paths across hosts. The home↔replica lease protocol is titogo's own, over the network. No change is needed beyond a test row.

**S6. Sidecar rows (Kind `sidecar`, merged from sidecar-members) are outside the pass, and status must say so.** `sidecarRouteFor` mints `DBPath: DefaultDBPath(dataDir, sidecarHost)` with `Socket ""` (provision/sidecar_reconcile.go, `sidecarRouteFor`). `ActivatorManaged` excludes them (Kind != instance) and so does Classify's conflict scan. The plan's status rows say "`external` for an unflagged socket instance route", and a sidecar is not one. Without an explicit skip, a sidecar would render as nothing or as `foreign`. `coSpawnStaged` (sidecar_reconcile.go:156) is a new hand-built flat-path reader of `<DataDir>/<host>-sidecar.env`, but its host is a sidecar host (`*.sidecar.internal`, sidecar.go:31), never a migratable row. No change needed.

**S7. Per-environment and private object storage (#412) add no on-box path.** 96d4dacb says "Nothing on a box changes: config delivery already resolves per (account, app, environment)". 91d16d0a is a console and manifest policy. Neither introduces a host-named file under `DataDir`. No impact.

**S8. The §9 sweep for new hand-built consumers since c1d40eeb found no new reader of a migratable host's flat shape.** The diff grep of non-test Go for `DefaultDBPath|DefaultSocket|.DBPath|.env"|config.env|-sidecar.env|.sock"` finds four:
- `movespass.go:393-397`, which now follows the row;
- `status` `DBBytes(r.DBPath)`, which follows the row;
- `sidecarRouteFor`, sidecar rows only;
- `coSpawnStaged` (S6).

Pre-existing readers listed in spec §9 are unchanged. No ruling needed.

**S9. `carlos add` on an existing nested host re-flattens it: a third unflag and flatten path the downgrade guards miss.** `runAdd` defaults `--db` to `DefaultDBPath` (cli.go:139). `Upsert`'s ON CONFLICT overwrites `socket`, `db_path` and `hibernate` (registry.go:987-1006). SetHibernate's own doc names it as the third writer (:1705-1708). A re-add of a nested host leaves the row flat and unflagged over nested files. The next wake restores a second database at the flat path. Spec §8.2 lists only `carlos route --hibernate=false` and the reconciler. §8.3 names `carlos add` as the repair for `foreign` rows, which is still fine, because `foreign` ≠ nested.

**S10. Recovery releases the host before converging it, and at start it does not converge at all.** Spec §3.3 and §8.2 say a journal that passes is completed to its own end and then "converged to the current intent in the same pass, **before it is released to the activator**". Plan `recoverHost` does `defer p.Fence.LiftFence(...)` and returns. `Converge` fences again later. At start, the plan runs `Recover` only. `fastTick`'s first successful serial read only seeds (agent.go:1624-1638), so no converge happens until someone bumps the serial. §3.3's forged-journal argument depends on that ordering. A flat tenant (same uid) can forge a valid `mode: flat` journal for a nested host. Recovery reverts it, releases it, and the host then runs unisolated until the next deploy.

**S11. Two spec-named artifacts are missing from the on-disk artifact list.** §8.2 step 3: "`-wal`, `-shm`, **the litestream shadow dir and `.corrupt`** are derived from it". The shadow dir is `<dir>/.activator-<host>.db-litestream` (lease/lease.go:287). The quarantine is `<db>.corrupt` (backend_exec.go:50, 314). Step 6 names "a quarantine" as an optional artifact. The plan's `artifactNames` has seven entries and calls itself "part of the on-disk contract". Adding entries later changes the journal format.

**S12. Renames and directory creation use path strings that a same-uid tenant can redirect.** `/data/.instances` is `0700 carlos` (bootstrap.go:69), and during the transition flat tenants run as `carlos` (spec §1.3). A tenant can plant `/data/.instances/<iid>` as a symlink before `EnsureDirs`. `os.MkdirAll` accepts an existing symlink to a directory, and `os.Rename(src, "/data/.instances/<iid>/<host>.db")` then lands the DB wherever the link points. Spec §3.2: the raw path is opened "only as `openat2(RESOLVE_NO_SYMLINKS | RESOLVE_BENEATH)` directory fds for `renameat`".

**S13. The intent object's wire shape differs from the spec and will be frozen by the first rolled box.** The spec says `control/layout.json` is "`{id, requested_at, mode, scope}`, the newest entry of an append log like `restart-scope.json`" (§8.2), i.e. `{schema, requests:[…]}` (restartscope.go:113-120). The plan reads a single flat object. Plan 5 writes the spec's shape. A plan-4 box reads `mode:""`, `ValidLayoutMode` fails, it logs "not converging", and it never converges.

**S14. Status placement, privacy and fleet boxes.** Box fields other than `Metrics` and `Sidecars` are "meant for the public page" (snapshot.go:300-305). The contract allowlist (contract_test.go:26-80) fails on a new key, and it handles hostname-keyed maps with `wildcardAnyKey` under `metrics` (:78-79). The plan puts a hostname-keyed `Layout` at the Box root. Separately, customer-fleet boxes publish under `fleets/<name>/status/boxes/` (fleets.go:997; apisystemupdate.go:233). A door reading only `status.LoadBoxes(ctx, s.cfg.Releases)` misses them.

**S15. `Converge` runs inline in the 2 s serial loop and stops awake hosts.** `fastTick` runs its legs synchronously (agent.go:1642-1669). `FenceAndStop` on an awake host waits for the stop, and the plan gives it up to 60 s per host. That stalls the restart, wake, schedule and system legs and the fast sweep. Under a standing nested intent, every freshly minted flat row that is awake gets bounced on the next serial move (every deploy or config write). Spec §8.2: a later-eligible row "is migrated on the next pass **when it is asleep**".

**Count:** 15 findings. 12 need action. S5, S7 and S8 are evidence of no impact.

---