# Census: the claims-on update gate and recover's generation census Status: PROPOSED. Every row carries `rev3=PROPOSED-claims-on-gate-recover`. The ruling is asked of the census owner on #280. The branch is `claims-on-gate-recover`, from main 3d444a1b. It fixes two defects a deployment harness found on a box running with CARLOS_HOST_CLAIM on: (a) the update gate rolled every update back, and its rollback too, because a claims-on edge takes about 22 s to listen and the gate waited 20 s; (b) carlos-system-recover's root generation census refused racily and, since the edge's guard drop-in `Requires=` the recover unit, kept the edge down. It adds one census overlay, `internal/provision/testdata/plan14-root-reads-claims-on-gate-recover.tsv`, loaded last in `TestPlan14RootReadCensus`, after scan-demand-revival. Every row is authority. There is no legacy row and no frozen declaration, so the seal stands at `a21dba12…f0cc262`. The `internal/host` envelope is re-pinned. The record-write census (`TestPlan9B8RecordWriteCensus`) does not change: no record write is added or moved. ## Ruling 6 No-census boxes see no change, and claims-off boxes see none in the gate. A claims-off box that retains a census (claims were on and were turned off) still uses root participation, so it receives the recover settlement changes below exactly as a claims-on box does; that is the existing retained-census rule, not a new one. - The gate's new window and fail-fast are keyed on CARLOS_HOST_CLAIM, read with `hostwrite.Enabled` from the update and recover units' environment (both load host.env). Claims off, the gate is exactly the 20 s one and reads no `systemctl show` inside its wait (`TestClaimsOffGateIsUnchangedByAnEdgeThatListensAt22s`, `TestClaimsOffGateWaitsItsFullWindowThroughACrash`). - The recover deferral applies only where root participation is required (a census). A no-census box's refused settlement fails recover as on main (`TestRuling6NoCensusRecoverIsUnchanged`). - The generation census itself still runs only where it ran before: it is reached only through the ledger's launch verifier, which a no-census box never consults. ## The units ### (a) The update gate | unit | change | executing identity | disposition | |---|---|---|---| | `internal/host/systemapply.go` `RunSystemApply` | re-pinned, no read of its own: picks the gate window through `systemApplyGate` (CARLOS_HOST_CLAIM by `hostwrite.Enabled`) | root apply and recovery, uid 0 | authority, D,F,J4, unchanged | The gate (`systemApplier.gate`, `gateCheck`, `gateFailFast`, `systemGateWaitFor`) reads only `systemctl is-active` and `systemctl show` answers it already read, and has no row. ### (b) Recover's generation census | unit | change | executing identity | disposition | |---|---|---|---| | `internal/host/guardtransaction.go` `systemApplier.checkGuardRecovery` | re-pinned: its reads are unchanged; the two no-journal settlements call `settleWithoutJournal`, which on a census box wraps a refused `controllersRecovered` in `errSettlementDeferred` | root system update and recovery, uid 0 | authority, D, unchanged | | `internal/host/systemapply.go` `systemApplier.Run` | re-pinned: its lock-file read is unchanged; under `--recover` a deferred settlement is logged and not fatal, and is not tried again at the end of the run | root apply and recovery, uid 0 | authority, D,F,J4, unchanged | | `internal/host/systemparticipation.go` `systemApplier.checkUnarmedParticipationRecovery` | re-pinned, no read of its own: with no journal open its settlement goes through `settleWithoutJournal` | root system update and recovery, uid 0 | authority, G,F, unchanged | | `internal/host/rootgeneration_kernel_linux.go` `generationKernel.absent` | re-pinned: same reads; a task refusal is typed (`generationTaskRefusal`, same text) so the retry can tell task-level refusals from structural ones | root census, uid 0 | authority, E2, unchanged | | `internal/host/rootgeneration_kernel_linux.go` `generationKernel.task` | re-pinned: same reads; a start time that moves mid-inspection refuses as `errGenerationTaskReplaced` (still `os.ErrPermission`); every reference check still runs on every accepted service task | root census, uid 0 | authority, E2, unchanged | | `internal/host/rootgeneration_census_linux.go` `generationCensusRetry` | re-pinned, no read of its own: whole passes; every pass and its blocking reads run under one 20 s deadline, and a pass that ends past it refuses; task-level refusals a later pass can clear retry with backoff from 250 ms inside it; reference, namespace, mount and structural refusals refuse at once | root census, uid 0 | authority, E2, unchanged | | `internal/host/rootgeneration_services_linux.go` `generationServiceReader` | re-pinned, now a binding with no read: the production `generationServiceEnv` | root census, uid 0 | authority, E2, unchanged | | `internal/host/rootgeneration_services_linux.go` `generationServiceEnv.reader` | new: `generationServiceReader`'s body. The exe `Readlink` moves first, for diagnostics only. Every refusal names check, unit, cgroup and exe. A non-root task, or a root task of a unit with a selected image (the guarded edge), whose exe reads ` (deleted)` goes to `deletedProgram` | root census, uid 0 | authority, E2 (its caller's) | | `internal/host/rootgeneration_services_linux.go` `generationServiceShow` | new: the `systemctl show` moved out unchanged | root census, uid 0 | authority, E2 (its caller's) | | `internal/host/rootgeneration_services_linux.go` `generationServiceTrust` | new: the unit-file `EvalSymlinks` and `OpenTrustedFile` moved out unchanged | root census, uid 0 | authority, E2 (its caller's) | | `internal/host/rootgeneration_services_linux.go` `generationServiceEnv.deletedProgram` | new root read: opens a service task's image through `/proc//exe`, `Fstat`s it and hashes it | root census, uid 0, in carlos-system-recover, carlos-system-update, instance-control's start and the guard-exec prelude | authority, F,J4 | ### Why the deleted-exe read is F,J4 `deletedProgram` makes the proof the guard makes before it execs a binary, on the same kind of input. The guard's reads are F,J4 root-input: `hostfloor` `OpenProtected`, `Disk.VerifyFile` and `ExecutingHash` (which opens `/proc/self/exe` and hashes it), and in this package `readRootFloorBinary`. `deletedProgram` opens the running image of another task the same way, requires a regular file owned by root, with no links left and no group or other write, hashes at most 512 MiB of it, and accepts only a digest equal to the signed router build of `/carlos` or `/carlos.prev`, which `readRootFloorBinary` proves from the identity cache the box holds. Classifying it at E2 with its caller would understate what it decides: a match is what lets an unselected inode count as the unit's program. It is reached for a non-root task, and for a root task of a unit with a selected image (the guarded edge, whose selected program is `/carlos`), when the unit selects `/carlos` and the exe link reads `/carlos (deleted)`. A root task of a unit with no selected image keeps main's path walk, which a deleted exe fails, so the strict root scan applies to it (`TestRecoverCensusDeletedExeOfARootTask`). Anything else refuses as before. Acceptance only lets the census go on to the reference checks: exe, cwd, root, descriptors, maps and mounts are still compared against the retired generations (`TestRecoverCensusStillChecksReferencesOfAnAcceptedDeletedExe`). ## Fix round 1 (Astra's review of 8eed5c19) | unit | change | disposition | |---|---|---| | `systemApplier.checkGuardRecovery` | every no-journal settlement (controllers census, completion or withdrawal, the census again inside `EndParticipation`) in the marker, no-marker and already-completed branches goes through `settleWithoutJournal`, which now defers only a result that is a census refusal and nothing else (`onlyCensusRefusal`); an unfinished transaction's marker is removed inside the settlement, after its census. Round 0's deferral of any controller failure is withdrawn | authority, D, re-pinned | | `systemApplier.checkUnarmedParticipationRecovery` | the same census-only rule | authority, G,F, re-pinned | | `productionGenerationKernelCensus` | every refusal typed `generationCensusRefusal`; each pass's lookup is built on the census deadline context and wires its recheck | authority, E2, re-pinned at main's class | | `productionGenerationAbsence`, `generationRetiredController`, `type generationKernel` | wire or carry the lookup's recheck | authority, E2, re-pinned at main's class | | `generationKernel.task` | holds a trusted task to its proven image: recheck before each use of the trust and at the end; no recheck wired refuses every trusted task | authority, E2, re-pinned | | `generationCensusRetry` | one 20 s deadline shared by every pass and its blocking reads; a pass ending past it refuses | authority, E2, re-pinned | | `generationServiceReader`, `generationServiceEnv.reader`, `generationServiceEnv.deletedProgram` | the lookup returns classify and recheck; the hashed descriptor is kept to bind the task; the hash stops at the deadline | re-pinned at their classes | | `generationServiceEnv.recheck` (new root read) | re-opens `/proc//exe` and requires the hashed inode (device, inode, mount ID, file handle), still root-owned, unlinked, not group or other writable | authority, F,J4, as `deletedProgram` | | `generationExeIdentity` (new) | `Fstat`, `Statx` and `name_to_handle_at` of an open image descriptor | authority, F,J4, as its callers | | `generationContextReader.Read` (new) | the hash's reader over the already open image; stops at the deadline | authority, F,J4, as its caller | No execution exclusion exists for a service task's own `exec`: root's launch exclusion (`participationRelease`, `Ledger.ExcludeLaunches`) covers claim launches only. The rechecks are sampled identity checks at observation points (before each use of the trust and at the end of the inspection), not execution exclusion: a replacement visible at a recheck refuses, but a task that execs an unsigned helper after the final `/proc//exe` open keeps its classification until the next pass. Main's live-exe path (the selected-program walk and plain non-root services) has the same race after its one observation; this branch extends that exposure to the deleted, root-owned, protected inodes whose bytes match the signed current or previous router build, which main refused outright. It does not admit an unsigned helper on its own. The binding applies only to deleted-image acceptance; live-image rechecks return at once. Main's path is unchanged here and needs a ruling. Still no legacy row, so the seal stands; `internal/host`'s envelope is re-pinned again. ## Fix round 2 (Astra's round 2 on ec057b9b) - **Post-journal settlement (finding 1a).** `closeGuardJournal` durably removes the journal, then settles (`endGuard`, `clearUpdateBegun`). That settlement ran unwrapped, so a census refusal there failed recover and the edge's dependency job with the files whole and the journal gone: the Run 11b/11d shape. It now goes through `settleWithoutJournal`, forward and backward, guarded and unarmed, and `Run` under `--recover` consumes the deferral from `recoverIfNeeded` without keeping the fatal error or retrying settlement. Non-census failures stay fatal. - **Acquisition barrier (finding 1b).** With no update record (a prepared transaction never begun), `EndParticipation`'s `BeginUpdate` withdrew the record it wrote when its census refused, so a deferral left `Acquire` open. `settleWithoutJournal` now calls `Ledger.HoldUpdate` first, under the transaction lock: root's update record kept, or written under the transaction's id, with no census of its own. `Acquire` reads the ledger census and `update.json`, so it stays refused while recover defers. A barrier that cannot be written leaves the refusal fatal. An already-completed transaction with no record left replays at once and runs no census. `Ledger.HoldUpdate` is a new unit at its siblings' class (D). - **The signed-image proof under the deadline (finding 3).** `readRootFloorBinary` becomes a wrapper (no read) over `readRootFloorBinaryContext`, whose hash stops between reads and whose identity scan stops before each identity once the census context ends. `generationContextReader` moves beside it. The two helpers are witnessed directly; the lookup is witnessed to hand the deadline to the proof; the plumbing from `readRootFloorBinaryContext` into the helpers is not witnessed by a non-root test (the proof needs root-owned files), so a mutant that drops the context inside it survives. - **Prose (finding 4).** The rows and this note now describe the rechecks as sampled identity checks, name the final-open race as shared with main's live-exe path, state the deadline's coverage and its gaps, and no longer say the three restored tests expect exit 0. ## Fix round 3 (Astra's round 3 on d16cb5f0) - **The legacy marker's settlement (finding 1b, unarmed).** With root participation, no armed floor, a legacy `update-begun.json` and neither an update record nor a journal, `checkUnarmedParticipationRecovery` returns without settling and the run's end settled the marker through `clearUpdateBegun` directly, logging any error: its `BeginUpdate` was refused by the census, which withdrew the record it wrote, so recover exited 0 with acquisitions open. `settleUpdateBegun` now settles through `settleWithoutJournal` with the `recover-legacy` barrier, and `settleMarker` decides what the outcome means for the run: a deferral under `--recover` is logged and left to a later run; a barrier that could not be written (`errAcquisitionBarrier`) joins the run's error in every mode; every other marker failure is logged as on main. `TestRecoverHoldsAcquisitionsForALegacyMarker`, `TestRecoverFailsALegacyMarkerWhoseBarrierCannotBeWritten`. - **Prose (findings 2 and 3 of round 3).** The deadline is described as cooperative cancellation at checkpoints with the uncancelled work named, and the boot-delay figures as estimates; every row that said a trusted task is "held to" its image now says it is rechecked at observation points and names what a recheck cannot see. ## Fix round 4 (Astra's round 4 on 2a823aef) - **A barrier prerequisite failure is a barrier failure.** In `settleWithoutJournal`, when `ParticipationRequired` itself failed (the store did not open, a retained census did not read) the original census refusal was returned bare, so the legacy marker's settlement logged it and recover exited 0 without a barrier. It now returns `errAcquisitionBarrier` with both errors kept, which `settleMarker` makes fatal in every mode; a box where participation is not required still falls back as before. `TestRecoverFailsALegacyMarkerWhoseBarrierPrerequisiteFails` injects the failure after `BeginUpdate` has withdrawn its record. - **Ruling 6 wording** qualified: a claims-off box with a retained census still uses participation and receives the settlement changes. ## The census window and the worst-case boot delay Each call of root's kernel census runs under one deadline, 20 s from its start (`generationCensusRetryFor`). Every pass gets a context that ends at the deadline. The deadline is cooperative cancellation at the implemented checkpoints, plus the rejection of late results: `systemctl show` runs through `exec.CommandContext`; the deleted-image hash checks the context between reads; (fix round 2) the signed-image proof, `readRootFloorBinaryContext`, checks it between reads of `/carlos` and `/carlos.prev` and before each cached identity. A pass that ends past the deadline refuses even if it succeeded, and no pass starts after it. Between checkpoints the work is not cancelled: the opens and metadata operations, the identity cache's directory enumeration (`Names`, a `ReadDir`), a whole identity record's read (`io.ReadAll`), a `systemctl show` already started, one `Read` of an image, and the versions-tree walk and mount checks before the first pass (bounded at 8192 entries). The figures below are therefore estimates, not bounds. On the recover path, with the census refusing every time, the edge's start waits at most for, in order: 1. the guard-exec recover prelude's census (`rootFloorDowngradeIdle`), when a moves floor is armed: 20 s; 2. recover's first controllers census: 20 s; 3. root's pending-controller recovery (`recoverRootExecution`), which runs under its own 30 s context, plus two cleanups of up to 30 s each; 4. recover's second controllers census: 20 s. Then recover defers and exits 0. That is about 90 s, and about 150 s when both cleanups run to their bounds, as estimates: each 20 s window can overrun by the uncancelled work named above. The recover unit's TimeoutStartSec is 10 min. instance-control's start runs its own census under the same 20 s window. ## Tests named in the rows - (a): `TestSystemGateWaitIsKeyedOnClaimsMode`, `TestSystemApplyReadsClaimsModeFromTheUnitEnvironment`, `TestClaimsOnGatePassesAnEdgeThatListensAt22s`, `TestClaimsOnGateWaitsUpTo120s`, `TestClaimsOnGateFailsFastOnACrashRestart`, `TestClaimsOnGateFailsFastOnAFailedOrInactiveEdge`, `TestClaimsOnRollbackGateUsesTheSameRule`, `TestClaimsOnRollbackGateFailsFastToo`, and the two claims-off tests above. - (b) fix 1: `TestRecoverDefersACensusRefusalWithNoJournalOpen`, `TestRecoverFailsACensusRefusalWithAJournalOpen`, `TestRecoverDefersACensusRefusalWithoutAnArmedFloor`, `TestApplyStillFailsACensusRefusalWithNoJournalOpen`, `TestRuling6NoCensusRecoverIsUnchanged`; `TestGuardRecoveryRefusesControllersAfterBegin`, `TestPlan9M8GuardRecoveryRecoversPendingWithoutJournal` and `TestSystemGuardRegionHAKeepsFailedCompletionEvidence` are unchanged from main: deferral is census-only (fix round 1), so a live or unreadable controller and a failed completion still fail as they did. - (b) fix 2: `TestRecoverCensusAcceptsADeletedExeOfTheCurrentOrPreviousBuild`, `TestRecoverCensusAcceptsTheEdgesDeletedExeInSupervise`, `TestRecoverCensusRefusesOtherDeletedExes`, `TestRecoverCensusRefusesADeletedExeNotOwnedByRoot`, `TestRecoverCensusStillChecksReferencesOfAnAcceptedDeletedExe`. - (b) fix 3: `TestRecoverCensusRetriesATransientRefusalWithinTheWindow`, `TestRecoverCensusRetriesOnlyTaskLevelTransients`, `TestPlan14GenerationCensusRetriesWholeObservation` (unchanged). - (b) fix 4: `TestRecoverCensusRefusalsNameCheckUnitCgroupAndExe`. The census tests drive the production service lookup over a fake `/proc`, replacing only `systemctl`, the unit files' ownership check, the signed-binary proof and the owner uid, which a non-root test cannot provide. No root-runner test was run for this branch.