# Move any app or instance to another region: Phase 1, a safe single-instance move **Date:** 2026-10-06 **Revision:** 7.24. - **Rev 7.24** lets a move target the home region (moves into home, part 2; plan `docs/superpowers/plans/2026-10-15-moves-home-region-part2.md`, amadan #236). Phase 1 moves between two non-fleet placements, into home only with host claims on (§2). §7.1 refuses a target of home on a console without host claims, or a home whose box sets a region code of its own. §16 F2 records home as a target as delivered by that plan. Paul's rulings of 2026-10-15, decided: **R1**, home targets only under host claims; **R3**, `--to` takes the home's configured name, with no `home` keyword, and `carlos move back` stays a follow-on; **R5**, no protocol version bump (a home box without the new T3 fails T3 and waits, fail closed; boxes roll before consoles); **R7**, a home target may be the console's own box (the records and the claim are durable, so a box lost mid-landing delays recovery and loses nothing). R2(a) and R6 bind the rehearsal runbook, not this spec. - **Rev 7.23** records Paul's ruling M4 (2026-10-10, moves Plan 5 delta pre-flight after claim Plan 10): **every close is record first, then the claim's release.** Rev 7.22 released the claim to `serving` first and cleared `move` second (§7.5, §19 K8). Claim Plan 10's move-recovery grant closes the record first, and a serving claim with a live move record can never acquire that grant, so the two orders could not both stand. Each close (the commit's, a rollback's, a revert's, an abort's) now checks that no obligation of the op remains, CAS-clears `move` into `last_move`, and then releases. A crash between the halves leaves the record at rest and the claim still under the op with nothing owed; the box passes are driven by records with a live move and do not come back, so the console's recovery pass releases from `last_move`. Amended: §7.5, §7.8 finalization, §7.9 step 5, §7.11, the `reverting` table's `opened` row, §14.2.2's model row and §19 K8. **The model** (`internal/sim/hostlife`, moves Plan 5 Task 7 fix round 1) closes the same way: a box's close CASes the record (`last_move`), and on every third step stops before the release; the recovery pass releases from `last_move` (`fixRecord`). A new invariant, K8, fails any move's release whose record still carries the move, and a new rule, `record-first-close`, whose mutation is rev 7.22's claim-first close, is caught by it (1,060 / 3,000 seeds, first seed 1; no witness). The 3,000-seed sweep of every rule otherwise matches its expected outcomes. The `claim-cas-opid` witness is restaged: its stalled goroutine is now the source between the record's close and its release (§14.2.5). - **Rev 7.22** answers **joint round 19** (`CODEX-FINDINGS-JOINT-R19.md`, "R19-1", the round's one finding, P2) and fixes the install manifest's keys. **R19-1:** rev 7.21 bound the prepared files manifest to "the holder location and generation" during `local`, but `holder.generation` still names the source's pre-move generation `S` until the probe's admission (§7.14, §9.4), and claim rev 6.30 binds the manifest to the **prepared** generation `P`; a manifest bound to `S` could never be `P`'s baseline and an ordinary cold landing would block. §7.6 now binds the manifest, and places it, under the installation's **prepared destination generation**: `g-` for a forward landing, the prepared return generation `g-` for a dirty rollback; `holder.generation` is unchanged until admission. Plan 2 Task 10's `local` step matches. **`install.json` keys:** §7.6 names the snake_case keys claim Plan 6's `install_contract.go` tags the shared `moves.InstallManifest` with (`snapshot_txid` for the Go field `N`). The model encodes no manifest binding (§14.2.6); no rule changed, gate only. - **Rev 7.21** answers **joint round 18** (`CODEX-FINDINGS-JOINT-R18.md`, cited "R18-n"; not clean: one P2, three P3s, no P1). **R18-1 (P2):** the first files manifest of an installation is not the session-bound publisher's, since no target session exists until `probing`; §7.6 now has the landing build the initial immutable files objects and a create-only **prepared manifest**, mirroring claim rev 6.30 (73b1b9dc) exactly: published by `op.exec` before any session exists, in a phase that permits installation; bound to `{location, generation, op id, installation stamp}`, naming exact versions, at sequence 0, never replaced, retired with its generation; the session-bound publisher starts only after the first admission, takes that manifest by exact version as its baseline and publishes from sequence 1 (region HA spec 1 §5.8's prepared-manifest build, `:809`, is the same rule). **R18-2 (P3):** §7.10's "before the flip" is stated as "the record has not entered `landing`", the flip's record CAS, distinct from the final commit. **R18-4 (P3):** M-C22 no longer promises that nothing is lost; it describes the retained remote data, the pending recovery and the declaration a restart may make (§20). R18-3 is the claim's (§11 there). The model encodes no publication timing beyond installing the files at the installation CAS, which is the prepared manifest's moment (§14.2.6); no rule changed and no sweep was run. - **Rev 7.20** aligns with claim rev 6.28 (4ffe8a0d), which adopts region HA's L1 to L20 at its sections (`CODEX-ANALYSIS-REGION-HA-L1-L20.md` beside it). **The fence reference** (§7.10, moves C7): every transition that relies on a termination fence writes `op.journal["fence."] = {"enrollment","key","sha256"}`; a `loss.` intent cites the same reference; `ReturnToSource` accepts a fence only through it, re-verifying the record; an automatic declaration needs no quarantine revision and no operator (L4). **A source terminated before the flip** (§7.10): the fence converts the `moving` op to `relocating` in one CAS (exec `"none"`, stamp and generation preserved, `relocating` at `pinning` in the source region, every obligation untouched, the lane's `park-replica` queued when not yet owed); in a one-box region it holds at `pinning`, safe but **not recovered**, until region HA enrolls a replacement box; the operator sees `waiting for a box in ` and one alert; after the flip the conversion is refused. This replaces rev 7.19's "the move aborts". **L13** (§7.6, §8.3): the files manifest is claim C3's, §6.2a's and P6-D3's: the session-bound publisher, origin-qualified manifest references, files reachability independent of the database baseline. **Scope ruling:** Phase 1 is for one-box regions; a move into or out of a region with more than one eligible box is refused at the door with new copy (M-C21, pending Paul) until region HA's own model and plan cover multi-box; the model stays one box per region (§14.2.6). The model gains rule `fence-converts-moving` and witness `wFenceConvertsMoving` (§14.2.5). - **Rev 7.19** records Paul's rulings of 2026-10-06 on amadan #236 as claim rev 6.27 (989ee349) §1.1 states them, and folds in region HA's requests L14 to L17 (spec 1 rev 5.4, branch `design-region-ha-rev5`, `2026-10-04-region-ha-box-placement-design.md` §9). New §0.5. **P7, cold-only admissions:** until a reviewed Litestream fix (#234) makes a clean-close certificate safe, every admission is cold, no certificate is issued, and nothing in Phase 1 depends on warm; §5.1 step 9's warm mode stays as specified future behaviour. **L14** (§7.10): a target terminated mid-move is taken off the move by the termination fence, as M16's evidence, automatically, under the before-commit rules. **L15** (§7.1, §7.3, §19.1): a move names its target box in the start CAS, chosen by region HA's `choose()`; readiness, evidence, install and recovery bind to it. **L16** (§5.1, §7.6): the install is one shared routine with relocation, fed by a sealed staged replica or a pinned set of remote versions. **L17** (§6.1, §8, §9.4): every installation writes a files manifest in its new generation; the archive at the cut stays; timer freezing and draining apply to `relocating` holds. **L18** (§14.2.6): the model stays at one box per region plus replacements; the two-box extension is stated as a limit, not done here. Nothing in the protocol of rev 7.18 changes otherwise. Conflicts and claim changes these raise are listed in `region-ha-conflicts.md` (the coordinator's record) for platform-1e. - **Rev 7.18** folds in **joint round 17**'s two P3s (`CODEX-FINDINGS-JOINT-R17.md`, cited "R17-n"); the round was **clean**, A, B and C ready for moves Phase 1 implementation planning, after seventeen rounds. Claim rev 6.26 (98b45332) fixes R17-2 on its side; the code is main at 67547f5. **R17-1:** the model's INV-23 now checks scope membership for every reference a scoped park archives or deletes, in the manifest it builds and in a persisted manifest it replays, whatever rules are on; an out-of-scope reference is a violation and is never acted on (rev 7.17 checked candidate keys, and only with the confinement rule off); the operations' own parks run under a scope of `holder.location`; `TestForeignManifestReplayRefused` replays a manifest naming another incarnation's key under a valid scope. **R17-2:** the scope grammar's `moves/` form is used by a delete's park of the staging namespace (§5.2) and by an at-rest sweep, each while no operation stages there, as claim rev 6.26 says. No rule change (107 rules; 73 witnesses): the gate alone, per §14.2.1; the mutation tables stand at rev 7.17's. - **Rev 7.17** answers **joint round 16** (`CODEX-FINDINGS-JOINT-R16.md`, cited "R16-n") and is written against **claim spec rev 6.24** (0b821896: the legacy identity, scope namespaces, the narrowed born-covered) **and rev 6.25** (413b12cb, which answers this revision's four notes: §5.2 now has a delete park only `holder.location` plus `moves/`, never the whole host prefix, as the model does); the code the model aligns with is **main at 67547f5**, which carries platform-1e's R16-1 and R16-3 fixes (c7e595b, "park, hostclaim: scopes stay in their own namespace; the legacy session-0 loss identity") and the first-admission fix (a created incarnation's first admission declares nothing; note 2 was an E bug); §22.23 has every disposition. **A park executes strictly within its scope** (R16-2): the model's sweep groups residue by the location it belongs to, an incarnation's own, the legacy `instances/` (never a child incarnation) or `moves/`, and queues one chore per location with that scope and its archive incarnation; the archive names the incarnation; every scoped chore's listing is confined to its scope and INV-23 checks every key a chore would archive for membership, not only the location's syntax; the delete parks the deleted incarnation's own location and the staging namespace, never the whole host prefix. **The legacy identity is session 0** (R16-3): an adopted host's pre-claim writer is `{session: 0, generation: "" (@root), box: holder.writer_box}`, valid in both `writes_lost` and `additional_writes_lost`; every other declaration has a session above 0, and a declaration always names a box, so a created incarnation's first admission, whose predecessor has no writer, declares nothing (INV-24); the model no longer gives adoption session 1; `TestLegacyLostTargetReturn` walks adopt → a move with `src_covered` false → a cold target admission → the target declared lost → the return declaring the target's session and the source's cut session 0. **An instance park never touches staging** (R16-1): `TestStagingSurvivesLegacyPark`. **Born covered, narrowed** (6.24 item 5): only an unscoped `park-replica` or a park scoped exactly to `holder.location`. Rule `scope-membership` (107 in all); witness `wSweepScopeMembership` (73 in all). Evidence under the policy of §14.2.1: the gate, the 30,000-seed baseline, one 3,000-seed sweep. - **Rev 7.16** answers **joint round 15** (`CODEX-FINDINGS-JOINT-R15.md`, cited "R15-1") and is written against **claim spec rev 6.23** (876d6908, §2's park scope: the location grammar, the archive incarnation, `evidence_at` always the original evidence or fencing bound, one deadline for every scope); E is merged to main as 99084d7, F merges next; §22.22 has the disposition. **Every remote chore carries the claim's scope from its queuing CAS** (R15-1): `location` exactly one of `instances/`, `instances//`, `moves//` (a move's staging or a rollback's return lane) or `moves/` (the staging sweep), no trailing `/` (INV-23); `incarnation` the location's own, the staging op's holder incarnation or, for a free host's `moves/` sweep, the newest retired one; **`evidence_at` the original bound**, the cut's stop for the commit's source park, the release or the declaration for the target region's, **the seal bound for a staging lane** (the time after which no staging writer, its lease holder included, can still write: the seal plus the staging lease), the listing for a sweep, the publication for a generation chore; **never the cleanup's start time**. **One deadline for every scope, staging included:** acknowledge only after a listing later than `evidence_at + H`; there is no separate staging contract (rev 7.15 bound a staging park at cleanup and then kept the seal's quiet time instead, which Astra's example exposed: sealed at 100, deadline 560, a cleanup first running at 600 must acknowledge at once, not at 660). Witness `wStagingScopeDeadline` (delayed cleanup through an agent restart; INV-9 under the mutation); `wParkScopeSurvivesRestart` takes its deadline from the persisted scope. Rules unchanged (106); witnesses 72. Evidence under the policy of §14.2.1: the gate, the 30,000-seed baseline, one 3,000-seed sweep. - **Rev 7.15** answers **joint round 14** (`CODEX-FINDINGS-JOINT-R14.md`, cited "R14-n"), against claim spec rev 6.22 (0bf2f49f) with platform-1e's answers to the six rev 7.14 notes, E f0127321 and F d7621a29, both MERGE-READY there; §22.21 has every disposition. **The start restores what the run walk selected** (R14-1): the open state carries the restore mode, the marker's generation through `generation.json` or a run generation rooted at `[1, 1]`, so the start and the recovery oracle use one selection; witness `wIntermediateBaseline` (R's first upload pending at its crash, an empty R2, R's upload landing). **One `writes_lost` plus `additional_writes_lost`, unique by `(box, session)`** (R14-2): the source's cut session joins the list under its persisted identity, the separate representation is gone, one alert per declaration, INV-21; witnesses before and after the target's first admission. **The horizon keeps its signature-age allowance** (R14-3): the park scope persists `request_limit`, and every acknowledgement waits `evidence_at + H` with `H` the signature-age limit plus `request_limit`; a park-sweep acknowledges only after a verification listing later than that (platform-1e's answer to note 2). **A running holderless sweep admits another region's chores** (R14-4; `TestJoinHolderlessSweep`). **Every remote chore runs under a persisted scope** (R14-5): generation chores take theirs from the publication, staging parks bind one before their first step, INV-22. Two fidelity limits stated in §14.2.6 (occupancy by current objects; the publication's precondition and trimming aligned with E). Rules `restore-selected-baseline`, `unique-declarations` (106 in all); witnesses `wIntermediateBaseline`, `wNoDuplicateDeclarationEarly`, `wNoDuplicateDeclarationLate` (71 in all). Evidence under the policy of §14.2.1: the gate, the 30,000-seed baseline, one 3,000-seed sweep. - **Rev 7.14** answers **joint round 13** (`CODEX-FINDINGS-JOINT-R13.md`, cited "R13-n") and is written against **claim spec rev 6.22** (0bf2f49f, which adopts rev 7.13's seven notes in rev 6.21 and answers R13-1 to R13-3; rev 6.15 and 6.16 added the persisted park scope and the holderless sweep), the library E on `feat/hostclaim-r713` at f0127321 and the generation selector F on `feat/hostclaim-genselect` at d7621a29; §22.20 has every disposition. **The model's park state is the claim's** (R13-4): every remote chore carries a persisted **park scope** `{location, incarnation, evidence_at, request_limit}`, bound in a CAS before its first step or at the sweep's queuing CAS, so a retry after an agent restart keeps the original evidence time and never restarts the horizon; a free host's at-rest sweep is a **holderless parking op** `{kind: parking, intent: {park_sweep: "v1"}}` with no exec and no holder, carrying only scoped remote chores, which blocks the successor create and frees the claim again in its last acknowledgement's CAS (`FreeAfterPark`); on a serving host the sweep's chores carry `session:`; INV-20, a create never claims a host over an outstanding park. **Coverage, completely** (R13-2): an entry becomes covered in exactly four ways, at birth (a warm entry; an entry admitted under a pending whole-location park of its predecessor's region), by publication (which requires an entry to name the published generation and walks every entry in order, skipping covered ones), by `CoverRegionPredecessors` in the three whole-location-park CASes, and by `CoverParkedPredecessors(region, generations)` in the sweep's queuing and acknowledgement CASes, only for the generations the sweep parks. **The run walk, precisely** (R13-1): from `holder.generation` back to the marker's generation, a link whose generation keeps its objects in another region is passed over, covered or not; a link in this region must be uncovered, and a covered local generation reached as a needed baseline refuses the start and alerts; `restoreCalc` stays the reference and `TestInterruptedCleanReturn` walks R → T → S and R2 → R → T → S. **`additional_writes_lost`** (R13-3): the lost-target return's extra declarations, one per target session since the cut, unique by `(box, session)`, each alerted. Rules `persisted-park-scope`, `holderless-park`, `covered-baseline-refused` (104 in all); witnesses `wParkScopeSurvivesRestart`, `wFreeSweepVsCreate`, `wCoveredLocalBaseline` (68 in all). Results in §14.2.3 and §14.2.5. **The evidence policy** (Paul, 2026-10-06; §14.2.1): each revision runs the gate and one mutation sweep at 3,000 seeds, serially and niced; baselines at 30,000 only; the 100,000-seed baseline and the 30,000- and 100,000-seed mutation sweeps run once, at a final pre-implementation checkpoint, when Paul says so. Rev 7.14's 100,000 baseline ran before the policy and is reported; its deeper sweeps were stopped and are marked not run. - **Rev 7.13** answers **joint round 12** (`CODEX-FINDINGS-JOINT-R12.md`, cited "R12-n") and is written against **claim spec rev 6.14** (dc146058, whose §6.4a has one operative subsection, "Admissions, coverage and retirement (operative, rev 6.14)", and answers R12-4 and R12-6; rev 6.13, b3825827, adopted rev 7.12's five notes) and the library E at 47426cc1 (unchanged since round 12 judged it merge-ready); §22.19 has every disposition. **Coverage is regional and recorded** (R12-3): every admission record carries `region` and `predecessor_region`; `PublishGeneration` walks the uncovered records (a predecessor holding nothing anywhere is covered with no chore, one in the publishing region gets its chore and is covered, one in another region stays uncovered with no chore, and one that is the live generation again is left uncovered too, a reading against item 5's first bullet: covered, nothing would retire it later), every cold admission publishes, a prepared generation's included, and **`CoverRegionPredecessors`** runs inside the CAS that queues a region's whole-location park: the commit's park of the source region, a rollback's park of the target region, the console's lost-target release (note 5's discharge, adopted). Two readings for the claim: a record admitted while such a park is already queued for its predecessor's region is **born covered**, and the sweep's CASes cover the records whose predecessors it parks. INV-17 checks, as a whole-location park completes, that no record whose predecessor was there is uncovered. **The loss oracle matches the declared identity** (R12-1): acknowledgements carry the writer session and generation of the process that gave them, a `writes_lost` record exempts the writes of the session it names and no other, the source-cut declaration stays separate, and a declared-lost target's **every** session since the cut is declared, certified or not (a probe session that certified and a warm continuation that did not are two, and no return lane carries either's writes); a two-session witness (INV-18). **The probe deadline is checked before each start and before readiness is accepted** (R12-2): an expired attempt never starts, a late readiness is not accepted, INV-19 at every probe acceptance, and `TestProbeWindow` walks k = 29, 30, 31. **One operative rule set for admission** (R12-4): §7.14 states it, mirroring claim rev 6.14's structure, and the passages that disagreed are corrected or marked history: the restore contract's "after installed and probed", the dirty-return region rows, the start table's failed-host marking (a serving wake's only; a probe never marks the host failed), the recovery limit (nine attempts, R12-5), and `holder.admissions` retention (conservative, no compaction). The absent-marker sweep fault has its own rule and mutant, `absent-marker-origin`, so `wOriginSwept`'s evidence no longer rides on `baseline-live`. Rules `foreign-region-uncovered`, `cover-at-park`, `session-scoped-loss`, `probe-window`, `absent-marker-origin` (101 in all); witnesses `wTwoSessionDeclaration`, `wRollbackBaselinePreserved`, `wCommitCoverage`, `wCommitInterrupted`, `wCommitReplayed`, `wProbeWindowAt` (65 in all). Results in §14.2.3 and §14.2.5. - **Rev 7.12** answers **joint round 11** (`CODEX-FINDINGS-JOINT-R11.md`, cited "R11-n") and is written against **claim spec rev 6.12** (b202ea4e, which answers rev 7.11's eight notes and R11-2 to R11-5) and the library E at 34224a29; §22.18 has every disposition. **`holder.admissions`** replaces `op.journal.writers`: the record of admissions lives in the holder, so it survives every operation, release and new operation, trimmed to eight covered records and never an uncovered one or the current predecessor. **An admission's predecessor is always the last record there** (session, box, generation), never the generation an operation chose for a restore (R11-4): a clean return resets `holder.generation` to the source's for the restore, but the predecessor is still the target's writer (INV-16). **`Coverage`** `""` | `sealed` | `clean-return` replaces `BaselineCoversPredecessor`: only `""` with no certificate records `writes_lost`, and the record always names its coverage; a clean rollback with `src_covered` declares nothing. **`PublishGeneration`** replaces `QueueGenerationRetire`: the one retirement path, bound to the observed operation and phase, legal while serving and inside any operation a writer runs in (R11-5), chores with `op = op.id` inside an operation and `session:` while serving, only of predecessors in `holder.admissions`; `""` and `"@root"` are one origin, so nothing retires an adopted host's replica before a cold admission has published a successor (R11-2). **The sweep reads an absent `generation.json` as the origin** (R11-1), as the residue oracle does, so an unfinished admission's chain protects an adopted host's replica; witness `wOriginSwept`. **The final oracle runs the recovery admission for real** when the serving session is uncertified and judges by what it declared; an admission that cannot run is counted as an unverified prospective declaration, never exempted (R10-6). **A probe attempt's five app restarts share its 30 s readiness window** (§7.7): the fifth failure or the window ends the attempt and the operation counts one of its three budgeted attempts; a probe never marks the host failed. §5.1, §7.8 and the authoritative-region table carry no instruction left that assigns `holder.generation` outside a cold admission's CAS or the clean return (R10-3). Rule `clean-return-identity` (96 in all); witnesses `wCleanReturnIdentity` and `wOriginSwept` (59 in all). Results in §14.2.3 and §14.2.5. - **Rev 7.11** answers **joint round 10** (`CODEX-FINDINGS-JOINT-R10.md`, cited "R10-n") and is written against **claim spec rev 6.10** (3f916b05, with rev 6.9.1's answers to rev 7.10's readings); §22.17 has every disposition. **The admitting CAS comes first** (R10-1): before the purge, the restore, the app start and readiness, so nothing the app acknowledges is ever outside a writer session; a session belongs to the **agent's replicator**, app restarts under the same agent stay in it, a new agent process is a new session and stops a running app it finds; failed starts are governed by a **retry policy** (five attempts under one session with a doubling backoff from one step, then the host is marked failed and gets no new session until an operator clears it). **A cold admission's generation is unfinished until `generation.json` names it** (R10-2): its journalled predecessor stays live, no chore retires it, and the sweep never parks it; the retire chores are queued by the marker's publication; a marker lagging behind `holder.generation` that names the predecessor of the latest admission, or of a chain of unfinished ones, is the normal state and no alert, any other disagreement leaves the location untouched and alerts. **§7.5, §7.6, §7.8 and §7.9 are rewritten around §7.14's protocol** (R10-3): the commit and the `probed` CASes move no generation; `holder.generation` is always the latest admission's. The admission is **API-shaped like the library's `Admission`** (R10-4): `KeepStamp` only when `holder.stamp > holder.session`, warm with an empty `NewGeneration`, refused in `creating`, `deleting`, `parking`, `purging` and `placing`, `BaselineCoversPredecessor` for a seal that covers the stopped predecessor (compared against the seal's own position; the journal says `baseline: sealed`, no `writes_lost`), the empty-generation ancestor restore only for the latest journalled unfinished admission. Serving-time chores use `op = "session:"` and an adopted host's root generation is `"@root"` (R10-5). **Disk-loss faults exempt nothing** (R10-6): every exemption comes from a durable `writes_lost` declaration, alert delivery is checked separately (INV-14), timers are never exempted by the database rule, and a write the remote holds is never declared lost while its writer's box is with us (INV-15). **Startup PUTs are session evidence** (R10-7): the first upload and the anchor enter the ledger, a retry after the client's wait makes the session uncertain for good, server completion and client response are distinct, and the bytes of a retry are the original's. **The budget prices the new protocol** (R10-8): a restore per probe attempt, the admission's overhead, a fresh generation's full upload on a retried probe; the 2 GiB example runs to 117.9 s and admits one attempt, the review's 2.5 GiB is refused at 131.2 s; the replication interval is the normal window a failed upload extends. **Counters are structured** (R10-9): the baseline logs `COUNTERS {json}`, the generator fails on a missing field, §14.2.3 and the change note are regenerated. Two gaps the rerun found beyond the review: a rollback is **clean only when the source's own replica covered its stopped database** at the seal, else the return carries the seal, whether or not the target wrote (§7.8, rule `seal-return`, INV-15); and a pre-flip round never forces a sync on a session whose start is unfinished. Rules `admit-before-start`, `baseline-live`, `seal-return` (95 in all); witnesses for the acknowledgement before admission, the parked baseline, the first-upload and anchor retries and the uncovered return (57 in all). Results in §14.2.3 and §14.2.5. - **Rev 7.10** answers **joint round 9** (`CODEX-FINDINGS-JOINT-R9.md`, cited "R9-n") under Paul's ruling to **simplify**, and is written against **claim spec rev 6.9** (f6180ae4); §22.16 has every disposition. **A box never reuses its local database.** Every admitted writer, warm or cold, in every phase, purges the local database (an error-returning purge; if it fails the writer does not start) and restores from the certified remote: warm, the same generation up to the certificate's `max_txid`; cold, the predecessor generation's remote. Lineage, provenance and the `warm-provenance` rule are gone (R9-1 closed by construction; its clean-close stale-database scenario is the witness `wStaleLocalAfterClose`). Warm continuation needs only the certificate for the observed session with a matching `max_txid`, and **any `op.exec` box** may continue (claim rev 6.8.2 reverses rev 7.9's same-box reading). **`holder.writer_box`** is set by every admission and survives operation replacement; a cold admission of an uncertified session records **`writes_lost {box: observed writer_box, generation, session, remote_max_txid}`** in its journal entry and the box alerts, so a crash with acknowledged, unuploaded writes loses them declared, never silently (R9-2). **Only warm and cold survive** (R9-3): a first install is a cold admission into an empty generation; `holder.generation` is the only authoritative generation in every kind and phase; a probe or recovery that opens replication is a cold admission that moves it; an operation's prepared generation is opened once, by its probe's cold admission, and never re-entered; a fresh generation roots at the writer's full first upload, made after the CAS and before its marker; a failed cold start leaves an empty, unpublished generation for the sweep and the next attempt is cold again. Admission happens after readiness and before any upload, and is **bound to the observed `op.id` and `op.phase`**, serving's `""` compared literally, refused in deleting, parking, purging and creating (R9-4; INV-13). Close evidence is **client-observed success** for every request class, level-0, compaction, snapshot and retention, with a sticky uncertainty for a timeout or a lost response (R9-5). §14.2.5's tables are regenerated from the retained verdicts (R9-6). The spec states plainly how today's activator differs (§7.14: production keeps the warm cache and restores only when the local copy is absent; this is a behaviour change, costing a restore per wake and the unuploaded writes on a crash). Rules `remote-baseline`, `writes-lost`, `admit-bind-op` (92 in all); witnesses for the stale local database after a clean close, the silent loss, the stale closure during a delete, a maintenance retry and a lost response (52 in all); the whole-start-check witness is retired, subsumed by regional restores. Results in §14.2.3 and §14.2.5. - **Rev 7.9** answers **joint round 8** (`CODEX-FINDINGS-JOINT-R8.md`, cited "R8-n") and is written against **claim spec rev 6.8** (5462d293), whose rewritten §6.4a, writer sessions and epochs, answers R8-1 to R8-4, and **rev 6.8.1** (a963d98d): only `op.exec` may open replication, in every op kind; §22.15 has every disposition. **R8-1 (P1):** rev 7.8's `closed.json` was consumed by deleting the version the writer read, which is not one-use under S3's semantics (an exact version delete removes it and exposes an older version; a delayed or replayed PUT recreates it); the certificate is now **per writer session**, `closed/.json`, never deleted, and what makes it one-use is the **admitting CAS** that moves `holder.session` forward: a certificate names a dead session from then on, whatever its stamp and `max_txid` say. **R8-2 (P1):** rev 7.8 checked replication state only at an ordinary wake; **every** path that opens replication is an `AdmitWriter` now (a committed landing's serving, a restarted probe, a reverted source, an opened rollback recovery, a push's writer), and a cold admission runs **inside** the operation it interrupts, keeping its kind, id, journal, participants and obligations (§7.14's phase table says which box may be admitted in which phase; `op.exec` is handed over before any probe or landing write). **R8-3 (P2):** a created incarnation's first wake is a cold admission into an **empty** generation: no snapshot `[1, 0]`, no anchor `[0, 0]` (the pinned encoder refuses a zero TXID, `TestFact_R8_3_EmptyBootstrap`); the app starts first and may create its database, and the first transaction's file is the first upload. **R8-4 (P2):** the warm database is the cold baseline only with **provenance**: a lineage file naming the journalled predecessor's `{generation, session}`, the predecessor's box being this box, and an integrity check; re-initializing a stamp record never writes lineage. **R8-5 (P2):** the clean close's evidence is every request of the writer's lifetime, level-0, compaction, snapshot and retention, with a sticky uncertainty flag for a timed-out original whose retry succeeded. **R8-6 (P3):** §14.2.5's retired-door rows separate the historical results from the current ones. Rules `writer-session`, `admit-everywhere`, `exec-writer`, `warm-provenance`, `valid-ltx`, `close-evidence`; witnesses for a dead session's certificate replayed, the collision during a committed landing's serving and during a probe restart, a non-exec writer, a warm database with a foreign lineage, an empty first wake, and a retry that succeeds while its original stays unresolved. 90 rules, 49 witnesses. The model faults this round found before the final numbers (a stalled worker replaying a finished build, the carried files untagged, a source that lost its disk during a rollback, the biased schedule's unbounded hold in the calm phase, a lost target's unfinished admission blocking the rollback, every close outside `serving` dropped) are in §14.2.2. Results in §14.2.3 and §14.2.5. - **Rev 7.8** answers **joint round 7** (`CODEX-FINDINGS-JOINT-R7.md`, cited "R7-n") and is written against **claim spec rev 6.6** (b10547e1), whose new §6.4a, writer epochs, answers R7-1; §22.14 has every disposition. **R7-1 (P1):** the hazard rev 7.7's note raised is real and now closed: litestream's level-0 PUT is unconditional and its key is prefix, level and TXID range only (`TestFact_R7_1_UnconditionalPut`), so a dead writer's in-flight upload for `[N + 1]` can replace the file the next writer uploaded for the same TXID, or make its upload loop skip it. **Two writers whose uploads could overlap never share a generation** (§5.1 step 9, §7.14): a writer that stops cleanly drains its uploads and writes `closed.json {stamp, max_txid}` once; a wake continues the generation only if that marker exists, its `max_txid` is the generation's remote maximum and its stamp is the box's own, and the writer consumes it as it opens; anything else is lost replication state and a **cold recovery**, an installing CAS into a fresh generation (new `holder.generation`, new `holder.stamp`), built under stop evidence from the warm database or the old generation restored, published, then served, with the old generation parked as a chore. Rules `recovery-generation`, `closed-marker`, `close-waits`; witnesses for both completion orders, the initialization/remote-position race, a stale marker, a crash before the marker, and a clean close that must wait. **R7-2 (P2):** the cold-restore oracle checks required writes by identity and compares the state with the replay of exactly the writes the restored files carry; the per-key mask is gone, with the review's counterexample as a witness. **R7-3 (P2):** every retention path keeps the newest level-0 file (`newest-l0`, witness with the local position ahead of the uploads). **R7-4 (P2):** a build is a durable intent, then acknowledged artifacts, then a completed manifest, and `generation.json` is published only from the completed state (§5.1 step 2, §7.6; INV-11, `build-completion`, witness with crashes between the steps). **R7-5 (P3):** the Store gate awaits each maintenance event it claims, compares row contents with updates and deletes, and forces a restore through compacted continuation files (§14.2.1). The model's full files replace the state they are applied to and carry every write they hold, which the rebuilt oracle needed. 84 rules, 42 witnesses (two rules the sweep no longer reaches at 3,000 seeds under writer epochs, `evidence-whole-tree` and `timer-freeze`, gained witnesses; the two door witnesses are retired). Results in §14.2.3 and §14.2.5. - **Rev 7.7** answers **joint round 6** (`CODEX-FINDINGS-JOINT-R6.md`, cited "R6-n") and is written against **claim spec rev 6.5** (d36adcfa), whose new invariant answers R6-4: `holder.stamp` always names the installation `op.exec` should run, so every CAS that changes `op.exec` writes it, a new installation's new gen or an existing installation's own stamp from the op journal; §22.13 has every disposition. Round 6 judged the claim ready for planning and found no P1. **R6-1 (P2):** the model's restore oracle wrote anchors into the system it checked and granted a position before the anchor PUT succeeded; the restore is now a pure calculation (`restoreCalc`) that every oracle uses, and the activator's initialization (`restoreInit`) opens replication only once the anchor PUT is **acknowledged**, reading the position from the listing, never from the restored TXID (§5.1 step 7; rule `anchor-acked`, witness: the anchor PUT held across attempted opens); every oracle asserts its purity (buckets, versions, queues, anchors, console, random draws unchanged), and every number in §14.2 is rerun. **R6-2 (P2):** `litestreamfacts` asserts exact facts and fails on any alternative (the newest level-0 file retained by identity, the next TXID exactly `N + 1`, every row, the original continuation gone, the rooted restore's exact refusal), and gains the missing integration gate: one anchored lineage through the production Store shape with compaction, snapshot rotation, time-based level-0 expiry, disk loss and repeated cold restores, exact positions and contents asserted (§14.2.1). **R6-3 (P2):** the `restored` phase **verifies** the database `building` produced instead of restoring it again, T11 prices regional S3 and the disk separately and the verification pass too, and the worked example is one set of numbers in the spec, the note and the test (2 GiB at 10 / 250 / 500 MiB/s: 53.6 s fixed, one attempt; §7.3, §7.6). **R6-4 (P3):** the return to source encoded as claim rev 6.5 states it (the flip keeps the source's stamp in the journal, the return CAS writes it back, a lost local record re-initializes from it, artifacts are tagged when the new stamp activates, `retire-artifacts` parks by tag; §6.4, §7.8; rule `return-stamp`, witness: a dirty return with a lost local record). **R6-5 (P3):** the anchor's encoding inputs, the header timestamp included, are persisted in the install manifest so a replayed PUT repeats identical bytes (§5.1 step 2; `TestFact_R6_5_AnchorReplayBytes`). Compaction in the model keeps the newest level-0 file, as litestream's retention does. 79 rules, 34 witnesses. Results in §14.2.3 and §14.2.5. - **Rev 7.6** answers the moves side of **joint round 5** (`CODEX-FINDINGS-JOINT-R5.md`, cited "R5-n") and is written against **claim spec rev 6.4** (697f0f9f), which answers the claim side of R5-4 (rev 6.3: `holder.stamp`, the record's initialization, two named retirements; rev 6.4: the stamp captured at work selection, `retire-artifacts` never moves a stamp, the legacy location's children); §22.12 has every disposition. Round 5 found that rev 7.5 described the pinned libraries from their documentation; rev 7.6 starts from **their observed behaviour**: `internal/sim/litestreamfacts/` drives litestream v0.5.14 and `superfly/ltx` v0.5.1 as pinned and asserts each fact this spec relies on (§14.2.6). **R5-1 (P1):** a snapshot-only install restarts replication at TXID 1 (fact R5-1a) and a later cold restore loses the new writes (R5-1b); the install now writes a **level-0 anchor** `[N, N]` beside the snapshot, which the pinned open takes as its position (R5-1c, d); `-txid` is follow-only (R5-1e). **R5-2:** the pinned writers set `HeaderFlagNoChecksum`, so there is no post-apply checksum to compare; verification is what the format supplies, the per-file CRC-64 trailer, plus SQLite's integrity check and the required TXID (§5.1 step 5); the whole-state checksum is gone from the model. **R5-3:** snapshot rotation and retention are specified and modelled; the restore roots at the **newest current successor snapshot** of the named generation at or past the pinned one, and only at the pinned version when no successor is current (R5-3d, e). **R5-4:** claim rev 6.4's stamp lifecycle, encoded exactly (§6.4): `holder.stamp` written by every installing CAS, the local record initialized only from a fresh read under the guard, no record refuses, `retire-artifacts` against `retire-installation`, and a dirty rollback's install carries its own stamp. **R5-5:** the sealed snapshot is **built on the target** from the staged replica; T11 prices the final delta within the bar and the target's restore, snapshot and anchor at the local rate, and `budget_test.go` shows a 2 GiB database over 10 MiB/s admitted that way and refused as a cross-region transfer (§7.3, §7.6). **R5-6:** the retention fixture populates 2 to 33 and asserts the injected incarnation absent. LTX keys follow the pinned S3 client's layout, `//-.ltx`. Two new rules, each with a witness (`restore-anchor`, `root-advance`). Results: 30,000 and 100,000 seeds clean; 77 rules, 63 caught alone, 11 in stated combinations, 2 liveness, 1 unreachable, none unexercised (§14.2.3, §14.2.5). - **Rev 7.5** answers the moves side of **joint round 4** (`CODEX-FINDINGS-JOINT-R4.md`, cited "R4-n") and is written against **claim spec rev 6.2** (30d572c4), which answers the claim side (R4-2 installation stamps, R4-3 the region-qualified live set, R4-4 retirement under purge) and the moves session's three questions (rev 6.1, 79a5243c); §22.11 has every disposition. **R4-1 (P1):** the restore contract is one implementable contract (§5.1): the sealed transfer is a verified final snapshot, `generation.json` names it by key, version, TXID and checksum, and the restore is litestream v0.5.14's planner from that snapshot, overlap allowed, failing closed on a missing snapshot, a gap or a checksum mismatch; §7.6 produces what the restore consumes. The model's databases are key-value maps with inserts, updates and deletes, its replica objects are LTX files with ranges, page images, checksums and compaction, and its restore is the planner, so a compacted continuation and a genuine gap each have a witness (`restore-overlap`, `restore-gap-closed`). **R4-2:** installation stamps as claim rev 6.2 states them, the op counter gone (§6.4); a stale writer across a retire-and-reinstall is the new witness (`fence-stamp`). **R4-3:** the live set is `{authoritative region, location, generation}`, with a phase table naming the authoritative region (§5.1), and a witness for an old regional PUT after adoption (`live-set-region`). **R4-4:** retirement under the purge's own claim, encoded. The random generator gains marker-hold and targeted-freeze schedules, a retention fixture and two successful-move oracles, so `generation-conditional`, `declare-lost-compute` and `sweep-namespace` are now caught by the sweep as well. Results: 30,000 and 100,000 seeds clean; 75 rules, 60 caught alone, 12 in stated combinations, 2 liveness, 1 unreachable, none unexercised (§14.2.5). - **Rev 7.4** answers the moves side of **joint round 3** (`CODEX-FINDINGS-JOINT-R3.md`, cited "R3-n") and is written against **claim spec rev 6** (3ea3f90c), which answers the claim side (R3-2, R3-3, R3-4); §22.10 has every disposition. On the spec: **R3-1**, the restore contract is an algorithm now (§5.1): `generation.json` names the installed snapshot by version id and TXID, and the restore replays the LTX chain that continues it, skipping what it already covers and refusing a conflict, never guessing; **R2-5**, a recovery install's `configured` phase renders current inputs and is never gated on `move.inputs` (§7.6, §7.8); **R3-7**, T11's permitted probe attempts depend on the admitted budget, and a test shows rev 7.3's formula refused every default move (§7.3, `budget_test.go`); claim rev 6's installation-scoped retired marker, ordered orphan cleanup and namespace sweep are taken into §6.4 and §19.4. On the model (§14.2): restore is a chain, not a union; the local fence is per installation and a deterministic UK→US→UK round trip serves on UK again; orphan cleanup is a plan with pauses; the sweep discovers locations from the namespace with the 32-entry cap modelled; a public request to a landing target before the commit is INV-5 (**R3-5**), which gives `commit-before-open` its own violation; `mutate.sh` fails on orchestration errors and missing witness output (**R3-6**); every witness names its failure and the gate proves it under every proper subset of its mutation set. Three new rules (`fence-by-installation`, `orphan-order`, `sweep-namespace`), each with a witness. Results: 30,000 and 100,000 seeds clean; 71 rules, 57 caught alone, 11 in stated combinations, 2 liveness, 1 unreachable, none unexercised (§14.2.5). - **Rev 7.3** adds **deterministic witness scenarios** to the model (§14.2.2, `witness_test.go`): one fixed, hand-scheduled scenario per rule the random sweep catches rarely or only in combination, 22 scenarios for 20 rules. Each forces exactly the interleaving its rule exists for, under a quiet RNG that injects nothing, holds with the rules in place, and violates an invariant under the rule's mutation, alone or in the stated combination. Witnesses never draw from a seed's RNG, so adding one reshuffles nothing, and the whack-a-mole of rev 7.1 (`lost-return-park`) and rev 7.2 (`generation-conditional`) ends. `mutate.sh` runs them beside the sweep and reports both; a rule counts as caught if either catches it. Nine rules are caught alone by their witness, `generation-conditional` and `declare-lost-compute` among them, which the sweep misses at 100,000; eleven hold alone and violate in their combination, which is the redundancy the combination table states, shown rather than inferred (§14.2.5). Found on the way: the purge drain iterated registrations in Go map order, the model's one source of nondeterminism; it drains in create-id order now and two runs agree exactly. Results: 30,000 and 100,000 seeds clean; 68 rules, 53 caught alone, 12 in stated combinations, 2 liveness, 1 unreachable, **none unexercised**. - **Rev 7.2** is written against **claim spec rev 5.2** (465fee96), which adopted CC22: the creator's claim CAS is `If-Match` the ETag read at registration (`If-None-Match: *` when the claim was absent), a 412 restarts the create from step 0 with a new registration, the purge fence is released by writing a free claim, and a new **C5a** says a claim object is never deleted. The model encodes that text: the 412 restarts the create instead of abandoning it, and a new rule, `claim-never-deleted`, lets the mutation delete a fence that found no claim; dropped, a paused creator's `If-None-Match: *` passes again (§14.2.2, §14.2.5). Rev 7.2 also closes the gap rev 7.1 left: a **biased schedule** in half the seeds makes the target's app write in the settle window, fails the settle check, and stops the target the moment its first return PUT is out, so declare-lost meets a partial return lane. `lost-return-park` (model 27) is caught alone (3 / 3,000, 32 / 30,000), where rev 7.1 reached it in no seed of 300,000. Results: 30,000 and 100,000 seeds clean; 68 rules (§14.2.5). The reshuffle this schedule causes moves two rare rules back: `declare-lost-compute` is combination-only again (0 / 100,000 alone, 3 / 100,000 with `watchdog-independent`), and `generation-conditional` (model 29) is unexercised (0 / 300,000; 2 / 100,000 on rev 7.1), listed as such in `expected-outcomes.txt` as rev 7.1 listed `lost-return-park` (§14.2.5). - **Rev 7.1** is written against **claim spec rev 5.1** (64eec93b), which adopted CC20 (the sweep lists `moves//`) and answered CC21 with a **purge fence** on the host claim instead of the creator-side check rev 7 proposed: S3 cannot condition one object's write on another's version. The model encodes the fence, a biased schedule that pauses a creator between its registration re-read and its claim CAS, and finds one more condition the fence needs: the creator's claim CAS must be conditional on the claim ETag it read **when it registered** (CC22), or a fence placed and released while it slept lets it through (§14.2.5, §19.4). It also fixes a moves-side fault the 100,000-seed run surfaced: `generation.json` is a reused key, and an earlier push's PUT completing late put an older generation back and left recovery stuck; the marker is now written If-Match the one it supersedes (model 29, §5.1). Results: 30,000 and 100,000 seeds clean; 67 rules, 51 caught alone (the fence and the ETag condition among them), 12 in stated combinations, the re-read alone shown insufficient (287 / 30,000), and `declare-lost-compute` now caught alone (§14.2.5). The mutation harness runs one prebuilt test binary under `$TMPDIR`, with no copies of the source (§14.2.1). - **Rev 7** was written against **claim spec rev 5** (4a458a9c on `design-replica-ownership`), which adopted CC15 to CC19. It: - closes the model's last gap: a **biased schedule** beside the uniform one recreates a host where it last served while the previous incarnation's requests are still in flight, and races door listings against the delete and the recreate. `location-per-incarnation` (claim C3a) is now caught alone, and both door rules with it (§14.2.5); - re-encodes claim rev 5: the at-rest sweep under an obligation the sweeping box owns, registration leases, orphan records at any time, `holder.generation`, signatures taken at the deadline check with an SDK retry that must not re-sign; - asks for two more claim changes, CC20 and CC21 (§19.4). Earlier revisions: - **Rev 6** (7ac666fe) answers joint Astra round 2 (`CODEX-FINDINGS-JOINT-R2.md`, df41156c; cited "J2-…"), whose three verdicts were "not ready", and is written against **claim spec rev 4** (a3c86fa5 on `design-replica-ownership`), which answers the claim-side findings. Rev 6: - **rebuilds the model to encode exactly what the specs say** (§14.2): restore and the start check no stronger than §5.1; record-first transitions split from their claim halves, with crash points between; independent processes for the host guard; a door listing with O1 and O2; creates and purges as concurrent console plans over two hostnames; requests that complete after `H`; a final oracle that only reads; and a mutation harness that counts only a parsed invariant violation as a detection (J2-R2-9, J2-R2-10); - specifies this spec's share of J2-R2-1: **operation-scoped generations** within the claim's `holder.location`, a restore contract that reads only the generation the claim names, and a **straggler sweep** for what lands after `H` (§5.1, §6.4); - fixes J2-R2-5: a recovery probe runs on the inputs it renders now, not the move's pinned ones (§7.6, §7.7); - fixes J2-R2-8: the start check also requires the generation the claim expects, so the third admission fence the model relied on is now specified (§5.1); - makes the record and the claim consistent: every move transition is one record CAS, and the claim follows it; any actor may write the claim half, which the record fully determines (§7.5, §19 K4); - makes the cut promise the enforced deadline (§7.4, T11); - records that `CARLOS_SECRETS_KEY` is on every flagship box and config sync is armed in every region (§1.2, §15); - lists CC15 to CC19 for platform-1e (§19.4). Rev 6's changes are marked "(J2-…)" for the review's findings and "(model N)" for the stricter model's. - **Rev 5** (4ec3282b) was written against **claim spec rev 3** (8b5bccb2 on `design-replica-ownership`), which adopted CC1 to CC13. It answers the coordinator's round on rev 4: - the model (§14.2) is extended until it exercises every rule rev 4 listed as a gap: stalled staging signers, paused registry writes that resume at the worst moment, an obligation runner beside the role worker, and admission from cached reads. It is re-encoded against claim rev 3, and reruns clean at 30,000 and 100,000 seeds, with the full mutation sweep (§14.2.5); - N2's second half is now proven: dropping the seal by version loses acknowledged writes, and dropping the staging horizon leaves residue. The model also found the rule both rest on: a staging writer signs only before its lease's `SafeUntil` (model 14, §7.3); - CC11 is now proven: an operation started over a pending park loses acknowledged writes (model 15); - T12 and the install wait on `quiet_after` cannot bite under claim rev 3: they are kept as defence in depth (§14.2.5); - §15 step 8 is done: every AWS behaviour the model assumes was checked against the real service; - one more claim change, CC14, goes to platform-1e (§19.4). Rev 5's changes are marked "(model N)" like rev 4's. Earlier revisions: - **Rev 4** (a80ff90b) was the first revision checked against an **executable model** instead of a prose round. The joint Astra review of the claim spec (rev 1) and this spec (rev 3), `CODEX-FINDINGS-JOINT-R1.md` (4316db73), found 12 new P1s and 3 P2s, and recommended a joint simulation before planning. Rev 4: - builds the model: `internal/sim/hostlife/`, a test-only Go package (§14.2). It encodes claim spec rev 2 (cf3a7861) and this spec, with the real IAM split, late S3 completion, crashes, partitions, whole-VM pauses, destroyed and restarted boxes, and old executables. It runs 30,000 seeds clean, and a mutation harness drops each rule in turn; - adopts the joint review's adjudicated text for D1, D3 to D8 and I-7, and answers the findings this spec owns: N2, N11, N13, N14, N15, and the moves side of the others. Changes are marked "(joint N)"; - fixes what the model found beyond the review, marked "(model N)" (§14.2.4); - lists the claim changes the model requires (§19.4), for platform-1e. - **Rev 1** (e41c4fef) went to Astra (GPT-6, via codex): 12 P1, 7 P2, 1 P3, "not ready". The findings are in `CODEX-FINDINGS-MOVES-R1.md` (9fbf4afe). - **Rev 2** (45b77691) answered them, cut the scope to Phase 1, took Paul's rulings M4 to M15, and rebuilt on #199 rev 3. Round 2, `CODEX-FINDINGS-MOVES-R2.md` (e5b67514), resolved or accepted the deferral of most of round 1. It found 6 new P1s and 4 P2s: "not ready". - **Rev 3** answers round 2. Changes are marked "(codex R2-N)"; rev 2's marks "(codex N)" stay. §22 records every disposition from both rounds. Rev 3 also: - takes Paul's rulings M16 to M18 (§0.1); - builds on **the host lifecycle claim** that Paul ruled on 2026-10-02: one object per host, updated by CAS, that serializes delete, park, move, push, the db door and wake. It is specified in `docs/superpowers/specs/2026-10-02-host-lifecycle-claim-design.md` (rev 1 at 653ad4b1 on `design-replica-ownership`; "the claim spec"). §19 says how moves use it and where this spec asks it to change. **Status:** DRAFT for review. Nothing here is built. **Discussion:** amadan `carlos/platform` #175. **Goal (Paul, 2026-10-02):** "I want region-default AND the ability to easily move any app or instance to a different region." The region default is `docs/superpowers/specs/2026-10-01-nearest-region-default-design.md` on branch `design-nearest-region`. **Builds on:** - #331, `docs/superpowers/specs/2026-08-28-instance-move-design.md`. - The moves design record (the "09-24 record"), `docs/superpowers/specs/2026-09-24-instance-moves-for-every-app-design.md` on branch `docs-instance-moves-design`, with `CODEX-RULINGS-INSTANCE-MOVES.md`. - On `main` (7edd4b33): P0 (the cut fence), P7 (the edge door), and the move-ready fix (16 MiB readiness bar, copy receipts for multipart ETags). - #199 rev 3, `docs/superpowers/specs/2026-10-02-replica-ownership-design.md` at 66f1dad3 on `design-replica-ownership` (the "#199 spec"). It failed review for the same reason as round 2's P1s; the claim spec supersedes it from its §2 onward (tombstones become the claim's `deleting` and `parking` operations). - The claim spec, `docs/superpowers/specs/2026-10-02-host-lifecycle-claim-design.md`, **rev 6.2 at 30d572c4** on `design-replica-ownership`. Rev 7.5 is written against rev 6.2; rev 7.4 against rev 6 (3ea3f90c); rev 7.2 and 7.3 against rev 5.2 (465fee96); rev 7.1 against rev 5.1 (64eec93b); rev 7 against rev 5 (4a458a9c); rev 6 against rev 4 (a3c86fa5); rev 5 against rev 3 (8b5bccb2); rev 4 against rev 2 (cf3a7861). Citations are to this worktree at 7edd4b33 unless they name another branch or repo. ## 0. Rulings and the shared primitive ### 0.1 Paul's rulings (2026-10-02). Not reopened here. | id | ruling | where Phase 1 spends it | |---|---|---| | M1 | **Who:** owners move their own apps and instances, self-serve, from the console and the CLI. Operators can do the same for any account. The platform refuses unsafe moves. | Phase 1 is operator-only (§12); owners come in F1 (§16) | | M2 | **Downtime:** a short cut window (tens of seconds of 503), an automatic health check after the cut, and **automatic rollback to the source if the target will not start**. Not live migration. | §7 | | M3 | **Stores move with the app.** | F4; Phase 1 leaves the app store in place and refuses private stores (§7.1) | | M4 | The files the move carries are a fixed set. | §6.1 | | M5 | The files archive is capped at 256 MiB, enforced while streaming and again at the cut. | §8.1 | | M6 | Person-started moves cut automatically once ready and validated; the API default stays manual; `--manual-cut` exists. | §12 | | M7 | The source copy is held for 7 days after a move; owners may end the hold early. | F3 | | M8 | An app move pauses when one of its instances rolls back, and keeps the target app region. | F5 | | M9 | An app store's final copy holds every consumer at 503, behind a durable barrier covering concurrent creates. | F4, F5 | | M10 | An app with sidecars is refused an app move until sidecar moves exist. | F5 | | M11 | Target pre-warming is deferred; eligibility is gated on the measured duration of the whole cut. | §7.3 (T11) | | M12 | Rehearsal order: a probe app, then a disposable ips instance, then kpmg, each with scoped approval. | §14.4 | | M13 | Owners only, not members; the member-minted instance-token bypass is closed. | §12; F1 | | M14 | One landing path in production. | §7.6 | | M15 | A landed copy is installed into the target's `instances/`, never served from `moves/`. | §5.1, §7.6 | | **M16** | **An operator-only `carlos move resolve --declare-lost `, refused after commit, proceeds as a clean rollback after a confirmation naming what is lost.** | §7.10 | | **M17** | **The return lane is approved.** Every region's `sibling_staging_arns` lists every other region's data bucket, with `moves/*` Put and Delete and prefix-conditioned List only. | §7.3 (T10), §15 | | **M18** | **Tito waits.** Move mutations are operator-only; instance-role tokens keep only the status read. | §12 | ### 0.2 The 09-24 record's rulings They bind unless M1 to M18 override them. **R4** ("rollback is a move back") is narrowed by M2: a landing that fails before the target admits traffic is undone automatically (§7.8); after the target opens, R4 stands. **Q5** (other accounts operator-only until #171) is consistent with Phase 1. ### 0.3 What #199 contributes - Replica data stays at `instances/` (`internal/activator/activator.go:138`). No record field carries storage identity, so this spec names a generation by the operation id that installed it, in object names and marker files (§5.1). - The incarnation, `provision.Incarnation(rec)`, including the zero-CreatedAt legacy identity, and the park helpers (copy with a manifest then delete; local rename), now specified in the claim spec §2 and §5. ### 0.4 The host lifecycle claim From claim spec rev 5.1 (§2 to §10 there), the parts this spec relies on: - **The object** `control/hosts/.json`, rewritten whole by CAS: a per-write revision `gen` (not a data generation, J-D2); a `holder`; `op` with `kind`, `id`, `phase`, `intent` (written in the start CAS), `journal`, `exec` and `participants`; `touched` (with `complete`); `inventory`; `obligations[]`; the `lost` quarantine; and `retired_incarnations`. - **Its invariants:** C1 one holder; C2 intent before effect, a recovery owner for every state, every CAS conditional on `op.id` as well as the ETag; C3 storage mutated only by the current operation and only on **exact object versions**, with a shared staging-work lease for cross-region staging; C4 fresh admission; C5 revocation needs stop evidence or a lost-box fence; C6 a read error never acts; C7 already-admitted work is drained (`hostguard`). Rev 3 limits C3's "late completion is harmless" to the operation's own version-bound calls, and covers the replication exception with a late-completion horizon (§6.2a there). Rev 4 adds **C3a: no storage location is shared between incarnations.** `holder.location` is `instances///` for a new incarnation and `instances//` for an adopted legacy host, set with the holder and immutable while it holds; restore reads only that location, plus this spec's generation within it, from the same fresh admission read. - **Op kinds** (§4 there): `moving`, `landing` and `reverting`/`restoring` belong to this spec. One operation runs at a time. - **Mechanisms:** stop evidence from system state, with every probe rendering held config (§6.1); parking by exact version into per-retirement destinations keyed `@` (§6.2); the late-completion horizon `H` and the per-region `quiet_after` that survives `free` (§6.2a); local retirement for both layouts (§6.3); the host guard, an exclusive `flock` shared by every OS process that writes host state, with a fresh claim read under it (§6.4); door listings that read the claim twice, O1 and O2 (§6.5); execution leases for liveness, and declare-lost by compute fencing only (§7); one `claim.Adopt` routine whose discovery parks every other region and every other box (§9); app admission registration, closed and drained by purge, which fences a drained registration's host claim before removing it (§5.1, §5.3, CC21); `cancelled_creates` (§5.1); the purge journal `reserved → adopted → claimed → torn-down → retired → done` (§5.3); park manifests with separate archive and deletion sets (§6.2); the floor on every executable restore path (§10). This spec's rules for using the claim are K1 to K13 in §19, named "K" so they never read as the claim spec's C1 to C7. **Execution holder** below means the claim's `op.exec` (rev 2 made it a first-class field, as rev 3 asked). Rev 4's model required CC1 to CC13 of claim rev 2, and claim rev 3 adopted them; rev 5's CC14 is in claim rev 3.1 and rev 4; rev 6's CC15 to CC19 are in claim rev 5, rev 7's CC20 and CC21 in claim rev 5.1 (CC21 as a purge fence), and rev 7.1's CC22 in claim rev 5.2, with C5a (§19.4). Claim rev 6 (3ea3f90c) answers the claim side of joint round 3 (R3-2, R3-3, R3-4) with no request from this spec; rev 7.4 encodes it. Claim rev 6.2 (30d572c4) answers joint round 4's claim side (R4-2 installation stamps, R4-3 the region-qualified live set, R4-4 retirement under purge) and the moves session's three questions; rev 7.5 encodes it (§5.1, §6.4, §14.2.2, §19.4). Until platform-1e adopts them, this spec states each as the behaviour it relies on. ### 0.5 Paul's rulings of 2026-10-06 (#236; claim rev 6.27 §1.1) and region HA's requests Claim rev 6.27 (989ee349) §1.1 records every ruling of amadan #236. The ones this spec acts on: - **P1, the floor is the first fully integrated release.** §11's `control/moves-protocol.json` names a real release that carries the claim, the moves and the region HA code together; no placeholder tag. - **P5, record-backed placement only.** A move's target is a record's region and a box the roster knows; nothing places by hand. - **P6, rename and transfer stay refused.** Out of scope here as before. - **P7, cold-only admissions.** Until a reviewed Litestream fix (#234) makes a clean-close certificate safe, **every admission is cold**: no `closed/.json` is written, no `PredecessorCertified` admission is made, and no plan task depends on warm. §5.1 step 9's warm mode, the certificate ledger and `KeepStamp`'s interaction with warm stay in this spec as the behaviour a later revision turns on; the model keeps both modes so the rule stays checked. The cost in Phase 1: every wake restores from the remote (one regional S3 read and one disk write of the database), writes a fresh generation's first upload (one disk read and one regional S3 write) and its `generation.json`, and `PublishGeneration` queues one `park-generation` chore for the dead predecessor per wake. T11 already prices every probe attempt as a cold admission (rev 7.11, R10-8), so the readiness budget is unchanged by P7; the per-wake chore is a claim CAS and a regional park after the horizon, outside the cut window. Region HA (spec 1 rev 5.4, §9) asks four things of this spec, folded in as follows; the fifth, L18, is stated as a limit: - **L14** (§7.10): the termination fence is M16's evidence, automatically. - **L15** (§7.1, §7.3, §19.1): the target box is chosen in the start CAS. - **L16** (§5.1 step 2, §7.6): one install routine, two sources. - **L17** (§6.1, §8.3, §9.4): a files manifest per installation. - **L18** (§14.2.6): two boxes per region in the model is not done here. ## 1. What happened in production ### 1.1 Finding 1: busy instances never reached `ready` (fixed on main) Multipart ETags never matched after `CopyObject`. Fixed in 2cbf6196 and 167d721b (merged as 7edd4b33): copy receipts (`internal/activator/backend_exec.go:1557`) and a 16 MiB readiness bar (`internal/host/movespass.go:215-235`, applied at `:387`). ### 1.2 Finding 2: a completed move could not start for 1h54m `kpmg.bdn.oncarlos.com` (account bdn, app ips) moved IE (home) to uk, and every wake on the UK box then failed with `instance not healthy within 30s` (`backend_exec.go:1443`). Both causes are confirmed. **(a) The per-host data directory does not move.** The cut mirrors the replica (`movespass.go:476-490`), purges database files only (`backend_exec.go:1566-1575`), and flips the record (`movespass.go:494`). The app keeps `instance.key` and `report.token` in `/data//`, which no platform code names. On the UK box the app minted a new key. **(b) Only the Ireland box holds `CARLOS_SECRETS_KEY` (codex 20).** The UK, US and AU journals log "config sync loop disabled: CARLOS_SECRETS_KEY not set" (`internal/host/agent.go:884`). Both config loops return at once (`agent.go:881-886`, `:1279`), the fast converge skips its config leg (`agent.go:1812`), and the spawn treats a missing config file as empty (`backend_exec.go:1146-1149`). Paul approved distributing the key to every region box. **Done (2026-10-03): the key is on all four flagship boxes, and config sync is armed in every region.** The fingerprint every box publishes (§10) still verifies it per box, and T2 still refuses any target without it: a new or rebuilt box can lack it again. With config sync live everywhere, bundle updates now reach a box during a move, which is why J2-R2-5's case is real (§7.6). ### 1.3 Finding 3: the rehearsal proved nothing `bdf/hello` has no key file, store, config or timers. §14.4 rehearses with an app shaped like ips. ### 1.4 Finding 4: SSM Parameter Store is regional carloku-infrastructure `region-uk.tf:61-64`: nothing replicates `/carlos/*` between regions. §10 shows that an instance needs only the box's secrets key from it. ## 2. Phase 1: what it does and what it refuses Phase 1 moves **one** exec-backed, hibernating `kind:"instance"` record between two non-fleet placements, into home only with host claims on (Ruling R1, rev 7.24), driven by an operator. It carries every piece of per-host state, proves the target can run it before the source stops, starts it on the target with no public traffic, and opens only after a commit. If the target cannot start it, the instance returns to the source automatically, carrying anything the target wrote. Phase 1 refuses everything §7.1 lists. It deletes nothing: what a move supersedes is parked under the operation's id, out of every live namespace, ready for F3's cleanup. ## 3. Words - **Source / target:** the box serving before the move, and after. "Box" means the one box whose registry owns the route (`OwnsRoute`, `movespass.go:180-185`). - **Move id / rollback id:** console-minted, unique, `mv_` or `rb_` plus 26 lowercase base32 characters. Each names one **operation**. - **Generation:** one installed copy of an instance's state (database, files, timers), identified by `(op id, incarnation)`. An instance that has never moved is in the **origin** generation (no markers anywhere). - **Handoff:** the transfer of the execution right from one box to another, with a new generation (§5). A move is a handoff; a dirty rollback is a handoff back; a future `carlos db push` is a handoff on one box. - **Hold:** a registry row status that admits no traffic, ticks or ordinary wakes (§5.2). - **Flip / commit / open:** the CAS that sets `region` to the target; the CAS after which the target is authoritative; the target row becoming `ready`. ## 4. Invariants Stated first, designed to, and proved by the simulation in §14.2. - **INV-1 One writer.** At any instant at most one process tree, on any box in any region, runs the instance with write access to its database, files or store. A box starts the instance only while the claim names it as the execution holder, and the right moves only on **stop evidence** proving the previous holder's whole process tree is gone (§5.3). A timeout never moves it (codex 1, R2-1). - **INV-2 No lost acknowledged write.** No write the instance acknowledged, to a client, a tick, its own background work or an external system, is lost by any outcome. A rollback carries back whatever the target wrote, and a source recovery never reinstalls over its own new writes (codex 2, R2-3). M16's `--declare-lost` is the one disclosed exception. - **INV-3 Convergence without a human.** A crash of any actor (a console or agent process restart, or a box reboot) at any step leaves a state from which the move reaches a terminal outcome with no human action. This holds even when the crashed activator has lost all memory (codex R2-2). The exception is a box that never returns, handled by M16 (§7.10). - **INV-4 Files match the database.** Every start, on any box, uses a files directory whose generation equals its database's generation, and the generation installed in the box's own bucket. Otherwise the start is refused (§5.1). Supporting invariants: - **INV-5 No admission before commit.** A row leaves a hold only by a conditional write naming the operation that holds it (codex 3, 6). - **INV-6 The environment that runs is the environment that was checked,** from inputs whose versions were pinned at readiness (codex 14, R2-8). - **INV-7 Install is never replayed over writes,** on the target or the source; and a completion marker is published only after everything it vouches for is durable (codex 5, R2-3, R2-6). - **INV-8 Phase 1 deletes nothing a recovery might need** (codex 8). - **INV-9 Bounded.** Every non-terminal state has a deadline after which it advances, recovers, or pages in a state that still satisfies INV-1 and INV-2. Recovery deadlines are separate from the promised cut window (codex R2-10). - **INV-10 Incarnation isolation.** When a move or rollback ends, no copy of the instance's data remains in a live host namespace (`instances//`, `moves//`, the flat DB path, the files directory) on any box or bucket except the one the claim names as current. Everything else is parked under the operation id (codex R2-5). **What the model checks (§14.2).** The coordinator's list for the joint model, and where each lives: | invariant | checked | here | |---|---|---| | at most one writer per host, across regions | after every step | INV-1 | | no acknowledged write lost (database writes and registered timers), except a declared loss | at the end, on the serving box's local state **and** on a cold restore from the bucket | INV-2 | | no recreated host sees a previous incarnation's database, key or config | at every spawn | INV-4, C1 | | no live-namespace residue after an operation ends | at the end, every bucket and box | INV-10 | | claim and record agree after recovery | at the end | C2 | | every crash converges with no human, except declared loss (M16) | at the end: the run must reach a quiescent state | INV-3 | | an app store exists while its host is held for anything but delete | after every step | claim §5.3 | | a door read returns only the incarnation it authorized | at every open | claim §6.5 | | a commit happens within the cut budget | at the commit | INV-9 | | an admitted cut never waits out `quiet_after` (rev 5) | at the landing's park | INV-9, T12 | | no IAM violation | at every request | M17 | | no stuck recovery | after every step | INV-3 | ## 5. The handoff protocol ### 5.1 Generations, markers and the restore contract (codex R2-6, R2-9, J2-R2-1, J2-R2-8) **Location and generation.** Claim C3a gives each incarnation its own replica location, `holder.location`: `instances///`, or `instances//` for an adopted legacy host. Within it, every installed state of the database is a **generation**, named by the operation that installed it: the origin generation (`""`, written by the host's own replication from its first wake), then one per committed move, per push, and per dirty rollback. Each generation has its own key prefix: | generation | replica objects | |---|---| | origin | `` itself, the layout litestream writes today (`ltx…`, a legacy host's existing objects) | | an operation `op` | `g-/…`: the installed snapshot, its anchor and every LTX object replicated on top of them | Within a generation prefix the objects are litestream's, in the layout the pinned S3 client writes (`s3/replica_client.go`, `ltx.FormatFilename`): `//-.ltx`, so a snapshot ending at TXID 7 is `0009/0000000000000001-0000000000000007.ltx` and a level-0 file for TXID 8 is `0000/0000000000000008-0000000000000008.ltx`. Rev 7.5 wrote `ltx//`, which is not the layout (R5-6). Rev 7.6 writes the real one in this spec and in the model. The claim names the current one, **`holder.generation`** (CC18): written only by a cold admission's `AdmitWriter` CAS (a landing's, a push's or a dirty rollback's probe into its prepared generation, every recovery into a fresh one, §7.14) and by a clean rollback's return CAS, which names the source's generation again for the restore (§7.8; R10-3: no commit, `probed` or install phase writes it). Restore reads it from the same fresh admission read as `holder.location` (claim C3a). **The restore contract (J2-R2-1; R3-1; R4-1).** A restore reads `holder.location`, generation `holder.generation`, in the box's own bucket, and nothing else. It never picks "the newest object under the prefix". Rev 7.4's algorithm required each next LTX object to start at `current TXID + 1`, which is not how LTX works (R4-1): compaction merges files into one covering `[Min of first, Max of last]` whose page images are the state at `Max`, deletes the inputs, and so leaves overlapping ranges and no individual intermediate objects. A snapshot ending at 100 with one current continuation `[91, 110]` is a healthy replica; rev 7.4 would have refused it, or restored through 100 and let a move commit without transactions 101 to 110. Rev 7.5 took **one implementable contract: the sealed transfer is a verified final snapshot, and the restore is litestream's own planner over that snapshot and the generation's later files**. Joint round 5 showed that rev 7.5 described `litestream` v0.5.14 and `superfly/ltx` v0.5.1, the versions this repository pins, from their documentation and got three things wrong (R5-1, R5-2, R5-3). Rev 7.6 writes the contract against the libraries' **observed** behaviour: `internal/sim/litestreamfacts/facts_test.go` drives the pinned libraries with the file replica client and asserts every fact cited below as "fact R5-n"; a library upgrade that changes one fails there first (§14.2.6). 1. **The sealed transfer is the staged replica at the required final TXID (R5-5).** At the cut, after the final checkpoint and `SyncAndWait`, the source's replication position is `N`: the `MaxTXID` of its newest level-0 file. The final mirror round (`staged`, §7.4) copies the delta of the source's replica into staging's `replica/`, within the 16 MiB bar, and the **seal** records `N` and the exact `versionId` of every current LTX file under `replica/`. Nothing larger crosses the region boundary at the cut: rev 7.5 shipped a whole snapshot of the stopped database after the stop, which a 2 GiB database over a 10 MiB/s path cannot do in a 120 s window, while T11 priced only the 16 MiB delta (R5-5). `N` is the **required final TXID** of the transfer. 2. **The target builds the generation's snapshot from the staged replica** (§7.6 `building`). It restores the sealed versions with the planner of step 4, in its own region, verifying each file's trailer (`ltx.Decoder.Verify`: the CRC-64 `FileChecksum`), and requires the result to reach exactly `N` and to pass `PRAGMA integrity_check`. From that database it writes two files into the move's own generation: the **snapshot**, one level-9 LTX file `[1, N]`, every page, at `g-/0009/<1>-.ltx`; and the **anchor** (step 7), one level-0 LTX file `[N, N]`, every page, at `g-/0000/-.ltx`. Both are written as the pinned writers write theirs: `HeaderFlagNoChecksum` set, pre- and post-apply checksums zero, the trailer's CRC-64 present (fact R5-2); the anchor's `WALOffset` is the WAL header size, which `Verify` requires (fact R5-1c). **A build is two durable states, never one object** (R7-4): first the **build intent**, `g-/build-intent.json`, written before any PUT, with the encoding inputs the builder will use (the header timestamp, the page size, the commit, `N`) so that a replay (a restarted `building`) encodes **identical bytes** (R6-5: the bytes depend on the timestamp, `TestFact_R6_5_AnchorReplayBytes`); then, only once the snapshot PUT and the anchor PUT have both returned, the **completed install manifest**, `g-/install.json`, with the artifacts' acknowledged identities (keys, the `versionId` each PUT returned, `N`, each file's CRC-64) and the verification results (`integrity_check` passed, the sha256 of the database file the build produced). Its keys are the shared `moves.InstallManifest`'s json tags (claim Plan 6's `internal/moves/install_contract.go`; rev 7.22): `v`, `op`, `host`, `snapshot_txid` (the Go field `N`), `snapshot_key`, `snapshot_version`, `snapshot_crc64`, `anchor_key`, `anchor_version`, `anchor_crc64`, `integrity_check`, `db_sha256`, `built_at`. The completion predicate is the manifest's existence, never the intent's: rev 7.7 said "the install manifest records the inputs before the PUTs" and "`building` is done when the install manifest exists", so a crash after the inputs and before the artifacts left a manifest that satisfied the completion predicate (R7-4). A replay reads the intent, encodes the same bytes, and reuses a snapshot or anchor it finds already current under its key rather than writing again (the model's cold admission does the same, §7.14). `generation.json` is published only from the completed state (step 3). The cost is three passes over the database, each across regional S3 and the disk, plus a verification read, which T11 prices (§7.3). 3. **`generation.json` names the snapshot**: `snapshot_key`, `snapshot_version`, `snapshot_txid = N`, `snapshot_crc64`. These are the snapshot identity: the **pinned root**. A second version under the same key, from a PUT that completed late, is identical bytes and is never read. 4. **The restore is litestream's planner** (`replica.go` `CalcRestorePlan`, v0.5.14), rooted in the named generation and nowhere else. Its **root** is the newest current level-9 file of the generation whose `MaxTXID ≥ snapshot_txid` (a **verified successor snapshot**, step 8), or, when no such file is current, the pinned snapshot opened by `snapshot_version`. From `current = root.MaxTXID` it takes, among the generation's current files at levels 0 to 8, the file with `MinTXID ≤ current + 1` and `MaxTXID > current` (overlap allowed: `ltx.IsContiguous`), preferring the highest `MaxTXID`, then the lowest `MinTXID`, then the higher level, applies it, sets `current = MaxTXID`, and repeats until nothing extends the range. A compacted `[91, 110]` over a snapshot at 100 is taken: its pages are the state at 110, so applying them over the state at 100 is exact. Files are taken from the current listing, as litestream lists them; nothing reads noncurrent LTX versions, because compaction's and retention's deletes make the superseded files noncurrent on purpose. Rev 7.5 said "every level" and rooted at the pinned snapshot for good; after retention that root has no continuation (fact R5-3d) and the restore would have refused a healthy replica (R5-3). 5. **Fail closed, by what the format supplies (R5-2).** The pinned level-0 writer, snapshot writer and compactor all set `HeaderFlagNoChecksum`; `PreApplyChecksum` and `PostApplyChecksum` are zero in every file, and `ltx.Trailer.Validate` requires them to be with that flag (fact R5-2, litestream `db.go:1948`, `db.go:2412`, ltx `compactor.go`). There is no database checksum to compare after applying a file; rev 7.5's step 5 would have refused every healthy continuation, or been skipped. What the format supplies is **per-file integrity**: the trailer's CRC-64 over the file, which `Decoder.Verify` checks, and which SQLite's own `PRAGMA integrity_check` (`CheckDB`, `backend_exec.go:313`) complements over the result. The restore refuses, with a member-safe reason and a page, and never produces a database, when: no root is current and the pinned version is missing or its `MaxTXID` is not `snapshot_txid`; a file fails `Verify`; a current file starts beyond `current + 1` once nothing extends the range (litestream's "non-contiguous ltx files": a gap); the result fails `integrity_check`; or, at an install, the result is not at the required final TXID. End-state checksums would need the three writers changed; Phase 1 does not change them, and the model carries none (§14.2.2). 6. **What a gap means.** Litestream does not advance past a failed upload, and a timed-out upload is retried with the same bytes before the next range, so a gap in a generation's current listing is a lost or not-yet-landed object, never a healthy replica. The refusal holds until the object lands, or pages an operator. 7. **The anchor: restore-to-replication initialization (R5-1).** Litestream's open (`db.go` `checkDatabaseBehindReplica`) takes a fresh box's replication position from the **newest remote level-0 file**; a `-txid` sidecar is read in follow mode only (fact R5-1e). A generation holding a snapshot and nothing at level 0 therefore starts replication at TXID 1: the first upload is `[1, 1]` (fact R5-1a), later writes land below `N`, and a cold restore rooted at `N` skips them and loses every write since the install (fact R5-1b; the R5-1 counterexample, confirmed). The remedy the pinned library supports unchanged is the **anchor** of step 2: a level-0 file `[N, N]` with every page, beside the snapshot. The open takes `N` as its position, the local WAL does not match it, and the next sync is a full upload at `N + 1` (fact R5-1c); a cold restore afterwards holds every row (fact R5-1d). Every install of a generation writes its anchor before any process may open the database: a landing (§7.6 `building`), a push's install, a dirty rollback's install on the source (§7.8). **The PUT must be acknowledged before replication opens, and the position is what the listing then shows** (R6-1): litestream derives its position from the remote level-0 files actually present, so an anchor still in flight is no anchor, and an activator that opened on the strength of having issued the PUT would replicate from TXID 1. The activator therefore waits for the PUT's response, retrying the same bytes after its client timeout (the original may still land, identical), and only then opens; it never sets the position itself. Rev 7.6's model did (`db.Pos = cur`, R6-1); rev 7.7's `restoreInit` returns nothing until the anchor is acknowledged and reads the position from the listing (rule `anchor-acked`; the witness holds the anchor PUT across forty attempted opens and requires the host not to open, §14.2.5). A **cold restore of a replica that has replicated since needs no anchor**: the newest level-0 file is never removed by TXID- or time-based retention (fact R5-1f, asserted by identity), so a fresh box continues at exactly `N + 1` (facts R5-1g, h; `TestGate_AnchoredLineageThroughStore` for three disk losses in a row). The wake writes one anyway whenever the restored TXID is above the newest current level-0 file, which costs nothing when it is not (the model's `restore-anchor`). Oracles never write one: a cold-restore check is the pure calculation of steps 4, 5 and 8 (R6-1). 8. **Rotation, retention and root advancement (R5-3).** Litestream writes a new snapshot `[1, P]` at its snapshot interval, and `EnforceSnapshotRetention` keeps the newest, returning `minSnapshotTXID` = the `MaxTXID` of the snapshot before the oldest kept one; the store then expires every level-1 and higher file whose `MaxTXID` is below it, and level 0 by time (fact R5-3a, b). After two rotations no current file continues from the pinned snapshot (fact R5-3f): a restore rooted there fails with "non-contiguous ltx files: have up to 1 but next file starts at 5" (fact R5-3d) while a restore from the newest snapshot holds every row (fact R5-3e). Protecting the pinned version alone does not protect its continuation; **the root advances**. A **verified successor snapshot** is a current level-9 file of the named generation with `MaxTXID ≥ snapshot_txid` that passes `Verify`; only the generation's own replicator writes under its prefix (C3a, and the generation name is the operation's), and a stale re-upload of a rotated snapshot is identical bytes under the same key, so the newest current one is the one litestream itself restores from. `generation.json` is not rewritten at rotation: the pinned root is the install's identity and the floor the successor must clear, not the only root (step 4). The sweep protects every current version under `holder.generation` and the pinned version (§6.4). **The newest level-0 file survives every retention path** (facts R5-1f to h; R7-3): compaction deletes its inputs bar the newest, and the expiry below the previous snapshot never takes the newest level-0 file, which is where a fresh box reads its position. Rev 7.7's model kept it only through compaction and expired it when the local position had run ahead of the uploads (R7-3; the model's `newest-l0`). 9. **Writer sessions and epochs: two writers whose uploads could overlap never share a generation, every path that opens replication is admitted before its app starts, and a box never reuses its local database (claim rev 6.10 §6.4a; R7-1, R8-1 to R8-5, R9-1 to R9-5, R10-1 to R10-7).** Litestream writes each file to a key built from the level and the TXID range only, and its PUT is unconditional (`TestFact_R7_1_UnconditionalPut`). So a writer A whose level-0 upload for `[N + 1]` is still in flight when A dies with its disk, and a writer B that restores the generation through `N` and uploads its own `[N + 1]`, collide: if A's PUT lands last it replaces B's file and the next cold restore loses B's acknowledged writes; if it lands after B initialized its position and before B's sync, B's upload loop takes the remote `[N + 1]` as already uploaded and skips B's (`replica.go` `Sync`), and B's write is never replicated at all. `lock.json` cannot revoke a PUT S3 has accepted, and waiting for the lease to expire proves nothing about requests already signed. Claim rev 6.9 states the rule and this spec owns the data path: - **`AdmitWriter`, one claim CAS before the app process starts** (claim rev 6.10, R10-1; this reverses rev 6.8.2's order, which let an app acknowledge work before it had a session: a background worker committed and acknowledged, the app crashed before readiness, the old certificate still stood and the next warm admission purged the acknowledgement silently). The order is: the CAS; the purge and the restore of the baseline; the app start; readiness; uploads. A **session belongs to the agent's replicator**, not to the app process: an app that crashes or restarts while the same agent keeps replicating stays within the admitted session and is not re-admitted; **a new agent process is always a new session**, so an agent that restarts under a running app stops it (the replicator that was admitted died with the old agent) and its next wake is a new admission, which declares what the app acknowledged meanwhile. `holder.session` is the claim `gen` of the CAS that admitted the current writer session; `holder.writer_box` is the box it admitted (R9-2). Every path that opens replication makes this CAS: an ordinary wake, a committed move's target serving while the claim is still `landing`, a probe and a restarted probe, a reverted source, an opened rollback recovery, a push's writer (R8-2). The library's call is `AdmitWriter(Admission{ObservedOpID, ObservedPhase, ObservedSession, ObservedGeneration, Box, Mode, NewGeneration, PredecessorCertified, RemoteMaxTXID, KeepStamp, Coverage, Region, PredecessorRegion})` (R10-4; `Coverage` since claim rev 6.12, `Region` and `PredecessorRegion` since rev 6.13, E 47426cc1), and this spec's model is shaped like it. It requires the observed `op.id` and `op.phase` (serving's `""` compared literally, so a closure computed under serving fails once any operation starts), the observed `holder.session` and `holder.generation`, and **`op.exec` to be this box**, in every op kind (claim rev 6.8.1); it is refused in **`creating`, `deleting`, `parking`, `purging` and `placing`** (R9-4, R10-4): a create's first writer is admitted after `CompleteCreate`, under `serving`, and nothing writes while a placement runs. After a conflict the box re-establishes its evidence, the certificate, the exec handover, rather than re-reading alone. It writes `holder.session = its resulting gen`, `holder.writer_box = this box`, and a record in **`holder.admissions`** `{session, box, generation, region, predecessor: {session, box, generation, stamp}, predecessor_region, mode, coverage, covered, writes_lost?}` (claim rev 6.14 §6.4a item 1; the library's `AdmissionRecord`). `region` is where the new writer's generation keeps its objects, the admitted box's region; `predecessor_region` is the last record's `region` (before any record, the writer box's region, `""` when the generation holds nothing anywhere, as after a create). **The predecessor is the last record there**, with its own session, box and generation; it never comes from the holder's fields, which a clean return has reset to the source's generation for the restore while the predecessor is still the target's writer (R11-4; rev 7.11 journalled the target's session and box with the source's generation, a writer that never was; INV-16). `holder.admissions` lives in the holder, so it survives every operation transition, release and new operation; its retention is **deliberately conservative** (claim rev 6.14 item 8, R12-6): covered records are trimmed from the front only, while more than eight remain, an uncovered record stops the trimming and everything behind it is kept, and there is no compaction. The admission **never replaces the operation**: kind, id, phase, participants, obligations and the rest of the journal are kept. - **Failed starts are governed by a retry policy, never by another admission** (R10-1). Within the admitted session the agent retries the app start with a doubling backoff from one step (1, 2, 4, 8, 16 steps in the model; seconds in production) up to **five attempts**; a start the environment is not ready for (config not rendered, the release not live) waits and counts no attempt. For a **serving wake**, at the fifth failure the box marks the host **failed**, abandons the session's start, and creates no new session for the host until an operator clears it (or a health signal does; the model's operator clears it after half a horizon). For a **probe**, the restarts share the probe attempt's 30 s readiness window (§7.7): the fifth failure or the window's end, whichever comes first, ends the attempt, the operation counts one of its budgeted attempts, and the host is not marked failed. Five attempts because the start's own faults (a missing key, a bad bundle) do not heal within the window and a crash loop would otherwise churn generations. The waits between attempts total 15 s (1 + 2 + 4 + 8); each attempt's own start and readiness time is on top, so five attempts are not a bound on a serving wake's duration, only on its count. An app's writes are never left in an interval without an admitted session: there is no app before the CAS, and the session stands until the next CAS. - **A box never reuses its local database (R9-1).** Before it starts, every admitted writer purges the local database with an error-returning purge; if the purge fails the writer does not start (a transient error: the next attempt purges again). It then restores from the certified remote baseline, under the lease, and starts the app on that; the restore is version-pinned and the planner's (steps 4 and 5). A warm database is never the baseline, whatever it holds: acknowledged writes it held that never uploaded are lost, and they are **declared** (below), never silently dropped. There is no lineage file and no provenance check. - **Warm continuation** is an admission in **warm** mode: legal only if `closed/.json` exists for the observed session and its `max_txid` equals the generation's remote maximum TXID over every level (`PredecessorCertified`). **Any `op.exec` box** may continue on it (claim rev 6.8.2): the certificate proves every upload of the session completed, whichever box made them. The admitting CAS moves `holder.session` forward, so that certificate names a dead session: a delayed, replayed or re-created certificate of any earlier session can never authorize anything again. The writer restores the same generation **up to the certificate's `max_txid`**, refusing to start on a restore that stops short of it, and reads its remote position at its first sync (litestream initializes it lazily). - **Everything else is cold**: no certificate for the observed session, a mismatched `max_txid`, any doubt. A cold admission is an installing CAS: in the same write a new `holder.generation` (`NewGeneration`) and `holder.stamp = the resulting gen`. The new generation is either the operation's **prepared** generation (a landing's or a push's `g-/`, built from the sealed archive with its snapshot `[1, N]` and anchor `[N, N]`; a dirty rollback's return generation), opened exactly once, by its probe, and never re-entered once it is `holder.generation` or an ancestor of it (a dead writer's uploads may still land in it); or a **fresh** id; warm passes an **empty** `NewGeneration` (R10-4). The writer restores the prepared generation from its pinned snapshot, or the predecessor generation's remote (`holder.generation` as observed) for a fresh one, in its own region; a fresh generation is **empty** until the writer's first upload, the full file `[1, 1]` of the restored baseline (fact R5-1c), made after the app started and before the marker; the bytes are captured once and a retry carries the same bytes (R10-7). When the observed session had no certificate and the baseline does not cover the predecessor's last acknowledged position, the CAS records **`writes_lost {box: holder.writer_box, generation, session, remote_max_txid}`** in its journal entry and the box raises an alert (R9-2): the predecessor's acknowledged writes beyond its uploads are gone with its database. When the baseline is a **seal that covers the stopped predecessor** (a landing's, a push's, a return's prepared generation, built from the stopped database's every page), there is nothing to declare: the admission's **`Coverage`** is `sealed` (claim rev 6.12, replacing rev 6.9.1's `BaselineCoversPredecessor`); the comparison is against the seal's own position, never a TXID from another region. A clean rollback's return with `src_covered` (§7.8) is covered too, `clean-return`: the unchanged target and the source's complete replica establish preservation, and the source's re-admission declares nothing. The coverage speaks for the **target's probe session** and for the **source's cut session** (`cut.session`, journalled at the stop) only: a later session of the recovered source that dies uncertified is declared like any other, and a return made clean only by a declared-lost target is not covered at all: it records `writes_lost` for **every session the target was admitted into since the cut**, certified or not (a probe session that closed cleanly and a warm continuation that died are two; the first's writes are durable only in the lost target's region, and no return lane carries either's; seed 33373 / 100,000, hidden until the session-scoped match of R12-1). **Representation** (claim rev 6.22 item 2, R13-3, R14-2): the admission entry carries `writes_lost` (one, the predecessor's session) and **`additional_writes_lost`** (a list, the other target sessions since the cut, and the source's cut session, under its persisted identity `cut.session`, when `src_covered` is false), each `{box, generation, session, remote_max_txid}`, **unique by `(box, session)` across both**: a target destroyed after the flip and before its first admission leaves the source's cut session as the predecessor itself, declared once (rev 7.14 declared it twice, through a separate field; INV-21); the caller supplies the list and the library records it in the admitting CAS; the admitting box raises **one alert per declaration** after the CAS and retries until delivered, and the console's durable loss record is the long-term copy, so trimming a covered entry never loses a declaration (the model: `admissionRecord.AdditionalWritesLost`, an alert each, INV-14 counts them). Only coverage `""` with no certificate records `writes_lost`, and the record always names its coverage, certified or not. A seal built from the **remote** because the disk was gone (`seal.from_db` false, §7.4) stops at the remote's position and covers nothing beyond it: the prepared generation's admission has coverage `""` and records `writes_lost` for the predecessor's acknowledged writes past it, as a cold admission from the remote would. The box is never inferred from `op.exec`. A write the host's remote holds, in any generation, is never declared lost while its writer's box is with us (INV-15): the lineage adopts the generation that holds it, a dirty return's lane, instead; a box declared lost may leave such writes behind, and so may a rollback whose target was declared lost (no return lane runs; the seal in the target's region is unreadable from the source); the record's `remote_max_txid` names where an operator finds them (§7.10). - **Only warm and cold exist (R9-3).** A first install, after a create or an adoption with nothing replicated, is a cold admission into an empty generation: no LTX file until the first committed transaction, never a zero-TXID file (`TestFact_R8_3_EmptyBootstrap`); the app starts first and may create its database, as the activator does today. There is no fresh, reserve, build or open transition. `holder.generation` is the only authoritative generation in every kind and phase: a probe or a recovery that opens replication is a cold admission that moves it, never an op-journal generation. A cold admission issues a new `holder.stamp` **except** when no writer was admitted since the current stamp was issued (`holder.stamp > holder.session`, the library's `KeepStamp`, claim rev 6.9.1): a landing's first probe after the flip keeps the stamp the flip issued and the target's record activated; a restarted probe, a push and a dirty return take a new one. - **A cold admission's generation is unfinished until `generation.json` names it** (claim rev 6.10, R10-2). Until then its baseline, the journalled predecessor, is part of the live set: no `park-generation` chore for it may be queued and the sweep never parks it (rev 7.10 queued the chore at the CAS, so a writer that died before its first upload could have its only baseline archived under it). The chores that retire superseded generations are queued by the **marker's publication**, through one **`PublishGeneration(observedOpID, observedPhase, observedSession, publishedGeneration, region, newID)`** CAS (claim rev 6.12, replacing `QueueGenerationRetire`): bound to the observed operation and phase like the admission, legal while serving and inside any operation a writer runs in (a restarted landing probe, a rollback's or a revert's source; R11-5), it **requires `holder.generation` to be the published generation and an entry to name it, then walks every entry of `holder.admissions` in order, skipping covered ones** (claim rev 6.22 item 5, R13-2; bounded by the list's length; warm entries are born covered, so it visits every uncovered cold entry): a predecessor whose `predecessor_region` is `""` (it holds nothing anywhere) is covered with no chore; one that is the published, live generation again (a clean return's source generation, a republished lagging marker's) is **left uncovered** with no chore (covered, nothing would retire it once a later admission supersedes it, since that admission's predecessor is the last record and not the generation it restored from: seed 41914 / 100,000 left the source's generation as residue; rev 7.13 note 6, adopted); one whose objects are in the publishing `region` gets one deduplicated `park-generation` chore there and is covered; one whose objects are in **another region** stays uncovered, with no chore, and the walk goes on. The walk finds what rev 7.12 enumerated: the predecessor; behind a lagging marker, every generation of the chain of unfinished admissions between the marker's generation and the new one, and the marker's; and the predecessor of any earlier cold admission not yet covered (its writer died after its marker landed and before the CAS). **Every cold admission publishes**, a prepared generation's included (rev 7.12's model left a prepared generation's predecessor to the operation, R12-3): a landing's publication in the target's region leaves the source's generation uncovered, and the commit covers it (below). Its chores carry `op = op.id` inside an operation and `op = "session:"` while serving. **Retirement happens only through it, and only of predecessors recorded in `holder.admissions`** (R11-2). An adopted legacy host's generation is the location root itself, named by the sentinel **`"@root"`** so a chore can name it; `""` and `"@root"` are one origin, so nothing retires the active legacy replica before a cold admission has created a successor and published it; `holder.generation` is `""` for such a host until its first cold admission, and the adoption writes `holder.writer_box` as the box running the pre-release process, so the first cold admission's `writes_lost` names the writer whose unuploaded writes it leaves behind (R10-5, R11-3). **That writer is the legacy identity** (claim rev 6.24, R16-3): `{session: 0, generation: "" (read as @root), box: holder.writer_box}`, adoption's `holder.session` being 0; it is valid in both `writes_lost` and `additional_writes_lost` (a lost-target return after a move cut at session 0 declares it there), every other declaration has a session above 0, and a declaration always names a box: a created incarnation's first admission, whose predecessor has no writer (`holder.writer_box` empty), declares nothing (INV-24; rev 7.13 to 7.16 gave adoption session 1 and declared a boxless record for a created incarnation's first admission). **Coverage, completely** (claim rev 6.22 item 5, R13-2): a record is covered when its predecessor's retirement is in hand in the region that holds the predecessor's objects, and an entry becomes covered in **exactly four ways**, no other transition covers anything: **at birth**, a warm entry (its predecessor is its own generation, which nothing retires; the next cold admission's record names it and queues the chore), or an entry admitted while the claim already holds a park of the whole location in its `predecessor_region`: an unscoped `park-replica`, or a park scoped **exactly to `holder.location`**; a scoped park of an old incarnation or of a staging prefix does not count (claim rev 6.24 item 5; that park retires the predecessor: a rollback's return admitted after the release queued the target region's park, which the CAS that queued it could not see; seed 1036 / 3,000); **by publication**, above; **by `CoverRegionPredecessors(op, phase, region)`** in one of the three CASes that queue a whole-location park of `region`, every uncovered entry whose `predecessor_region` is `region`: the commit's `park-superseded` of the source region (§7.9), a rollback's `park-superseded` of the target region at the target's release (§7.8), and the console's release of a declared-lost target (§7.10), which retire those generations wholesale and so queue nothing; and **by `CoverParkedPredecessors(op, phase, region, generations)`** in the CAS that queues an at-rest `park-sweep` and in the CAS that acknowledges it, only the uncovered entries whose predecessor is one of the listed generations, the ones that sweep actually retires (a predecessor with a sweepable object at the queuing; one whose objects it retired on the way, at the acknowledgement): a sweep never covers a predecessor it did not park. A predecessor whose objects live in another region (a move's source generation, seen by the target's publication) is **never queued into its region from here**, since the rollback may still need it (seeds 1036 and 1296 / rev 7.12; witness `wRollbackBaselinePreserved`). INV-17: when a whole-location park of `holder.location` completes in a region, no record whose predecessor's objects were there is uncovered. A dead generation holding nothing yet still gets its chore in the publishing region, as does one whose objects were parked already: a dead writer's late upload or certificate may land in it within the chore's horizon. A failed cold start leaves an empty, unpublished generation; the next attempt is cold again and the marker it publishes retires it. - **A marker that lags `holder.generation`** (an absent marker is the origin `""`, for the sweep as for the restore, R11-1) is read through `holder.admissions`: when it names the predecessor of the latest admission, or the predecessor of a chain of admissions each of which died before its marker (two in a row happen: the second's purge failed after its CAS), that is the normal unfinished state, every generation of the chain and the marker's is live, and there is no alert. **The run walk, in one region's bucket** (claim rev 6.22 §6.2a, R13-1): the next admission follows each record's `predecessor` back from `holder.generation` to the marker's generation, newest first. A link whose generation keeps its objects in **another region** is passed over, covered or not: it holds nothing in this bucket, and its own region's park retires it (a clean rollback's fresh source admission R, admitted after the target region's park was queued and dead before its first upload, walks R, then the target's T in the other region, born covered under that park, then the source's S, and restores S; `TestInterruptedCleanReturn` walks that and R2 → R → T → S after it). A link **in this region** must be uncovered: a covered local generation is retired or being retired here, so reaching one as a needed baseline **refuses the start and alerts** (rev 7.13 read objects alone and would restore the marker's retired generation as an empty database; witness `wCoveredLocalBaseline`); an uncovered local link that holds an object is the baseline, the nearest such; one that holds none (a fresh generation that never uploaded) is passed over; else the marker's generation. **The start restores what the walk selected** (R14-1): the marker's generation through `generation.json`, as every ordinary restore; a run generation the walk selected **rooted at `[1, 1]`**, its first full upload, never through the marker, which still names the run's baseline (rev 7.14's start selected R from R2 → R → S and then refused it because the marker named S, while the oracle restored it; the marker is never published backwards). The model's `restoreCalc` over the marker's generation and `restoreWith` rooted at `[1, 1]` over a selected one are the reference the selector F mirrors. An empty `holder.generation` for any other reason is unexplained missing data: refuse the start and alert. A `holder.generation` that holds objects has its marker published again (from the operation's build journal for a prepared generation, rooted at `[1, 1]` otherwise) and the chores queued, before anything else. Any other disagreement between the marker and the claim leaves the location untouched and alerts (§6.4). - **A clean close is a one-use certificate.** A writer session stopping cleanly first waits until **every request it issued over its lifetime** returned **success to the client, within the client's wait** (R8-5, R9-5, R10-7): every level-0 upload, every compaction and snapshot upload, every retention delete, **and the start's own PUTs**, the fresh generation's first upload and a warm restore's anchor, which enter the same ledger. A request the client timed out on, whether or not it later completed on the server (a lost response), one the server refused, and any retry after a timeout make the session **uncertain for good**; server completion and the client's response are distinct, and only the latter advances the start. A session whose start has not opened replication, or whose replicator died with its agent, certifies nothing. A session that can prove completion writes `//closed/.json` `{session, box, max_txid, closed_at}` once (`If-None-Match`), then releases its lease; nothing of the host may run when it certifies (the unit stop kills the cgroup; a descendant that escaped it makes the close fail rather than wait, since its writes would be lost silently at the next purge, and the next wake is cold and declares). A certificate PUT not acknowledged within the client's wait ends the close uncertified: the PUT may still land, and a later wake that finds it may continue warm, since the drain was complete; the next wake is cold. **No certificate is ever deleted or reused**. Litestream's `Replicator.Close` drops the initial `SyncAndWait` error (`internal/lease/lease.go`), so certificate eligibility needs its own evidence, never `Close`'s return value. An operation that takes the writer over, a push, a move's cut, a delete or a purge, drops a pending close: the generation is being replaced or parked. - **Writers are bound to their generation.** The replica client is constructed for the `{holder.location, holder.generation, session}` the writer was admitted with and refuses any key outside `//`: level-0 files, compaction, snapshots, retention. It never re-reads the generation, so a new admission is a new writer. A dead writer's late uploads land in the old generation, or in the current one **before** the session that superseded it opened; a cold successor has a new generation, and a warm successor exists only after the old session certified that every one of its requests had completed. - **Conditional PUT alone would not do** (R7 review): if A's file won a create-only race, B would have to detect differing bytes and fail closed; the remote-position shortcut would have to check identity; multipart completion would need the same condition; and a delete marker makes a key creatable again. Disjoint generation prefixes need none of that. The model encodes all of it (§14.2.2): both completion orders and the race are witnesses whose late upload lands harmlessly in the dead generation with the rule and loses B's write without it; a dead session's certificate with a matching stamp and `max_txid` authorizes nothing; the collision during a committed landing's serving and during a probe restart is caught by the admission inside the operation; a non-exec writer is refused; a valid certificate over a stale local database restores the remote; a crash with an unuploaded write is declared; a stale closure during a delete is refused; an empty first wake and an empty install write no file; a retry that succeeds while its original stays unresolved, a maintenance retry, a lost response, a first-upload retry and an anchor retry write no certificate; a background acknowledgement right after the start, then a crash, is declared by the next admission (under rev 6.8.2's order it is purged silently); a crash before the first upload leaves the predecessor live until the marker names the new generation (with the chore at the CAS the predecessor is parked and the recovery restores nothing); a rollback whose source replica stopped short of the stopped database returns the seal. The cost is a restore at every wake, warm or cold, a full first upload per fresh cold generation (none for an empty one), three local passes for an operation's prepared generation as §7.3 prices an install, a restore per probe attempt, and a dead generation to park. So: - a request that completes late can only add an object to **its own** generation. A late snapshot or anchor PUT repeats identical bytes (another version, never opened); a late LTX PUT from an old executor, or from a dead writer of the same incarnation (step 9), lands in a generation the claim no longer names. Neither is ever restored into the current state; - the claim moves to a new generation at its writer's **cold admission**, before the restore, the app start and the probe (§7.14; rev 7.12's wording here, "only after that generation is fully installed and probed", described rev 7.9's protocol and is superseded, R12-4). A restore still never meets a half-written generation: a prepared generation is complete, snapshot and anchor, before any admission opens it, and a fresh one is empty until its own writer's first upload, which is the baseline's full file and lands before the marker names the generation; a restore reads the baseline the admission names, never a generation under construction; - what lands in a dead generation after its retirement is residue, not data: the straggler sweep (§6.4) retires it. **Against litestream.** `lease.Restore` (`internal/lease/lease.go:404`) delegates to `Replica.Restore`, whose planner lists each level without versions and opens LTX files by key. Phase 1 needs three things beside it, none a change to the pinned library: the root rule of step 4 (the newest current successor snapshot of the named generation, else the pinned version opened by `versionId`, never another generation's or "the latest under the location"); the builder of step 2 (restore the staged versions, write the snapshot and the anchor with `ltx.NewEncoder` as `litestreamfacts` does); and the wake's anchor check of step 7. The rest of the planner, overlap rule, preference order and the non-contiguous error included, is used as it is. The repository's compaction ladder (`lease.go:55`: levels 0 to 3) is what produces the overlapping ranges step 4 handles. `internal/sim/litestreamfacts/` verifies the library behaviour this section relies on (facts R5-1 to R5-3, §14.2.6); §15 step 8 verified the S3 behaviour (claim §13 rows 2, 3 and 10). **In the model** (§14.2.2) a database is a map of keys to values with a lineage TXID and, separately, a **replication position**; transactions insert, update and delete; an LTX file carries its level, its range in replication numbering and the final image of every key it touches, and no checksum (R5-2); the serving process ships one level-0 file per sync at `position + 1`; compaction merges three level-0 files into a level-1 file and deletes the inputs, then rotates a snapshot `[1, position]` and enforces retention as step 8 states (R5-3). The restore is steps 4, 5, 7 and 8 as written: the root is the newest current successor snapshot at or past the pinned one, the position is read from the newest current level-0 file, and the anchor is written when the restored TXID is above it. The model drops the contract (`restore-by-generation`: the newest object under the location) and loses acknowledged writes; it drops the overlap rule (`restore-overlap`: rev 7.4's `min == current + 1`) and loses a compacted continuation; it drops the gap refusal (`restore-gap-closed`) and wakes a host without durable transactions; it drops the anchor (`restore-anchor`) and a snapshot-only install that wrote, replicated and lost its disk loses every write since the push; it drops root advancement (`root-advance`) and a host whose replica rotated four times refuses to wake (§14.2.5). Rev 7.4's model carried a full transaction history in every object, which no LTX file has; rev 7.5's carried a whole-state checksum no pinned file has and a position that litestream never derives from a snapshot (R5-1, R5-2). **Markers.** A generation is named in three markers. None is a record field. | marker | where | written | content | |---|---|---|---| | `generation.json` | `generation.json`, beside `lock.json` | **last**, after the install manifest is complete and verified (§7.6); **conditionally**: If-Match the marker it supersedes, or If-None-Match when there is none (model 29) | `{"op","incarnation","host","box","installed_at","manifest_sha256","snapshot_key","snapshot_version","snapshot_txid","snapshot_crc64"}`: the pinned root, the install's snapshot identity and the floor a successor snapshot must clear (R3-1, R4-1, R5-3) | | `.gen` | beside the local database | after a restore completes | the generation's name as plain text, one line: the format main already writes (`hostbox.WriteGenerationMarker`). It carries no incarnation and no location; the claim's holder vouches for both (Paul's ruling, moves Phase 1 Plan 3) | | `.files-gen` | inside the files directory | **last**, after files, `.env`, timers and the carried registry columns are all durable (§7.6) | `{"op","incarnation"}` | `generation.json` is the one reused key a generation writes: everything else lives under the generation's own prefix. Without the condition, an earlier publication that completes late puts an older generation back: the claim names the new generation, the bucket names the old one, and every wake and recovery probe refuses: the start check compares the markers with the claim's generation and refuses forever (model 29; the model finds it after a push in 2 seeds of 100,000, §14.2.5). With the condition the late write fails with 412 (claim §13 rows 5 and 6), and the later generation stays. All three are **protocol metadata**: excluded from the archive, from every content digest, and from F4's definition of the files directory (§6.1). Absent everywhere means origin. The claim's `gen` is a per-write version (claim spec §2), not a generation, so it never appears in a marker. **The authoritative region (claim rev 6.5 §6.2a, R4-3).** The live set is `{authoritative region, holder.location, holder.generation}` plus the versions this spec names for the generation (the snapshot version in `generation.json`, noncurrent included; every current version under the generation, successor snapshots and anchors among them). Objects of the generation are live only in the authoritative region's bucket; a matching key in any other region is residue, and the sweep parks it. The authoritative region is the region of the holder's current installation: | claim state | authoritative region for `holder.generation` | |---|---| | `serving`, `pushing` | `op.exec`'s region | | `moving` (before the flip), `reverting` | the source's region | | `landing` after the flip, before the target's probe is admitted | the source's region: `holder.generation` is still the pre-move generation, and the target's prepared generation is not yet named | | `landing` from the target's probe's cold admission on, committed or not | the target's region: `holder.generation` is the generation that admission chose, the prepared one or, for a restarted probe, a fresh one restored from it (§7.14; rev 7.11's row said "before the commit", R10-3) | | `restoring`, clean, from the return CAS on | the source's region: the return CAS named the source's generation again for the restore, and the source's re-admission moves to a fresh generation there | | `restoring`, dirty, until the source's probe is admitted | the target's region, for `holder.generation`, which is still the target's last admission: a dirty return resets nothing (only a clean return's CAS names the source's generation again), and the generation stays the target's until the source's probe is admitted. In practice nothing is live there: the target stopped, and the target region's whole-location park is queued at the release with the records whose predecessors are there covered in that CAS (§7.8). The return generation is built in the source's region but not yet named (rev 7.12's row assigned the pre-move generation here, before any source admission, R12-4) | | `restoring`, dirty, from the source's probe's cold admission on | the source's region, for the return generation, which that admission named; its record's predecessor is the target's writer, in the target's region, born covered under the target region's park | | `creating`, `deleting`, `parking`, `purging`, free | none: nothing is live | **The start check (INV-4, J2-R2-8).** Before every spawn (`Start`, `Seed`, `ProbeWake`), the activator requires: 1. the box's own `generation.json` (absent is origin) to name **the generation the claim expects**: `holder.generation`, as the admission's CAS left it, for every start; a probe's first admission publishes the operation's prepared generation (the move id, the rollback id, the push id) and a restarted probe's a fresh generation restored from it (§7.14; rev 7.11's clause named the prepared generation for every probe, R10-3); a lagging marker is read through `holder.admissions` (§5.1 step 9); and its incarnation to be the holder's; 2. `.files-gen`, when the files directory exists, to name the holder's incarnation and either `holder.generation` or its cold predecessor (the newest cold admission into `holder.generation`, warm self-links skipped), which is the prepared generation a restarted probe was restored from; no older generation counts; 3. `.gen` to name the same generation, exactly: the plain name carries no location, which the claim's `holder.location` vouches for. A warm database that does not is from an earlier stay on this box: the activator parks it (§6.3) and restores cold. Rev 5's check compared the three markers with each other only. Astra showed (J2-R2-8) that after a commit, before the source is retired, the source's own three markers still agree with each other, so the "third fence" the model reported in rev 5 did not exist in the text: the model's check was stronger than the spec. Rule 1 makes it exist. A mismatch refuses the wake with a member-safe reason and pages. A marker that merely exists proves nothing (codex R2-6). The bucket read rides the wake's existing lease round trip to the same prefix. This replaces "staging wins" (`backend_exec.go:1581-1588`); `promoteStagedReplica` is deleted (M15). ### 5.2 Holds A hold status is one the edge does not serve (`internal/edge/edge.go:595`), ticks refuse (`activator.go:2785-2789`), and no ordinary wake reaches: `moving` (source), `landing` (target), `restoring` (source, recovering), `releasing` (target, during a rollback). The `routes` table gains an additive column, `hold_op`, set whenever a hold is written. Leaving a hold is one conditional statement: ```sql UPDATE routes SET status='ready', hold_op='' WHERE host=? AND hold_op=? ``` A late write from a finished operation matches nothing (codex 6). Reconcile mints held rows for every participant the claim names and never mints `ready` while the claim holds a move operation (`moving`, `reverting`, `landing` before the commit, or `restoring`) (§9.6, §19 K9). ### 5.3 Stop evidence (codex 1, R2-1, R2-2) The execution right moves only on stop evidence. Rev 2 relied on the activator's in-memory `FenceAndStop` (`internal/activator/maintenance.go:50`). Round 2 showed two gaps: a flat child's descendant that outlives it (`childProc.Stop` waits for the main process only, `backend_exec.go:1502-1543`), and evidence that cannot be produced after an agent restart, because `FenceAndStop` returns `ErrStopUnproven` when it has neither an entry nor a remembered stop history (`maintenance.go:90-100`). Evidence comes from **durable, observable state**, never from memory. Claim spec §6.1 (rev 2 and rev 3) adopts the joint review's D3 text, and this section follows it (joint D3): > All lifecycle-controlled spawns require strict containment. Stop evidence > requires a closed admission fence, drained spawn attempts, an empty > complete process tree, closed replicators and no remaining database/WAL/SHM > holders. An absent host cgroup is insufficient where an earlier spawn > could escape containment. and > Reconstruction uses a cleanup-only operation that can reopen, identify, > kill and drain an existing populated cgroup without granting spawn > authority. `PrepareStrict` remains the empty-group spawn API. A pre-move > best-effort child is not certified stopped by ordinary parent exit. `PrepareStrict` refuses a populated group (`internal/cgroup/strict_linux.go:206`), so the cleanup-only operation is a separate call, `cgroup.ReopenAndDrain`. **Every spawn under a move is strict.** Flat children already run in a per-host cgroup, `carlos-exec@` under the agent's delegated subtree (`internal/cgroup/cgroup.go:37`, `:243`; `Delegate=` in `internal/host/units/carlos-edge.service`), but placement is best-effort: a failed placement spawns "unaccounted" in the agent's own cgroup (`cgroup.go:248-252`). The strict path exists (`internal/cgroup/strict_linux.go`: `PrepareStrict`, `Populated`, `KillAndEmpty`). Under a move, every spawn of the host (the source's recovery, the target's probe, and the source's ordinary wakes while the claim says `moving`) uses `PrepareStrict` and refuses to spawn if it cannot. A nested child already runs in its own `carlos-instance@` unit. **The source's pre-move child.** Its cgroup placement was best-effort. The source's readiness (§7.3, S2) therefore requires its current or next child to be strict: before `ready`, it stops an awake non-strict child (an ordinary idle stop) so the next wake is strict. **Evidence** is `ReconstructStopEvidence(host)`, a new activator call that reads only the box's current state, so it gives the same answer after an agent restart as before: 1. **Kill and drain the tree.** Flat: write `cgroup.kill` on `carlos-exec@` and wait until `cgroup.events` says `populated 0` (`KillAndEmpty`). Nested: stop `carlos-instance@` (and its seed and sidecar units) through the instance-control channel and require each unit inactive and its cgroup unpopulated. 2. **No stray process.** The agent's own cgroup (`cgroup.procs` of the service's leaf) holds only the agent's own process. A strict spawn never leaves anything there; anything else is unattributable and refuses evidence with an alert. 3. **No open database.** No process on the box holds the instance's database, WAL or SHM open (a scan of `/proc/*/fd` for the paths, restricted to the `carlos` uid). 4. **Lease released.** The host's `lock.json` lease (`internal/lease/lease.go:86`) is absent, released, or expired by more than the lease's renewal margin. 5. **Replica final.** After 1 to 4, a final checkpoint of the local database (if any) into the box's own `instances/`, and its sha256. **A durably never-spawned operation.** If the box's journal for the operation shows no spawn was ever attempted (the target's `landing_phase` before `probing`), steps 1 to 4 still run: an empty tree proves the same thing whether or not a spawn happened. "No entry in memory" is never evidence on its own (codex R2-2). The box writes the result to the deployment bucket at `moves/////stopped-.json`: box label, incarnation, the claim `gen` it read, each step's observation, and the database, files and timers digests. It then CASes the claim's execution holder to `none` (§19 K3). The next holder reads the evidence and the claim before it starts anything. **Evidence also starts a clock (model 1).** Ordinary replication is the one storage write claim C3 does not bind to a version: the serving process's litestream PUTs go to reused keys (litestream v0.5.14 `LTXFilePath`, names by TXID range, restarting at 1 for a new lineage). A PUT signed before the evidence can still complete afterwards. So every retirement of a location that a replicator wrote acknowledges only after a verification listing taken later than `evidence time + H`, where H is the late-completion horizon (S3 refuses a request whose signature is more than 15 minutes old, so H is 15 minutes plus the longest request). The model drops this rule and finds residue in a freed host's namespace. ## 6. State inventory ### 6.1 Files on a box (codex R2-9) | # | state | where | disposition | |---|---|---|---| | F1 | SQLite DB, WAL, SHM | `/.db` (`provision.go:1219`) or nested `InstanceDir/.db` | carried as the replica; installed as a generation (§7.6) | | F2 | litestream meta dirs | `-litestream`, `.activator--litestream` (`backend_exec.go:1570-1571`) | rebuilt on restore | | F3 | `.gen` | beside the DB | protocol marker (§5.1); **local-parked with the database**, flat or nested | | F4 | **files directory** | flat: `strings.TrimSuffix(dbPath, ".db")` = `//`; nested: every entry in `InstanceDir` except F1 to F3, F5 to F11 and #199's markers | **carried** (§8), excluding `.files-gen` | | F5 | `.files-gen` | inside F4 | protocol marker (§5.1), never archived or digested | | F6 | `.env` | `paths.go:50`; `CARLOS_ADMIN_TOKEN` (`admintoken.go:115`) and operator overrides | **carried** in the archive, as its own entry | | F7 | `.config.env` | `syncexec.go:71-76` | **rendered** for the held row (§9.1), pinned by T3 | | F8 | `-sidecar.env` | `paths.go:51` | **refused** (§7.1) | | F9 | launch files, sockets, log sockets, exit records | `provision.go:1282-1294` | rewritten per start | | F10 | quarantined restore `.corrupt` | `backend_exec.go:313-324` | local-parked with the database | | F11 | layout journal | `/layout/.json` (`provision.go:1303`) | the cut waits while present (`movespass.go:447`) | | F12 | operation journals | `/.moves///` (new) | protocol state for this box's phases (§7.6, §7.8); parked at finalization | ### 6.2 Registry state (codex 16) Every `routes` column (`internal/registry/registry.go:34`): | column(s) | class | Phase 1 | |---|---|---| | Host, Slug, Kind, App, Account, Region, Channel, Environment, Hibernate, StatusNamed, Steering, Gate, Seed | reproduced | from the record by `routeFor` (`internal/provision/reconcile.go:364-376`) | | Socket, DBPath | reproduced | derived for the target row's layout; **Phase 1 lands flat** (`routeFor` mints flat) | | Provisioned | reproduced | true | | Status, hold_op (new) | protocol | §5.2 | | **Replication** | **carried** | box-local (`registry.go:138-145`, `reconcile.go:1100-1113`) | | **NextDue** | **carried** | the app's own next wake, probed at stop (`activator.go:2019-2030`) | | Version, PrevVersion | reset | materialized for the held row (§9.2) | | LastError, DNSStatus, DNSCheckedAt, CertExpiresAt, CreatedAt | reset | recomputed by their passes | | Caps, Unit, Backing (unit), Platform, Tunnel, FleetBox, Pool, Canary, Addr, Addrs, Health | **refused** if set | not the platform's to move | | Replica, Replicas | **refused** | a replica, or a home with replicas (Q2) | | IdleSeconds, MaxRunSeconds, Args | n/a | sidecar columns | The `timers` table (`registry.go:565-579`) is carried as a snapshot (§9.4). ### 6.3 Other host-keyed state (codex R2-9) | state | where | disposition | |---|---|---| | restore-verification verdict | `control/restore-verify.json`, `hosts[]` (`internal/dbverify/dbverify.go:10`, `:33`) | the host's entry is **removed** at every install (landing and dirty rollback), so an earlier generation's "ok" never vouches for a new one; the next drill writes a fresh verdict | | config ack | `control/config-ack//.json` (`internal/appconfig/configack.go:111-118`) | move-bound acks (§9.1); the source's parked at finalization | | copy receipts | `move-receipts//.json` (source bucket) | keyed by operation | | snapshots | `move-snapshots///` (source bucket) | §8.3 | ### 6.4 Parking Phase 1 uses the claim spec's park (§6.2 there) and local retirement (§6.3 there), with the move's own destinations: - **One destination per retirement event (joint D5, N7).** Superseded data goes to `superseded////{instances,moves}/`, and local state to `.parked/superseded-///`. The landing's park of an earlier stay, a failed target's cleanup, the source's retirement after commit and each staging retirement are separate events with separate manifests, even inside one operation. The model drops the retirement ids and the park never finishes: the second event finds the first event's immutable manifest and stops on unlisted keys. - **What a move parks is its location, not the host namespace.** The landing's park of an earlier stay, the source's park after commit, a rolled-back target's park and a push's park all name `holder.location` in their own region (C3a); staging parks name `moves///`. - **Exact versions (joint N1, J2-R2-6).** The snapshot takes every version under the location, noncurrent included. The manifest records two sets (claim §6.2): the **archive set**, every non-marker version, copied by `versionId`; and the **deletion set**, every version, delete markers included, which is what a retry replays, by `VersionId`. The model replaces version deletes with bare-key deletes (delete markers on a versioned bucket) and finds parks that never finish, in 2,325 of 3,000 seeds (rev 5). - **The live lock is not data (joint N6).** The parker takes the location's replica lease first, and renews its own `lock.json` while it works. That key is excluded from the snapshot, the manifest and the emptiness check, and released by the parker's own version after the completion CAS. The model includes the lock and the park never finishes. - **Progress accumulates (model 2).** A copy's destination key carries the source version id (`@`, claim §6.2), so a retry skips copies that landed and the manifest is written once every copy exists. Restarting the snapshot from scratch on every retry, with many versions and late completions, does not converge in the model. - **Verify after the horizon (model 1).** A park acknowledges only after a verification listing taken later than `evidence time + H` (§5.3, claim §6.2a) and, for a staging retirement, later than the staging horizon (§7.3). Anything found then is parked as a further retirement event of the same obligation, and the verification repeats. - **Local retirement** takes the host guard and writes a retired marker first (claim C7), so a stale background write for that incarnation is refused. **Installation stamps** (claim rev 6.5 §6.4, R3-3, R4-2, R5-4): an installation's stamp is the claim `gen` produced by the CAS that made it current, and **that CAS writes it into `holder.stamp`** (claim rev 6.5 §2), so the claim is the durable record of every installation's stamp and the recovery source for every box. The CASes that install, and so write `holder.stamp`, are: a create's serving CAS (claim §5.1 step 4); adoption (claim §9); a landing's flip (§7.5); a push's probe CAS (F7); a dirty rollback's source probe CAS (§7.8); a reinstall after quarantine. A dirty rollback installs a new generation and **never keeps its predecessor's stamp** (rev 7.5's model did, R5-4). The stamp is kept for as long as the installation exists, serving included. Each box keeps, under the guard, a record per host incarnation: `{active_stamp | none, retired_stamps}`. **Initialization (claim rev 6.5 §6.4):** a box creates or refreshes its record only from a fresh claim read under the guard, only when that read names it `op.exec` or the landing participant (the phase table, §5.1), setting `active_stamp = holder.stamp`: a create's first wake, adoption on the serving box after the adoption CAS, a landing in its `checking` phase, a box whose record was lost, on restart. Rev 7.5's model initialized the record from the claim revision at reconciliation, which intervening claim writes change (R5-4). **No local record: refuse.** Every host-state writer (config sync, release adoption, the activator's admission) carries `{incarnation, stamp}` **captured from the box's local record when it selected its work** (claim rev 6.5's answer 1: the fresh claim read under the guard is still required, for the incarnation, the operation and the config and release identity, but the stamp is the selection's), and is admitted only if its stamp **equals** the active stamp; a lower stamp, a retired stamp, any stamp while the active stamp is `none`, and any stamp while the box has no record, are refused (rev 7.5's model admitted a writer when no record existed, R5-4). **Two retirements, named at every call** (claim rev 6.5 §6.4): `retire-artifacts(S)` parks the local artifacts tagged with stamp `S` and **moves no stamp**: neither `active_stamp` nor `retired_stamps` changes (rev 6.3 had it add `S` to the retired set; rev 6.4's answer 2 withdraws that). It parks **by tag**: every local artifact carries the stamp of the installation that produced it (claim §6.3's inventory), `retire-artifacts(S)` parks the ones tagged `S` and leaves the rest, and a caller that expected another tag has made a mistake the model records (R6-4). Where the files directory is carried into the new generation (a push, F7; a landing's `restored` phase; the activator's park of a warm database before a cold restore) the retirement is of the database alone. It is what a landing that has activated its own stamp in `checking` uses in `local` to park its predecessor's database, files and `.env`, what the activator uses on a warm database from an earlier stay, and what a dirty rollback's install uses on the source, whose pre-move stamp is still active at that point. Only two things move a stamp: **activation**, which puts the old `active_stamp`, if any, into `retired_stamps` and then sets the new one (so the dirty rollback's pre-move stamp retires when its probe CAS's new stamp activates), and `retire-installation`, which moves `active_stamp` into `retired_stamps`, sets `none` and parks every artifact; it is the local park at a delete, a move away (the source's finalization, §7.9), a rolled-back target's `cleaning` and quarantine (CC12). Nothing calls `retire-installation` to clean up a predecessor: rev 7.5's §7.6 reused the general local park in `local`, which read literally retires the installation `checking` had just activated and refuses its own probe (R5-4). **A cold admission is an installing CAS too** (claim rev 6.9 §6.4a; §5.1 step 9, §7.14): it writes a new `holder.generation`, a new `holder.stamp` and `holder.writer_box` in one write, with the predecessor's `{session, box, generation, stamp}` journalled in the same write; the box's record activates the new stamp from its next fresh read, and the carried files are retagged with it at the CAS, whether or not the start completes (the first rev 7.9 model left the files under the dead writer's stamp, and the next dirty install parked an artifact of a retired stamp, seed 453 / 30,000; rev 7.10's first model retagged them only at the start, seed 62 / 300); the restored database carries it; the dead writer's stamp retires with it. The purge of the local database that precedes every restore (§5.1 step 9) is not a retirement event: it deletes a file no installation will run again. There is no lineage file (rev 7.10). The admission's writer opens the new generation without a certificate of its own; its clean close writes the first one. **The return to source (claim rev 6.5, R6-4).** `holder.stamp` always names the installation `op.exec` should run, so **every CAS that changes `op.exec` writes `holder.stamp` in the same write**: a CAS that makes a new installation current writes the gen it produces; a CAS that makes an existing installation current again writes that installation's own stamp, taken from the op journal. The flip records the source's pre-move stamp `S` in the journal (`op.source_stamp`) as it writes the target's `F`; a rollback's return CAS (§7.8 `claimed`: `op.exec` back to the source) writes `holder.stamp = S`, never `F`. The source's initialization then finds `holder.stamp` equal to its `active_stamp` and changes nothing; a source whose local record was lost (a reboot) re-initializes to `S` from the same fresh read. A dirty rollback's install runs under `S`, its artifacts tagged `S`; the probe CAS issues `R` with `holder.generation`, and **activation of `R` tags the generation's artifacts `R`** and retires `S`. `F` was only ever active on the target and retires there (`retire-installation` at `cleaning`). Rev 7.6's model returned execution with `holder.stamp` still `F`, so the source activated the target's installation and tagged its artifacts `F` (R6-4); rev 7.7 encodes rev 6.5 and records any activation of a stamp the claim issued for another box as INV-6 (`return-stamp`, with a witness whose source has lost its record, §14.2.5). Op ids are opaque and never compared (claim rev 6.1 Q1). Rev 7.4's marker compared operations by an order the production system does not have and admitted its own installation only by a `- 1` in the model (R4-2); rev 7.3's marker fenced the whole incarnation, and the move back never passed its probe. The model keeps the stamp record exactly as claim rev 6.5 states it (every installing CAS writes `holder.stamp`, `initRecord` under the guard, refusal without a record, activation the only move of the old stamp, `retire-artifacts` a named no-op on the record), requires the UK→US→UK round trip to serve on UK again unmutated, records a landing probe refused by its own installation record as a violation, and shows a stale config pass admitted across a retire-and-reinstall only when the stamp check and the host guard's fresh check are both gone (§14.2.5, `fence-by-installation`, `fence-stamp`). No park destination is an ancestor of a live prefix (codex 8). **A park is its own journaled phase**, complete before any install into the vacated namespace begins (codex R2-6). **The straggler sweep (J2-R2-1, CC15, model 17; claim rev 5 §6.2a).** S3's signature limit bounds when a request is accepted, not when it completes (claim C3a), so a request accepted before a retirement's horizon listing can still land after it. Within one incarnation the restore contract (§5.1) makes such a straggler harmless to data. It is still residue, and no later retirement of that location may ever run. Claim rev 5's at-rest sweep owns it: for a host with no operation running, each region's box lists the host's namespace and parks what is not live, as a further retirement event, under a `park-sweep` obligation the sweeping box owns. **The chore carries a persisted park scope** (claim rev 6.23 §2; R13-4, R14-5, R15-1): `{location, incarnation, evidence_at, request_limit}`, written at the queuing CAS, so every retry, an agent restart's included, uses the same location, archive incarnation and **original** evidence time; a retry's own clock is never evidence, and a restart never starts the horizon again (rev 7.13's model kept the evidence time in a box-local map that the restart cleared; witness `wParkScopeSurvivesRestart`). **`location`** is exactly one of `instances/`, `instances//`, `moves//` (a move's staging or a rollback's return lane, the op id with no `/`) or `moves/` (used by a delete's park of the staging namespace, §5.2, and by an at-rest sweep, each while no operation stages there; claim rev 6.26, R17-2), nothing else and no trailing `/` (the model's INV-23 checks every scope against that grammar). **`incarnation`** names the archive: the location's own, the staging op's holder incarnation, or, for a free host's `moves/` sweep, the newest retired incarnation. **`evidence_at` is always the original evidence or fencing bound**, never the cleanup's start time: the cut's stop evidence for the commit's park of the source region; the release's stop, or the declaration's `fenced_at`, for the target region's; **a staging lane's own seal bound**, the time after which no staging writer, its lease holder included, can still write (the seal plus the staging lease), persisted when the lane is sealed or fenced; the sweeping box's complete listing for a sweep; the publication that supersedes a generation for a generation chore. A chore queued without a scope (a delete's or an adoption's `park-replica`, whose CAS has no evidence time yet) binds one in a CAS before its first step runs, once its stop evidence is in hand, and never again; INV-22, no remote chore runs a step unscoped. **One deadline for every scope** (R15-1): acknowledge only after a verification listing later than **`evidence_at + H`**, with **`H` = the signature-age limit plus `request_limit`** (claim §6.2a, §2: 15 minutes plus 120 s in production; `sigAgeH + requestLimitH = horizonH` in the model, 45 + 15 = 60 steps). There is no separate staging contract: rev 7.15 bound a staging park at cleanup and then kept the seal's quiet time instead of that bound, so a lane sealed at step 100, deadline 560, whose cleanup first ran at 600 would have waited until 660 (witness `wStagingScopeDeadline`: the cleanup acknowledges at once, through an agent restart, since the persisted deadline has passed). Rev 7.14 wrote `evidence_at + request_limit` here, dropping the signature-age allowance (R14-3). The sweep parks at once and acknowledges, with its straggler pass, only after a **verification listing later than `evidence_at + H`** (platform-1e's answer to rev 7.14 note 2: a park-sweep is a retirement event and is not exempt from the horizon). **On a serving host** the chore's op is `session:`, as for `PublishGeneration`. **On a free host** (no holder) the sweep is a **holderless sweep** (claim rev 6.15 §6.2a): one CAS sets `op = {kind: parking, id: , intent: {park_sweep: "v1"}}` with no `exec` and no holder and queues only scoped remote chores owned by that op; it is the only holderless claim that may carry obligations, so it blocks `free`'s successor create while it runs (the create waits, it is not refused), and its last acknowledgement frees the claim again in the same CAS (`FreeAfterPark`), keeping `quiet_after` (witness `wFreeSweepVsCreate`; INV-20: a create never claims a host while a park of its namespace is outstanding). Another region's sweep while the op runs **joins** it with its own scoped chores under the same op id (R14-4, platform-1e's preferred reading of rev 7.14 note 1; `TestJoinHolderlessSweep` has two regions' residue parked under one op and one `FreeAfterPark`); any other operation running suppresses new sweeps; the console's recovery pass leaves the op to its runners and only replays `FreeAfterPark` for one whose chores are all acknowledged. **It discovers locations from the namespace itself** (claim rev 6 §6.2a, R3-4), not from `retired_incarnations`, which is capped at 32 entries and exists only to stop an incarnation being reused: it lists `instances//` with delimiter `/`, every page, refusing on a listing error; a child prefix of 64 hex characters is an incarnation location, residue unless it is `holder.location`; a level directory directly under the host prefix is the legacy location, residue unless `holder.location` is the host prefix itself; and within `holder.location`, every generation other than `holder.generation` is residue. **What it finds is grouped by the location it belongs to** (claim rev 6.24 §2, R16-2): each incarnation location its own, the legacy `instances/` (a level directory or generation directly under the host prefix, never a child incarnation), `moves/` for staging residue; **one chore per location**, scoped to that location with its archive incarnation (the location's own; the holder's for the legacy location or the staging sweep; the newest retired on a free host), and the archive is written under that incarnation. **A park executes strictly within its scope**: its listing is confined to the scope's namespace (an `instances/…` scope never lists or parks anything under `moves/`, the legacy `instances/` scope excludes child incarnations, a staging lane's scope only its own lane), and every key a chore would archive must be a member of its scope (the model's INV-23 checks membership for every reference a park archives or deletes, in the manifest it builds and in a persisted manifest it replays, whatever rules are on, and an out-of-scope reference is never acted on, R17-1; rev 7.16 scoped the whole `instances//` listing as `instances/` under the serving incarnation and archived a deleted incarnation's late object from there; witness `wSweepScopeMembership`). The same confinement governs a delete's parks, which cover the deleted incarnation's own location and the staging namespace, never the whole host prefix (claim rev 6.25 §5.2); an older incarnation's residue is this sweep's. **The live set is region-qualified** (claim rev 6.5 §6.2a, R4-3): `{authoritative region, holder.location, holder.generation}` plus the versions §5.1 names for the generation, the snapshot version `generation.json` names included, is protected in the authoritative region's bucket only (the table in §5.1); within `holder.generation` every current version is live, the successor snapshots and the anchor among them (§5.1 steps 7 and 8); and, in claim rev 6.4 §6.2a's words, **a child prefix that is not an incarnation id (64 hex) belongs to the host prefix's own location**: a level directory `0000` to `0009` or a generation prefix `g-/` directly under the host prefix is the legacy location's, never an incarnation location (the model's `locationOf`; rev 7.6's first baseline misread `g-/` under a legacy host as an incarnation and swept the live generation, caught in 2 seeds of 3,000); a matching key in any other region is residue and is parked, so a replica PUT for a legacy host serving in UK that lands in US after US's retirement is parked, not exempted. Claim §6.2a owns the sweep-side rule: the sweep never parks or deletes an object version in the live set, never touches the version `generation.json` names, never treats the marker as residue, reads the live set under the same fresh claim observation it uses for everything else, and touches nothing in a location whose marker disagrees with `holder.generation`, raising an alert instead, **except** for the one disagreement that is normal (claim rev 6.10, R10-2): a marker that names the journalled predecessor of the latest admission, or of a chain of admissions each of which died before its marker, while `holder.generation` is ahead of it; then every generation of the chain and the marker's are live, protected, and no alert is raised (§5.1 step 9). The model keeps the real cap and shows a replication PUT of an incarnation 33 recreates old landing after the list forgot it (`sweep-namespace`), and an old US PUT for a legacy host serving in UK parked only when the live set is region-qualified (`live-set-region`, §14.2.5). Being an obligation, it holds off any new operation over that region until it acknowledges (CC11), and every delete names a snapshot version, so nothing written later is touched. **`moves//` too (CC20, in claim rev 5.1).** Claim rev 5's list left out staging. A staging request accepted before its lease ended can complete after the staging cleanup, and after the move's release nothing lists `moves///` again. The sweep also parks everything under `moves//` while no operation runs, when no move is staging there. The model drops this and leaves staging residue for good (§14.2.5). A periodic listing per host and region is the cost (rate: once a day, and after each operation's release). The horizon of §6.2a stays: it is what makes INV-10 hold **when an operation ends**, for every request that completes within `H`; the sweep covers only what completes after it. The model checks both: INV-10 at every release and free, excepting only versions created after `H`, and INV-10 everywhere at the end of the run. ## 7. The move ### 7.1 Request and refusals `POST /api/cli/apps/{account}/{app}/instances/{host}/move` `{"region":"uk","cut":"auto"|"manual"}`, operator-only (§12). The door CASes the claim from `serving` (holder = this incarnation, execution holder = the source box) to `moving` with op id = the move id (§19 K2). **The target box is named in that CAS** (region HA L15, ruled H12): the door computes region HA's `choose(eligibleBoxes(region, pool), app)` (spec 1 §5.2: prefer a box running no instance of the app, then the most headroom, then the SHA-256 tie-break) over the roster and the edges snapshot. **Phase 1 scope (rev 7.20):** a move **into or out of a region with more than one eligible box is refused** at the door, with M-C21, until region HA's own model and plan cover multi-box regions; so in Phase 1 `choose()` always names the region's one box, and the source is the one box of its region. The refusal is a scope rule, not a safety rule: the protocol below is written for any target box the start CAS names. The choice is made once and recorded as `op.participants[1]` and `op.intent.target`; readiness (T8, T9), evidence, install and recovery bind to that box, and nothing substitutes another box: a target that cannot complete is replaced only by an explicit fenced transition (§7.10, L14), never by recovery picking a new one (P5-D2, A1-10). It is refused (400 or 409, with the sentence), and every box step re-checks: - not exec-backed `kind:"instance"`, addr-backed, pooled, fleet-placed, non-hibernating (`internal/console/apiinstancemove.go:53-72`); - a replica (`:187-190`, `provision.go:978-980`), or a home with replicas (new); - a target of home on a console without host claims, or a home whose box sets a region code of its own (`apiinstancemove.go:203-209`; F2, rev 7.24, Ruling R1), the current region, or one outside the manifest's `regions` (`apiinstances.go:1797`). A home target is named by the home's configured name; there is no `home` keyword (Ruling R3); - a source `-sidecar.env`, or any refused registry column (§6.2); - a private store bound to the instance's environment (codex 12; F4); - the claim not `serving`, or any box in either region below the protocol floor (§11). ### 7.2 Record fields Additive under `move` (`provision.go:907-917`). The claim carries the cross-operation serialization (§19); the record carries this move's phase journal, because only a move reads it. ```json "move": { "id": "mv_…", "incarnation": "…", "from": "", "to": "uk", "cut": "auto", "state": "landing", "requested_by": "…", "requested_at": "…", "updated_at": "…", "ready_at": "…", "inputs": {"bundle_etags": {"…": "…"}, "overlay_etag": "…", "release": "…", "files_plain": "sha256:…", "env_report": "sha256:…"}, "cut_phase": "flipped", "cut_at": "…", "cut_budget_s": 95, "source": {"replication": "stream", "next_due": 0, "db_sha256": "…", "files": {"plain": "sha256:…", "sealed": "sha256:…", "bytes": 1824, "entries": 4}, "timers_sha256": "…", "evidence": "…/stopped-source.json"}, "flipped_at": "…", "landing_deadline": "…", "landing_phase": "probing", "landing": {"installed_db_sha256": "…"}, "committed_at": "…", "opened_at": "…", "source_parked_at": "…", "rollback": {"id": "rb_…", "requested_at": "…", "reason": "…", "target_phase": "returned", "dirty": true, "evidence": "…", "source_phase": "probing", "recovered_at": "…", "opened_at": "…", "target_parked_at": "…", "release_deadline": "…"}, "reason": "…" } ``` Every box transition is a CAS on the record guarded by `move.id` and `move.incarnation`, **and** revalidates the claim's op id, kind and phase first (§19 K5). ### 7.3 Preparing and readiness **Staging layout (codex R2-7).** Each artifact class has its own subprefix under `moves///` in the target bucket: | subprefix | content | writer | |---|---|---| | `replica/` | the mirrored litestream replica; the only subtree `MirrorPrefix` touches | source rounds | | `files/` | sealed archive and manifest | source | | `timers/` | the timers snapshot | source | | `move.json` (outside every subtree) | `{"move_id","incarnation","host","from","to"}`, create-only | source, before the first round | `MirrorPrefix` runs on `replica/` only, so it can never delete an archive, a snapshot or `move.json`. Landing installs only `replica/` into `instances//` (§7.6). Receipts are `move-receipts//.json` in the source bucket. **The staging-work lease, sealing and draining (joint D4, N2).** M17's push-only grant stays: the source can put and delete in the target's `moves/*` but cannot read it, so it cannot hold the target's `lock.json`. Exclusion between staging writers uses the claim's shared staging-work lease in the deployment bucket, keyed by target region and host (claim C3). Every round, the final round and every staging retirement take it first, then revalidate the claim's op id and phase. That is enough for two workers of one operation (a restarted agent beside a stalled one), but a lease cannot fence a request already signed, so staging safety rests on two further rules: - **Seal by version.** Each PUT's response carries the new object's `VersionId`. The final round records the exact versions of `replica/…`, `files/…` and `timers/…` in the claim's op journal: the **seal**. Landing copies those versions by `versionId` and nothing else. A stale round that lands later creates a newer version, which the seal does not name, or a delete marker, which a copy by `versionId` ignores. So nothing has to be drained for correctness. - **Sign only before `SafeUntil`, and sign at the check (model 14, model 18).** A staging writer checks its lease's `SafeUntil` immediately before it signs each request, after any pause, and drops the request if the time has passed. Revalidating at the start of the round is not enough: a goroutine that revalidated and then paused can sign long after its lease ended. **The signature is computed at that check**, with the check's time as its `X-Amz-Date`, and never recomputed, including by the SDK's automatic retry, which must resend the original signature (claim rev 5, CC19): a pause between the check and the send, even a whole-VM freeze that no in-process bound can promise against, then ends in S3 refusing the request once its signature is older than the signature-age limit (403 `RequestTimeTooSkewed`, verified, §15 step 8). `SafeUntil` therefore leaves room for the signature's whole valid life: `SafeUntil = Until − clock skew − 15 min`. Rev 5's rule signed at send time; the stricter model freezes a box between a passed check and the send, and the write lands after the staging cleanup (1 seed in 100,000; CC19, §19.4). - **Retire staging after its writers' horizon.** Staging is retired (by the target-region box, §7.11) only after `seal time + staging-lease TTL + H`, when no writer that held the lease can still have a request in flight. That bound holds only because of the rule above: a request is signed at a check made before `SafeUntil − 15 min`, and S3 accepts it only within 15 minutes of that, so it is accepted before `Until` and completes within `H` of acceptance, or it is a straggler that completes later, which only the straggler sweep can retire (§6.4). All three are proven by the model (§14.2.5). A staging round's goroutine can pause between revalidation and signing (one round in three), past its lease; a cut round's lost-response retry can leave a paused duplicate (one in three). Dropping the seal by version loses acknowledged writes: the install copies a stale round's older snapshot. Dropping the staging horizon or the `SafeUntil` check leaves staging in the serving region or in a region the instance left. **Where correctness rests on the pause assumption (model 14).** Claim rev 3 keeps its clock and pause assumption "only for liveness", because every destructive request is version-bound. A staging PUT is not destructive, but its residency is: the staging horizon is a bound on signing time, and the only thing that enforces it is the `SafeUntil` check, whose margin must cover the clock skew and the longest pause between check and signature. The worst case is garbage under `moves///` (never live data, because the seal names versions), but in a region the instance left that is a residency failure. CC14 asks the claim spec to say so (§19.4). **Snapshot rounds (source).** Each round the source builds the files archive (§8) when the plaintext digest changes, and publishes its **launch report** `moves/////source.json`: every variable of the **composed** launch environment its next spawn would get, which is `childEnv` (`backend_exec.go:1158-1196`) over `instanceEnv` (`:1135-1153`) plus the agent-owned variables (`:975-993`), including the non-`CARLOS_*` variables `childEnv` inherits from the agent (codex R2-8). Each value is reported as an HMAC-SHA256 under a key derived (HKDF) from `CARLOS_SECRETS_KEY`, so the report reveals names and equality only. **Readiness report (target).** For every record whose `move.to` is its placement, the target writes `moves/////target.json`: | check | passes when | |---|---| | T1 protocol | this box is at or above the protocol floor (§11) (codex 7) | | T2 secrets key | this box publishes `secrets_key_fp` equal to the console's (§10) | | T3 environment | the target composes the environment it would launch with for a **flat** held row on this box (rendered bundle through §9.1, the carried `.env` from the newest archive, the agent-owned variables, and its own inherited agent variables), and every source variable matches by HMAC, with these classes: **region** (`CARLOS_REGION`): must differ as expected; **layout** (`CARLOS_CONTROL_SOCKET`, `CARLOS_PURGE`): must be present iff the flat layout provides it, value not compared; **box** (an allow-list of host-specific inherited variables such as `INVOCATION_ID`, `JOURNAL_STREAM`): not compared; **everything else**: equal. A `.env` variable that shadows a rendered one is reported. The report records the **input versions** it used: every bundle etag, the overlay etag, the archive's plaintext digest, the release version and the launch report's digest (codex R2-8) | | T4 storage | for each store bound in that composed environment (`internal/objstore/objstore.go:214-218`), with its credentials: list one key, put, get and delete `.carlos-move-probe/` | | T5 archive | the newest archive opens with this box's key, matches its manifest, has no special file, and is under the cap | | T6 disk | free space (`readDiskFrac`, `internal/host/probes.go:209`) covers twice the replica, plus the archive, plus 2 GiB, and stays under 85% | | T7 release (amendment, 2026-10-12) | the pinned release materializes for the held row (§9.2). With host claims on, T7 stages the pinned release into the shared version store and verifies it there (D3's admit, §4.1's authorization, the stage bound to its digest and D3's bind on the staged bytes) but does not publish `live/`: a `moving` claim grants no held write, and the link is this host's file. The publication moves to the landing's `configured`, under its `held-release` selection (§9.2). With host claims off, T7 is unchanged. A target that cannot write its version store under the claim reports that the release is not staged yet, because there is no root stage pass to relay it to. Paul ruled it on 2026-10-12 (Q2.1 of the moves pre-flight after claim Plan 12), so that a move can become ready while host claims are on | | T8 one owner | exactly one hosting box publishes the target placement | | T9 claim | the claim is `moving` with this move id, its holder is the record's incarnation, and this region and box are in `touched` (added by CAS before the first staging write, claim spec §2) | | T10 return lane | this box can put and delete `moves/_lane/` in the source's bucket (M17) | | T12 quiet region | the claim's `quiet_after` for the target region (CC1) has passed, so the landing's park of an earlier stay never waits inside the cut window. A move back into a region the instance left within H waits at `ready`, not mid-cut. **Defence in depth (model, rev 5):** under claim rev 3 no state reaches it, because every retirement acknowledges only after its own horizon and CC11 refuses a move over a pending chore; the model drops T12 and no cut ever waits (§14.2.5). It stays, because it costs one comparison and guards any future path that acknowledges early | | T13 landing armed (amendment, 2026-10-11) | the target box reports that it can land the move and roll it back (`landingArmed` and `rollbackTargetArmed`); a report without the check counts as unarmed. The source also refuses to start the cut unless its own recovery is armed (`recoverySourceArmed`), checked at `requested` before the staging lease and again at `admitted` before the stop; a refusal returns the move to `ready` with the reason shown and is never stuck. Paul ruled it on 2026-10-11 so that the switch is not the only guard against a cut nothing can land. An old source ignores T13, so every box runs a release with it before moves are enabled | | T11 budget | the **whole** stop-to-open estimate fits the operator's cut window (default 120 s) (codex R2-10, M11), from three measured rates, the **cross-region copy path**, **regional S3** (reads and writes from a box in the bucket's region) and the box's **disk**: stop + final checkpoint and `SyncAndWait` + final mirror (at most the 16 MiB bar at the cross-region rate) + archive build and upload + **the target's build (§7.6 `building`, R5-5, R6-3): the restore of the staged replica (the whole database read from regional S3 and written to disk), the snapshot write and the anchor write (each the whole database read from disk and written to regional S3)** + **the verification of the built database (§7.6 `restored`: one disk read for its sha256)** + render + **the probe attempts the window permits, each with its own settle** + the open (`open_budget`, 5 s) (joint N15, J2-N15). Rev 7.5 made the sealed transfer a snapshot of the stopped database and still priced the cross-region copy at the 16 MiB bar: a 2 GiB database over a 10 MiB/s path needs about 205 s for that copy alone, so T11 admitted a move the deadline then rolled back (R5-5). Rev 7.6 built the snapshot on the target and priced three passes at one local rate, then restored the generation cold a fourth time in `restored` without pricing it: the review's 3 GiB at 200 MiB/s with one probe came to 108.68 s priced and 124.04 s run (R6-3). Rev 7.7 verifies the built database instead of restoring it again and prices every byte moved. **Rev 7.11 prices the probe's own protocol (R10-8):** every probe attempt is a cold admission that purges the built database and **restores the prepared generation again** before the app starts (one regional S3 read and one disk write of the whole database), plus the admission's overhead (the CAS, the purge, `generation.json`: 2 s), the probe and its settle; every attempt after the first is a **fresh generation** whose first upload is the whole database again (one disk read and one regional S3 write). Rev 7.10 priced the previous protocol, three build passes, a verification and the probe alone, so the review's 2.5 GiB at the example rates was admitted at 113.8 s against the 120 s window and would have run to 131.2 s (129.2 s before the admission's overhead). **The worked example**, the same in this spec, the change note and `budget_test.go`: 2 GiB at 10 MiB/s across regions, 250 MiB/s to regional S3, 500 MiB/s on disk gives 53.6 s of fixed costs (restore 12.3, snapshot 12.3, anchor 12.3, verify 4.1, the rest 12.6), a first attempt of 64.3 s (restore 12.3, admission 2, probe 30, settle 20), 117.9 s in all, and admits **one** attempt by 2.1 s; a second attempt would add 76.6 s; the cross-region snapshot transfer (204.8 s) is refused by the same arithmetic, so is the review's 3 GiB workload at 200 MiB/s (rev 7.6's arithmetic admitted it), and so is 2.5 GiB. **The permitted attempts depend on the budget (R3-7):** the attempts admitted are those whose summed costs fit the window after the fixed costs, `attempt(1) = restore + admission + 30 s + 20 s`, `attempt(i > 1) = attempt(1) + full upload`, at most three, and the instance is eligible only if that is at least one; the probe loop (§7.7) runs at most that many. Rev 7.3 required all three attempts to fit, 150 s of probing against a 120 s default window, so no default move was ever eligible; `budget_test.go` (§14.2) states the arithmetic and shows a tiny instance admitted with two attempts, the boundary at 70 s of fixed costs, and never more than three. The estimate is recorded per phase as `cut_budget_s`, and admission sets an absolute `cut_deadline` (§7.4) | **Source readiness.** S1: a probe request (§7.7) to the readiness path on the instance as it runs today answers 200 to 399 from its own process (codex 14); an app that fails is refused with "declare `moves.ready_path`". S2: its next spawn is strict (§5.3). **S1 is load-bearing (model 3):** a source box that cannot run the instance now (in the model, one without `CARLOS_SECRETS_KEY`, finding 2b's state) can be cut, and its pre-flip recovery then never passes its probe. The model drops S1 and finds moves stuck in `reverting`. `ready` requires a round at most the 16 MiB bar, a passing target report and source readiness, each under 10 minutes old. ### 7.4 The cut, journaled (codex 4, R2-8, R2-10) Each phase is written to `cut_phase` before its side effects. | phase | entered by | action | failure and recovery | |---|---|---|---| | `requested` | cut door or the source (auto-cut), CAS `ready` → `cutting` | none | | | `admitted` | source CAS, stamping `cut_at` | **only if** the report and source readiness are fresh **and** every pinned input version (`move.inputs`) still matches the current bundle etags, overlay etag, release and launch report; otherwise CAS back to `ready` so the checks rerun (codex R2-8) | abort is legal until `admitted` and refused after; both CAS the same field | | `stopping` | source | row `moving`, `hold_op = move id`; `ReconstructStopEvidence` (§5.3) | its budget, then `reverting` | | `stopped` | source | final checkpoint and `SyncAndWait`, so the replica holds every acknowledged transaction; record the **required final TXID** `N`, the `MaxTXID` of the newest level-0 file (§5.1 step 1); read `Replication` and `NextDue`; snapshot the timers rows to `move-snapshots///timers.json` **and** `…/timers/` in staging; write `stopped-source.json` with `N`; CAS the claim's execution holder to `none` (§19 K3) | | | `staged` | source | byte-final mirror round into `replica/`, within the bar; the **seal**: `N` and the `versionId` of every current LTX file under `replica/`, of `files/…` and `timers/…`, in the op journal (§7.3), **and `src_covered`** (rev 7.11): whether the source's own replica of `holder.generation` reached the stopped database's position with no pending write, which decides a rollback's clean return (§7.8); **and `from_db`**: whether the seal was built from the stopped database itself, which decides whether the landing's admission has anything to declare (§5.1 step 9). A round never forces a sync on a session whose start has not opened replication (§7.14): it mirrors the remote as it stands | 3 deferred rounds or its budget: `reverting` | | `archived` | source | build the final archive from the stopped directory, seal, write create-only to `move-snapshots/`, record both digests, copy into `files/` | over the cap, a special file, or its budget: `reverting` | | `flipped` | source, the flip CAS | `region = to`, `state = landing`, `landing_deadline`; and the claim: `op.kind = landing`, execution holder = target (§19 K4) | | **One absolute deadline, equal to the promise (joint N15, J2-N15).** The M2 promise is the window from the source's stop to the target's open. Admission writes `cut_deadline = cut_at + cut_budget − open_budget` into the move journal. Every phase before the commit checks it, on the source and on the target alike, and **the commit is a record CAS conditional on `now ≤ cut_deadline`** (§7.9): that CAS is the decision, and the claim follows it unconditionally, so the deadline cannot be passed between the two writes. After the commit the target is authoritative and its open is a local conditional update on its own box, bounded by `open_budget`; an open that overruns it is reported against the promise, but nothing is rolled back after a commit. Exceeding the deadline before the commit starts recovery: `reverting` before the flip, a rollback after it. Rev 5 bounded the commit and called that the promise; the promise was stop to open (J2-N15). The recovery's own deadlines (`reverting` 15 min, the release deadline 15 min) are separate from the promised window (codex R2-10). Rev 3 checked the deadline only while probing; the model found commits far past the budget after slow installs, and finds them again when the deadline is dropped. The source does not purge its database or files at the cut; finalization parks them (§7.9). **`reverting` (pre-flip recovery, codex 4, joint N11).** The record still names the source. The source CASes the claim to `op.kind = reverting` with itself as execution holder, holds the row as `restoring` with `hold_op = move id`, and runs a **journaled** recovery, `op.journal.rev_phase`: | phase | action | replay | |---|---|---| | `evidence` | entered from `stopping`: evidence was never obtained, so something may still run. Compute it (§5.3) and start nothing until it holds. No files baseline is checked, because the move has not touched the files | replayable | | `verify` | entered from `stopped`, `staged` or `archived`: check files and timers against the digests recorded at `stopped` | replayable **until `probing` is journaled** | | `probing` | render held config and materialize the release (§7.7), then probe | restarts only the probe. **Never re-runs `verify`**: the probe's process may have written, and those writes are now the authoritative state (INV-2) | | `probed` | conditional open on `hold_op = move id` | replayable | | `opened` | the claim waits for the staging obligation (§7.11); then the record clears (`last_move` = `rolled_back`) and the claim releases to `serving` (record first, §7.5) | | Rev 3 compared digests again on re-entry, so a crash after the recovery probe had written left the move stuck on a "mismatch" that was really the app's own work (joint N11). The model reproduces that when it drops the journal. Fifteen minutes of failed probes: `stuck`, held. **Source digests at `stopped`.** The source records `files.plain` (§8.3) and `timers_sha256` at `stopped`, before any phase that can fail, so every recovery has a baseline. ### 7.5 The flip, and how the claim follows the record (J2: record/claim ordering) **Every move transition is one record CAS, and the claim follows it.** The record carries the move's journal, so the record CAS is the decision: the flip (`archived` → `flipped`), the commit (`probed` → `committed`, §7.9) and a rollback (`landing` → `rolling_back`, §7.8) are each one CAS on the record, guarded by move id and incarnation. The commit and a rollback CAS the same field, so exactly one wins. The claim half is a second CAS that the record fully determines: the op id, the claim's prior kind and phase, and its new kind, phase and execution holder: | record transition | claim CAS that follows | execution holder | |---|---|---| | `flipped` | `moving`/`sealed` → `landing`, `landing_phase = checking` | `none` → target, citing the source's evidence (`stopped-source.json`) | | `committed` | `landing` → phase `committed`, plus the obligations of §7.9; **`holder.generation` is not touched**: it is the generation the latest admission chose, the target's probe (§7.14), and a commit that wrote the prepared generation back would put an obsolete one into authority after a restarted probe (R10-3) | unchanged (target) | | `rolling_back` | `landing` → `restoring` under the same op id, `rollback.id` from the record; `holder.generation` untouched here too | unchanged until the target's evidence (§7.8) | **Any actor that reads both objects and sees the record ahead writes the claim half**: the actor that wrote the record, on its next step; the other participant; or the console's recovery pass (claim C2's recovery owner for this state). The CAS is conditional on the op id and the prior claim state, so two actors racing to write it cannot both succeed, and neither can write anything the record did not decide. Until it lands, the claim still names the previous state: after the flip nobody holds the execution right (the source gave it up at `stopped`, K3); after the commit the target is not yet admitted to serve; after a rollback decision the target is still the executor and must still give evidence. Each is safe. This replaces rev 5's wording, in which §7.5 had the source complete the handoff while K4 reserved it to the receiving participant (J2-D7). K4 now says the execution holder changes only in the claim CAS that mirrors a record transition, citing the evidence that transition names (§19 K4). The model writes the record half and the claim half as separate steps, with crashes, console loss and the recovery pass between them (§14.2.2). The source row stays held (`moving`); reconcile keeps it because the claim names this box as a participant (§9.6). **The close is record first too (rev 7.23, Paul's ruling M4).** The actor that closes first checks that no obligation this op created remains, then CASes the record: `move` is cleared and `last_move` names the move and its outcome (`done`, `rolled_back` or `aborted`). The claim's release to `serving` (§19 K8) is the second CAS, determined by `last_move`: the op id, its kind and phase, and the executor the outcome names. Until it lands, the claim still holds the op, so nothing else starts on the host. After a crash between the two, no box revisits the host (the box passes are driven by records with a live move), and the console's recovery pass writes the release from `last_move`. Rev 7.22 had this the other way round, claim first; claim Plan 10's recovery grant can act only while the claim holds the move op, so a claim released over a live `move` could never be finished by it. **Late record writes.** Every move write to the record is a conditional CAS. One that completes after its client gave up either still matches, and repeats a decision already taken, or fails on the ETag. ### 7.6 Landing and install (codex 5, 13, R2-6) The target's reconcile mints the row held as `landing` with `hold_op = move id` (§9.6). The target's worker runs these phases. Each is written to `landing_phase` before its side effects, and its local work is journaled in `/.moves///` (F12). | phase | action | done when | replay | |---|---|---|---| | `checking` | read `stopped-source.json`; require the claim's `op.kind = landing` with this move id and execution holder = this box; require this box in `touched.boxes` (CAS-add before any local write); read `move.json` and require its incarnation; under the host guard, from that fresh read, set the local record's `active_stamp = holder.stamp`, the stamp the flip CAS wrote (claim rev 6.5 §6.4, §6.4 here); acquire the lease on `instances//` in its own bucket (claim spec C3) and hold it through `published` | | replayable | | `parking` | bucket-park whatever is at `holder.location` in its own bucket into `superseded///` (an earlier stay of this incarnation) | the park manifest exists **and** its deletion set is deleted | replayable by the claim's rule (manifest present: replay the deletion set) | | `building` | **build the generation from the staged replica** (§5.1 step 2, R5-5): restore the sealed versions under `replica/` with the planner into the generation's database file, verifying each file's CRC-64 trailer, requiring exactly `N` and `PRAGMA integrity_check`; write the **snapshot** `[1, N]` to `g-/0009/<1>-.ltx` and the **anchor** `[N, N]` to `g-/0000/-.ltx` (§5.1 step 7); in this order (R7-4, §5.1 step 2): **the build intent** `g-/build-intent.json` with the encoding inputs (the header timestamp, page size, commit, `N`), before any PUT; the snapshot PUT, acknowledged; the anchor PUT, acknowledged; then **the completed install manifest** `g-/install.json` with the snapshot's key, the `versionId` each PUT returned, `N`, each CRC-64, `integrity_check`'s result and the sha256 of the database file the build produced (R4-1, R5-1, R5-2, R6-5) | the **completed** install manifest exists; the intent alone is not completion | replayable at every boundary: staging is immutable after the flip, the generation is new, and the build never deletes. A replay reads the intent, encodes identical bytes, and reuses a snapshot or anchor it finds current under its key; a crash after the intent, after either PUT, or before the manifest leaves no completion, so `published` never runs; a PUT that completes late lands in this generation again as another version of identical bytes, which nothing reads (§5.1). The model journals the same states (`install-copy`, `install-anchor`, `published`) and refuses, as INV-11, a `generation.json` naming a snapshot or anchor that has not landed | | `published` | the build is complete and recorded; **`generation.json` is not written here** (R10-3; rev 7.9 wrote it): the probe's cold admission publishes it, with op, incarnation, the install manifest's sha256 and the snapshot identity (`snapshot_key`, `snapshot_version`, `snapshot_txid`, `snapshot_crc64`, §5.1), after its CAS and before replication opens (§7.14) | the journal records the phase | replayable | | `local` | `retire-artifacts` (claim rev 6.5 §6.4, §6.4 here: never `retire-installation`, which would retire the stamp `checking` activated) any warm database (with `.gen`), files directory and `.env` of an earlier stay into `.parked/superseded-/`; extract the archive's files; write `.env` from the archive; `ReplaceTimers` from the snapshot (§9.4); set `Replication` and `NextDue`; remove the host's restore-verification entry (§6.3); fsync all; **then** write `.files-gen` | `.files-gen` names this op and incarnation, and the journal records each sub-step | each sub-step is idempotent and journaled; `.files-gen` is written last, so its presence vouches for all of them (codex R2-6) | | `configured` | render through §9.1 and materialize through §9.2; **for a forward install (this landing, a push)** the move-bound ack's bundle etags must equal `move.inputs` | ack matches | replayable; a mismatch is a rollback ("configuration changed during the move"), and the source's recovery then runs on the new bundle (§7.7, J2-R2-5). **A recovery install (§7.8) runs this phase on current inputs and records them; it is never gated on `move.inputs`** (R2-5) | | `restored` | **verify the database `building` produced** (R6-3): its sha256 equals the manifest's, `CheckDB` (`backend_exec.go:313`) passes, and `.gen` is written naming this generation and location; record `landing.installed_db_sha256`. No second cold restore: the file was built by the planner over the sealed versions on this box and verified at `N`, and a cold restore from the generation would yield the same pages by construction (the snapshot is every page of that file). The anchor's PUT was acknowledged in `building`, so litestream's open reads position `N` from the listing and replication starts at `N + 1` (§5.1 step 7); a `building` replayed after a crash rebuilds and the sha256 is checked again | | replayable until `probing`; a sha256 or `CheckDB` mismatch is a rollback | | `probing` | the probe (§7.7) | | **never replays an earlier phase** (INV-7); a crash restarts only the probe | | `probed` | probe passed | | | Any failure from `checking` to `probing`, or `landing_deadline`, triggers a rollback (§7.8). **One install routine, two sources (region HA L16; rev 7.19).** `building` and `published` are one routine shared with region HA's relocation (spec 1 §5.8): its input is a **pinned set of remote versions** (key, version id, level, TXID range) plus the required final TXID, and its output is the prepared generation's snapshot, anchor, build intent and completed install manifest exactly as above. A move feeds it the seal's `replica/` versions and `N`; a relocation feeds it the versions `genselect.SelectPrepared` or the generation's current listing pins and that listing's maximum TXID. The anchor rule, the two durable states, the replay rule and the marker rule are the routine's, so both callers get them once. **Every installation writes a files manifest in its new generation** (L17, §8.3; claim rev 6.28 C3, §6.2a, P6-D3; rev 7.21 after R18-1): the first manifest is **not** the session-bound publisher's, because no target writer session exists until `probing` (claim C3 binds that publisher to an admitted `{location, generation, session}`). It is the **prepared manifest**, as claim rev 6.30 (73b1b9dc) states the rule and region HA spec 1 §5.8's remote build (`:809`) uses it: **before any session exists, `op.exec` may publish an installation's initial files state in a phase that permits installation**: a move's `local`, a relocation's `installing`, or region HA's prepared build. That state is **create-only content objects plus one create-only prepared manifest**: in `local`, after the archive is extracted and verified, the target writes the installed files, `.env` and the wake state (the timers snapshot, `NextDue`, `Replication`) as region HA §5.6's immutable objects under `files/

/objects/` and then the one manifest `files/

/manifest.json`, **bound to `{location, P, op id, installation stamp}`** where **`P` is the installation's prepared destination generation, never `holder.generation`** (rev 7.22, R19-1): `g-` for a forward landing, the prepared return generation `g-` for a dirty rollback (§7.12), the same name the install manifest and the generation prefix of §5.1 use. `holder.generation` still names the source's pre-move generation `S` until the probe's cold admission moves it (§7.14, §9.4) and is **not** changed by `local`; a manifest bound to or filed under `S` could never be `P`'s baseline, so the first landing admission would block (claim rev 6.30 requires the prepared generation). The binding's other members are the move id (or rollback id), the holder location and the flip's stamp. The manifest is published **naming exact versions, with sequence 0, never replaced, and retired with its generation**; it is journalled in the op journal like the install manifest, before `.files-gen` is written, and a replay reuses the objects it finds by digest and the manifest by key (create-only, so a late second write fails). The **session-bound publisher starts only after the first admission** (the probe's, §7.14): it **takes that manifest by exact version as its baseline and publishes from sequence 1**; until then nothing but the install writes under `files//`. Manifest entries carry **origin-qualified references** (claim P6-D3): an object is named by its own incarnation location and region, so copying a manifest into a new generation never reinterprets its references, and a cross-region landing's prepared manifest names destination-local objects it materialized and verified from the sealed archive. **Files reachability is independent of the database baseline:** a cold admission establishes its files manifest from the newest valid manifest in the admitted predecessor chain, read by exact version, before the app starts; missing or malformed manifest data is never an empty-files bootstrap. The archive at the cut stays as the sealed transfer and the rollback's evidence. ### 7.7 The probe (codex 14) `ProbeWake(host, op)`, legal only on a row held with `hold_op = op` and only while the claim names this box as execution holder: - **first, render held config and materialize the release** (`RenderHeld`, `MaterializeHeld`, §9.1, §9.2), for every kind of probe: the landing, the source's pre-flip and rollback recoveries, and a push (model 4). Config sync and adoption skip held rows, so an instance created shortly before the operation began may never have been rendered. In the model, pushes and reverts of such an instance never pass their probe until this rule is applied to every probe; - **check the actual launch environment at spawn** (joint N8): recompose it exactly as the spawn will, including inherited agent variables after any restart. **The landing probe** requires the input versions pinned at admission (`move.inputs`) and the executable. **A recovery probe** (the source's pre-flip revert and its rollback recovery, §7.4, §7.8) requires the inputs it rendered itself, now, and records them in the journal (`recovery.inputs`); it is not bound to `move.inputs` (J2-R2-5). Rev 5 bound every probe to the pinned inputs: a bundle published after admission rolled the landing back and then failed every recovery probe forever, because recovery renders the current bundle. The pin exists so the target runs what the source ran (T3); on the source itself, the current bundle is what it would run had there been no move. A bundle that is itself broken fails the recovery probe as it would fail any wake: `stuck`, held, and paged; - a strict spawn (§5.3), with `CARLOS_MOVE_PROBE=1` (advisory); - the answer must come from the spawned tree: the socket peer's credentials (`SO_PEERCRED`) name a process in the host's cgroup; - the readiness path (`moves.ready_path` from the instances manifest, `/` otherwise) answers 200 to 399 within 30 s (`backend_exec.go:36`); - a 20 s settle, the process still running, and a second request; - up to three attempts for a landing, and never more than T11 admitted for this budget (§7.3, R3-7); a recovery probe (a revert's or a rollback's source) has **nine** before `stuck` (§7.8, §7.10; R12-5). **One attempt is one cold admission** (§7.14): its app may be restarted under the retry policy of §5.1 step 9 within the attempt's **30 s readiness window**, which runs from the first start (the waits total 15 s, each start's own time is on top), and the fifth failure or the window's end, whichever comes first, ends the attempt; the next attempt, if the budget allows one, is a new cold admission into a fresh generation restored from the dead attempt's. **The deadline is checked before each start and before readiness is accepted** (R12-2): an attempt whose window passed while the agent was paused, or between a failed start and its retry, does not start the app at all (rev 7.12 checked only while the environment was unready or after a failed start, so a successful start after the window made an expired attempt the probed writer); an app that answers readiness at or after the deadline is stopped and the attempt fails; and the probe's acceptance checks the attempt it accepts answered within its window (INV-19). At the boundary, readiness k steps after the attempt's first start: k = 29 is accepted, k = 30 fails before the start, k = 31 fails at the deadline with the environment still unready (`TestProbeWindow`). A probe never marks the host failed; the operation's rollback (or `stuck`) is its outcome. The budget's attempts and the policy's five restarts are different things: at most three (or nine) admissions, each with at most five starts inside 30 s. The absolute `cut_deadline` (§7.4) still bounds the whole. The probe's activator entry is **held**: `RetireExcept` (`internal/edge/edge.go:924`, `activator.go:2624`) keeps it, and the edge never admits a request to it (codex 13). ### 7.8 Rollback (codex 1, 2, 3, R2-3, R2-4, R2-5) **Triggers:** a landing failure (the target), or `landing_deadline` (the source or the console's sweep). Both are a record CAS `landing` → `rolling_back`, guarded by move id; the commit is a CAS from `probed` on the same record, so exactly one wins. The CAS sets `region = from`, `rollback.id`, `reason`, `release_deadline = now + 15 min`, and the claim moves to `op.kind = restoring` under the same op id, with the execution holder still the target until its evidence arrives. Both boxes stay participants, so **reconcile keeps both rows held** (§9.6, §19 K9): the target row is not retired by the region change (codex R2-4). **Target release**, journaled as `rollback.target_phase`: | phase | action | |---|---| | `releasing` | row `releasing` with `hold_op = rollback id`; `ReconstructStopEvidence` | | `released` | digests after a final checkpoint: database sha256, files plaintext, timers. **Clean** when all three equal what was installed (`landing.installed_db_sha256`, `source.files.plain`, `source.timers_sha256`) or `landing_phase` never reached `probing`, **and the seal recorded `src_covered`** (§7.4: the source's own replica reached its stopped database's position); otherwise **dirty**. Rev 7.11 adds the second condition (rule `seal-return`, INV-15): when the source's final upload was still in flight at the stop, the seal holds a write the source's replica does not, and a clean return would restore the source without it, declaring lost a write the staging and the landing's built generation hold. Write `stopped-target.json`; CAS the claim's execution holder to `none` | | `returning` (dirty only) | the handoff back: after a final checkpoint and `SyncAndWait`, write a snapshot of its stopped database (one level-9 LTX file `[1, N']`, every page, as the pinned writers write it, §5.1 step 2) into the source bucket's `moves///replica/0009/<1>-.ltx`, seal it by `versionId` with `N'`, publish its files archive and timers snapshot under `files/` and `timers/`, write `move.json`. **A target that never restored a database** (dirty only for want of `src_covered`) returns the staged seal as it was given, pinned by the versions the cut journalled, and the staged files and timers with it. The return is under the release deadline (15 min), not the cut window, so the whole snapshot crosses the region here | | `returned` (dirty only) | every artifact published | | `cleaning` | only after the source records `recovered_at` (below): **bucket-park its `instances//` and its staging `moves///` into `superseded///`**, local-park its database, `.gen`, files and `.env`, delete its row (conditional on `hold_op = rollback id`), clear its timers rows | | `cleaned` | CAS `rollback.target_parked_at` | The `cleaning` phase is what leaves no copy of the incarnation in the target's live namespace, so a later delete and recreate there cannot restore it (codex R2-5, INV-10). It waits for the source's recovery because, until then, a source failure might need the target's state again. **Source recovery**, journaled as `rollback.source_phase` and in `/.moves///` (codex R2-3): | phase | action | replay | |---|---|---| | `waiting` | until `released` (clean) or `returned` (dirty) and the claim's execution holder is `none` | | | `claimed` | CAS the claim: `op.kind = restoring`, execution holder = this box, citing the target's evidence (§19 K4), **and `holder.stamp = op.source_stamp`**, the source installation's own stamp the flip journaled (claim rev 6.5: a CAS that makes an existing installation current again writes that installation's stamp; R6-4); **clean, with `src_covered`: `holder.generation = op.journal.src_gen`**, the source's generation again, for the restore. The predecessor of the source's re-admission is still the target's writer, the last record of `holder.admissions` (claim rev 6.12, R11-4), and its coverage is `clean-return`: nothing is declared. The source's fresh read then matches its record, or re-initializes a lost one, to `S` | | | `restoring` | row `restoring`, `hold_op = rollback id` | replayable | | clean: `verified` | files and timers digests equal its `stopped` digests (a mismatch is `stuck`, INV-4); its warm database's `.gen` equals its pre-move generation | replayable | | dirty: `parking`, `building`, `published`, `local`, `configured`, `restored` | the target's install phases of §7.6, on the source, from `moves///` (`building` restores the returned snapshot, requires `N'`, and writes the rollback generation's snapshot and anchor), with destination `superseded///` for what it supersedes, **with two differences: `configured` and the release materialization run on the inputs the source renders now, recorded as `recovery.inputs`, and are never gated on `move.inputs`** (§7.7, J2-R2-5); and the stamps: the install runs under the source's pre-move installation `S`, which the `claimed` CAS wrote back into `holder.stamp` and which stays active; `local` runs `retire-artifacts(S)` on the pre-move database, tagged `S`, which moves no stamp (a leftover tagged with a **retired** stamp, left by a cold admission that issued its stamp and then failed its purge, is parked too, rev 7.11); the return generation's database and files are tagged `S` as they are installed; **the probe's cold admission** (§7.14), which names the return generation in `holder.generation`, issues the **new `holder.stamp` `R`** in the same write (`KeepStamp` false: a writer was admitted since `S` was issued), and the source's fresh read under the guard activates `R` before the open, tagging the return generation's artifacts `R` and retiring `S` then (claim rev 6.5 §6.4: a dirty rollback installs a new generation and never keeps its predecessor's stamp, R5-4, R6-4; rev 7.5's model kept it, rev 7.6's activated the target's; rev 7.11's row still said the `probed` CAS wrote them, R10-3). Rev 7.3 said "exactly" and so inherited the landing's `configured` gate: a bundle published after admission, a target that wrote, and a dirty rollback would then have failed before the recovery probe (R2-5). The model's `recovery-current-inputs` witness is that case (§14.2.5) | as §7.6 | | `probing` | the probe, an `AdmitWriter` (§7.14): dirty, cold into the return generation from its pinned seal (`baseline: sealed`, no `writes_lost`); clean, warm or cold into the source's own generation, which the return CAS named again | **never replays an earlier phase**: once its recovery process has started, its own new writes are authoritative (INV-2, INV-7); a probe that dies before its session opened replication, or in its settle, is restarted, a new admission | | `probed` | the phase CAS alone: **`holder.generation` and `holder.stamp` moved at the probe's cold admission** (§7.14; a dirty return's generation and its new stamp, claim rev 6.5 §2 and 6.9 §6.4a), and the `probed` CAS writes neither (R10-3; rev 7.10's row still named the return generation here). Dirty: the return lane's `park-staging` chore is queued now | | | `recovered` | CAS `rollback.recovered_at` | | | `opened` | the conditional open on `hold_op = rollback id`; CAS `rollback.opened_at` | | **Obligations.** The target's `released` CAS also adds to the claim, in the same write, a `park-superseded` obligation for the target box and region (its replica, staging and local state; destination `superseded///`), which its `cleaning` phase discharges, and in the same write runs `CoverRegionPredecessors` for the target region (claim rev 6.14 item 6, R12-3): every record whose predecessor's objects are the target's (a restarted probe's record names the dead probe's generation there) is covered, with no chore, since the park retires the whole location. The console's release of a declared-lost target (§7.10) queues the same park and covers the same way. The source's return admission comes after this CAS, so its record, whose predecessor is the target's writer, is **born covered** under the park the claim already holds (§5.1 step 9). A dirty source recovery adds one for its own region and box for what its install supersedes. These are the claim spec's durable chores (§2 there); §19.3 asks for the kind. **Finalization:** when `recovered_at`, `opened_at` and `target_parked_at` are all stamped, and no obligation this op created remains, whichever actor stamps last writes the terminal log event (`rolled_back`, with every timing, digest and reason), CAS-clears `move`, and then releases the claim to `serving` (§19 K8; record first, §7.5). Until then `move` and the claim stay, so nothing else can start on the host (codex 6). The source serves from `opened` on; the held claim blocks only other operations. Nine failed recovery probe attempts (three times the landing's budget of three; `probeTries*3` in the model): `stuck`, held (§7.10). Rev 7.12's sentence here said three, against its own model description (R12-5). **Waiting.** If `release_deadline` passes with no target evidence, the move stays in `rolling_back` ("waiting for uk to release"), paged, safe: nothing runs anywhere. It converges when the target returns (INV-3). ### 7.9 Commit and finalization (codex 6) 1. **Commit** (target): record CAS `probed` → `committed`, conditional on `now ≤ cut_deadline` (§7.4). The commit is also refused once a rollback reason is recorded, and it is bounded by the earlier of `cut_deadline` and `landing_deadline` (§7.8 makes the second a rollback trigger; Plan 2 ruling P2-deadline). Then the claim follows (§7.5): phase `committed`, plus the obligations in the same claim write, `park-superseded` for the source's location in its region and box, and `park-staging` for the target's staging (§19.3). **`holder.generation` is not written** (R10-3): it is whatever the latest admission chose, the prepared generation `g-/` when the first probe passed, a fresh generation restored from it when a probe was restarted (§7.14). Rev 7.9's commit wrote `move id` here; after a restarted probe that had acknowledged writes into its fresh generation, that put the obsolete prepared generation back into authority and the next restore lost them. 2. **Open** (target): conditional open on `hold_op = move id`; CAS `opened_at`. From here the target serves on the session its probe was admitted with; the claim's `landing` op with phase `committed` admits its later wakes as `AdmitWriter`s like serving's (§19 K6, §7.14). 3. **Source park** (source, holding the lease on its own `instances//`, claim spec C3): bucket-park its `instances//` into `superseded///`; local-park its database (with `.gen`), files, `.env`, quarantine and journals; delete its row (conditional on `hold_op = move id`) and its timers rows; CAS `source_parked_at` and remove its obligation. 4. **Target staging park** (target, holding the lease on its own `moves//`): `moves///` into `superseded///`; remove its obligation. 5. **Close** (when both obligations are gone): terminal log event `done`; CAS-clear `move` (`last_move` = `done`); then release the claim to `serving` with the target as execution holder (§19 K8; record first, §7.5). ### 7.10 Stuck and declare-lost (M16) **The termination fence is M16's evidence, automatically (region HA L14, ruled H10; rev 7.19).** Region HA §5.9 defines the termination fence: a regional worker observes state exactly `terminated` for the box's instance id, account and region as the roster records them, and writes the create-only fence record `control/box-fences/.json`; the claim puts the box on `lost` citing it (claim L2 to L4). For a move whose **target** is fenced that way, the move takes the fence as the evidence `--declare-lost` would have supplied, with no operator: the same legality table below applies (the target owes stop evidence, or released dirty and has not returned), **under the same before-commit rule**: before the record's commit CAS the move rolls back clean through the declared-loss path, with `fenced_at` as every scope's evidence time and the fence record as `rollback.evidence`; after the record's commit the target is authoritative and the move is not what recovers the instance: the move finishes its own close (its remaining steps are parks run by regional workers), and the instance's recovery is region HA's relocation, outside this spec. **A source terminated before the flip (rev 7.20; claim rev 6.28 §4; R18-2; claim rev 6.30).** "Before its flip" means **before the authoritative record enters `landing` through the flip CAS** (`archived → flipped`, which sets `state = landing`, §7.4). That is distinct from, and earlier than, the final commit CAS (`probed → committed`, §7.9): a record at `landing` whose claim half is still `moving/sealed` is already past the flip, and the conversion is refused there as everywhere after it. The fence converts the pre-flip `moving` operation to `relocating` in **one CAS**: it records the fence reference (below), sets `op.exec = "none"`, preserves `holder.stamp` and `holder.generation`, starts `relocating` at `pinning` in the source's region, and leaves every obligation untouched; the move's staging lane, `park-replica` scoped `moves//` and owned by the target region, keeps its op id and stays executable (the conversion queues it when the move had not yet, as an abort would), and it blocks neither the conversion nor the relocation's release. Readiness, evidence and recovery bind to the replacement box `choose()` names when the relocation leaves `pinning`. After the flip (the record's `landing` state) the conversion is **refused** and this section's rules for a lost target apply. **In a one-box region** the converted operation has no eligible target: it sits at `pinning` with exec `"none"` and the lane's park still owed until region HA enrolls a replacement box (automatic ASG enrollment, H16 and H17). That state is safe, since no writer exists and the data is pinned in the source region's bucket, but the host is **not recovered until a box exists**. The operator sees it as `relocating / pinning: waiting for a box in ` in `carlos move status` and the fleet view, and one incident alert is raised when the conversion finds no live box in the region (§13); the alert is not repeated. The operator's `--declare-lost` stays for the stopped, unclear and vanished boxes the fence does not cover (claim D2). **The fence reference (rev 7.20; moves C7, claim rev 6.28 §7.3).** Every transition that relies on a termination fence writes `op.journal["fence."] = {"enrollment", "key", "sha256"}`: the box's enrollment, the key `control/box-fences/.json`, and the SHA-256 of the exact bytes the transition verified. A `loss.` intent written by the same transition cites the same reference. `ReturnToSource` and every other consumer accept a fence only through that reference, re-reading the record at `key`, requiring its digest to equal `sha256` and its enrollment and cloud identity to match the roster's; the key is never reused for another box or enrollment within the operation. An automatic declaration from a fence needs no quarantine revision and no operator (L4); the terminated enrollment is permanently ineligible. **What a declared-lost box leaves behind (rev 7.11).** Its acknowledged writes beyond its uploads are declared by the next admission's `writes_lost` (the record's `remote_max_txid` names where an operator finds them in the seal or the box's generation); its **registered timers** are declared by the console's declare-lost record itself and alerted, since no return lane will carry them; neither is ever exempted silently (R10-6). When the target of a rollback is the lost box and the seal was the one holder of the **source's** writes beyond its own replica (`src_covered` false, §7.4, §7.8), the return is clean for want of a target and the source's re-admission from its own generation declares its **cut session** as well, a second `writes_lost` record (`{box: source, generation: source's, session: the cut's, remote_max_txid}`) with its alert: the seal in the lost target's region is unreadable from the source and no lane brings it. - **`stuck`:** the source's recovery probe failed. Held. `carlos move resolve --host --retry` reruns the recovery's `probing` phase. - **`--declare-lost ` (M16)**, operator-only. `carlos move resolve --host --declare-lost ` is the move-facing form of the claim spec's `carlos ops host resolve --declare-lost ` (§7.2 there). The joint review's D6 text governs it (joint D6, N3): > A claim CAS is bookkeeping, not fencing. The resolve command must > establish verified compute fencing, or an independently enforced > execution-lease expiry, before reopening the source. Recover validated > durable target data where available, not only an already-published > return. Retain M16's precommit restriction, exact-participant check, > source refusal and explicit loss confirmation. **This spec takes the first branch only: compute fencing (model 5).** The verb requires the operator's cloud credentials to show the box `stopped` or `terminated`, verified through the cloud API by the CLI. An expired execution lease is not enough, even with the self-fencing watchdog: a whole-VM pause stops the watchdog with everything else, and on resume the application can write before the watchdog fires. The model allows the lease-expiry branch and finds two writers when a frozen target resumes beside the recovered source (1 seed in 30,000; 2 when the watchdog also runs inside the agent, so an agent pause defeats it too). The lease stays useful for liveness: a partitioned or hung participant fences itself. It is legal in exactly these states, and refused otherwise: - **the target owes stop evidence** (landing, or a rollback before the target released): it proceeds as a clean rollback, after a confirmation naming the box, the flip time, and that the target's writes since then are lost (M-C18); - **the target released dirty but had not returned its writes** (model 6): the same, and the confirmation says so. Rev 3 missed this state; the model finds a move waiting forever on a return that a destroyed box can never send. **The same CAS adds a `park-staging` obligation for the return lane, `moves///` in the source's region, owned by the source region's box, not before the lane's staging horizon** (model 27): the lost target may have published part of its return, and the rollback now finishes as a clean one, which parks nothing else there; - **a destroyed box owes only local chores** (`park-local`): the obligation is removed and nothing else changes. This one is legal after commit, because no data authority is involved (model 6); - refused after `committed` in the first two states, refused for the source, refused for any box that is not that exact participant. "After `committed`" means after the **record's** commit CAS, whether or not the claim has followed it yet (§7.5). Before the commit the verb's rollback is the same record CAS as any other rollback, so it cannot race a commit. The box goes on `lost`. **A returning box is quarantined** (claim §7.2): before anything else it retires every local artifact for the host under the host guard, and only then can an operator `--readmit` it. The model starts stopped boxes again after they were declared lost; without the quarantine they would keep stale local state for the host. ### 7.11 Abort Legal in `preparing`, `ready`, and `cutting` with `cut_phase = requested` (CAS). A move whose readiness never comes aborts itself after a bound, so an unreachable target never needs a human. **Who retires what (joint N13).** The source cannot read the target's `moves/*` and cannot write the target's `superseded/` (M17 grants it Put, Delete and prefix-scoped List on `moves/*` only), so it cannot park target staging. Abort therefore adds a `park-staging` obligation **owned by the target region's box**, not before the staging horizon of §7.3. The source parks its own snapshots. (Plan 5 note, ruling K19: "parks" here means the source retires them from its own bucket, every version of `move-snapshots///` through `DeleteVersion`; that prefix is outside the park scope grammar, and F3's retention of snapshots covers completed moves only.) The claim stays `moving` until every obligation has acknowledged; then the record clears (`last_move` = `aborted`) and the claim releases to `serving` (record first, §7.5). The model gives the source the target's park and records an IAM violation on every abort. ### 7.12 Deletes and parks under the claim - **Delete during a move is refused** (409): the delete doors and `PurgeApp` CAS the claim from `serving` to `deleting` (claim spec §5), so they cannot start while a move op holds it (§19 K7). A purge of an app with a moving instance waits. An operator who must delete aborts first, or waits for the commit or rollback to finish. - **App purge (joint N9, model 7, 8).** Claim §5.3 reserves the app first, then claims every host, then tears down stores. The model found two more rules it needs. **Purge adopts first:** a legacy host of the app with a record but no claim yet is invisible to a purge that looks only at claims, so the purge tore down a store under a serving instance; purge must run the one adoption routine (claim §9) for every unclaimed record of the app before deciding. **The reservation is the purge's journal:** the console's recovery pass finished a stalled purge's delete and freed the host, `free` dropped the reservation, a create for the same app went through, and the stalled purge then tore down the new incarnation's store. The reservation records the purge's phase; recovery drives those phases in order; only the purge's own completion removes it, never `free`. **The purge fence (claim rev 5.1, CC21):** a creating console can renew its registration, pause, and resume after purge's drain has removed that registration as expired. S3 cannot make the creator's claim write conditional on the registration object, so before purge removes a registration whose host claim does not name the create, it CASes that host's claim from free or absent to `{op: {kind: purging, id}}` with no holder, and releases it at `done` by writing a free claim. The creator's CAS from free or absent then fails. The model found the fence needs one more condition, and claim rev 5.2 adopted it (CC22, §19.4): the creator's CAS is If-Match the claim ETag it read when it registered, or If-None-Match: * if the claim was absent then; a 412 restarts the create from step 0, with a new registration; and a claim object is never deleted (C5a there), because deleting a released fence would let the paused creator's If-None-Match: * succeed. The creator's re-read of its registration just before the CAS stays, as a cheap extra; the model shows it is not sufficient alone (§14.2.5). - **Recreate during a move** cannot happen: create requires the claim free or absent (claim spec C2). - **A delete after a move** parks every region in `touched.regions` (claim spec §5). The move added each region and box to `touched` before writing there, so nothing it left is missed, and what it superseded is already out of the live namespace (§7.8, §7.9, INV-10). The claim's delete park copies `moves//` and `instances//` as separate subtrees, which is compatible with per-op staging (§7.3). ### 7.13 Crash at every step The crashed actor restarts with **no activator memory**; everything it needs is in the record, the claim, its journals (F12) and the box's actual process state (§5.3). | state / phase | source crash | target crash | console crash | |---|---|---|---| | preparing, ready | next round | next report | stateless | | cutting: requested, admitted | resumes the phase | n/a | | | cutting: stopping | `ReconstructStopEvidence` from cgroup and unit state | n/a | | | cutting: stopped to archived | resumes; the sealed archive is reused from its create-only key | n/a | | | flipped, record CAS done, claim CAS not | completes the claim CAS | waits for the claim | | | reverting | resumes the recovery, held | n/a | | | landing: checking to restored | holds `moving` | replays the phase per §7.6 | | | landing: probing, probed (the settle window) | | kills the tree, restarts the probe, which is an `AdmitWriter` within the landing (§7.14): cold, into a fresh generation restored from the dead probe's, since the dead probe's session has no certificate; the same for a push's, a revert's and a rollback's probe | | | any: the agent restarts under a running app | stops the app: the session's replicator died with the old agent and a new agent process is a new session (claim rev 6.10); the next wake is an `AdmitWriter`, cold, which declares what the app acknowledged meanwhile (§7.14) | the same | | | any: the admitted writer dies between its CAS and its marker | the generation is unfinished; its predecessor stays live; the next wake is a cold admission whose baseline is the predecessor, or the generation itself when it holds the first upload (§7.14) | the same | | | committed, the claim still `landing` | resumes the park | the target's next wake is an `AdmitWriter` within the landing (§7.14) | | | committed | resumes the park | opens | | | rolling_back: target phases | | resumes its phase; evidence from actual state | the deadline sweep is a CAS | | rolling_back: source phases | resumes its phase from its journal; never reinstalls after `probing`; a source whose database went with the crash verifies nothing and is admitted at its probe like any writer (§7.14) | | | | stuck | held; operator | | | ### 7.14 Writer admission (claim rev 6.10 §6.4a; R7-1, R8-1, R8-2, R9-1 to R9-4, R10-1 to R10-5) Every path that opens replication is an `AdmitWriter` (§5.1 step 9), one claim CAS made **before the app process starts** (claim rev 6.10, R10-1), by the box the claim names as `op.exec`, and never by another. Two modes exist, **warm** and **cold**; nothing else (R9-3). **Which box may be admitted in which phase**, in which mode, into which generation, and the CAS that handed it `op.exec` first: | op kind, phase | the box admitted | mode and generation | `op.exec` handed to it by | |---|---|---|---| | `serving` | the serving box, at every wake | warm into `holder.generation` on a certificate for `holder.session`; else cold into a fresh generation restored from `holder.generation` | the create's claim CAS, a push's release, a move's finalization or rollback release (§7.9, §7.8) | | `pushing: probing` | the pushing box, as its probe starts | cold into the push's prepared generation `g-/` (its snapshot and anchor), opened once; a restarted probe is cold into a fresh generation restored from it | the push's claim CAS (`op.exec` = the serving box) | | `moving` (before the flip) | the source, at every wake; the target writes nothing | warm or cold as for `serving`; the source's clean closes certify | the move's claim CAS (`op.exec` = source) | | `reverting: probing, probed` | the source, as its probe starts and at every wake after | warm or cold as for `serving` (`holder.generation` is still the source's) | the revert CAS, which cites the stop evidence of the cut (§7.5) | | `landing: probing, probed` | the target, as its probe starts and at every restart of it | cold into the landing's prepared generation `g-/`, opened once; a restarted probe is cold into a fresh generation restored from it | the flip CAS, which cites the source's stop evidence (§7.5) | | `landing`, committed, not yet finalized | the target, at every wake | warm or cold as for `serving` | the same flip CAS | | `restoring: probing, probed, opened` (the source's rollback) | the source, as its probe starts and at every wake after | dirty: cold into the return generation prepared from the target's sealed copy, opened once; clean: the return CAS named the source's generation again, and the probe is warm or cold into it as for `serving` | the return CAS (`op.exec = op.src`, claim rev 6.5), after the target's release wrote `op.exec = -1` with stop evidence | | `restoring`, the target's phases | nobody: the target is stopping and never opens replication again | | | | `creating`, `deleting`, `parking`, `purging`, `placing` | nobody: the admission CAS refuses these kinds (R9-4, R10-4, INV-13); a create's first writer is admitted after `CompleteCreate`, under `serving` | | | Two writers never overlap because `op.exec` moves only through a CAS that cites stop evidence or a lost-box fence (§5.3, §7.10), and the admission CAS requires `op.exec == this box` (rule `exec-writer`) and is bound to the observed `op.id`, `op.phase` and kind (rule `admit-bind-op`; the model's general CAS condition on `op.id`, C2, already refuses a closure whose operation changed, so the binding adds the phase and the kinds). **`holder.generation` is the only authoritative generation** in every kind and phase: an operation's prepared generation (`g-/`, a rollback's return generation) is built by the operation's install phases and becomes `holder.generation` at its probe's cold admission, never before; the flip and the commit move no generation (rev 7.9's commit did). A clean rollback's **return CAS names the source's generation again** (`op.journal.src_gen`, recorded at the flip; only with `src_covered`, §7.8), since the target's probe had moved `holder.generation` to the target's and the target wrote nothing; the source's probe then restores from its own region's remote of it, is covered `clean-return`, and journals the **target's writer** as its predecessor, the last record of `holder.admissions`, never the holder's reset fields (claim rev 6.12, R11-4, INV-16). **The admitted writer's start**, on its box, in this order; it is the agent's own loop around one claim CAS, not a set of claim transitions, and a new agent process never resumes it (a new agent is a new session): | step | action | if it fails | |---|---|---| | decide | read the claim; `op.exec` must be this box and the kind one that admits; the certificate for `holder.session` decides warm or cold; a cold admission's generation is the operation's prepared one if `holder.generation` is not it or a descendant of it, else a fresh id; `KeepStamp` when `holder.stamp > holder.session`; `Coverage` `sealed` for a prepared generation from a `from_db` seal, `clean-return` for a clean rollback's return with `src_covered`, else `""`; a lagging `generation.json` is read through `holder.admissions` (below) | nothing happened | | **`AdmitWriter`** | the CAS of §5.1 step 9, **first**: session, writer_box, the `holder.admissions` record with the last record as its predecessor, its own region and the predecessor's region; cold: generation, stamp unless kept, `writes_lost` only for coverage `""` with no certificate; **no chore**: the predecessor is the unfinished generation's baseline until the marker names the new one (R10-2); born covered when the claim already holds a whole-location park for the predecessor's region (a rollback's return); the carried files are retagged with the new stamp here | a conflict: re-establish the evidence and decide again | | purge | delete the local database, with an error-returning purge | the start waits and purges again; the session stands | | restore | warm: `holder.generation` from this region's remote, to at least the certificate's `max_txid`; cold, prepared: the prepared generation from its pinned snapshot and anchor (the operation's build journal); cold, fresh: the baseline from this region's remote, the predecessor or, behind a lagging marker, the nearest generation of the chain that holds an object | the start waits; a warm restore short of `max_txid` is an alert, the remote is incomplete | | start, readiness | the app starts on the restored database and answers its readiness path; nothing it acknowledges is outside the session. A probe checks its attempt's deadline before the start and before accepting readiness (§7.7, R12-2) | the retry policy (§5.1 step 9): a doubling backoff from one step, five attempts. **A serving wake**: at the fifth failure the host is marked failed, the start abandoned, and no session is created until an operator clears it. **A probe**: the fifth failure, the window's end or a late readiness ends the attempt, the operation counts one budgeted try, and the host is never marked failed (claim rev 6.14; rev 7.12's row applied the marking to probes too, R12-4) | | first upload (cold, fresh, a non-empty baseline) | the full file `[1, 1]` of the restored database, captured once and acknowledged to the client; the generation has a root; it is in the session's ledger (R10-7) | retried with the same bytes after the client's wait, whether the original timed out or completed unseen; the session is uncertain for good | | marker (cold) | `generation.json` names the new generation: the pinned snapshot identity for a prepared one, `snapshot_txid = 0` for a fresh one (its root is `[1, 1]`); never before the root exists (INV-11). **Then `PublishGeneration`** (§5.1 step 9), for every cold admission, a prepared generation's included, bound to the operation and phase it observed, legal in `serving` and in every operation a writer runs in: the walk over the uncovered records queues one `park-generation` per dead generation whose objects are in this region, the predecessor's, the chain's and any uncovered record's, and covers them; a predecessor in another region stays uncovered for that region's whole-location park or publication | retried after the timeout; the writer waits | | anchor (warm, when the restored position is above the level-0 maximum) | `[N, N]`, acknowledged to the client; in the ledger (R10-7) | retried with the same bytes; the session is uncertain for good | | open | replication opens; the replica reads its remote position at its first sync; every later upload is the session's. An app stopped idle before this point is not closed: the replicator opens, ships what the app wrote, then closes cleanly | | #### Admissions, coverage and retirement (operative, rev 7.14) This list, with §5.1 step 9 and the two tables above, is the operative rule set for writer admission (R12-4), mirroring claim rev 6.22 §6.4a's operative subsection item for item. Where any other passage of this spec reads differently (the restore contract's older wording in §5.1, the §7.8 rows as rev 7.11 had them, the authoritative-region table, the per-revision blocks of §14.2.2, which are history by construction), this text wins. 1. **The record.** Each admission appends `{session, box, generation, region, predecessor: {session, box, generation, stamp}, predecessor_region, mode, coverage, covered, writes_lost?}` to `holder.admissions`. The predecessor is the last record, never the holder's fields (INV-16); `region` is the admitted box's region, where its generation's objects are; `predecessor_region` is the last record's `region`, before any record the writer box's region, `""` when the generation holds nothing anywhere; a warm admission continues the same generation in the same region, so its region equals its predecessor record's (a clean return's first admission is cold: an old source certificate does not certify the target's session). Born covered: item 5. 2. **Coverage** `""` | `sealed` | `clean-return`, as §5.1 step 9 states it: `sealed` for a `from_db` seal of the stopped predecessor, `clean-return` for a clean rollback with `src_covered`, a target that gave its evidence and no declared-lost box, speaking for the target's probe session and the source's cut session only; `""` otherwise, and without a certificate it records `writes_lost` (behind a lost target, one record for every session the target was admitted into since the cut, and a second for the source's cut session when `src_covered` is false, under its persisted `cut.session`): `writes_lost` (one) plus `additional_writes_lost` (a list), each `{box, generation, session, remote_max_txid}`, unique by `(box, session)` across both, one alert per declaration (R13-3, R14-2; INV-21). The legacy identity `{session 0, generation "" (@root), holder.writer_box}` is valid in both; every other declaration has a session above 0, and every declaration names a box, so a predecessor with no writer is not declared (INV-24, claim rev 6.24, R16-3). A `writes_lost` record declares the writes of the session it names and no other (INV-18, R12-1). 3. **The baseline.** A prepared generation from its pinned snapshot in the admitting box's region, opened once; otherwise the predecessor generation's remote in this region; in an unfinished run the newest generation of the run that holds an object **in this region**, else the marker's, walked as §5.1 step 9 says (R13-1): a covered foreign link is passed over, a covered local baseline refuses and alerts; the start restores the selected run generation rooted at `[1, 1]`, the marker's through `generation.json` (R14-1); an empty generation outside an unfinished run is refused and alerted. 4. **Stamps.** A cold admission takes a new stamp except under `KeepStamp` (`holder.stamp > holder.session`); a clean return's CAS may name the source's generation and stamp again, leaving `holder.session` alone. 5. **Coverage, completely.** An entry becomes covered in exactly four ways (R13-2): at birth (a warm entry; an entry admitted while the claim holds an unscoped `park-replica`, or a park scoped exactly to `holder.location`, in its predecessor's region, claim rev 6.24); by publication; by `CoverRegionPredecessors` (item 6); by `CoverParkedPredecessors` (item 6). **Publication:** after the marker names the generation, one `PublishGeneration(observedOpID, observedPhase, observedSession, publishedGeneration, region, newID)` requires `holder.generation` to be the published generation and an entry to name it, then walks every entry in order, skipping covered ones, as §5.1 step 9 states: no objects anywhere, covered, no chore; live again, left uncovered, no chore; in `region`, one deduplicated chore, covered; another region, uncovered, no chore, the walk continues. Every cold admission publishes. Retirement happens only through it or through item 6. 6. **Parks cover what they retire.** `CoverRegionPredecessors(op, phase, region)` runs inside the CAS that queues a region's park of the whole location: the commit's park of the source region, the target's release and the console's lost-target release for the target region. It covers every record whose predecessor's objects are that region's and queues nothing. `CoverParkedPredecessors(op, phase, region, generations)` runs in the CAS that queues an at-rest `park-sweep` and in the CAS that acknowledges it, and covers only the records whose predecessor is among the generations that sweep parks. INV-17 checks the composition as each whole-location park completes. A foreign-region predecessor is never queued into its region from another one. 7. **Starts.** The admitting CAS first; a new agent process stops a running app, admits a new session and restarts it; failed starts are the retry policy's, five under one session with a doubling backoff; the failed-host marking is a **serving wake's** at the fifth failure; a probe attempt ends at the fifth failure, at its 30 s window, checked before each start and before readiness is accepted, or at a late readiness, counts one budgeted try (three for a landing, nine for a recovery) and never marks the host failed. 8. **Retention and lost boxes.** `holder.admissions` is trimmed conservatively: covered records from the front only, to eight, never past an uncovered one; no compaction. A declared-lost box's timers are declared on the console's record and alerted; adoption sets `holder.writer_box` to the pre-release writer's box. 9. **Park state** (claim rev 6.23 §2; R13-4, R14-3 to R14-5, R15-1; §6.4). Every remote chore carries a persisted park scope from its queuing CAS (a delete's or adoption's park binds one before its first step), `{location, incarnation, evidence_at, request_limit}` in the claim's grammar; `evidence_at` is the original evidence or fencing bound (a staging lane's is its seal bound, the seal plus the lease), never the cleanup's start; every retry uses it; every acknowledgement, a staging lane's included, waits a listing later than `evidence_at + H`, `H` the signature-age limit plus `request_limit`. A park executes strictly within its scope's namespace (claim rev 6.24 §2, R16-1, R16-2): the sweep's chores are grouped by location, each with its archive incarnation; an `instances/…` scope never touches `moves/` and the legacy `instances/` scope excludes child incarnations; INV-23 checks membership for every reference a chore archives or deletes, fresh or replayed from a persisted manifest (R17-1). A free host's at-rest sweep is a holderless `parking` op with `intent.park_sweep`, no exec and no holder, which blocks the successor create until `FreeAfterPark` frees the claim in its last acknowledgement's CAS; on a serving host sweep chores carry `session:`. **History.** The paragraphs that follow and §14.2.2's per-revision blocks record how these rules were reached; where they differ from the list, the list governs. An admitted writer that dies between its CAS and its marker leaves `holder.generation` naming an **unfinished** generation `generation.json` does not (R10-2): its journalled predecessor is the live baseline, no chore retires it and the sweep protects it. The next admission, by this box or another `op.exec` box, reads the lag through `holder.admissions` (§5.1 step 9): the marker names the latest admission's predecessor, or the predecessor of a chain of unfinished admissions, so this is the normal state and no alert; a `holder.generation` that holds objects (the first upload landed) has its marker published again, from the operation's build journal for a prepared generation, and the retire chores queued; an empty one is skipped and the baseline is the nearest generation of the chain that holds an object, else the marker's; the new marker retires them all. An empty `holder.generation` the journal does not explain is unexplained missing data: the start is refused and the box alerts. A clean rollback's return leaves the marker naming a descendant of `holder.generation`; the source publishes the source generation's marker again from its last version before anything else. `holder.admissions` lives in the holder, so an admission unfinished at a release (an abort released over a source admitted a tick earlier) stays explained through the next operation. A marker that lands after its writer died, before the writer's `PublishGeneration`, leaves the predecessor unretired: the next admission's `PublishGeneration` retires the predecessor of every record not yet covered. A pre-flip round never forces a sync on a session whose start is unfinished: there is no open replicator, and a forced upload would write the first upload's key with other bytes. **Behaviour change against today's activator.** Today `Restore` materializes the local database from S3 only when it is absent, "a no-op when the warm-cache copy is still on disk", and `doWake` relies on that: "when it's still on disk it is the warm cache: on one box, with the lease held, the local copy can only be what we ourselves checkpointed at the last sleep" (`internal/activator/activator.go`). Rev 7.10 changes this: every wake restores, warm or cold. It costs a restore per wake, one pass over the database from regional S3 at the local rate (§7.3 prices it, and prices the probe's restore per attempt), and it loses the acknowledged writes a crash left unuploaded, which the warm cache preserved on a same-box restart; those are declared through `writes_lost` and an alert. **The loss is normally within the replication interval** (§7.3, the sync class), the window between syncs; a failed or stalled upload extends it back to the last upload that landed, so the bound is the interval in operation and the age of the last durable upload in a fault (R10-8). Two more changes against today's activator (rev 7.11): the admitting CAS precedes the app start, so a start that fails leaves a session and a retry policy behind it rather than nothing; and an agent that restarts under a running app stops it and re-admits, where today's activator reattaches to the running unit. What it buys is one rule with no provenance machinery: a box's database is always the certified remote's, so a cached, truncated or foreign copy can never be served (R9-1), and the claim alone decides who writes what. A cold admission costs the three local passes of an install (§7.3) for an operation's prepared generation, one full upload for a fresh one, none for an empty one, and a dead generation to park. It runs after every crash, OOM kill, lease takeover or lost disk, after a close that could not certify (a retry, a timeout, a lost response, R9-5), and at the first wake of a created incarnation or an adopted legacy host, whose origin generation has no writer yet. It does not run after a certified close. **The source that lost its disk** during a revert or a rollback has nothing to verify against the digest it recorded at the stop: its recovery skips the digest check and its probe is admitted like any other writer (rev 7.9, R8-2); a target that never restored a database returns nothing dirty (rev 7.10: the probe's admission restores it, so a target whose probe never started has none). ## 8. The archive (codex 19) ### 8.1 Content A tar of the files directory (F4, without `.files-gen`) and `.env` (F6), in sorted path order, with owners and times zeroed and modes kept, so equal content gives equal plaintext. Regular files and directories only; a symlink, device, FIFO or socket refuses with its name. The 256 MiB cap is enforced while streaming (M5). Then zstd. ### 8.2 Sealing For the deployment's config keypair (P-256 ECDH, `internal/appconfig/crypto.go:44-59`): a fresh 256-bit data key; AES-256-GCM over 1 MiB chunks, each nonce the chunk index, the last chunk flagged; the data key sealed with `appconfig.Seal` (`crypto.go:86`, fresh ephemeral key and nonce); a manifest with the sealed data key, the sealed object's sha256, the plaintext sha256, the per-file list (path, size, sha256), the op id and the incarnation. Encryption is randomized and never made deterministic. ### 8.3 Identity, publication, retention - **`plain`** (plaintext digest) identifies content; **`sealed`** (ciphertext digest) identifies one immutable object. Content checks use `plain`; fetches use `sealed`. - The `plain` digest of the files directory as stopped is computed at `stopped` without sealing, so recoveries have a baseline before any archive exists (§7.4). - **One sealed object per cut:** written create-only to `move-snapshots///files-.{sealed,json}`, then its digests recorded in the record, then copied to staging `files/`. A crash before the record write leaves an unreferenced object; after it, replay reuses the recorded object (codex 19). - **Retention:** the source's snapshots stay until F3; staging is parked at finalization (§7.9, §7.8). - **The files manifest (region HA L17; rev 7.19).** The archive is the transfer; durability between moves is region HA §5.6's files publisher: immutable objects `files//objects/-` and one manifest per generation, replaced with `If-Match`. Every installation, a landing's and a dirty return's included, publishes its installed files directory and `.env` as that generation's first manifest in `local`, before `.files-gen`; the serving writer then keeps it current per the app's `files_durability`. The classifier treats `files//` as that generation's (claim L13), so a `park-generation` chore retires it with the generation, except objects the files live set names. ## 9. Changes to existing code (codex 13, 15) ### 9.1 Config render for a held row `SyncExec` skips non-`ready` rows (`internal/appconfig/syncexec.go:643-647`) and writes an ack only when a file changes (`:1159`). New: `appconfig.RenderHeld(ctx, rt, op)` renders the same file, writes it, and writes a **move-bound ack** with the op, the file's sha256 and the bundle etags, changed or not. `SyncExec` keeps skipping held rows, so the two never race on one file. ### 9.2 Release materialization for a held row `release.Adoptable` excludes non-`ready` rows (`internal/release/adopt.go:816-819`). New: `release.MaterializeHeld(rt, op, version)` stages exactly the pinned release (`move.inputs.release`). **Amendment, 2026-10-12 (Paul's ruling on T7, Q2.1).** With host claims on, the stage and the publication are split. T7 (§7.3) stages the pinned release and verifies it without touching `live/` (`release.StageHeldVerified`). The landing's `configured` step, and a dirty return's, publishes it under the claim's `held-release` selection, whose identity names exactly that version (`release.MaterializeHeldUnder`), and then reads it back as T7 does with claims off (`release.VerifyHeldLive`). The held render in the same step runs under a `held-config` selection (`appconfig.RenderHeldUnderBefore`, which keeps carry (c)'s journal before the write), and a merge that is not the pinned inputs is the "configuration changed" rollback before anything is selected or written. With host claims off, `MaterializeHeld` and `RenderHeld` run as before. The landing still refuses at `checking` under the claim (`RollbackClaimWired`, ruling G16) until this box's moves source is registered with the claim. ### 9.3 The activator - `ProbeWake` (§7.7); `RetireExcept` keeps held entries. - `ReconstructStopEvidence` (§5.3); strict spawns under a move. - The start check (§5.1) in `Start`, `Seed` and `ProbeWake`, reading the claim. - Restore writes `.gen` and parks a warm file of another generation. `promoteStagedReplica` and its call (`backend_exec.go:290`) are deleted. ### 9.4 Timers frozen, drained and restored (codex 15, R1-15 partial) - **Drained:** entering `stopping`, the source waits until the runner has no delivery in flight for the host (the runner's per-host claim, `internal/schedule/runner.go:154`) before taking evidence, so a delivery that started before the hold cannot advance a row after the snapshot. - **Frozen:** the runner skips, with no bookkeeping, any held row (today a blocked delivery still advances `attempts`, `runner.go:345-351`), and the converger neither upserts nor prunes timers for a host whose claim op is `moving` (`internal/schedule/converge.go:114`, `:223-241`). - **Replaced exactly:** `Registry.ReplaceTimers(host, rows)` deletes and inserts in one transaction, preserving `next_at`, `due_at`, `attempts` and `registration`. `UpsertTimer` resets them on rearm (`internal/registry/timers.go:46-55`), so it is not used here. - **The runner's own prune (joint N14).** A second actor deletes timer rows: the runner's daily sweep (`internal/schedule/runner.go:260`), whose `PruneTimers` removes aged completed once-timers even for a live host (`internal/registry/timers.go:183`). It cannot be frozen without stalling every other host, so the **snapshot is canonical instead**: the timers digest covers active rows and their bookkeeping, and excludes disposable history (completed once-timers). Pruning history therefore never changes the digest a clean recovery compares. The model includes history in the digest and finds clean rollbacks stuck on a "mismatch" the runner caused; it lets a held row's refused deliveries advance bookkeeping and finds the same. - **Carried timers are acknowledged state.** A once-timer the app registered is a write the app was told succeeded. The model counts it as one for INV-2, and finds timers lost when the files marker is written before the timers are imported (R2-6). - Declared schedules converge again once the claim returns to `serving`. - **`relocating` holds (region HA L17; rev 7.19).** The same freezing and draining apply to a row held by region HA's `relocating` op: the runner skips the held row with no bookkeeping, the converger neither upserts nor prunes the host's timers while the claim op is `relocating`, and the relocation's release flush carries the timers snapshot as §7.4 `stopped` does. ### 9.5 Edge None in Phase 1. Platform-domain hosts get the 503 pending page during the window (#174, `internal/edge/pending.go`); custom domains get 404 until F1. ### 9.6 Reconcile (codex R2-4) - **Mint:** while the claim holds a move operation, a participant box mints the host's row held (`landing` on the target after the flip, `restoring` on the source during recovery), never `ready`. - **Removal:** never removes a row of a host whose claim names this box as a participant, whatever the record's region (`reconcile.go:1114-1159`; claim spec §8, "rows that a running operation lists as participants are retained"). The move worker deletes it at its own finalization step. - **Orphan retirement** (claim spec §8, a row whose incarnation is not the claim's holder) does not act on a participant row of a move op; a move never changes the holder, so the case cannot arise for a live move, and the guard makes that explicit (§19 K9). ## 10. The secrets key and the shared fingerprint The platform reads no SSM parameter at runtime. An exec instance's secrets reach it through its config bundle, opened with the box's `CARLOS_SECRETS_KEY`, so a target needs only the deployment's key. **Shared with nearest-region R9.** One edges-file field for both specs: - **name:** `secrets_key_fp`, in `edgesFile` (`internal/host/edgesfile.go:155-176`, written at `:603-605`); - **format:** `"sha256:"` and 64 lowercase hex digits of SHA-256 over `(*ecdh.PublicKey).Bytes()` (the uncompressed P-256 point) of the box's `CARLOS_SECRETS_KEY`; - **not omitempty**, like `hibernation_configured`: absent means unknown, `""` means no key; - the console computes the same over `CARLOS_SECRETS_PUBKEY`. A box is key-ready when its value equals the console's. ## 11. Versions and the protocol floor (codex 7) - Every box publishes `moves_protocol: 1`; the console reports its own. The request door and T1 refuse unless every box in both regions and the console are at or above it, and every box is claim-aware (§19 K10). - Before the first production move the deployment writes `control/moves-protocol.json` `{"min": 1, "min_release": "vX.Y.Z"}`, create-only. **Every path that activates an executable** refuses a release below it: `carlos system apply`, explicit rollback, and **automatic boot recovery** of `carlos.prev` (`internal/host/systemapply.go:901`, restoring through `:817`), which validates `.prev` and any outstanding update journal before activating (joint N12; claim §10). Any downgrade of a box that is a participant in a held operation is refused. The console roll recipe states the same floor. The model lets a reboot restore an old executable and fails in 2,696 of 3,000 seeds (rev 5), mostly as parks an old executable can never let finish. - **The floor is published before any protected production rehearsal** (joint N12). Rev 3's rollout order had it the other way round (§15). ## 12. The operator surface (M16, M18) **API.** The existing doors (`internal/console/authcli.go:625-628`). Every move mutation requires an operator (`Store.IsOperator`, `internal/console/store.go:1858`); others get the family's 404. Instance-role tokens keep the status GET only (M18), which closes the member-minted token path (`apiinstances.go:1554`) for moves (codex 11). New: `POST …/move/resolve` (§7.10). `GET …/move?detail=1` adds phases, reports, evidence and reasons. **CLI.** `docs/site/cli.md` gets the heading in the same change (`internal/docsite` fails the gate otherwise). ``` carlos move --host --to [--manual-cut] [--no-wait] carlos move status --host carlos move cut --host carlos move abort --host carlos move resolve --host --retry | --declare-lost ``` `--wait` is the default; exit 0 done, 3 rolled back, 4 stuck or waiting, 5 refused. ## 13. Observability - **Move log:** one object per event under `moves/////` in the deployment bucket, beside the reports and evidence. - **Timings:** prepare, cut, land, and **offline** (`cut_at` to the open, or to the source's reopen after a rollback), each against its budget. - **Paging:** `rolled_back` (warning); `stuck`, `rolling_back` past `release_deadline`, a refused start check, and a refused stop evidence (stray process) (incident); `preparing` with no promotable round for 60 minutes (warning); a move converted to `relocating` by a termination fence with no live box in its region (incident, once; M-C22 is the operator's line, rev 7.20). - **Member stream:** each transition is an event on the app's platform log (`emitEvent`, `internal/activator/logsink.go:31-36`). ## 14. Tests ### 14.1 Unit and integration Every transition and guard; each readiness check failing alone (T4 against MinIO); every journaled phase crashing between its individual side effects and replaying; `ReconstructStopEvidence` against a real cgroup v2 hierarchy with a forked descendant that closes its descriptors and outlives its parent; the start check; `RenderHeld`, `MaterializeHeld`, `ReplaceTimers`; reconcile keeping participant rows. ### 14.2 The executable joint model Built for rev 4, as the coordinator directed and the joint review recommended: `internal/sim/hostlife/`, a deterministic randomized Go simulation. **It is test-only**: every file is a `_test.go` file, so `go build ./...` never sees it, and it imports only the standard library. Rev 5 re-encodes it against claim spec rev 3 (8b5bccb2) and this spec as they would be implemented, and every rule is a named, mutable guard (`rules_test.go`). #### 14.2.1 Running it ``` go test ./internal/sim/hostlife/ # the gate: 300 seeds, about a second HOSTLIFE_SEEDS=30000 go test ./internal/sim/hostlife/ -run TestHostLifecycle -v -timeout 60m HOSTLIFE_SEEDS=100000 go test ./internal/sim/hostlife/ -run TestHostLifecycle -v -timeout 300m HOSTLIFE_SEED=1796 go test ./internal/sim/hostlife/ -run TestHostLifecycle -v # one seed, full trace ``` **Rev 6: strict classification (J2-R2-10).** `mutate.sh` reports CAUGHT only when the run completed and reported a seed that violated an invariant, with the seed number parsed from its output. A build failure, a panic, a timeout or unparsable output is ERROR, never a detection; every run's log is kept. `expected-outcomes.txt` beside the script declares the rules not expected to be caught alone (liveness-only, unreachable, combination-only, unexercised), each with its reason. The sweep exits nonzero on any ERROR, any unexpected MISSED, and any expected miss that is now caught (STALE). **The evidence policy (Paul, 2026-10-06).** The deep sweeps were using half the shared box's cores and starving other agents. From rev 7.14 on: each revision runs **the gate** (300 seeds plus the witnesses) and **one mutation sweep at 3,000 seeds**, serially, with `HOSTLIFE_PAR=2` under `nice -n 19`, never alongside another sweep or a baseline; **baselines run at 30,000 seeds only**; the 100,000-seed baseline and the 30,000- and 100,000-seed mutation sweeps run **once**, at a final pre-implementation checkpoint, and only when Paul says so. A table cell or column those sweeps would have filled says "not run" for the revision, never a placeholder and never a number carried over from an earlier revision. Rev 7.14's 100,000-seed baseline had already run, clean, before the policy was set and is reported in §14.2.3; its 30,000- and 100,000-seed mutation sweeps were stopped mid-run and are not evidence. **Rev 7.14: `TestInterruptedCleanReturn`, INV-20.** `go test` runs the positive case of the run walk across regions beside the witnesses: a clean rollback's fresh source admission dies before its first upload and the recovery walks R → T → S, then one more interrupted admission walks R2 → R → T → S; both restore the source's data. INV-20 is checked in the create's claim CAS: no obligation may be outstanding when a create claims a host. **Rev 7.13: INV-17 at the park, `TestProbeWindow`.** The oracle for regional coverage runs when a whole-location park of `holder.location` completes in a region, not at the end: a record whose predecessor's objects were there and that is still uncovered is INV-17 at that CAS (a record born after the park is not in question). `go test` runs `TestProbeWindow` beside the witnesses: the landing probe's environment ready k steps after the attempt's first start, for k = 29, 30, 31. **Rev 7.12: the final oracle runs the recovery.** When the run is quiescent with the serving host at rest after a session that did not certify, the final oracle no longer exempts the acknowledged, unuploaded writes as a prospective declaration (R10-6: that recorded and alerted nothing, with `writes-lost` off too). It wakes the host, runs the cold admission for real, drives its chores to quiescence, and judges the database by what the admission declared; a recovery that cannot run within the bound is counted (`final: unverified prospective declaration`) and the local check is skipped for it, never exempted. **Rev 7.11: the rare-at-depth outcome.** `expected-outcomes.txt` may list a rule as `rare `: it has no witness of its own and the random sweep reaches it only at N seeds or more; below N its miss is reported as EXPECTED-MISS, at N or more a miss is MISSED, and a catch is never STALE. Three rules moved below the 3,000-seed line this round (`hostguard`, `stale-hold-release`, `strict-spawn`, §14.2.5) and are listed so at 30,000. **Rev 7.11: structured counters (R10-9).** The baseline logs one `COUNTERS {json}` line beside the human-readable outcome map; the table generator reads it and fails on a missing required field. Rev 7.10's generator parsed the map with a regular expression that split `admissions: cold:57938` at the wrong colon and published zeros for every counter whose name carried one. **Rev 7.1: one binary, no copies.** Mutation checks never touch the source. The test binary is built once (the only step that writes the Go build cache); `mutate.sh` then runs it once per rule with `HOSTLIFE_MUTATE=` (or `a,b` for a pair, `a,b,c` for a triple), in parallel. It refuses a binary older than the source. Its scratch (the binary, each run's log, the verdicts) lives under `$TMPDIR/hostlife-sim` and is removed once, after the whole sweep: ``` go test -c -o "$TMPDIR/hostlife-sim/hostlife.test" ./internal/sim/hostlife/ internal/sim/hostlife/mutate.sh 3000 # every rule internal/sim/hostlife/mutate.sh 30000 rule-a,rule-b # a combination ``` **Rev 7.3: the witnesses.** `TestWitnesses` runs in the gate: every witness scenario (§14.2.2) must hold. Under `HOSTLIFE_MUTATE=` it runs that rule's witnesses and reports each as violated or held; a witness that must violate under the mutation and holds fails the test. `mutate.sh` runs both tests for every rule and prints one verdict line with the two results side by side: ``` CAUGHT lost-quarantine: sweep 0 of 3000; witness violated CAUGHT fence-etag: sweep 17 of 3000 (first seed 341); witness none EXPECTED-MISS exec-admission: sweep 0 of 3000; witness held ``` A rule is caught if either catches it. A witness run that fails is an ERROR (the witness is broken, not the rule). `HOSTLIFE_WITNESS=` runs one rule's witnesses with their full trace. **Rev 7.4: the gate proves each witness's claim in full, and the harness cannot pass on an empty result.** Every witness names the failure its scenario produces (a substring of the violation). With no mutation the gate runs each witness in place (must hold), under **every proper subset** of its stated mutation set (must hold: a rule a combination names is shown necessary, not assumed) and under the whole set (must violate with its named failure, not merely some invariant); 26 witnesses, 92 runs, under a second. `mutate.sh` now fails on a scratch directory it cannot create and on an `xargs` failure, requires exactly one well-formed verdict per requested mutation, and reports a rule that has witnesses but produced no witness output as an ERROR, never as "none" (R3-6). `go test` also runs `TestReadinessBudget`, T11's arithmetic (§7.3), `TestProbeWindow`, the probe deadline's boundary (§7.7, rev 7.13), `TestInterruptedCleanReturn`, the run walk across regions (§5.1 step 9, rev 7.14), `TestJoinHolderlessSweep`, two regions' residue under one holderless op (§6.4, rev 7.15), `TestLegacyLostTargetReturn`, the legacy identity through a lost-target return, `TestStagingSurvivesLegacyPark`, sealed staging beside a legacy instance park (rev 7.17), and `TestForeignManifestReplayRefused`, a persisted manifest naming another incarnation's key refused under a valid scope (rev 7.18). **Rev 7.6: the pinned libraries, observed.** `internal/sim/litestreamfacts/` is a second test-only package: it drives litestream v0.5.14 and `superfly/ltx` v0.5.1 as this repository pins them, with the file replica client the lease tests use, and asserts every library fact §5.1 cites as "fact R5-n" (replication position after a snapshot-only install and with the anchor, `-txid` follow-only, level-0 retention, the checksum flags, snapshot rotation and retention, the rooted-restore gap). It needs no S3. A library upgrade that changes a fact fails there before it reaches the model or the spec: ``` go test ./internal/sim/litestreamfacts/ # about 9 s ``` **Rev 7.7: exact facts, and the anchored lineage through the Store (R6-2).** Rev 7.6's facts logged what they saw and some accepted either outcome: the level-0 expiry test passed when replication restarted and seven rows of nine survived, the retention test logged the newest restore's row count without asserting it, the anchor test required a position past `N` rather than exactly `N + 1`. Every fact is now an exact assertion that fails on any alternative behaviour: the newest level-0 file retained by identity through TXID- and time-based retention; the next upload exactly `[N + 1, N + 1]`; every row restored; the original continuation gone after the second rotation; the rooted restore refused with litestream's own message for the gap after TXID 1; the newest restore's exact row count. `TestFact_R6_5_AnchorReplayBytes` shows the anchor's bytes identical for the same inputs and different for another timestamp (R6-5). `TestGate_AnchoredLineageThroughStore` is the integration gate the review asked for: one anchored lineage through `internal/lease`'s Store shape (a DB with its own metadata directory, a replica client, `litestream.NewStore` with the compaction ladder, the snapshot and level-0 retention monitors), every interval shortened from hours to fractions of a second, three rounds of eight writes with level-1 to level-3 compaction, snapshot rotation and time-based level-0 expiry running, then disk loss and a cold restore; every sync advances the position by exactly one, a reopened box continues at exactly the newest remote level-0 TXID plus one, and every cold restore holds exactly the rows written. What the file client cannot show is stated in the package doc and in §14.2.6. **Rev 7.9: the empty bootstrap, observed (R8-3).** `TestFact_R8_3_EmptyBootstrap` starts with neither a database file nor a replica: the pinned encoder refuses a header `[1, 0]` ("maximum transaction id required") and `[0, 0]` ("minimum transaction id required"); a restore from an empty replica fails ("cannot calc restore plan: transaction not available"); litestream opened on an absent database against an empty replica syncs without error and uploads nothing; after the first transactions and one sync the replica holds level 0 `[1, 1]` alone (a TXID is one sync's WAL frames, not one SQLite transaction), and a cold restore then holds the row. The spec's empty generation (§5.1 step 9) rests on these five facts. **Rev 7.8: the Store gate asserts what it claims (R7-5), and the PUT fact (R7-1).** The gate awaits each maintenance event as a condition rather than sleeping past it (a snapshot rotation, level-1, level-2 and level-3 compactions, snapshot retention leaving exactly one snapshot, time-based level-0 expiry); its rows have distinct bodies with an update and a delete per round, and every cold restore is compared row by row; a tail with the monitors stopped writes, compacts into level 1, expires every level-0 file but the newest, and forces a restore that must take the level-1 file beyond the newest snapshot. `TestFact_R7_1_UnconditionalPut` reads the pinned S3 client's source: its `PutObjectInput` carries no `If-Match` or `If-None-Match`, and its key is prefix, level and TXID range only, the two facts §5.1 step 9 rests on. The package doc now says that S3 lists LTX files in filename order and ages them by `LastModified`, where the file client uses the encoded timestamp as the mtime. **Rev 7.7: the oracles are pure (R6-1).** Every invariant check that restores (INV-2 at each release, the final oracle) calls the restore calculation and nothing else, and asserts afterwards that the buckets' version counter and every version, the pending requests, the boxes' upload queues and anchors, the console's plans and the random source's draw count are what they were before; a check that changed any of them is its own violation. The final oracle's start check initializes the stamp record on a copy of the box. Rev 7.6's cold-restore oracle wrote anchors into the system it was checking (R6-1); every number in §14.2.3 and §14.2.5 is from the pure model. #### 14.2.2 What it models **Rev 6 (joint round 2): the model encodes the specs, nothing stronger.** Astra's §2 listed where the rev 5 model was stronger or more atomic than the text. What changed, and what did not: | J2 fidelity point | rev 6 model | |---|---| | restore stronger than the spec (an exact base pointer, historical lineage selection) | restore reads `holder.location` and `holder.generation` and replays every object of that generation (§5.1). The mutation restores the newest object under the location | | the start check stronger than the spec (compared with the committed lineage) | the start check compares the box's own three markers with each other and, as a separate rule, with the claim's generation (§5.1); rev 5's check is the mutation | | claim transitions followed by an atomic record write | record-first flip, commit and rollback, then the claim half as a separate step, written by the actor, the other participant or the recovery pass; the close record first too, then the claim's release, which the recovery pass finishes from `last_move` after a crash (§7.5, rev 7.23) | | every request completes by `issue + H` | one long-tail request in 20 completes up to 11 `H` late, after publication and after retirement; the restore contract keeps them out of data, the sweep retires them (§5.1, §6.4) | | check and sign, check and local write, atomic | staging writers sign at the check and may then pause, a frozen box without bound (§7.3); host-guard writers are independent processes (`carlos-config-sync`, `carlos-adopt`) that survive an agent crash, pause before the guard and re-validate under it (claim §6.4); door listings read O1, pause, select, read O2 (claim §6.5) | | atomic `localPark` | one artifact per step, the retired marker first | | creates and purges atomic, one host | creates are console plans (register, claim, record, serve, deregister), purges close and drain admission, plans run concurrently, the console crashes between any two; a second hostname takes part in creates and purges (claim §5.1, §5.3) | | record writes synchronous | a create's record PUT can lose its response and land later, rarely very late (claim §5.1) | | config an incarnation integer | bundle versions, published at any time, pinned at admission (§7.6, §7.7) | | park manifest of current versions | archive and deletion sets (claim §6.2) | | legacy discovery never meets local residue | a legacy host's earlier move can leave a database and files on another box; adoption discovery parks them (claim §9) | | a final oracle that replicates before checking | the oracle only reads: cold-restore checks of every durable write at every release and at the end, INV-10 at every operation's end, no restored database holding another incarnation's writes (§14.2.3) | | mutation infrastructure errors counted as detections | strict classification (§14.2.1) | | the return lane outside the lease | the return lane takes the staging-work lease of the source's region, with paused signers and the forward path's horizon (§7.8) | **Rev 7: a biased schedule beside the uniform one.** In half the seeds the scheduler aims at the interleavings C3a and the door exist for, which the uniform schedule never reached (deleting and recreating a host took longer than any straggler): - a freed host is recreated at once, in the region where it last served; - at the delete, the executor's last replication PUT is accepted and then completes at its worst moment: when another incarnation holds the host with an executor, or a create, in that region. Any other long-tail request of an earlier incarnation's replica data may complete at that moment too, within `H` or after it, and the early-completion fault leaves such requests alone; - a door listing is authorized at the delete, and stalls, before selecting or between signing and opening, until the object its key names belongs to another incarnation. **Rev 7.1: the purge fence, and a creator that pauses at its worst moment.** The model now encodes claim rev 5.1 §5.3 step 1b: before the drain removes an expired registration whose host claim does not name the create, purge CASes that host's claim from free or absent to a fence `{op: {kind: purging, id}}` with no holder; a fence already held by another purge makes it wait; a claim held by anyone else needs no fence, because the creator's CAS from free fails against it anyway. The purge id is kept in the reservation, so a purge resumed by the recovery pass releases its own fences at `done`, and the pass leaves a holderless `purging` claim alone. The creator's re-read of its registration is a step of its own, and a new fault class, in half the seeds, pauses the console there, after the re-read and before the claim CAS, for 100 to 300 steps, past the registration's 60-step expiry, so a purge can drain the registration, fence, tear the store down and release while it sleeps. The second host's claim carries an epoch that every write advances, the model's ETag, and the creator's CAS is conditional on the epoch (or the main host's claim revision) it read when it registered (CC22). `generation.json` is written with a conditional PUT, If-Match the version it supersedes or If-None-Match when there is none, and the model's S3 answers 412 when the key's current version differs at completion, whether the request completes at once or late (model 29). **Rev 7.2: CC22 as claim rev 5.2 states it, and a target stopped mid-return.** The creator's 412 no longer abandons the create: the plan drops its registration and starts again at step 0, with a new create id, a new registration and a new read of the claim, as rev 5.2 §5.1 says (never a re-read and a retried CAS alone). Over 100,000 seeds 14,136 creates restart this way, after any claim write between the registration and the CAS: a sweep's acknowledgement, another create, a fence. A purge fence records whether it found a claim at all. Its release writes a free claim (rev 5.2 §5.3, C5a); the mutation of the new rule `claim-never-deleted` deletes a fence that found no claim instead, so the host's claim is absent again, and the second host's epoch returns to 0, which is how the model spells absence there. The schedule gains a third bias, in half the seeds, for the interleaving model 27 exists for. While a landing waits out its settle window, the target's app does real work (its writes are acknowledged like any other), and the settle check then fails as it does on a `FailSettle` box, so the rollback that follows is dirty and the target owes a return. The moment the target has issued the first PUT of its return lane, accepted or still in flight, or the second with the return not yet journaled, the cloud stops the box. The target is destroyed, which is what declare-lost's compute fencing verifies, and the operator declares it lost on a later call, with part of the lane in the source's region. Rev 7's uniform schedule reached this once in 100,000 seeds and rev 7.1's never; it now happens 106 times in 100,000 seeds, and dirty rollbacks rise from the 74 per 100,000 rev 7.1 recorded to 96. **Rev 7.3: deterministic witness scenarios.** Beside the random sweep, `witness_test.go` holds one fixed, hand-scheduled scenario per rule the sweep catches rarely or only in combination (22 scenarios, 20 rules; §14.2.5 lists them). A witness builds a clean world (three boxes with the key, no fault armed, no legacy host) under a **quiet RNG**, whose every draw takes the branch that injects nothing: no late completion, no stalled signer, no duplicate, no second process. It then scripts the interleaving its rule exists for, one protocol step at a time: it names the actors of each step (a box's worker, obligation runner and reconcile, the console's plans in order, the recovery pass, the operator), stops a box, crashes an agent, freezes a VM or pauses an agent, has the application write, holds one S3 request until it says so (the model's `issue` honours a witness's hold), places a paused registry open, or wakes a stalled worker goroutine with an old read. Then it drives to quiescence and runs the same final oracle as a seed. A milestone the scenario cannot reach, with no violation to explain it, is recorded as INV-3. Witnesses never draw from a seed's RNG: the model's RNG became an interface so the quiet one can stand in, `moveTo` and `resumeStale` are callable directly, and the random path's draws are unchanged, which the identical 300-seed gate of the rev 7.2 binary and this one confirms. Each witness must hold with every rule in place (it runs in the gate) and violate an invariant under its rule's mutation, alone or in the stated combination; `HOSTLIFE_MUTATE` applies the mutation and `mutate.sh` reports the witness beside the sweep (§14.2.1). A witness that holds alone and violates with its partners is the redundancy the combination table states, shown rather than inferred. **Found on the way: the model was not deterministic.** The purge drain iterated an app's registrations in Go map order, so a seed with two registrations at a drain could differ from run to run; the same rev 7.2 binary gave 49,743, 49,744 and 49,745 completed operations over 30,000 seeds. It drains in create-id order now, two runs agree exactly, and the sweep counts in §14.2.5 are rerun on the deterministic model. It was the only behaviour-affecting map iteration. **Rev 7.4: claim rev 6 and the moves side of joint round 3.** - **Restore is a chain (R3-1).** A history is an ordered list of transaction ids; every replica object carries the whole history up to its checkpoint. `generation.json` names the generation's snapshot by version id (recorded by the install, `op.BaseVer`); the restore opens that version and then takes, among every version of the generation's LTX key, those whose history extends the chain (longest last), skips those that are a prefix of it, and refuses on one that is neither. A staging round's snapshot is the local database only if its history continues the replica's. Rev 7.3 took the union of every version's writes, which no restore can do. The baseline is unchanged by the chain: every durable write stays reachable, which is the property the protocol owes. - **The local fence is per installation (R3-3).** `{incarnation, op}`: retirement writes the retiring operation's sequence; admission and the host-guard passes compare the operation they validated under with it (each pass records that sequence when it selects a row); the landing's `checking` phase replaces the fence on a fresh read that shows the move current and this box its target. Op ids carry the model's global counter, so "at or before" is the id order. - **Orphan cleanup is a plan with pauses (R3-2).** The recovery pass and a create that finds the record key occupied each run their own cleanup plan: read the record (ETag, incarnation); on a later step read the claim fresh and classify; on a later step `DeleteIfMatch`. Record ETags are unique per write now (rev 7.3 reused values, which would have made the conditional delete meaningless). The cleanup asserts that it never deletes the record of the claim's holder or creating incarnation. - **The sweep discovers locations from the namespace (R3-4)** and `retired_incarnations` is capped at 32, anti-reuse only. The legacy location is the host prefix itself. - **Public delivery is modelled (R3-5).** The edge routes on the row: a wake of a ready row on a landing target before the commit is INV-5, checked whenever a request reaches the edge. `commit-before-open` has its own violation now, alone, in the sweep and in its witness. - **Not taken in rev 7.4**, stated in §14.2.6: the target's cleanup is still queued at release rather than after `recovered_at`; a create still refuses while a record PUT is pending on the server, which a real console cannot see; the release check still omits timers and unreplicated local writes; same-operation workers still share a lease identity; the timing scale is unchanged. **Rev 7.5: joint round 4 and claim rev 6.2.** - **The database and its replica have LTX semantics (R4-1, `ltx_test.go`).** A database is a map of keys to values with a lineage TXID; a transaction inserts a key named by its write, or (every fifth write) updates the newest live key, or (every fifth but one) deletes the oldest, so a restore's fidelity depends on final page images, not on a list of ids. A replica object is an LTX file: a level (0, 1, or 9 for a snapshot), a transaction range, the final image of every key it touches (0 is a delete), and the database checksum after `Max`. The oracle alone knows which acknowledged writes a file covers. The serving process ships its pending transactions as one level-0 file, retries the same range widened after a timeout, and compacts three current level-0 files into one level-1 file before deleting the inputs (as `ltx.Compactor` and litestream's retention do; the versions stay noncurrent, which the park's version-exact snapshot still sees). The sealed transfer and the dirty return are snapshots of the stopped database. **Restore is litestream's planner** (§5.1 step 4) from the snapshot `generation.json` names, with the gap refusal and the checksum check (step 5). The oracle replays the incarnation's acknowledged transactions in order into the state a restore must equal: at every release and at the end a cold restore of the live generation must reach every durable write's TXID and equal that state (INV-2, cold); the serving box's own database must equal the replay of every acknowledged write, a write that never reached a database counting as the insert the application believed it made (INV-2, local). Rev 7.4's objects carried the full transaction history and could show neither a compacted continuation nor a gap. - **Installation stamps (R4-2, claim rev 6.2 §6.4).** Each box keeps `{active_stamp | none, retired_stamps}` per incarnation. A box's claim read carries the active stamp at the read; admission compares it with the active stamp now, so a stale worker's read from before a retirement is refused. Config-sync and adoption passes record the stamp they selected under and the host guard admits them only at the active stamp. A create's installation stamp is the gen that made it serving here; a landing's is the flip CAS's gen (`op.FlipGen`), set in `checking` under the guard on a fresh read. Retirement (local park, quarantine) moves the active stamp into the retired set. The op counter ordering, the `- 1`, and "an empty serving op means now" are gone. The oracle records a landing probe refused by its own installation record (INV-3, the R3-3 fault) and a host-guard write by a pass whose stamp is not the active one (INV-6). - **The live set is region-qualified (R4-3, claim rev 6.2 §6.2a).** A generation's files and marker are live only in the authoritative region's bucket; `deadGeneration` returns residue for a matching key in any other region. - **Retirement under purge (R4-4, claim rev 6.2 §5.3 step 4).** Under `deleting` and `purging` alike, the record version is recorded in `op.intent` first (`RecVer`), the record is deleted only if its version matches, and the CAS to `parking` is legal only from the same kind, with the record authoritatively gone, and, for a purge, with the purge journal at `torn-down` or later. A purged host never returns to `serving`. - **Orphan cleanup with no claim object** is explicit (claim rev 6.2 §5.1 step 5, Q2): a record with no claim is a legacy host, adopted (§9), never an orphan. - **The random generator (R4 §3).** Three additions, each drawn from the seed's RNG in half the seeds: a **marker-hold** schedule holds a `generation.json` publication until a newer marker is current, which is model 29's interleaving; a **targeted freeze** freezes the landing target in its probe or settle window and keeps it frozen until the source runs again, which is model 5's; and a **retention-boundary fixture** starts a seed with `retired_incarnations` at its cap and a replication object of an incarnation the list forgot, as a straggler. Two oracles make a rolled-back move an unaccepted terminal where it should be: the probe-fence oracle above, and the round-trip witness. These reshuffle every seed against rev 7.4; the witnesses are the stable evidence. paginated listings, separate service, wall and monotonic clocks, enrollment and console and updater versions, spawn separate from readiness, and same-operation lease identities. - **State.** The claim object (CAS on its revision, op id conditions), records, the console's purge reservations and app stores, three regions' **versioned** data buckets, and each box's registry row, local database, files directory, `config.env`, live link, timers, processes, retired markers and execution-lease file. - **The real permission split.** A box has full rights on its own bucket; on another region's bucket only Put, bare Delete and CopyObject **into** `moves/*` from its own bucket. No Get, no version listing, no version delete. Every request is checked; a violation is an invariant failure. - **Late S3 completion.** A request completes at once (70%) or is "submitted, client cancelled, completed later", within a long-tailed horizon, after the issuer has moved on, crashed or been replaced. A quarter of the long tail lands at exactly `H`, the latest S3's signature-age limit allows; staging PUTs take the long tail one time in three, other requests one in eight, because the cross-region path is the slow one. - **Paused goroutines (rev 5).** Four kinds, each killed by an agent crash or reboot, as a real goroutine is: - a **staging round** that revalidated under the staging-work lease and paused before signing (one round in three), and a **cut round's lost-response retry** whose first attempt paused (one in three). The pause runs up to `2 × staging TTL + H`, so it can outlast the lease; a frozen or paused box holds it longer. The `SafeUntil` check runs when the goroutine resumes (§7.3); - a **registry hold or open** that passed its checks and paused (one hold in 25, one open in six): mostly under 150 steps, one in four up to 2,000. Half the paused opens are woken **at the worst moment**: while a newer operation that has already committed holds the row, so that operation no longer re-asserts its hold; - a **stalled role worker** replaying an old read (rev 4's actor); - a **stalled door read**, with the same pause lengths, which resumes early at its worst moment: when its location's current object belongs to another incarnation. Rev 4's door reads never stalled for long, and rev 5's slower lifecycle hid `door-by-version` until they did. - **Obligations beside roles (rev 5).** Each box runs its obligations in a runner separate from its role worker, as the agent will, so a park can run while the same box's role work for a new operation proceeds. Rev 4's single worker serialized them and hid CC11's race. - **Box faults.** Agent crash (flat children die with the service, nested ones survive, memory is lost), reboot, whole-VM freeze, agent-only pause, partition from the control bucket, clock skew, destruction (a stopped instance) followed by either replacement or a restart with the old disk, and boot recovery to an old executable. - **The application.** Main processes and descendants (strict or escaped) that write the database and register once-timers on their own, so writes happen during probes and recoveries. - **Actors.** Create, delete, purge (store teardown included), push, the db door (authorize, stall, open), wake, idle stop, replication, reconcile (with legacy adoption), config sync and release adoption (each with a decision taken now and acted on later), the timer runner (delivery and its own prune), the self-fencing watchdog, the move worker (all of §7), rollback (clean and dirty), abort, a stalled second worker replaying an old read, the console's claim-recovery pass, and an operator (declare-lost, readmit). - **Two phases per seed.** 2,000 chaos steps with user operations and faults, then up to 6,000 calm steps: faults linger for a while (so the operator meets a frozen or partitioned participant), then heal; stopped boxes return or are replaced; ops gives every box its secrets key. - **Scale (rev 5).** One step is the unit of time. `H` = 60 steps; the execution lease 40, renewed within 15; the cut budget 320 (220 in rev 4); the staging-work lease 400, longer than the cut budget, as in production (rev 4 had 30, which made the staging horizon shorter than any landing and so unobservable); a move aborts if readiness has not come within 800. Moves are three times as frequent as in rev 4. - **What claim rev 3 changed in the model.** C3's harmlessness only for the operation's own version-bound calls; every retirement verifies after `evidence time + H`, and stragglers go into a further retirement event; `quiet_after` per region, kept across `free`, raised by each acknowledgement, and waited for by a create's first wake, a landing's install and a push's install; park destinations keyed by source version; one adoption routine with discovery parks; declare-lost by compute fencing only; the purge journal's phases; `RenderHeld` and `MaterializeHeld` before every probe. **Rev 7.6: joint round 5, the pinned libraries and claim rev 6.4.** - **Replication position and lineage are separate (R5-1, `ltx_test.go`).** A database has a lineage TXID (the oracle's) and a replication position `Pos`, litestream's TXID numbering in the generation: the serving process ships one level-0 file `[Pos + 1, Pos + 1]` per sync (full pages at position 0, as litestream's first sync is), and the position advances as the upload is issued, because a timed-out upload is retried with the same bytes and the next write takes the next TXID. A cold restore reads its position from the newest current level-0 file of the generation (`db.go` `checkDatabaseBehindReplica`), which is 0 for a snapshot-only install; when the restored TXID is above it, the restore writes the **anchor** `[cur, cur]` with every page before the database may be opened (`restore-anchor`, §5.1 step 7). The model's seal ships the pending writes first (`SyncAndWait`), so the snapshot is at the position. The mutation leaves the position where litestream's open finds it, and the witness (a push's snapshot-only install, three writes replicated, the disk lost, a cold wake) loses every write since the push: INV-2. - **No whole-state checksum (R5-2).** An LTX file carries its level, its range and the final image of every key it touches, nothing else: the pinned writers set `HeaderFlagNoChecksum` (fact R5-2). The oracle's state comparison and the planner's gap refusal are the model's whole verification, as §5.1 step 5's per-file CRC-64 and `integrity_check` are the implementation's. - **Snapshot rotation and retention (R5-3).** A compaction (three current level-0 files into one level-1 file, inputs deleted) is followed by a snapshot `[1, Pos]` when the position has moved past the newest snapshot, then retention: every older snapshot is deleted, and level-1 and level-0 files whose `Max` is below the previous snapshot's `Max` (litestream's `minSnapshotTXID`) are deleted, all by key, so the versions stay noncurrent. The restore roots at the newest current snapshot of the generation at or past the pinned one (`root-advance`), and at the pinned version only when none is current; the origin generation, which has no pinned snapshot, roots at its newest snapshot as litestream does. The mutation roots at the pinned snapshot for good; the witness (a push, then four rotations, then the disk lost) refuses to wake on a gap the retention made: INV-3. - **Claim rev 6.4's stamp lifecycle (R5-4).** `holder.stamp` is a claim field written by every installing CAS: create step 4 (the console and the recovery pass finishing it), adoption, the flip, a push's probe CAS, a dirty rollback's source probe CAS. A box's local record is created or refreshed only by `initRecord`: a fresh claim read under the guard that names the box `op.exec` or the landing participant, `active_stamp = holder.stamp`, run at the top of every role pass and at the recovery pass's mint. A writer for a host with no record is refused (rev 7.5's model admitted it). The two retirements are two helpers: `retireArtifacts(inc, S)` names the retirement and moves no stamp (the landing's `local`, the activator's park of a warm database from an earlier stay, a dirty install on the source); `retireStamp(inc)` moves the active stamp into the retired set and sets none (the local park at a delete, a move away, quarantine); activation (`install`) moves the old active stamp into the retired set before setting the new one, as claim rev 6.4's answer 2 has it. Passes capture the stamp from the record when they select their work (answer 1). The landing's `checking` installs from `holder.stamp`, not from a claim revision it computed. Local artifacts (`localDB.Stamp`, `filesDir.Stamp`) carry the stamp of the installation that produced them. - **The S3 layout (R5-6).** LTX keys are `//-.ltx` (`ltxKey`, `ltxKeyRe`); the sweep's `locationOf` treats a child prefix that is not an incarnation id (a level directory, a `g-/` prefix) as the host prefix's own location, and a decimal child (the claim's 64 hex characters) as an incarnation location (claim rev 6.4 §6.2a). - **Uploads are sequential (R5-1, step 6).** Each sync writes its level-0 file locally and the replica uploads the local files in TXID order, never sending the next before the one ahead completed; an upload the client has waited eight steps for is retried with the same bytes, and the original may still land as an identical version. A gap in the current listing is therefore always a file that has not landed. Rev 7.6's second baseline shipped the next file while the first was in flight and lost a cold restore to the gap in 9 seeds of 30,000. - **The retention fixture (R5-6)** populates `retired_incarnations` with 2 to 33 and panics unless the injected incarnation 1 is absent and the list is at its cap. - **The budget (R5-5, `budget_test.go`)** prices the final delta at the bar, then the target's restore of the staged replica, snapshot write and anchor write at the local rate; a 2 GiB database over 10 MiB/s is admitted with one attempt at 50 MiB/s local and refused as a cross-region snapshot transfer (205 s). - **Not taken in rev 7.6**, stated in §14.2.6: the model has no bytes, so the staged replica is one object holding the stopped source's state and the install copies it into the generation as the snapshot; where the snapshot is built is a cost the budget test prices, not a protocol step the model distinguishes; the seal names one version, not one per staged file. **Rev 7.7: joint round 6 and claim rev 6.5.** - **Restore split in two (R6-1).** `restoreCalc` is the restore contract as a pure calculation over one region's bucket: §5.1 steps 4, 5 and 8, returning the database and the position it restored to, or nothing when the contract refuses. `restoreInit` is the activator's: the calculation, then the restore-to-replication initialization. When the restored position is above the newest remote level-0 file it issues the anchor PUT and returns nothing until one of its PUTs is acknowledged (retrying the same bytes after `uploadTimeout`); the wake tries again later. Only then does it read the position, from the listing (`anchor-acked`). A box's outstanding anchor PUTs are forgotten with its agent (the restarted activator issues them again, identical). The mutation of `anchor-acked` opens at once with whatever the listing shows, which is litestream's behaviour with no anchor present; the witness holds the anchor PUT across forty attempted opens of a push's install and requires the host not to open. - **The oracles assert their purity (R6-1).** `fingerprint` and `pure`, around `checkDurable` and the final oracle (§14.2.1). - **Claim rev 6.5's return to source (R6-4).** `op.SrcStamp` is the source installation's stamp, journaled by the flip; the return CAS (`waiting → claimed`, `op.exec` back to the source) writes `holder.stamp = op.SrcStamp` (`return-stamp`). `issueStamp` records, for every stamp an installing CAS writes, the box whose installation it names; `activate` (initialization and reinstall) refuses, as INV-6, a stamp issued for another box, retires the old active stamp, sets the new one and tags the artifacts of the generation the claim names with it. `retireArtifacts(b, inc, S)` parks the database and files tagged `S` and nothing else; `retireDatabase` the database alone where the files are carried; callers assert the tags they expect (a landing never parks its own active stamp's artifact; a dirty install parks the pre-move database while its stamp is active). A reboot loses the local stamp record in one case in eight; the next fresh read re-initializes it. Witness: a dirty return whose source lost its record (the source must serve under the probe CAS's stamp, its database so tagged, the pre-move stamp retired). - **Compaction keeps the newest level-0 file**, as litestream's retention does (facts R5-1f to h), so the model no longer creates anchor-required cold restores that the library would not. - **The budget (R6-3, `budget_test.go`)** prices regional S3 and the disk as separate rates and the `restored` phase as a verification read; the review's 3 GiB workload is reproduced (108.68 s with three passes at one rate, 124.04 s with the fourth) and refused by rev 7.7's arithmetic. **Rev 7.18: joint round 17 (R17-1).** `parkStep` and `parkDepth` take the chore's scope; `member(scope, id, what, ref)` is INV-23's second half, checked for every reference the fresh manifest would archive or delete and for every reference of a replayed persisted manifest, with no rule gating; an out-of-scope reference is a violation, excluded from the sets and never deleted. The rule `scope-membership` keeps the listing's confinement as the implementation; the candidate-key check that ran only with the rule off is gone. The operations' own parks (a landing's install park of the target region's earlier stay, a dirty return's and a push's) run under `opScope(c, prefix)`, `holder.location` with the holder's incarnation, confined by `confine(prefix)`. `TestForeignManifestReplayRefused`: incarnation 2 serving, a persisted manifest at a sweep chore's destination naming a late object of incarnation 1 under the chore's valid `instances//2` scope; one step of the chore refuses it with INV-23 and the object survives (a later at-rest sweep may park it under incarnation 1's own scope, which is the right chore). No rule changed, so the gate alone was run (§14.2.1). **Rev 7.17: joint round 16 and claim rev 6.24.** - **Scope membership (R16-2).** `scopeOf(k)` is the scope location a key belongs to (`moves/` for staging; an incarnation location; the legacy `instances/` for a level directory or generation directly under the host prefix) and `inScope(k, loc)` its membership (a staging lane's scope by its own prefix). `sweep()` groups the sweepable keys by `scopeOf` and queues one `park-sweep` per location, scoped with `incOf(claim, loc)` (the location's own incarnation; the holder's for the legacy location or the staging sweep; the newest retired on a free host); `Dest()` for a scoped sweep is `parked/sweep////`. The runner confines every scoped chore's listing to its scope (`keep` composes `!inScope`), and INV-23 checks each key a chore would archive for membership before the park step. `parkingObligations` (the delete) parks `holder.location` and `moves//`, not the whole host prefix. Rule `scope-membership` (the mutation is rev 7.16's whole-prefix grouping and listing); witness `wSweepScopeMembership`: incarnation 1 deleted, incarnation 2 serving, a late object under `instances//1/g-old/`; the chore that parks it is scoped to `instances//1` with incarnation 1; under the mutation INV-23 fires on the delete's `instances/` park archiving a child incarnation's object. - **The legacy identity (R16-3).** `adopt()` writes `Session: 0` (the stamp stays 1); the sim's and the witnesses' pre-release processes, acknowledgements and certificate fixtures carry session 0; INV-24 in `admitCommit`: every declaration names a box, and session 0 comes only with generation `""`; a created incarnation's first admission, whose predecessor is `{session 0, box -1}`, records no `writes_lost` (`admission: no predecessor writer, nothing declared`; rev 7.16 recorded a boxless one). `TestLegacyLostTargetReturn`: adopt, the pre-claim writer's last write held off the replica, a move cut at session 0 with `src_covered` false, the target admitted cold and declared lost, the return's record declaring `{target, probe session}` and `{source, session 0}` once each. - **Staging beside a legacy park (R16-1).** `TestStagingSurvivesLegacyPark`: a legacy host whose earlier stay left a replica in the target region; the landing's install park, scoped to the legacy instance location, leaves the sealed `moves///db` and `files` current and the landing is probed. - **Born covered, narrowed (claim rev 6.24 item 5).** `admitCommit` counts an unscoped `park-replica`, or a `park-superseded` of `holder.location` whose scope, if any, is exactly that location. - **Disclosed difference (the sweep's walk).** `deadGeneration` protects every generation of an unfinished run in the authoritative region without the run walk's region or coverage filtering, which main applies when it selects a baseline; the sweep's protection is the wider, conservative set (§14.2.6). **Rev 7.16: joint round 15 and claim rev 6.23.** - **The scope on every chore, from its queuing CAS (R15-1).** `w.scoped(obligation, evidenceAt, inc)` attaches `parkScope{Location: scopeLocation(prefix), Inc, EvidenceAt, RequestLimit}` at every queuing site: the commit's source park (`op.StopAt`), the release's target park (the release's step), the lost-target release (`fenced_at`, the declaration's step), every staging lane (`stagingBound(quiet)`, the seal bound: the lane's quiet time less `H`, so the `staging-quiet` mutation's "now" yields a bound already past), the sweep (the listing, now; the archive incarnation the holder's or, on a free host, the newest retired), the generation chores (the publication, now). `scopeLocation` strips the park prefix's trailing `/`; INV-23 checks every scope's location against the grammar (`scopeLocationRe`; the model's incarnation ids are decimal). The delete's and adoption's `park-replica` still bind theirs before the first step, once the stop evidence is in hand. Under the `persisted-park-scope` mutation chores are queued unscoped and the runner binds the cleanup's own clock, as rev 7.13 did. - **One deadline (R15-1).** `parkScope.deadline() = EvidenceAt + sigAgeH + RequestLimit`; the runner's quiet for every scoped chore is the larger of its queued quiet time and that deadline (a staging lane's regardless of `quiet-before-ack`, since its queued quiet time was that bound already); the acknowledgement's `quiet_after` uses it too. Rev 7.15's staging exclusion is gone. Witness `wStagingScopeDeadline`: a committed move's staging lane, deadline `seal + stagingTTL + H`; the target region's obligation runner is withheld until 40 steps past it, then the target's agent restarts and the cleanup begins: the acknowledgement must come within 20 steps and never before the deadline; under the mutation the fresh bound waits another `H` (568 → acknowledged at 669 in the trace, the round's 560/600/660 example in the model's steps). `wParkScopeSurvivesRestart` reads its deadline from the chore's scope (the cut's stop plus `H`) and restarts the agent 20 steps before it. **Rev 7.15: joint round 14.** - **The selected baseline is restored (R14-1).** `openState.SourcePin` carries the restore mode the run walk chose: `-1`, the marker's generation through `generation.json` (`restoreCalc`); `0`, a run generation rooted at `[1, 1]` (`restoreWith(..., 0)`), as `restoreAsNext` always restored it. The restore stage calls `restoreWith` with it. Rule `restore-selected-baseline` (the mutation is rev 7.14's `restoreCalc` on the selected source, which the marker refuses); witness `wIntermediateBaseline`: S named by the marker; R's first upload held in flight, R crashes; R2 admitted over the empty R and crashed before uploading; R's upload lands; the next start selects R from R2 → R → S and restores it (`restore: run generation rooted`); under the mutation the start cannot restore it and the valid recovery stalls (INV-3). - **One plus a list, unique (R14-2).** `admissionRecord.LostSrc` is gone; the source's cut session, under `op.CutSession`, joins `AdditionalWritesLost` through one `add` that refuses a `(box, session)` already in `Lost` or the list; one alert per entry; INV-21 after the CAS. Rule `unique-declarations` (the mutation appends without the check, as rev 7.14's separate path did); witnesses `wNoDuplicateDeclarationEarly` (the target destroyed after the flip and before its first admission, `src_covered` false, the source's cut session uncertified: declared once) and `wNoDuplicateDeclarationLate` (after the target's probe: the predecessor is the target's session, the list carries the cut session once, nothing twice). - **The horizon's parts (R14-3).** `horizonH = sigAgeH + requestLimitH` (45 + 15); `parkScope.RequestLimit` persists the request bound; every quiet and acknowledgement computation is `EvidenceAt + sigAgeH + RequestLimit`; the park-sweep's acknowledgement waits that, its straggler pass the verification listing. - **Joining (R14-4).** `sweep()`'s outer guard admits a holderless sweep's claim; `sweep joined a running holderless op` counts; `TestJoinHolderlessSweep`. - **Every remote chore scoped (R14-5).** `park-generation` chores carry `Scope{EvidenceAt: now}` from the publication (both sites, the `baseline-live` mutation's included) and their quiet is the scope's horizon; `park-staging` binds a scope before its first step, keeping `SealQuiet`; INV-22 in the runner: no remote chore runs a step unscoped under `persisted-park-scope`. - **Fidelity.** The publication's precondition is that **any** entry names the published generation, as A and E read it (rev 7.14 required the last); trimming is E's: covered entries from the front only while more than eight entries remain. Occupancy stays by current objects (§14.2.6). **Rev 7.14: joint round 13 and claim rev 6.22.** - **The park scope and the holderless sweep (R13-4; claim rev 6.15).** `obligation.Scope *parkScope{Location, Inc, EvidenceAt, Limit}`. A `park-replica` or `park-superseded` chore binds its scope in a CAS the first time its owner has the stop evidence (`park scope bound`), and every later step, an agent restart's included, reads `EvidenceAt` and `Limit` from the claim; the sweep writes the scope at its queuing CAS and acknowledges after `EvidenceAt + Limit` with its straggler pass. `sweep()` on a free claim sets `op = {Kind: "parking", ID, ParkSweep: "v1", Exec: -1}` in the CAS that queues its chores; another region's sweep joins the running op; on a serving host the chores carry `session:`; the obligation runner's acknowledgement CAS performs `FreeAfterPark` (`op = {}`) when it removes the op's last chore (`free after park`, equal to `holderless sweeps` in every run); the console's recovery pass leaves such an op to its runners and replays `FreeAfterPark` only for one whose chores are gone. `createSteps` waits on a holderless sweep instead of abandoning the create, and its claim CAS checks INV-20 (no obligation outstanding). Rules `persisted-park-scope` (the mutation keeps rev 7.13's box-local evidence map, which `forget()` clears at an agent restart) and `holderless-park` (the mutation queues rev 7.13's bare `sweep` chores on the free claim and drops the create's wait on bare obligations); witnesses `wParkScopeSurvivesRestart` (a committed move's source park, 45 steps, the source's agent restarts, the acknowledgement must come within the commit's evidence plus the horizon and slack; INV-9 under the mutation) and `wFreeSweepVsCreate` (a deleted host, late residue in its region, a create requested at once; the create waits for `FreeAfterPark`; INV-20 under the mutation). - **Coverage, completely (R13-2; claim rev 6.22 item 5).** `publishGeneration` requires the last record to name the published generation and walks every entry in order, skipping covered ones, with the explicit bound `e.Session > session` (never reached under the CAS's session check); warm entries are born covered in `admitCommit` (`admissions born covered: warm`); born covered under a pending park applies to every mode; `coverRegionPredecessors` and INV-17 apply to every entry; `coverSweptPredecessors` became `coverParkedPredecessors(n, region, generations)` with `sweptPredecessors(n, region, retired)` supplying the list (sweepable predecessors at the queuing CAS, retired ones at the acknowledgement); trimming counts covered entries only. - **The run walk (R13-1; claim rev 6.22 §6.2a).** `unfinishedRun(c, bg)` returns the run's cold records newest first; `runBaseline(b, c, run, bg)` walks their predecessor links: a link whose `PredRegion` is not this region is passed over, covered or not; a covered link in this region refuses the start with an alert (`start refused: covered local baseline`); the nearest local link holding an object is the baseline, else the marker's. `hasFiles` reads current objects (rev 7.13 counted a key's presence, so a parked generation still looked like a baseline). `restoreAsNext`, the oracle's baseline, walks the same way. Rule `covered-baseline-refused` (the mutation ignores the covered flag and selects by objects alone); witness `wCoveredLocalBaseline` (an unfinished run g2 → g1 whose record is covered and whose g1 was retired: refused under the rule; under the mutation the walk passes over the empty g1 and restores the marker's g1 as an empty database, INV-2); `TestInterruptedCleanReturn` (above). `wOriginSwept` names the refusal as its failure under `absent-marker-origin`: the swept origin is a covered local baseline since rev 7.14, so the recovery refuses with an alert rather than restoring nothing. - **`additional_writes_lost` (R13-3).** `admissionRecord.LostMore` is `AdditionalWritesLost`, each entry alerted; the model already declared every target session since the cut (rev 7.13). **Rev 7.13: joint round 12 and claim rev 6.14.** - **Regions on the record (claim rev 6.14 item 1, R12-3).** `admissionRecord` gains `Region` and `PredRegion` (`-1` for a predecessor holding nothing anywhere); `admitInput` gains `Region` and `PredecessorRegion`, mirroring E's `Admission` (47426cc1). `predecessorRegion(holder)` is the last record's box's region, before any record the writer box's, else wherever `holder.generation` holds an object, else `-1`. - **The publication walk (item 5).** `publishGeneration(b, c, observedOpID, observedPhase, session, published, region, newID)` loses its `dead` set and `uncovered`: it walks every uncovered cold record, covers one whose predecessor holds nothing anywhere, leaves one whose predecessor is the published generation itself (live again; covered, as item 5's first bullet reads, nothing retired it later, seed 41914 / 100,000), queues one deduplicated chore for one whose predecessor is this region's and covers it, and leaves one whose predecessor is another region's. The marker stage calls it for every cold admission, prepared generations included; the lag path's republish calls it without a dead set. Trimming is E's: covered records from the front only while more than eight remain, an uncovered record stops it. Rule `foreign-region-uncovered` (the mutation queues the chore into the predecessor's region); witness `wRollbackBaselinePreserved`: a probed landing whose target published, the chore's quiet time brought forward (the schedule seeds 1036 and 1296 / rev 7.12 found with a landing open past the horizon), the source region's runner within the settle window, then a clean rollback; under the mutation the return restores an empty database (INV-2). - **Coverage at whole-location parks (item 6).** `coverRegionPredecessors(n, region, why)` is composed into three CASes: the commit's (`followRecord`, the source region's `park-superseded`), the target's release (`releaseTarget`, the target region's) and the console's lost-target release (the same park). A cold record is born covered when the CAS that admits it finds a `park-superseded` of `holder.location` for its predecessor's region already queued (seed 1036 / 3,000: the return's record is admitted after the release). The sweep covers what it parks: `coverSweptPredecessors` in the CAS that queues the `park-sweep` (a predecessor with a sweepable object) and in the one that acknowledges it (a predecessor whose objects it retired). Rule `cover-at-park` (the mutation leaves every one of these uncovered); INV-17 in the `park-superseded` acknowledgement CAS: an uncovered cold record whose predecessor's objects were in that region, other than the live generation. Witnesses `wCommitCoverage` (the commit's CAS covers the target's record, one park queued), `wCommitInterrupted` (the record flips to committed, every actor pauses past the horizon, whoever follows the record later performs the one CAS), `wCommitReplayed` (`followRecord` after the commit is refused, one park, coverage stands); under the mutation INV-17 at the source region's park. - **The declared identity (R12-1).** `ack` gains `Session` and `Gen`, set from the **process** that acknowledged (`proc.Session`, the session the app was started under; a write into a purged database is acknowledged into nowhere and is still that session's, seed 899 / 3,000 under a database-based first cut); the push's mark, the legacy writer (session 1) and every witness helper carry them. `exemptDeclared(b, lost, holder, newGen, inv15)` applies a record through `declaredBy`: box, incarnation, kind, and the record's session and generation; the source-cut record goes through the same function separately, and so does each record of `admissionRecord.LostMore`: behind a declared-lost target the return's admission declares every session the target was admitted into since the cut (`q.Box == op.Tgt && q.Session > op.CutSession`), certified or not (seed 33373 / 100,000: the probe session certified and its writes, durable only in the lost region, were hidden behind the record for the warm continuation). Rule `session-scoped-loss` (the mutation is rev 7.12's match by box and incarnation); witness `wTwoSessionDeclaration`, a constructed-state oracle test: two sessions of one box each acknowledged an unuploaded write, a record for the second exempts only its own, else INV-18. - **The probe deadline (R12-2).** In the app stage, `late` is computed once per step from `openState.startedAt`; the attempt fails before the start when the window has passed, whether the environment is ready (`probe attempt failed: deadline before the start`) or not (`environment`), and after a successful start whose readiness came at or after the deadline the main process is stopped and the attempt fails (`readiness after the deadline`; the model answers readiness in the start's step, so this check is the belt to the first check's braces). `box.lastProbe{startedAt, readyAt}` records the attempt; `probeAccepted` is INV-19 at the four `probed` transitions (revert, landing, restoring, push). Rule `probe-window` (the mutation makes no deadline check at all); witness `wProbeWindowAt` (the target's environment, by the knob `box.envUnreadyFor`, ready exactly 30 steps after the first app stage; under the mutation the expired attempt is accepted, INV-19); `TestProbeWindow` runs k = 29 (accepted, 29 steps), 30 (failed before the start) and 31 (failed at the deadline, unready), each followed by the retried admission and the run's end. - **The absent-marker mutant (R12 evidence).** Rule `absent-marker-origin`: `deadGeneration` reads an absent `generation.json` as the origin; the mutation reads it as agreement with `holder.generation` (rev 7.11's sweep), with nothing else changed. `wOriginSwept` moves to it from `baseline-live`, whose mutation also queued the premature chore. - **The recovery limit (R12-5).** Nine: `probeTries*3` in `probeAttemptFailed`, unchanged; §7.8's sentence corrected. - **Model faults found this round** (before the final numbers; the 3,000-seed gate and the first 100,000-seed baseline found them): a lost target's earlier, certified session went undeclared behind the record for its last (seed 33373 / 100,000, above); a live predecessor covered at a republished marker's publication was never retired once superseded (seed 41914 / 100,000, above); the sim's legacy host spawned its pre-release processes with no session, so their acknowledgements escaped adoption's session-1 declaration under the exact match (seed 44450 / 100,000: a fault of the sim's fixture, not of the protocol); the console's lost-target release queued the target region's park without covering (seed 1036: the third composition site); acknowledgements took their session from the database under the process, which a purge had removed (seed 899); the `generation-conditional` witness started its second push while the first push's publication chore still blocked operations in the region (`obligations-block-ops`), since a prepared generation's predecessor now gets its chore at the publication; the witness waits for the obligations to clear. **Rev 7.12: joint round 11 and claim rev 6.12.** - **`holder.admissions` (claim rev 6.12).** `holder.Admissions []admissionRecord{Session, Box, Gen, Predecessor, Mode, Coverage, Covered, Lost, LostSrc}` replaces `op.Writers`; the release and the move's and push's start carry nothing. `lastAdmission(holder)` is the next admission's predecessor; `admitCommit` journals it (session, box, generation) and INV-16 refuses a record whose predecessor is not the last admitted writer. `PublishGeneration` trims covered records beyond the last eight; an uncovered record and the newest are never trimmed. - **Coverage (R11-4).** `admitInput.Coverage` `""` | `"sealed"` (a prepared generation from a `from_db` seal) | `"clean-return"` (a rollback with `DirtyKnown && !Dirty && SrcCovered`, the source re-admitted into its own generation, and no declared-lost target); `writes_lost` only for `""` without a certificate. Rule `clean-return-identity` (the mutation takes the predecessor from the holder's fields and gives the clean return no coverage); witness `wCleanReturnIdentity`: a probed target that wrote nothing, a clean rollback, the source's re-admission journals the target's writer and declares nothing; under the mutation INV-16. `dirty-return`'s named failure is INV-2 local at the final recovery: treated as clean with the source's replica covering the stop, the return is covered and declares nothing, and the target's write is gone from the serving database. - **`PublishGeneration` (R11-5).** `publishGeneration(b, c, observedOpID, observedPhase, session, published, region, newID, dead)` replaces `queueGenerationRetire`: bound to the observed `op.id` and phase like the admission, legal in serving and in every operation kind, its chores carry `op = op.id` inside an operation and `session:` while serving; the marker stage and the lag path's republish call it; `uncovered` reads `holder.admissions`. - **The absent marker is the origin (R11-1).** `deadGeneration` reads a missing `generation.json` as `""` before testing the chain, as `residueOf` always did; witness `wOriginSwept` (under `baseline-live` then; under `absent-marker-origin` since rev 7.13): an adopted host's origin replica, a cold admission that crashes before its first upload, the ordinary sweep for three horizons with no wake, then the recovery; under the mutation the sweep parks the origin and the recovery restores nothing (INV-2 local). - **The final oracle runs the recovery (R10-6).** `final()` wakes an at-rest, uncertified serving host, drives the cold admission and its chores to quiescence, then judges; `final: recovery admission run` and `final: unverified prospective declaration` count the cases. It found a false coverage at once: a return made clean by a declared-lost target was covered `clean-return` and the target's write went undeclared (the `lost-return-park` witness under no mutation). - **A probe attempt's window (§7.7).** `openState.startedAt`, `probeStartWindow = 30`: in a probe, the fifth failed start or the window's end abandons the attempt (`probeAttemptFailed` counts one budgeted try: a landing rolls back at three, a revert or a rollback's source is stuck at nine), the environment wait included; a probe never marks the host failed. `recovery-current-inputs`'s witness reaches `stuck` through this path now. - **Adoption names the writer (R11-3).** Unchanged since rev 7.11 (`WriterBox: eb.ID` at `adopt`). - **Model faults found and fixed this round** (all before the final numbers; the 100,000-seed baseline found them): a `clean-return` coverage applied to a later session of the recovered source, whose uncertified death then went undeclared (seed 95491: the coverage speaks for the target's probe session and the source's cut session only, `op.CutSession`); `PublishGeneration` queued a dead generation's chore in the publishing region alone, so a move's source generation covered by the target's publication was never parked in the source's region (seeds 33646 and 40148: coverage is per region, and a chore is queued only where the generation holds objects; a first fix that queued chores into the other regions parked the source's own generation under a rollback, seeds 1036 and 1296 / 3,000, and was withdrawn; an empty dead generation still gets its chore, since a dead writer's late upload or certificate may land in it, seeds 15273 and 20810 / 30,000). **Rev 7.11: joint round 10 and claim rev 6.10.** - **The admitting CAS first (R10-1, claim rev 6.10).** `spawn` decides (`admitWriter`), makes the CAS (`admitCommit`) and only then opens an `openState` whose stages the agent's own loop drives (`resumeOpen`, `resumeStart`): `restore` (the purge, then the baseline), `app` (the start and readiness, under the retry policy: `startAttemptsMax = 5`, a backoff of `1 << (fails - 1)` steps from `startBackoff0 = 1`, then `hostFailed` with the session's start abandoned; the operator clears a failed host after half a horizon), `first`, `marker`, `anchor`, `open`. A session is the agent's: `localDB.Epoch` records the agent process that was admitted; a restarted agent (`forget` bumps `box.Epoch` and clears `open`) neither replicates nor certifies for it, and its first role step stops an app it finds running. An app stopped idle before `open` is not closed: the replicator opens, ships what the app wrote (`flush`), then closes. Rule `admit-before-start` (the mutation runs the app before the CAS, rev 7.10's order); witness `wAckBeforeAdmission`: a background worker (`w.startHook`) acknowledges a write right after the start and the box crashes before replication opens; admitted first, the next cold admission declares it; under the mutation the next warm admission purges it silently (INV-2 local). - **Unfinished generations and live baselines (R10-2).** `admitCommit` queues no chore. The `marker` stage, once `generation.json` names the new generation, calls `queueGenerationRetire` with the dead set: `openState.Dead` (the predecessor; behind a lagging marker the chain's empty generations and the marker's) plus the predecessor of every earlier cold entry of the writers journal not yet marked `Retired` (a writer that died after its marker landed and before the chore CAS; the CAS marks every earlier entry covered). `unfinishedChain` reads a lagging marker through the journal: `holder.generation` must be the latest entry's, and each earlier cold entry's predecessor the next, down to the marker's generation; `spawn`, `deadGeneration` (the sweep), `residueOf` (the live set) and `restoreAsNext` (what the next admission restores, the oracle's baseline at a release) share it. An empty `holder.generation` the chain does not explain is refused with an alert (`start refused: unexplained empty generation`); a marker the chain does not explain leaves the location untouched and alerts once. The release of an operation and the start of a move or a push carry `op.Writers` over, so an unfinished admission at a release stays explained; `owed` treats the predecessor of an uncovered entry as owed a chore, as a pending chore's keys are. `forget` clears `b.open`, the retry count and the backoff. Rule `baseline-live` (the mutation queues the chore at the CAS, as rev 7.10 did); witness `wBaselineParked`: a cold admission into a fresh generation, the box crashes before the first upload, the predecessor's retirement runs out its quiet time; under the mutation the predecessor is parked and the recovery restores nothing (INV-2 local). - **Commit and `probed` move no generation (R10-3).** Unchanged in the model since rev 7.10 (`holder.generation` moves only at a cold admission's CAS); the spec's §7.5, §7.6, §7.8 and §7.9 were brought to it. - **The library's `Admission` shape (R10-4).** `admitInput` has `KeepStamp` (set when `holder.stamp > holder.session`; `admitCommit` issues a stamp otherwise) and `BaselineCoversPredecessor` (true for an operation's prepared generation built from the stopped database, false when its seal was built from the remote because the disk was gone: `op.SealFromDB`, `op.ReturnFromDB`); warm passes an empty `NewGeneration`; `admitWriter` and `admitCommit` refuse `creating`, `deleting`, `parking`, `purging` and `placing`; a `baseline: sealed` entry records no `writes_lost`; the empty-generation ancestor restore needs the chain. `holder.writer_box` is `-1` before any writer. - **Serving-time chores and the root generation (R10-5).** `queueGenerationRetire` names `op = "session:"` when the claim is `serving` or the op id is empty, and `"@root"` for the origin generation; the `park-generation` chore maps `"@root"` back to the root's own keys. - **Faults exempt nothing (R10-6).** The crash-with-disk fault, the idle disk-loss fault and the witness helper `loseDisk` remove physical state only. Exemptions come from a `writes_lost` declaration (writes of the declared box not durable in the new lineage), from the final oracle's would-be declaration for an uncertified serving session, and from a declared-lost box's registered timers (the console's declaration names and alerts them). INV-14: every `writes_lost` record raised an alert. INV-15: a write the host's remote holds is never declared lost while its writer's box is with us (the dirty-return witness's named failure since this round). - **Startup PUTs in the ledger (R10-7).** The first upload and the anchor are `track`ed; a retry after the client's wait, whether the original timed out or completed unseen (a lost response), sets the sticky uncertainty; the stage advances only on client-observed success; the bytes are captured once (`openState.blob`) and a retry carries them, and the writes acknowledged since the capture stay pending. Witnesses `wFirstUploadRetry` and `wAnchorRetry` (close-evidence). - **The budget (R10-8).** `cutEstimate` prices `RestoreAttempt`, `Admit` and `FreshUpload`; `attemptsAdmitted` sums attempts; the tests pin the 2 GiB example at 53.6 s fixed, 64.3 s for the first attempt, 117.9 s in all, one attempt, and the review's 2.5 GiB at 113.8 s under rev 7.10's arithmetic and refused under rev 7.11's. - **Structured counters (R10-9).** The baseline logs `COUNTERS {json}`; `tables.py` reads it and fails on a missing field. - **The rollback's return carries the seal when the source's replica is short (rule `seal-return`, INV-15).** The seal CAS journals `src_covered` (the source's replica of its generation reached the stopped database's position, no write pending, the session's start complete); `releaseTarget` makes the return dirty when it is false; a target without a database returns the staged seal by version (`sealVersion`). Witness `wSealReturn`: the source's last upload is held when the cut stops it, the landing is rolled back after its build, and treated as clean the source declares a write the landing's built generation holds. - **Probes that die restart.** A push's, a revert's and a rollback's `probed` phase restart a probe whose process is gone instead of releasing or opening over it (the landing's already did); every `probed` phase waits for `b.open == nil`. - **Model faults found and fixed this round** (all before the final numbers; a 300-seed gate passed each intermediate build, the 30,000- and 100,000-seed baselines found them): a first upload or anchor whose completion the client never saw was never retried (seed 571 / 30,000); a rollback after the retry policy marked the target failed left its abandoned start to resume once the operator cleared the host, and two boxes ran the host (seed 25581 / 30,000: a start is abandoned when `op.exec` moves away or the host is marked failed); a purge that failed after the CAS and a crash before the marker made two unfinished admissions in a row, which the single-predecessor reading refused forever (seed 28216 / 30,000: the chain); a push released over a dead probe whose marker never landed (seed 487 / 30,000); an idle stop before replication opened certified position 0 over two unshipped writes (seed 2663 / 30,000); a pre-flip round forced a sync on a session whose first upload was in flight and wrote its key with other bytes (seed 17056 / 30,000), and on a dead session's database after an agent restart, where the skipped upload counted as durable (seed 12582 / 100,000); a first upload retried with fresh bytes carried a write the original did not, and the original landed last (seed 7574 / 30,000); a restarted agent continued a dead session's replication and its first sync was skipped at the dead session's remote position (seed 8528 / 30,000); the predecessor of a writer that died between its marker landing and its chore CAS was never retired (seeds 899, 4159, 26817 / 30,000); an abort released over a source admitted a tick earlier and the oracle restored an unfinished generation (seed 1052 / 30,000: the journal survives the release, the oracle restores as the next admission would); a declared-lost target's registered timers were lost undeclared (seed 4120 / 30,000); a clean rollback restored the source from a replica short of its stopped database and declared a write the seal held (seed 17056 / 30,000: `seal-return`); a cut or a push on a box without a database sealed an empty database when its remote's marker lagged (seeds 3987 and 14322 / 30,000: the seal is what the next admission would restore, and nothing is sealed over data that cannot be read); a seal built from the remote after a lost disk was taken to cover the predecessor and a pending write went undeclared (seed 34260 / 100,000); a declared-lost target that returned had its declared writes judged by INV-15 (seed 54594 / 100,000: the operation's lost box counts); a dirty install parked a dead installation's leftover under a newer active stamp (seed 65090 / 100,000: a retired stamp's artifact may be parked); the journal-wide retire set first queued a chore per generation and starved two restoring operations of their release (seeds 542 and 2412 / 3,000: uncovered entries only); a rollback whose target was declared lost before any lane ran declared a write the seal in the lost region held, which INV-15 refused (seed 8035 / 30,000: a lost box's rollback is INV-15's exception), and left the source's own cut-session writes undeclared (seeds 16395 and 27191 / 30,000: the source's re-admission declares its cut session too, §7.10); an adoption left `holder.writer_box` unset and the first cold admission's declaration named the wrong box, so the legacy process's writes lost with its disk went undeclared (seed 94288 / 100,000: the adoption names the box running the pre-release process). **Rev 7.10: joint round 9 and claim rev 6.9.** - **No local database reuse (R9-1, claim rev 6.9).** `spawn` admits every non-legacy start through `admitWriter` (the decision, no CAS) and `admitCommit` (the `AdmitWriter` CAS): it purges the box's database (`PurgeFails`, a transient fault injected at reboots and healed in the calm phase: the writer does not start), restores from the remote (`restoreCalc` of `holder.generation` for warm, to at least the certificate's `max_txid`; `restorePinned` of the operation's prepared generation from its build journal; `restoreCalc` of the predecessor for a fresh cold generation), checks readiness, makes the CAS, then finishes the start as an `openState` on the box: the full first upload `[1, 1]` for a fresh generation with content, the marker, the warm anchor when the restored position is above the level-0 maximum, each a PUT acknowledged before the next, retried with the same bytes after the client's timeout and driven by the agent's own loop (`resumeStart`), not by wakes; `atRest` counts an open in progress. A warm admission's first upload is full (fact R5-1c applies to every restore now), so `compact` makes its output full when any input is. `warmCertificate` became `certificate`: no box check (claim rev 6.8.2). Rule `remote-baseline` (the mutation keeps a local database of the generation it opens); witness `wStaleLocalAfterClose`. - **`holder.writer_box` and `writes_lost` (R9-2).** `holder.WriterBox` is written by every admission; a cold admission with `PredecessorCertified` false records `writesLost{Box: observed writer_box, Gen, Session, RemoteMaxTXID}` in its journal entry, declares the predecessor box's acknowledged, non-durable writes lost (the oracles' exemption) and logs the alert; timers are not writes. Rule `writes-lost` (the mutation loses them silently); witness `wSilentLoss`. - **Warm and cold only (R9-3).** `op.Rec`, `recStep`, `op.InstGen`, `op.Fresh`, the `open` and `fresh` modes and the recovery build are gone. `installGen` names an operation's prepared generation; its probe is a cold admission with `NewGeneration` the prepared one, never re-entered once `holder.generation` or an ancestor of it (`ancestorGen`), else a fresh id; the landing's prepared generation keeps the flip's stamp (`op.FlipGen`), every other cold admission issues one. The flip, the commit, the push's probed CAS and the dirty rollback's probed CAS move no generation; the return CAS names `op.SrcGen` (recorded at the flip) again for a clean rollback and the source republishes that generation's marker from its last version (`republishMarker`); a lagging marker is accepted only for this box's own unfinished admission or the source after a return, and an abandoned empty generation yields to its nearest ancestor with files (`parentGen`). A `park-generation` chore for a fresh generation's predecessor, in the admitting box's region. The landing's and the push's `published` phases publish nothing; the landing's `restore` phase computes the installed digest from a pure restore; a target that never restored a database returns nothing dirty. - **The admission bound to the operation (R9-4).** `admitInput` carries `ObservedOpID`, `ObservedPhase` (`phaseOf`), `ObservedSession`, `ObservedGeneration`, `Box`, `Mode`, `NewGeneration`, `PredecessorCertified`, `RemoteMaxTXID`; the CAS refuses a changed op id or phase and the kinds `deleting`, `parking`, `purging`, `creating`; INV-13 in `admitCommit` (no writer admitted into those kinds). Rule `admit-bind-op` (the mutation compares session and generation alone); witness `wStaleClosureDelete`, with `claim-cas-opid` as its partner, since the model's general CAS condition on `op.id` (C2) already refuses a closure whose operation changed. - **Close evidence by client outcome (R9-5).** `track` records every request whose success the client did not see at once and marks a refusal uncertain; `settled` requires every tracked request done within the client's wait (`doneAt - at < uploadTimeout`), marks a timeout or a completion after the wait (a lost response) uncertain for good, and covers level-0, compaction, snapshot and retention alike; the pre-cut source certifies (`moving` is inside); the idle stop kills the cgroup and the close waits while any process lives. Witnesses `wMaintenanceRetry` and `wLostResponse`. - **Witnesses moved.** `wEmptyBootstrap` exercises the empty install through a move of the never-written incarnation (a push's seal carries its own mark and is never empty; the valid-ltx mutation now reaches the anchor `[0, 0]`); `restore-overlap`'s named failure is INV-3 (a restore short of the certificate's `max_txid` refuses to start, where rev 7.4's rule lost four writes); the admission scenario's named failure is INV-1 again; the whole-start-check witness is retired (with every restore from the box's own region, the stale source of a cross-region move finds nothing to restore; `expected-outcomes.txt`); `wGenerationConditional` holds the probe's marker publication; `wDirtyReturn` replicates the target's write so no declaration can cover it; `wRollbackJournal` and `wProbeRestart` read `holder.generation`. - **Model faults found and fixed this round** (all before the final numbers): a lagging marker refused every later start (the admitted writer's own unfinished start was not recognized); a cold admission restored an abandoned empty generation and lost the data (its parent is the baseline); the position-0 build of a never-replicated seal (the TXID-1 rule, rev 7.9) and an empty seal treated alike; the files tag checked against the marker refused probes after the admission retagged them; the purge fault, left on, starved a box for the whole calm phase (one-shot); a prepared generation opened twice (seeds 43 and 87 / 300); a target that never restored a database returned a spurious dirty seal that the source then restored, empty (seed 62 / 300); a descendant surviving the idle stop acknowledged writes the certificate did not cover and the next purge lost silently (seed 17 / 300); every close in `moving` was dropped and the pre-cut source churned generations (seed 62 / 300); the admitted writer waiting for its marker was resumed only by a wake (seed 5094 / 30,000); the landing's prepared generation got a second stamp and the probe was refused by its own record (seed 1117 / 30,000); the return's lagging marker (seed 9554 / 30,000); a compaction of a full input into a delta lost a deletion (seed 27371 / 30,000); a close that waited for an escaped descendant, and one that waited for a certificate PUT late by three horizons, held a revert's probe until it stuck (seed 97950 / 100,000: the close is refused, not delayed); a stalled probe step carried the stamp active at its read, before a cold admission's installing CAS, and the box's record refused its own probe (seed 67671 / 100,000: the resumed step re-reads its record with the claim). Under the new start, `strict-spawn` is caught alone by the sweep (a non-strict descendant surviving the idle stop acknowledges writes the purge then loses), so it leaves `expected-outcomes.txt`. **Rev 7.9: joint round 8, claim rev 6.8 and 6.8.1.** - **Writer sessions (R8-1, claim rev 6.8 §6.4a).** `holder.session` is the gen of the admitting CAS; `admitWriter` decides the mode (`open`: the box's own cold admission has published; `warm`: the certificate of the observed session is intact, `warmCertificate`; `fresh`: an install's first probe into the generation it built; else `cold`) and `admitCommit` makes the CAS as the process starts, after the start and health checks, bumping `holder.session` and appending the writers journal entry with its predecessor. The writer records `db.Session`, writes `box.lineage[inc] = {generation, session}` and reads its remote position lazily at its first sync (`RemotePos = -1`). The certificate is `closed/.json` `{session, box, max_txid}`, written once with `If-None-Match` and never deleted; the live set keeps every session's certificate of the live generation. Rule `writer-session` (the mutation takes the newest certificate of any session, as rev 7.8 took `closed.json`). Witness `wDeadSessionCertificate`: two sessions certify, the third dies with its disk and an upload in flight; the dead certificates stand with a `max_txid` that still matches the remote maximum; the next wake is cold under the rule and continues the generation under the mutation, where the late upload replaces its file (INV-2). - **Admission on every path (R8-2, claim rev 6.8 §6.4a and 6.8.1).** `spawn` admits every non-legacy start: ordinary wakes, a committed landing's wakes while the claim is still `landing`, a probe, a probe restarted in its settle window (the landing's `probed` phase respawns a probe that died, as §7.13 says), a reverted source, a rollback's source probe and its wakes after `opened`, a push's probe. A cold admission runs **inside** the operation: `op.Rec` (`recState`: generation, base, session, box, phase, predecessor, the build's artifacts) is cleared at publish; `recStep` runs before the operation's own step on the admitted box; the console refuses a move, push or delete while it is in progress; the generation that moves is `holder.generation` for a wake and the operation's install generation (`op.InstGen`, read through `opGen`) for a probe into a generation under construction. The admission CAS requires `op.exec == this box` (rule `exec-writer`); `admit-everywhere` is the rule that every path is admitted (the mutation gates ordinary wakes alone, rev 7.8's rule). The crash-with-disk fault fires in every operation whose `op.exec` is the box. Witnesses: `wCommittedLandingCrash` (the committed target dies with its disk and an upload in flight before finalization; its next wake is cold within the landing, or under the mutation continues the generation and the late upload replaces its file), `wProbeRestart` (the landing's probe dies with its disk and an upload in flight; the restarted probe is cold into a fresh install generation, or under the mutation reuses it and the landing's restore loses the restarted probe's write), and the admission scenario (`wAdmissionTriple`) with `exec-writer` as a fourth fence: with all four gone the non-exec source's wake is a cold admission whose installing CAS moves `holder.stamp` under the target (INV-6), on its way to two writers (INV-1); the three box-side fences alone no longer hold the scenario, since under writer sessions the source's wake has no certificate for the current session and would be admitted cold rather than refused by its generation check, and `exec-writer` alone holds it, so the four are one combination. - **The empty generation (R8-3).** `snapshotOf` encodes a database with transactions but no replication position at TXID 1 (litestream's first sync) and only a database with no transaction at all at 0; a build or install at position 0 writes no snapshot and no anchor and publishes a marker with `snapshot_txid = 0`; `restoreCalc` of such a generation yields an empty database; INV-12 in `issue` refuses any LTX PUT with a zero TXID. Rule `valid-ltx` (the mutation writes `[1, 0]` and `[0, 0]`, rev 7.8's build). Witness `wEmptyBootstrap`: a create's first wake, no LTX file before the first transaction, the first transaction as the first upload, a disk loss and a cold wake. - **Provenance (R8-4).** `warmProvenance`: the lineage file names the journalled predecessor's `{generation, session}`, the predecessor's box is this box, and the database stands at or beyond its generation's remote position (the integrity check's stand-in: its uploads came from it). Otherwise the build restores the predecessor generation and declares the warm database's unreplicated acknowledged writes lost. `initRecord` writes no lineage. Rule `warm-provenance` (the mutation takes any database of the predecessor generation, rev 7.8's rule). Witness `wWarmLineageMismatch`: a stale copy of the database with a foreign lineage replaces the box's; the cold admission restores from the remote under the rule, and under the mutation serves the copy, missing a durable write (INV-2). - **Close evidence (R8-5).** `localDB.Issued` tracks every request of the writer's lifetime (level-0 uploads, compactions and their retention deletes, snapshots and their retention), `Uncertain` is set when the client gives up on an upload and retries it, and `settled` is every tracked request done, the queue drained and not uncertain; `closeStep` refuses to certify an uncertain session. Rule `close-evidence` (the mutation checks the level-0 queue alone, rev 7.8's drain). Witness `wRetryUnresolved`: one upload is held, the client times out and retries, the retry lands, the original stays outstanding; the clean stop writes no certificate under the rule and writes one under the mutation. - **Model faults found and fixed this round** (all before the final numbers): the first rev 7.9 build made the admission CAS before the health check, so failing starts admitted sessions in a storm (13 / 300 seeds); a stalled worker's replay of a finished cold admission's build took stop evidence of the running writer and replaced its database with the build's snapshot (seed 10382 / 30,000; the revalidation step now includes the writer state); the cold build left the carried files under the dead writer's stamp, and the next dirty install parked an artifact of a retired stamp (seed 453 / 30,000, INV-6); a source that lost its disk during a rollback or revert compared a missing database with its stop digest and stuck (seed 95 / 300); the local oracle refused a write declared lost that landed anyway and came back through a cold admission's remote baseline (seed 5863 / 30,000; it now accepts what the files carry, as the cold-restore oracle does); a seal of a database with transactions and position 0 was taken for empty and the landing installed nothing (seed 19023 / 30,000); the biased schedule's unbounded hold, one per park of a dead generation, chained across the whole convergence window (seed 20077 / 30,000; chaos only now); a lost target's unfinished cold admission inside the landing blocked the source's rollback probe for good (seed 58795 / 100,000; the next holder of `op.exec` abandons it and parks its generation); every clean close outside `serving` was dropped, so a rollback's reopened source was admitted cold at every wake and the rollback never released (seed 84087 / 100,000; a writer inside an operation certifies like any other, and the close is dropped only for a push, a move's cut, a delete or a purge, since the first correction let a push's writer certify into the superseded generation, seed 4493 / 30,000). The diagnostics added for these (why an admission is cold, which baseline a build takes, what keeps a run from converging) stay in the model. **Rev 7.8: joint round 7 and claim rev 6.6.** - **Writer epochs (R7-1, claim rev 6.6 §6.4a).** `idleStop` is a clean close: the writer drains its uploads (`closeStep` runs while no process runs), then PUTs `closed.json {stamp, max_txid}` once (`If-None-Match`); a crash while draining forgets the close (`forget`), and an operation that takes the writer over drops it. At an ordinary wake `replicationState` continues the generation only if the marker exists, its `max_txid` is the generation's remote maximum over every level and its stamp is the box's active stamp; the writer consumes the marker (a version delete) as it opens; a wake that finds the previous writer still draining waits. Otherwise `startRecovery` is the installing CAS (`op.kind = recovering`, a fresh id, `holder.generation` and `holder.stamp` in the same write) and `recoverStep` the data path: stop evidence, the baseline (the warm database when it is the old generation's, else `restoreCalc` of the old generation, or a snapshot already current in the new generation on a replay), the snapshot, the anchor and the installer's `closed.json` each acknowledged (`ackedPut`), `generation.json`, a `park-generation` obligation for the old generation after the dead writer's horizon, release. The database replicates only into its generation (`ltxKey(db.Loc, db.Gen, ...)`), its position is what the new generation's listing shows, and its first upload carries every page. The upload loop skips a local file at or below the remote position, as litestream's does, and the writer believes it replicated (`markDurable`), which is the initialization/remote-position race. Rules `recovery-generation` (the mutation is rev 7.7's same-generation restart), `closed-marker` (the mutation takes any marker as intact and consumes none), `close-waits` (the mutation writes the marker at once). Witnesses: `wLateUploadAfter` (A's upload lands after B's), `wLateUploadRace` (A's lands after B's initialization and before B's upload: B's skip), `wExemptKeyRestored` (R7-2's counterexample), `wCrashBeforeMarker`, `wStaleMarker`, `wCloseWaits`; each checks the cold restore the moment the late upload lands, before any snapshot of the warm database could repair the replica. A created incarnation's first wake is a cold recovery (its origin generation has no marker), so are a legacy adoption's and every wake after a crash; the random sweep counts them (`cold recoveries`). The agent-crash fault takes the disk with it in one case in three, so the sweep reaches a dead writer's upload in flight across a replacement. - **The oracle checks writes by identity (R7-2).** `checkDurable` requires every durable, non-exempt write to be in the restored files' log and compares the state with `expectFrom`, the replay in acknowledgement order of exactly the writes those files carry; a write declared lost that landed anyway is replayed because it is there, no key is masked, and a later required write re-establishes its key. A full file (a snapshot, an anchor, the first upload after a cold initialization, a compaction of one) replaces the state it is applied to and carries every write it holds, which the first rev 7.8 baseline needed (seed 12436 / 30,000: a full level-0 file carried a deletion's effect without the write). - **The newest level-0 file survives every retention path (R7-3)**: compaction keeps its newest input and the expiry below the previous snapshot skips the newest level-0 file (`newest-l0`). Witness: the local position runs ahead of the uploads across two compactions and rotations. - **Builds have intent, acknowledged artifacts and completion (R7-4).** A landing, a push and a dirty rollback journal `install-copy` (snapshot acknowledged), `install-anchor` (anchor acknowledged), then `published`; `publishGen` refuses, as INV-11, a marker naming a snapshot version that does not exist or an anchor that is not current (`build-completion`: the mutation publishes from the intent). Witness: a push's build with its snapshot, then its anchor, held, and the agent crashing in between each. - **Two witnesses the sweep's new shape needed.** With a cold recovery at every crash, `evidence-whole-tree` and `timer-freeze` fell to 0 / 3,000; each has a witness now: an adopted legacy host with an escaped descendant whose move must take evidence of the whole tree (INV-1), and a registered timer whose refused deliveries while the row is held must not advance the runner's bookkeeping (a clean rollback's digest check, `stuck`). - **Retired witnesses.** `door-reread` and `door-by-version` are subsumed: a recreated incarnation's first wake is a cold recovery into a fresh generation, which a listing scoped to the generation it was authorized for never selects, so the scenario holds with C3a dropped too; C3a stays as defence in depth (`expected-outcomes.txt`). - **Model faults found and fixed this round** (all before the final numbers): the first build of a cold recovery after a lost disk restored a generation whose stale marker a failed start had already consumed (the marker is consumed as the writer opens, not at the check); a pending clean close outlived a push and wrote a marker into a dead generation; the recovered database kept stamp 0 and files-gen named the old generation; a descendant wrote during a build (stop evidence); a replayed build rebuilt the snapshot from another baseline (reuse); the crash-with-disk fault first left a nested descendant alive to acknowledge writes into no database, which the oracle rightly called lost (nothing survives a lost volume). #### 14.2.3 Invariants and results The invariants are the table at the end of §4, with three checks rev 6 adds and one it makes stricter: - **INV-2, cold restore, at every release and at the end.** A restore of the claim's location and generation in the serving region, computed and never written, holds every acknowledged write that a completed replica object of that generation, or of any generation it superseded, ever held. Rev 5 wrote the serving box's database into the bucket and then restored it (J2-R2-9). - **INV-10 at every operation's end.** At each release to `serving` and each `free`, nothing outside the live set remains, except a version created by a request that completed after `H` (only the sweep can retire those) and what another pending chore owes. At the end of the run, nothing at all. - **C3a.** No database restored or started for an incarnation holds a write another incarnation acknowledged. - **A disk-loss fault** takes the serving box's database file while the app is idle. Writes past the last completed checkpoint are lost by design and declared so; every durable write must come back. **Rev 7.18** changed no rule and ran the gate only (§14.2.1): 300 seeds, zero violations, plus a 3,000-seed check, zero violations; the numbers below are rev 7.17's, on model b9e97783. **Rev 7.17, on the final model (joint round 16, claim rev 6.24: scope membership, the legacy identity, the narrowed born-covered): 30,000 seeds, zero violations** (84 s; no 100,000-seed run under the evidence policy of §14.2.1). Across the 30,000 seeds: 45,047 completed operations, 20,484 cold and 4,803 warm admissions, 8,051 first admissions with no predecessor writer and nothing declared, 3,320 declarations of the legacy identity (session 0), 11,878 holderless sweeps, 1 regions joining a running one, 11,878 `FreeAfterPark` transitions, 148,792 scopes bound before a chore's first step, 8,678 writes_lost records declaring 2,409 writes (72 for a lost target's other sessions, 8 for the source's cut session), 225 completed moves, 621 clean and 24 dirty rollbacks, 97 reverts, 74 declared losses, 7,327 pushes, 27,194 delete-park-free cycles, 2,745 final recovery admissions (0 unverified). Every oracle left the system unchanged. **Rev 7.16, on the final model (joint round 15, claim rev 6.23: the park scope on every chore from its queuing CAS, one deadline): 30,000 seeds, zero violations** (71 s; no 100,000-seed run under the evidence policy of §14.2.1). Across the 30,000 seeds: 45,059 completed operations, 20,487 cold and 4,803 warm admissions, 148,810 scopes bound before a chore's first step (the delete's and adoption's parks; every other chore scoped at its queuing), 11,350 holderless sweeps, 1 regions joining a running one, 11,350 `FreeAfterPark` transitions, 0 starts restoring a selected run generation, 16,733 writes_lost records declaring 2,408 writes (72 for a lost target's other sessions, 8 for the source's cut session), 224 completed moves, 621 clean and 24 dirty rollbacks, 97 reverts, 74 declared losses, 7,327 pushes, 27,197 delete-park-free cycles, 2,746 final recovery admissions (0 unverified). Every oracle left the system unchanged. **Rev 7.15, on the final model (joint round 14: the selected baseline restored, declarations unique, the horizon's parts, joining, every chore scoped): 30,000 seeds, zero violations** (97 s; no 100,000-seed run under the evidence policy of §14.2.1). Across the 30,000 seeds: 45,026 completed operations, among them 20,602 cold and 5,021 warm admissions (3,594 covered `sealed`, 235 covered `clean-return`; 5,021 warm entries born covered, 126 born covered under a pending park, 359 predecessors left uncovered by a publication for their own region, 0 live again and left uncovered, 8,065 holding nothing anywhere, 229 records covered at the commit's park, 81 at a rollback's release, 1 at a lost-target release, 179 by the sweep; 160,230 park scopes bound before a chore's first step, 11,341 holderless sweeps, 2 regions joining a running one and 11,341 `FreeAfterPark` transitions; 0 starts that restored a selected run generation rooted at `[1, 1]`; 0 starts refused on a covered local baseline; 5,964 clean closes, 440 closes refused as uncertain, 330 wakes that waited for a draining writer, 16,012 local databases purged, 1,267 starts that waited for a failed purge, 16,854 writes_lost records declaring 2,455 acknowledged writes lost (78 of the records for a lost target's other sessions, 8 for the source's cut session behind a lost target), 8,276 full first uploads, 160 markers published again, 1,061 probes restarted, 143 probe attempts ended (136 by the deadline check before the start), 474 app starts retried within their session, 0 hosts marked failed; 2,763 final recovery admissions run by the oracle, 0 unverified), 226 completed moves, 622 clean and 22 dirty rollbacks, 96 pre-flip reverts, 74 declared losses, 7,327 pushes, 27,180 delete-park-free cycles and 1,485 lost disks (1,071 with a crash). Every oracle left the system unchanged. **Rev 7.14, on the final model (claim rev 6.22: the persisted park scope, the holderless sweep, coverage in four ways, the run walk across regions): 30,000 seeds, zero violations; 100,000 seeds, zero violations** (169 s and 1021 s; the 100,000-seed run finished before the evidence policy of §14.2.1 was set). Across the 100,000 seeds: 150,124 completed operations, among them 69,017 cold and 16,058 warm admissions (12,090 covered `sealed`, 763 covered `clean-return`; 16,058 warm entries born covered, 384 born covered under a pending park, 1,157 predecessors left uncovered by a publication for their own region, 0 live again and left uncovered, 27,264 holding nothing anywhere, 748 records covered at the commit's park, 275 at a rollback's release, 3 at a lost-target release, 609 by the sweep's `CoverParkedPredecessors`; 497,273 park scopes bound before a chore's first step, 38,098 holderless sweeps and 38,098 `FreeAfterPark` transitions; 0 starts refused on a covered local baseline; 19,236 clean closes, 1,430 closes refused as uncertain, 0 stale certificates, 984 wakes that waited for a draining writer, 52,808 local databases purged, 4,085 starts that waited for a failed purge, 56,370 writes_lost records declaring 8,101 acknowledged writes lost (175 of the records in `additional_writes_lost`), 27,597 full first uploads, 105,879 start steps that waited for their marker, 608 markers published again, 3,614 probes restarted, 486 probe attempts ended by their window, a late readiness or their fifth failure (456 refused before the start by the deadline check), 1,554 app starts that failed and were retried within their session, 0 hosts marked failed by the retry policy, 90 seals whose source replica was short and 38 returns that carried the seal; 9,339 final recovery admissions run by the oracle, 0 unverified), 735 completed moves, 2,105 clean and 89 dirty rollbacks, 318 pre-flip reverts, 264 declared losses, 24,755 pushes, 90,312 delete-park-free cycles and 4,827 lost disks (3,474 with a crash); 9,221 LTX compactions, 6,478 snapshot rotations, 583 level-0 upload retries, 28,962 anchors; 26,123 local stamp records lost and re-initialized; 46,605 requests completed after `H`; 35,592 staging signatures refused as stale; 14,432 creates restarted on a 412. Every oracle left the system unchanged. **Rev 7.13, on the final model (claim rev 6.14: regions on the record, the publication walk, `CoverRegionPredecessors` at whole-location parks, the declared identity, the probe deadline before the start): 30,000 seeds, zero violations; 100,000 seeds, zero violations** (621 s and 1906 s). Across the 100,000 seeds: 148,012 completed operations, among them 68,153 cold and 15,537 warm admissions (12,021 covered `sealed`, 757 covered `clean-return`; 1,123 predecessors left uncovered by a publication for their own region, 735 records covered at the commit's park, 267 at a rollback's release, 3 at a lost-target release, 586 by the sweep, 365 born covered under a pending park, 2 predecessors live again at a publication and left uncovered, 26,793 holding nothing anywhere; 18,651 clean closes, 1,384 closes refused as uncertain, 0 stale certificates, 970 wakes that waited for a draining writer, 52,025 local databases purged, 4,124 starts that waited for a failed purge, 55,583 writes_lost records declaring 8,001 acknowledged writes lost (181 of the records for a lost target's earlier sessions), 27,328 full first uploads, 106,044 start steps that waited for their marker, 590 markers published again, 3,578 probes restarted, 474 probe attempts ended by their window, a late readiness or their fifth failure (438 of them refused before the start by the deadline check, 0 at a late readiness), 1,543 app starts that failed and were retried within their session, 0 hosts marked failed by the retry policy (serving wakes only), 93 seals whose source replica was short and 45 returns that carried the seal; 9,360 final recovery admissions run by the oracle, 0 unverified), 721 completed moves, 2,000 clean and 91 dirty rollbacks, 311 pre-flip reverts, 249 declared losses, 24,340 pushes, 89,267 delete-park-free cycles and 4,725 lost disks (3,423 with a crash); 8,881 LTX compactions, 6,208 snapshot rotations, 527 level-0 upload retries, 28,430 anchors; 26,014 local stamp records lost and re-initialized; 47,177 requests completed after `H`; 33,122 staging signatures refused as stale; 30,530 creates restarted on a 412. Every oracle left the system unchanged. **Rev 7.12, on the final model (claim rev 6.12: `holder.admissions`, `Coverage`, `PublishGeneration`, the origin's absent marker, the final recovery admission, a probe attempt's window): 30,000 seeds, zero violations; 100,000 seeds, zero violations** (673 s and 2231 s). Across the 100,000 seeds: 150,033 completed operations, among them 71,784 cold and 16,875 warm admissions (13,689 covered `sealed`, 919 covered `clean-return`; 20,049 clean closes, 1,568 closes refused as uncertain, 0 stale certificates, 1,063 wakes that waited for a draining writer, 56,566 local databases purged, 4,443 starts that waited for a failed purge, 57,206 writes_lost records declaring 8,884 acknowledged writes lost, 29,236 full first uploads, 108,295 start steps that waited for their marker, 663 markers published again, 3,883 probes restarted, 541 probe attempts ended by their window or fifth failure, 2,324 app starts that failed and were retried within their session, 0 hosts marked failed by the retry policy (a probe no longer marks one, and no serving wake failed five times in these seeds), 111 seals whose source replica was short and 51 returns that carried the seal; 9,326 final recovery admissions run by the oracle, 0 unverified), 825 completed moves, 2,199 clean and 101 dirty rollbacks, 356 pre-flip reverts, 287 declared losses, 25,894 pushes, 89,204 delete-park-free cycles and 4,945 lost disks (3,672 with a crash); 9,418 LTX compactions, 6,608 snapshot rotations, 564 level-0 upload retries, 30,446 anchors; 26,101 local stamp records lost and re-initialized; 49,321 requests completed after `H`; 33,312 staging signatures refused as stale; 30,514 creates restarted on a 412. Every oracle left the system unchanged. **Rev 7.11, on the final model (claim rev 6.10: the admitting CAS before the app start, a session per agent process, the retry policy, unfinished generations with live baselines, the library's admission shape, declarations as the only exemptions, startup PUTs in the ledger, the uncovered return): 30,000 seeds, zero violations; 100,000 seeds, zero violations** (388 s and 1190 s). Across the 100,000 seeds: 149,786 completed operations, among them 61,925 cold and 16,748 warm admissions (19,934 clean closes, 1,543 closes refused as uncertain, 0 stale certificates, 1,008 wakes that waited for a draining writer, 55,816 local databases purged, 4,418 starts that waited for a failed purge, 48,474 writes_lost records declaring 8,841 acknowledged writes lost, 29,100 full first uploads, 55,026 start steps that waited for their marker, 532 markers published again, 3,854 probes restarted, 1,371 app starts that failed and were retried within their session, 256 hosts marked failed by the retry policy, 112 seals whose source replica was short and 51 returns that carried the seal), 824 completed moves, 2,199 clean and 98 dirty rollbacks, 353 pre-flip reverts, 287 declared losses, 25,677 pushes, 89,192 delete-park-free cycles and 4,927 lost disks (3,649 with a crash); 9,331 LTX compactions, 6,543 snapshot rotations, 557 level-0 upload retries, 30,198 anchors; 26,074 local stamp records lost and re-initialized; 43,228 requests completed after `H`; 32,872 staging signatures refused as stale; 30,501 creates restarted on a 412. Every oracle left the system unchanged. The model faults the new protocol exposed before these baselines are listed in §14.2.2. Rev 7.10's published counters for this paragraph were wrong (R10-9: the generator misparsed every name with a colon); its retained baseline reads 57,938 cold and 17,025 warm admissions, 1,283 closes refused as uncertain, 6,761 starts refused by a failed purge and 56,380 waits for the marker, and the zero-violation verdict stands. **Rev 7.10, on the final model (claim rev 6.9: no local database reuse, warm and cold only, writer_box and writes_lost, the admission bound to the operation, close evidence by client outcome): 30,000 seeds, zero violations; 100,000 seeds, zero violations** (84 s and 279 s). Across the 100,000 seeds: 149,973 completed operations, among them 0 cold and 0 warm admissions (20,116 clean closes, 0 closes refused as uncertain, 0 stale certificates, 1,003 wakes that waited for a draining writer, 202,837 local databases purged, 0 starts refused by a failed purge, 57,938 writes_lost records declaring 5,817 acknowledged writes lost, 25,140 full first uploads, 0 starts that waited for their marker, 0 markers published again, 179 probes restarted), 1,433 completed moves, 1,715 clean and 143 dirty rollbacks, 352 pre-flip reverts, 305 declared losses, 25,627 pushes, 89,152 delete-park-free cycles and 6,234 lost disks (4,763 with a crash); 9,706 LTX compactions, 6,802 snapshot rotations, 581 level-0 upload retries, 30,362 anchors; 26,035 local stamp records lost and re-initialized; 41,855 requests completed after `H`; 32,791 staging signatures refused as stale; 30,496 creates restarted on a 412. Every oracle left the system unchanged. The model faults the new rules exposed before these baselines are listed in §14.2.2. **Rev 7.9, on claim rev 6.8 and 6.8.1 (writer sessions, admission on every path, the empty generation, provenance, close evidence): 30,000 seeds, zero violations; 100,000 seeds, zero violations** (145 s and 327 s). Across the 100,000 seeds: 147,788 completed operations, among them 43,976 cold admissions (21,499 warm, 24,385 opened after a cold build, 26,228 fresh installs; 25,754 clean closes, 644 closes refused as uncertain, 2 stale certificates, 3,930 wakes that waited for a draining writer, 30,732 builds from the remote and 14,731 from the warm database, 1,459 warm databases refused for want of provenance, 171 probes restarted, 3 cold admissions abandoned when their box lost `op.exec`), 1,126 completed moves, 1,396 clean and 121 dirty rollbacks, 266 pre-flip reverts, 258 declared losses, 24,666 pushes, 89,526 delete-park-free cycles and 7,561 lost disks (4,935 with a crash); 19,392 LTX compactions, 13,463 snapshot rotations, 945 level-0 upload retries, 0 uploads skipped at the remote position, 54,460 anchors; 25,969 local stamp records lost and re-initialized; 49,209 requests completed after `H`; 32,144 staging signatures refused as stale; 30,756 creates restarted on a 412. Every oracle left the system unchanged. The model faults the new protocol exposed before these baselines are listed in §14.2.2. **Rev 7.8, on claim rev 6.6 (writer epochs; the identity oracle; the newest level-0 file kept; builds with intent and completion): 30,000 seeds, zero violations; 100,000 seeds, zero violations** (60 s and 183 s). Across the 100,000 seeds: 174,931 completed operations, among them 26,380 cold recoveries (24,846 clean closes, 303 stale markers, 2,303 wakes that waited for a draining writer, 22,252 build steps that found their snapshot already current, a replay or a wait), 1,306 completed moves, 1,538 clean and 85 dirty rollbacks, 315 pre-flip reverts, 249 declared losses, 25,252 pushes, 89,404 delete-park-free cycles and 1,839 lost disks; 25,843 LTX compactions, 18,113 snapshot rotations, 1,159 level-0 upload retries, 536 uploads skipped at the remote position (a writer's own landed file, never another's), 42,449 anchors and 0 anchor waits; 26,075 local stamp records lost and re-initialized; 45,522 requests completed after `H`; 32,547 staging signatures were refused as stale; 30,706 creates restarted on a 412; 72 targets were stopped mid-return. Every oracle left the system unchanged. The model faults the rebuilt oracle and the new protocol found before these baselines are listed in §14.2.2. **Rev 7.7, on claim rev 6.5 (the return to source; the pure restore oracle, the acknowledged anchor, compaction keeping the newest level-0 file, the lost local record): 30,000 seeds, zero violations; 100,000 seeds, zero violations** (35 s and 115 s). Across the 100,000 seeds: 155,280 completed operations, among them 1,739 completed moves, 1,871 clean and 89 dirty rollbacks, 426 pre-flip reverts, 332 declared losses, 29,995 pushes, 89,101 delete-park-free cycles and 1,563 lost disks; 18,465 LTX compactions, 12,889 snapshot rotations, 751 level-0 upload retries, 32,387 anchors written at cold restores and 12,307 wakes that waited for an anchor's acknowledgement; 26,127 local stamp records lost with a reboot and re-initialized; 38,225 requests completed after `H`; 33,527 staging signatures were refused as stale; 30,646 creates restarted on a 412; 99 targets were stopped mid-return. Every oracle left the system unchanged (the purity assertion, §14.2.1). One oracle fault found and fixed before these baselines (§14.2.2): a write declared lost with a disk whose upload landed anyway and was applied by a later restore; the comparison now masks such keys (seed 68282 / 100,000 under rev 7.7's first binary). **Rev 7.6, on claim rev 6.4 (`holder.stamp`, initialization under the guard, the two retirements; the replication position, the anchor, sequential uploads, snapshot rotation and retention, root advancement; no whole-state checksum; the pinned S3 layout): 30,000 seeds, zero violations; 100,000 seeds, zero violations** (39 s and 120 s: without the whole-state checksum the page-image model runs at rev 7.4's speed again). Across the 100,000 seeds: 154,958 completed operations, among them 1,823 completed moves, 1,884 clean and 95 dirty rollbacks, 386 pre-flip reverts, 330 declared losses, 30,057 pushes, 89,042 delete-park-free cycles and 1,491 lost disks; 14,530 LTX compactions, 10,184 snapshot rotations, 737 level-0 upload retries and 34,658 anchors written at cold restores; 38,827 requests completed after `H`; 33,363 staging signatures were refused as stale; 30,693 creates restarted on a 412; 120 targets were stopped mid-return. No restore was refused for a gap with every rule in place. Three faults the rebuilt model found in itself before these baselines, each fixed in the model step (§14.2.2): the sweep misread `g-/` under a legacy host as an incarnation location and parked the live generation (2 / 3,000); the replicator shipped the next level-0 file while the one ahead was still in flight, so a cold restore met a gap (9 / 30,000, 19 / 100,000); and, before that, a seal advanced the position without shipping its file. Rev 7.5's explanation of its own fix below, the `-txid` sidecar, was wrong (R5-1, fact R5-1e); the position comes from the newest level-0 file, which is why the anchor exists. **Rev 7.5, on claim rev 6.2 (installation stamps, the region-qualified live set, retirement under purge; LTX files with compaction and litestream's restore planner; the marker-hold and targeted-freeze schedules and the retention fixture): 30,000 seeds, zero violations; 100,000 seeds, zero violations** (4 m 46 s and 15 m 03 s: page images and checksums cost about nine times rev 7.4's run time). Across the 100,000 seeds: 155,367 completed operations, among them 1,851 completed moves, 1,890 clean and 88 dirty rollbacks, 377 pre-flip reverts, 315 declared losses, 30,228 pushes, 89,170 delete-park-free cycles and 1,412 lost disks; 12,568 LTX compactions; 38,599 requests completed after `H` (the held marker publications among them); 33,549 staging signatures were refused as stale; 30,730 creates restarted on a 412; 121 targets were stopped mid-return. No restore was refused for a gap or a checksum mismatch with every rule in place. Rev 7.5's first baseline run found a model fault the oracle was built for: after a cold restore the model's replicator kept position 0 and shipped a `[1, TX]` file carrying only the new pages; the planner preferred it, the checksum check refused the restore, and the oracle reported the durable writes lost (2 / 30,000, 5 / 100,000). The restored database now replicates from the TXID it reached, as litestream's `-txid` sidecar has it. **Rev 7.4, on claim rev 6 (the restore chain, the installation-scoped fence, the ordered orphan cleanup, the namespace sweep, public delivery): 30,000 seeds, zero violations; 100,000 seeds, zero violations** (42 s and 100 s). Across the 100,000 seeds: 165,084 completed operations, among them 1,790 completed moves, 1,881 clean and 95 dirty rollbacks, 432 pre-flip reverts, 382 declared losses, 31,588 pushes, 94,712 delete-park-free cycles and 1,426 lost disks. 24,258 requests completed after `H`; 30,949 staging signatures were refused as stale; 14,114 creates restarted on a 412; 117 targets were stopped mid-return. The orphan plans reshuffle the seeds against rev 7.3 (a plan is one more console entry to pick from), which is what the witnesses are for. No restore was refused for a conflicting history in any seed: with every rule in place no generation ever holds two histories. **Rev 7.3, on rev 7.2's model with the deterministic purge drain: 30,000 seeds, zero violations; 100,000 seeds, zero violations** (35 s and 121 s). Across the 100,000 seeds: 165,834 completed operations, among them 1,613 completed moves, 2,072 clean and 96 dirty rollbacks, 443 pre-flip reverts, 388 declared losses, 31,682 pushes, 95,195 delete-park-free cycles and 1,416 lost disks. 24,270 requests completed after `H`; 30,884 staging signatures were refused as stale; 14,143 creates restarted on a 412; 106 targets were stopped mid-return. Every witness holds (22 scenarios, under a second). The rev 7.2 figures below differ by a few operations: the drain's map order (§14.2.2). **Rev 7.2, on claim rev 5.2 (CC22 with the restart and C5a; the biased schedules for the recreate, the paused creator and the target stopped mid-return): 30,000 seeds, zero violations; 100,000 seeds, zero violations** (32 s and 100 s). Across the 100,000 seeds: 165,856 completed operations, among them 1,611 completed moves, 2,072 clean and 96 dirty rollbacks, 443 pre-flip reverts, 388 declared losses, 31,686 pushes, 95,205 delete-park-free cycles and 1,418 lost disks. 24,266 requests completed after `H`; 30,885 staging signatures were refused as stale; 14,136 creates restarted on a 412; 106 targets were stopped mid-return by the biased schedule; 175 landings failed the settle check (rev 7.1: 74 dirty rollbacks). 573 landings rolled back because a config bundle was published during the move. **Rev 7.1, on claim rev 5.1 with CC22, the purge fence, the paused creator in half the seeds, the conditional marker: 30,000 seeds, zero violations; 100,000 seeds, zero violations** (73 s and 259 s, with the mutation sweep running beside them). Across the 100,000 seeds: 166,106 completed operations, among them 1,719 completed moves, 2,027 clean and 74 dirty rollbacks, 473 pre-flip reverts, 287 declared losses, 31,885 pushes, 95,232 delete-park-free cycles and 1,374 lost disks. 24,294 requests completed after `H`; 30,994 staging signatures were refused as stale. 564 landings rolled back because a config bundle was published during the move. There are fewer completed operations than rev 7's 180,848, nearly all of the difference in delete-park-free cycles (103,652 then): the paused creator is the one delay rev 7.1 adds, and it holds a create back for up to 300 steps in half the seeds. **Rev 7, on claim rev 5 (the biased schedule in half the seeds): 30,000 seeds, zero violations (28 s); 100,000 seeds, zero violations (117 s).** Across the 100,000 seeds: 180,848 completed operations, among them 1,774 completed moves, 2,277 clean and 77 dirty rollbacks, 565 pre-flip reverts, 345 declared losses, 33,722 pushes, 103,652 delete-park-free cycles and 1,493 lost disks. 24,973 requests completed after `H`; 30,551 staging signatures were refused as stale (most after an SDK retry that waited out a long pause). 666 landings rolled back because a config bundle was published during the move. Rev 6's figures were 30,000 and 100,000 seeds clean on claim rev 4. There are fewer completed moves than in rev 5 (3,887) for two reasons the stricter model makes real: bundle publications during a move now roll it back (J2-R2-5), and every transition takes two writes. #### 14.2.4 Faults the model found beyond the joint review Each was a failing seed first, then a rule; each rule's mutation is in §14.2.5. | id | fault | fix | owner | |---|---|---|---| | model 1 | A replicator's PUT signed before stop evidence landed after a park had verified the prefix empty, leaving residue in a freed host's namespace. Claim C3's "late completion is harmless" does not hold for its own replication exception | verify after `evidence + H` (§5.3, §6.4) | claim | | model 2 | A park that restarts its snapshot on every retry never finishes with many versions and late completions | destinations keyed by source version; skip copies that landed (§6.4) | claim | | model 3 | A source that cannot run the instance (no secrets key) was cut, and its pre-flip recovery could never pass | S1 is a hard gate (§7.3) | moves | | model 4 | Pushes and reverts of an instance created just before the operation never passed their probe: the passes skip held rows | every probe renders held config and materializes the release (§7.7) | moves, push | | model 5 | Declare-lost on an expired execution lease let a frozen target resume beside the recovered source | compute fencing only (§7.10) | claim | | model 6 | A target that released dirty and was destroyed before returning left the move waiting forever; a destroyed box owing only local chores blocked release | declare-lost covers both (§7.10) | claim, moves | | model 7 | A purge that looked only at claims missed an unclaimed legacy record of the app and tore down its store while it served | purge adopts first (§7.12) | claim | | model 8 | The recovery pass freed a purged host, `free` dropped the reservation, a create slipped in, and the stalled purge destroyed the new store | the reservation is the purge's journal (§7.12) | claim | | model 9 | A stale local execution-lease file kept self-fencing a box after its operation ended | a fresh read with no operation ends the fence | claim | | model 10 | An adopted legacy host kept pre-rollout residue in other regions for as long as it served | discovery parks at adoption; `touched.complete` after (§19.4) | claim | | model 11 | The purge path adopted without discovery | one adoption routine for every actor | claim | | model 12 | A hold left behind by a finished operation (a revert re-held its row after opening) blocked serving | reconcile releases a hold whose operation ended | claim | | model 13 | The cut deadline, checked only while probing, let slow installs commit far past the budget | every phase before the commit (§7.4) | moves | | model 14 (rev 5) | A staging round revalidated under its lease, paused, and signed after the lease ended; its PUT landed after the staging retirement had verified the prefix empty, leaving staging in the serving region or in a region the instance left | check `SafeUntil` immediately before signing (§7.3); CC14 | moves, claim | | model 15 (rev 5) | With obligations run beside role work, a move started while a `park-replica` of its target region was still running; the park retired the landing's fresh versions, and a cold restore lost acknowledged writes (or the landing never converged) | CC11, already in claim rev 3 §4: now proven | claim | | model 16 (rev 5) | A push whose row was already open (a reconcile acting on a cached claim had released its hold) never released: its open is conditional on its own `hold_op` and the row had none. Found under the `fresh-admission` mutation, unreachable with fresh admission; the open should be idempotent anyway | a push's open, like the landing's, accepts a row already open (F7) | push | | model 17 (rev 6) | With requests completing after `H`, residue under a freed host and in dead generations that no later retirement ever lists | the straggler sweep at rest (§6.4); CC15 | claim, moves | | model 18 (rev 6) | A box froze between a passed `SafeUntil` check and the send; signed at send time, its staging PUT was accepted after the lease and landed after the cleanup | sign at the check, `SafeUntil = Until − skew − 15 min` (§7.3); CC19 | claim, moves | | model 19 (rev 6) | A console that crashed between registering a create and its claim CAS left a registration that every later purge waited on forever; with an expiry alone, a create that had claimed before the expiry was dropped from the purge's host set and served after its store was torn down | registrations are leases, and the drain removes one only if expired and unclaimed; CC16 | claim | | model 20 (rev 6) | A create's record PUT landed after `cancelled_creates.until`, and the orphan blocked every recreate | orphan whenever its incarnation is retired; CC17 | claim | | model 21 (rev 6) | With the start check exactly as rev 5 wrote it, a paused open made the old source ready after the commit and, with the exec check also gone, it served beside the target (Astra's J2-R2-8 trace) | the start check compares the claim's generation (§5.1); CC18 | moves, claim | | model 22 (rev 6) | A bundle published after admission rolled the landing back, and then every recovery probe failed against the pinned inputs: stuck forever (J2-R2-5) | recovery probes run on current inputs (§7.7) | moves | | model 23 (rev 6) | A staging round or a push that took its writes from the local database alone, on a box that had lost it, staged an empty snapshot and parked the durable replica | a staging round is the replica plus the final checkpoint (§7.3); a push likewise (F7) | moves, push | | model 24 (rev 6) | The files directory missing on a box whose markers agree: rev 5's model refused it, rev 5's spec does not (`.files-gen` "when the files directory exists"), and the app mints a new `instance.key` | already covered: `files-carried` is now caught alone (§14.2.5); the start check is unchanged | moves | | model 25 (rev 7) | Claim rev 5's at-rest sweep lists retired locations and dead generations, both under `instances//`; a staging request completing after the move's cleanup left `moves///` residue for good | the sweep also lists `moves//` (§6.4); CC20 | claim | | model 26 (rev 7) | With registrations renewed and drained as claim rev 5 says, a console that paused between renewing and its claim CAS resumed after the drain; the host served after its store was torn down | the create re-checks its registration as the last step before its claim CAS; CC21. Claim rev 5.1 answers with a purge fence on the host claim instead, keeping the re-read as a cheap extra (§19.4) | claim | | model 27 (rev 7) | A target that had begun its dirty return was declared lost; the rollback finished as a clean one and nothing parked the partial return lane in the source's region (1 seed in 100,000 on rev 7; none of 300,000 on rev 7.1; rev 7.2's biased schedule stops a returning target on purpose, and the rule is caught alone, §14.2.5) | declare-lost parks the lane (§7.10) | moves | | model 28 (rev 7.1) | With claim rev 5.1's purge fence, a creator paused across a whole purge resumed after the fence was released; its claim CAS "from free", read fresh, succeeded, for an app whose store was gone | the CAS is conditional on the claim ETag read when the create registered; CC22, adopted in claim rev 5.2 with C5a (a claim object is never deleted) | claim | | model 29 (rev 7.1) | An earlier push's `generation.json` PUT completed after the later push's and put the older generation back; the claim and the bucket disagreed and recovery was stuck | `generation.json` is written If-Match the marker it supersedes (§5.1) | moves | #### 14.2.5 Mutation results **Rev 7.20** adds one rule, `fence-converts-moving` (108 in all), with one witness, `wFenceConvertsMoving` (74 in all): a source terminated before the flip in a one-box region; the fence converts the move to `relocating` at `pinning` (exec none, stamp and generation kept, the lane's park owed), the operator is alerted once, the claim does not churn while no box exists, and the relocation completes on the replacement box. Under the evidence policy of §14.2.1 the gate ran (every witness and the default seeds), `mutate.sh 3000 fence-converts-moving` at `HOSTLIFE_PAR=2` under `nice -n 10` reported `CAUGHT fence-converts-moving: sweep 0 of 3000; witness violated` (the random sweep never terminates a source, so the witness alone catches it, as designed), and one unmutated `HOSTLIFE_SEEDS=3000` run at `HOSTLIFE_PAR=2` passed. No other rule changed and no other sweep was run. Every rule was dropped in turn, with the one prebuilt binary of §14.2.1, on the final rev 7.17 model (b9e97783; rev 7.18 changed no rule and ran no sweep, §14.2.1), with the strict classification, **in two ways**: the **random sweep**, 3,000 seeds each (the 30,000- and 100,000-seed sweeps of earlier revisions are not run under the evidence policy of §14.2.1, Paul 2026-10-06; their columns say so); and the rule's **witness scenarios** (§14.2.2, `witness_test.go`) under the same mutation, for the 60 rules that have them. A rule is caught if either catches it; `mutate.sh` reports the two separately, and this section does too. The model has **107 rules**: rev 7.16's 106 plus `scope-membership` (claim rev 6.24 §2, R16-1, R16-2). `mutate.sh 3000`, run serially at `HOSTLIFE_PAR=2` under `nice -n 19`, exited 0: no ERROR, no unexpected MISSED, no STALE; every live combination was rerun this round. Seed numbers reproduce any sweep line with `HOSTLIFE_SEED`; `HOSTLIFE_WITNESS=` replays a witness with its trace. Rev 7.13's counts, the last with the deeper sweeps, are in the rev 7.13 spec (1f511c42); rev 7.14's in a4389280, rev 7.15's in 63fc6a71, rev 7.16's in 6f2a93c9. **Every table below is regenerated from the retained verdict files** (R9-6; `~/.cache/nearest-region/rev717-sweeps/`), none is carried over. | | rules | |---|---| | caught alone by the sweep at 3,000 seeds | 71 | | caught alone by the sweep at 30,000 (and by their witness, where one exists) | not run for rev 7.17 (evidence policy; Paul 2026-10-06); without a verdict this round, the rare rules the sweep reaches only at 30,000: 1 (`hostguard`) | | caught alone by the sweep at 100,000 (and by its witness, where one exists) | not run for rev 7.17 (evidence policy; Paul 2026-10-06) | | caught alone by their witness only; the sweep at 3,000 misses them | 22 (`absent-marker-origin`, `admit-before-start`, `baseline-live`, `close-evidence`, `close-waits`, `closed-marker`, `covered-baseline-refused`, `declare-lost-compute`, `evidence-whole-tree`, `generation-conditional`, `lost-quarantine`, `markers-last`, `newest-l0`, `orphan-order`, `persisted-park-scope`, `probe-window`, `remote-baseline`, `restore-gap-closed`, `restore-overlap`, `restore-selected-baseline`, `seal-return`, `session-scoped-loss`) | | caught only in a stated combination: the witness holds alone and under every proper subset, and violates with the partners | 8 | | subsumed: the witness holds under every combination tried, or the scenario is unreachable by construction | 2 (`door-reread`, `door-by-version`, by C3a and by writer epochs; historical results only, below). `start-check`, subsumed in rev 7.10, is caught alone by the sweep again since rev 7.11: with the whole check gone the lagging-marker reading is skipped and an empty generation is restored as if it held the data | | liveness only | 2 (`readiness-key`, `config-before-probe`) | | unreachable under claim rev 5 | 1 (`readiness-quiet`) | | unexercised | **0** | **What rev 7.17 moved.** The new rule `scope-membership` 1,252 / 3,000 (a park archives a key outside its scope, INV-23). The rules the scoped delete and sweep touched: `sweep-namespace` 984 / 3,000, `straggler-sweep` 1,192 / 3,000, `sweep-staging` 38 / 3,000, `legacy-discovery` 367 / 3,000, `touched-incomplete` 0 / 3,000, `location-per-incarnation` 29 / 3,000, `live-set-region` 2 / 3,000, `quiet-before-ack` 152 / 3,000, `holderless-park` 562 / 3,000, `writes-lost` 16 / 3,000, `cover-at-park` 30 / 3,000. **What rev 7.16 moved.** No new rule; `persisted-park-scope` witness; sweep 0 / 3,000 gains the staging-deadline scenario, and the scope's queuing-time binding touched `quiet-before-ack` 152 / 3,000, `straggler-sweep` 770 / 3,000, `sweep-namespace` 541 / 3,000, `sweep-staging` 38 / 3,000, `staging-quiet` 1 / 3,000, `lost-return-park` 1 / 3,000, `holderless-park` 561 / 3,000, `cover-at-park` 29 / 3,000. **What rev 7.15 moved.** The two new rules: `restore-selected-baseline` witness; sweep 0 / 3,000 (the start refuses the baseline the walk selected and the recovery stalls) and `unique-declarations` 3 / 3,000 (the source's cut session declared twice, INV-21). The rules the scope and declaration changes touched: `persisted-park-scope` witness; sweep 0 / 3,000, `holderless-park` 561 / 3,000, `covered-baseline-refused` witness; sweep 0 / 3,000, `writes-lost` 11 / 3,000, `lost-return-park` 1 / 3,000, `quiet-before-ack` 132 / 3,000, `straggler-sweep` 778 / 3,000, `sweep-namespace` 542 / 3,000, `cover-at-park` 29 / 3,000, `baseline-live` witness; sweep 0 / 3,000. **What rev 7.14 moved.** The three new rules: `persisted-park-scope` witness; sweep 0 / 3,000 (an agent restart mid-park starts the horizon again from the restart's clock), `holderless-park` 561 / 3,000 (a create claims a host over an outstanding park, INV-20) and `covered-baseline-refused` witness; sweep 0 / 3,000 (the run walk restores a retired local baseline as an empty database). The rules the coverage and park changes touched: `cover-at-park` 29 / 3,000, `foreign-region-uncovered` 4 / 3,000, `absent-marker-origin` witness; sweep 0 / 3,000 (its witness names the covered-baseline refusal now), `baseline-live` witness; sweep 0 / 3,000, `writes-lost` 13 / 3,000, `clean-return-identity` 13 / 3,000, `lost-return-park` 1 / 3,000, `obligations-block-ops` 21 / 3,000, `sweep-namespace` 542 / 3,000, `sweep-staging` 36 / 3,000, `straggler-sweep` 770 / 3,000, `quiet-before-ack` 152 / 3,000, `hostguard` 0 / 3,000, `stale-hold-release` 1 / 3,000, `strict-spawn` 13 / 3,000. The rules rev 7.13 caught only at 30,000 or 100,000 have no sweep verdict this round; the summary table lists them. **What rev 7.13 moved.** The five new rules: `foreign-region-uncovered` 2 / 3,000, 34 / 30,000 (a chore queued into the source's region from the target's publication parks the rollback's baseline), `cover-at-park` 34 / 3,000, 337 / 30,000 (a whole-location park that covers nothing leaves records uncovered, INV-17 as it completes), `session-scoped-loss` witness; sweep 0 / 3,000, 0 / 30,000, 0 / 100,000 (the random sweep cannot see an oracle-precision mutation: its witness constructs the two-session state), `probe-window` witness; sweep 0 / 3,000, 0 / 30,000, 0 / 100,000 (an attempt accepted after its window, INV-19) and `absent-marker-origin` witness; sweep 0 / 3,000, 0 / 30,000, 0 / 100,000 (`wOriginSwept`'s fault, isolated from `baseline-live`'s chore). `baseline-live` fell to witness-only (witness; sweep 0 / 3,000, 0 / 30,000, 0 / 100,000; 2 / 100,000 on rev 7.12: its mutation's premature chore no longer finds a baseline nobody else retires, now that every publication walks the uncovered records), and `declare-lost-compute` is caught at 30,000 this round (1 / 30,000; 1 / 100,000 on rev 7.12). Caught alone: 88 of 101. The rules the coverage change touched: `writes-lost` 13 / 3,000, 110 / 30,000, 368 / 100,000, `clean-return-identity` 15 / 3,000, 112 / 30,000, 332 / 100,000, `seal-return` 0 / 3,000, 8 / 30,000, 22 / 100,000, `dirty-return` 1 / 3,000, 9 / 30,000, `lost-return-park` 2 / 3,000, 27 / 30,000, 64 / 100,000, `obligations-block-ops` 29 / 3,000 (a prepared generation's predecessor now gets its chore at the publication, so an obligation blocks the next operation for a horizon after every push), `recovery-current-inputs` 33 / 3,000, `hostguard` 0 / 3,000, 2 / 30,000, 2 / 100,000, `stale-hold-release` 0 / 3,000, 14 / 30,000, 52 / 100,000, `strict-spawn` 9 / 3,000, 110 / 30,000, 374 / 100,000. **What rev 7.12 moved.** The new rule `clean-return-identity` 13 / 3,000, 125 / 30,000, 387 / 100,000 (a clean rollback after a probed target is a schedule the sweep reaches; under the mutation the false predecessor is INV-16 at the source's re-admission). The final oracle's real recovery and the probe window moved a few counts: `dirty-return` 0 / 3,000, 11 / 30,000, `lost-return-park` 0 / 3,000, 29 / 30,000, 74 / 100,000, `writes-lost` 14 / 3,000, 125 / 30,000, 411 / 100,000, `baseline-live` 0 / 3,000, 0 / 30,000, 2 / 100,000, `seal-return` 0 / 3,000, 12 / 30,000, 27 / 100,000, `recovery-current-inputs` 40 / 3,000, `hostguard` 0 / 3,000, 2 / 30,000, 2 / 100,000, `stale-hold-release` 0 / 3,000, 14 / 30,000, 47 / 100,000, `strict-spawn` 0 / 3,000, 1 / 30,000, 11 / 100,000; `dirty-return` and `lost-return-park` now need 30,000 seeds (the final oracle's recovery declares what the hypothetical declaration used to exempt, and the serving database is judged by it), and `newest-l0` fell back to witness-only (0 / 100,000; 1 / 100,000 on rev 7.11). The witness-only rules keep their shape: `admit-before-start` witness; sweep 0 / 3,000, 0 / 30,000, 0 / 100,000, `remote-baseline` witness; sweep 0 / 3,000, 0 / 30,000, 0 / 100,000, `close-evidence` witness; sweep 0 / 3,000, 0 / 30,000, 0 / 100,000, `closed-marker` witness; sweep 0 / 3,000, 0 / 30,000, 0 / 100,000, `close-waits` witness; sweep 0 / 3,000, 0 / 30,000, 0 / 100,000, `restore-gap-closed` witness; sweep 0 / 3,000, 0 / 30,000; the pair `admit-bind-op` + `claim-cas-opid` 0 / 30,000, 0 / 100,000; the admission four 183 / 30,000, 644 / 100,000. **What rev 7.11 moved.** The three new rules: `admit-before-start` is witness-only (sweep 0 / 3,000, 0 / 30,000, 0 / 100,000: a background acknowledgement in the few ticks between the start and readiness, then a crash before replication opens, is a schedule the generator does not produce), `baseline-live` 0 / 3,000, 0 / 30,000, 2 / 100,000 (a crash between the CAS and the first upload, with the predecessor parked by the chore at the CAS), `seal-return` 0 / 3,000, 10 / 30,000, 26 / 100,000 (a source upload in flight at the cut, then a rollback). `start-check` (13 / 3,000, 181 / 30,000) and `start-check-claim-gen` (12 / 3,000, 140 / 30,000) are caught alone again: the lagging-marker reading hangs on the generation clause, so the rev 5 check restores an empty generation as if it held the data; both leave `expected-outcomes.txt`. The orders changed by the new start moved several counts: `hostguard` 0 / 3,000, 2 / 30,000, 2 / 100,000 and `stale-hold-release` 0 / 3,000, 15 / 30,000, 46 / 100,000 fell below the 3,000-seed line; `writer-session` 1 / 3,000, 19 / 30,000, 61 / 100,000, `admit-everywhere` 705 / 3,000, 6,772 / 30,000, 22,881 / 100,000, `writes-lost` 11 / 3,000, 122 / 30,000, 395 / 100,000, `recovery-generation` 9 / 3,000, 98 / 30,000, 361 / 100,000, `anchor-acked` 13 / 3,000, 103 / 30,000, 358 / 100,000, `restore-anchor` 58 / 3,000, 590 / 30,000, 1,954 / 100,000, `return-stamp` 74 / 3,000, 772 / 30,000, 2,583 / 100,000, `root-advance` 11 / 3,000, 115 / 30,000, 422 / 100,000, `location-per-incarnation` 28 / 3,000, 360 / 30,000, 1,179 / 100,000, `dirty-return` 1 / 3,000, 25 / 30,000, `strict-spawn` 0 / 3,000, 1 / 30,000, 11 / 100,000 (back below the 3,000 line, with `hostguard` and `stale-hold-release`; the three are `rare 30000` in `expected-outcomes.txt`, §14.2.1). The witness-only rules keep their shape: `remote-baseline` witness; sweep 0 / 3,000, 0 / 30,000, 0 / 100,000, `close-evidence` witness; sweep 0 / 3,000, 0 / 30,000, 0 / 100,000, `closed-marker` witness; sweep 0 / 3,000, 0 / 30,000, 0 / 100,000, `close-waits` witness; sweep 0 / 3,000, 0 / 30,000, 0 / 100,000, `restore-gap-closed` witness; sweep 0 / 3,000, 0 / 30,000; `newest-l0` (0 / 3,000, 0 / 30,000, 1 / 100,000) and `live-set-region` (2 / 3,000, 13 / 30,000) are reached by the sweep now; the pair `admit-bind-op` + `claim-cas-opid` 0 / 30,000, 0 / 100,000. The admission four 143 / 30,000, 503 / 100,000; the three box-side fences 140 / 30,000 with the witness held, caught through the generation clause's own catch, so the triple leaves `expected-outcomes.txt` too. **What rev 7.10 moved.** The restore at every wake makes the random sweep reach what it could not before. `writer-session` is caught by the sweep (3 / 3,000, 18 / 30,000, 55 / 100,000; rev 7.9: once in 100,000), `admit-everywhere` everywhere (698 / 3,000, 6,691 / 30,000, 22,531 / 100,000: a stale source continuing the generation now collides at once), `recovery-generation` (11 / 3,000, 77 / 30,000, 266 / 100,000; rev 7.9: 0 at every depth), `rollback-journal` (1 / 3,000, 6 / 30,000), `lost-quarantine` (1 / 3,000, 6 / 30,000), `markers-last` (1 / 3,000, 7 / 30,000) and `timer-freeze` (3 / 3,000, 27 / 30,000). Of the three new rules, `writes-lost` is caught everywhere (13 / 3,000, 95 / 30,000, 318 / 100,000: a silent loss shows in the serving database at once), `remote-baseline` is witness-only (0 / 3,000, 0 / 30,000, 0 / 100,000: the stale cached copy with a matching certificate is a schedule the generator does not produce) and `admit-bind-op` is caught only with `claim-cas-opid` (the pair 0 / 30,000, 0 / 100,000, witness-only: the model's C2 already binds every CAS to `op.id`). `strict-spawn` is caught alone now (3 / 3,000, 24 / 30,000: a non-strict descendant surviving the idle stop acknowledges writes the next purge loses), so it leaves the combination-only list; `start-check` leaves the witnesses, subsumed by regional restores. The admission four 4 / 30,000, 20 / 100,000, the three box-side fences 0 / 30,000 with the witness held (`exec-writer` alone holds the scenario). Rev 7.8's remaining writer-epoch rules keep their shape: `closed-marker`, `close-waits` and `close-evidence` witness-only at every depth, `newest-l0` 0 / 3,000, 0 / 30,000, 0 / 100,000. Rev 7.7's rules hold their rates (`anchor-acked` 12 / 3,000, 95 / 30,000, 346 / 100,000; `return-stamp` 73 / 3,000, 630 / 30,000, 2,161 / 100,000; `restore-anchor` 69 / 3,000, 627 / 30,000, 2,107 / 100,000; `root-advance` 19 / 3,000, 163 / 30,000, 489 / 100,000; `location-per-incarnation` 29 / 3,000, 357 / 30,000, 1,158 / 100,000). Twelve rules the sweep misses at 3,000 are caught by their witnesses, and the sweep's zeros for them say nothing against the hazards (R7-2's point stands). **What rev 7.9 moved.** Of the six new rules, `valid-ltx` is caught everywhere (558 / 3,000, 5,273 / 30,000: a position-0 build writes a file the encoder refuses, and INV-12 sees it at the PUT), `exec-writer` only in the admission combination (its witness holds alone and under every proper subset of the four), and four are witness-only at 3,000 and 30,000: `writer-session` (0, 0, then **1 / 100,000**, seed 63342: the random sweep reached a dead session's certificate that still matched the remote maximum, the R8-1 schedule), `admit-everywhere` (0 / 3,000, 0 / 30,000, 0 / 100,000), `warm-provenance` (0, 0, 0) and `close-evidence` (0, 0, 0). Rev 7.8's writer-epoch rules keep their shape: `recovery-generation`, `closed-marker` and `close-waits` 0 / 3,000, 0 / 30,000, 0 / 100,000, `newest-l0` 0, 0, 1 / 100,000, `build-completion` 655 / 3,000. The admission scenario's named failure is INV-6: with the four fences gone the non-exec source's wake is a cold admission whose installing CAS moves `holder.stamp` under the target, before the two writers would collide (INV-1); the four-rule combination 7 / 30,000, 17 / 100,000, the four with `start-check` in place of its generation clause 7 / 30,000, and the three box-side fences alone **0 / 30,000 with the witness held**: under writer sessions the old source's wake has no certificate for the current session, and `exec-writer` alone refuses it (`expected-outcomes.txt`). Rev 7.7's rules hold their rates (`anchor-acked` 110 / 3,000, 1,218 / 30,000, 4,303 / 100,000; `return-stamp` 48 / 3,000, 503 / 30,000, 1,774 / 100,000; `restore-anchor` 75 / 3,000, 854 / 30,000, 2,905 / 100,000; `root-advance` 68 / 3,000, 643 / 30,000, 2,267 / 100,000; `location-per-incarnation` 28 / 3,000, 336 / 30,000, 1,081 / 100,000). Sixteen rules the sweep misses at 3,000 are caught by their witnesses, six of them by the sweep at 30,000 and two more at 100,000; the sweep's zeros for the writer-session rules say nothing against the hazards, which the witnesses are the evidence for (R7-2's point stands). **What rev 7.8 moved.** The writer-epoch rules were **witness-only**: `recovery-generation` and `closed-marker` 0 / 3,000, 0 / 30,000, 0 / 100,000. The random generator reaches a dead writer's upload in flight across a replacement (the crash-with-disk fault, 1,839 lost disks in 100,000 seeds) but not the two completion orders that cost the next writer a write: the late upload must land after the next writer's own upload for the same TXID, or between its initialization and its sync, and must then be met by a cold restore before a snapshot of the warm database repairs the replica. The six witnesses are the evidence for R7-1 (§14.2.2), and the sweep's zeros say nothing against the hazard (R7-2's point about "not found in 100,000 seeds" stands). `close-waits` 1 / 100,000 and `newest-l0` 1 / 100,000 are caught by the sweep at depth and by their witnesses; `build-completion` 676 / 3,000, 6,645 / 30,000, 22,205 / 100,000 is caught everywhere (the mutation publishes before the artifacts land, and INV-11 sees it at once). Rev 7.7's rules hold their rates (`anchor-acked` 73 / 3,000, 794 / 30,000, 2,789 / 100,000; `return-stamp` 52 / 3,000, 518 / 30,000, 1,869 / 100,000; `restore-anchor` 37 / 3,000, 374 / 30,000, 1,275 / 100,000; `root-advance` 101 / 3,000, 934 / 30,000, 3,367 / 100,000). `location-per-incarnation` (29 / 3,000, 361 / 30,000, 1,153 / 100,000) is caught less often: with every recreated incarnation's writer in a fresh generation, a shared location reuses no key, and the door combinations (`door-reread` + `location-per-incarnation` 414 / 30,000, `door-by-version` + `location-per-incarnation` 361 / 30,000) are the partner's own count; their witnesses are retired (§14.2.2). `rollback-journal` fell to 0 / 30,000 and is witness-only this round; `evidence-whole-tree` (4 / 30,000) and `timer-freeze` (29 / 30,000) gained witnesses. Five rules are witness-only: `recovery-generation`, `closed-marker`, `rollback-journal`, `live-set-region` and `restore-gap-closed`. **The witnesses.** One scenario per rule the sweep catches rarely or only in combination, and one per rule a review round added; two for `registration-check`, one per partner. "Alone" is the witness under the rule's own mutation; "with partners" under the stated combination. The gate also runs every proper subset of each combination and requires the witness to hold there; each witness violates with the failure it names. | rule | the scenario | witness alone | witness with partners | sweep alone | |---|---|---|---|---| | `generation-conditional` (model 29) | push p1's first `generation.json` PUT is accepted and its response lost; the agent publishes again and p1 completes; push p2 completes; the stale PUT lands | **violated**: the marker names p1, the claim p2, and the serving box cannot start | | witness; sweep 0 / 3,000 | | `lost-return-park` (model 27) | the target writes in the settle window, the settle check fails, the target is stopped after its first return PUT and declared lost | **violated**: INV-10, the lane in the source region at the release | | 1 / 3,000 | | `declare-lost-compute` (model 5) | the target is frozen in its settle window and its execution lease expires; the VM resumes once the source has recovered | **violated**: INV-1, two boxes run the host | | witness; sweep 0 / 3,000 | | `watchdog-independent` (§7.1) | as above, with the agent alone paused, so the application keeps running | held | with `declare-lost-compute`: **violated**, INV-1 | 0 / 3,000 | | `lost-quarantine` (CC12) | the target is stopped while landing, declared lost, and started again with its old disk | **violated**: INV-10, local state for a host served elsewhere | | witness; sweep 0 / 3,000 | | `markers-last` (R2-6) | the source has a registered timer; the target's agent crashes between extracting the files and writing the marker | **violated**: INV-2, the timer is lost | | witness; sweep 0 / 3,000 | | `rollback-journal` (R2-3) | a dirty rollback; the source's agent crashes after its probe spawned the instance, which wrote and replicated into the return generation | **violated**: INV-2 on cold restore | | 1 / 3,000 | | `seal-by-version` (J-N2) | a pre-cut staging round completes after the seal; the source wrote and replicated between them | **violated**: INV-2 on cold restore; the landing installs the stale snapshot | | 2 / 3,000 | | `staging-quiet` (J-N2) | a staging round completes after the landing's staging park acknowledged, before the release | **violated**: INV-10 at the release | | 1 / 3,000 | | `dirty-return` (INV-2) | the target writes during its probe and replicates; the rollback is dirty and the return lane carries the write | **violated**: treated as clean with the source's replica covering the stop, the return is covered `clean-return`, nothing is declared and the serving database lacks the target's write (INV-2 local at the final recovery; rev 7.11 saw INV-15 at the declaration) | | 1 / 3,000 | | `commit-before-open` (INV-5; R3-5) | a public request reaches the target's edge while its probe runs | **violated**: INV-5 | | 3 / 3,000 | | `claim-cas-opid` (C2) | the source closes a rollback's record and its goroutine stalls before the release, holding the rollback's last read (rev 7.23: before it, a goroutine stalled before a claim-first release); the recovery pass releases from `last_move`, a purge claims the host, and the goroutine resumes into its release | held | with `revalidate-before-effect`: **violated**, the store is torn down under a serving host | 0 / 3,000 | | `exec-admission`, `hold-op-open`, `start-check-claim-gen`, `exec-writer` (J2-R2-8; claim rev 6.8.1) | a paused open from an earlier operation lands after the commit while the source's row is still held; a request wakes the source | held, each, and under every proper subset | all four: **violated**, INV-1 (the non-exec source is admitted cold into a fresh generation restored from its own region's old files and serves beside the target) | 0 / 3,000 | | `start-check` (INV-4) | the same, dropping the whole start check | retired in rev 7.10: with every restore from the box's own region, the stale source finds nothing of the target's generation to restore and cannot start; subsumed (`expected-outcomes.txt`) | | 0 | | `strict-spawn` (J-D3) | the source runs a main process and a descendant placed as its parent was; the move cuts | held | with `evidence-whole-tree`: **violated**, INV-1 | 13 / 3,000 | | `touched-incomplete` (J-N5) | a legacy host with a pre-release replica in another region is adopted, then deleted | held | with `legacy-discovery`: **violated**, INV-10 after `free` | 0 / 3,000 | | `registration-check` (CC21) | the first create of a host registers and its console pauses; a purge of the app drains the expired registration, tears the store down and finishes; the creator resumes | held | with `fence-etag`: **violated**; with `purge-claim-fence`: **violated** | 0 / 3,000 | | `fence-by-installation` (claim rev 6.4 §6.4, R3-3) | UK→US, then US→UK; **unmutated, the instance must serve on UK again, running** | **violated**: the landing probe on UK is refused by the box's own installation record (INV-3) | | 170 / 3,000 | | `fence-stamp` (claim rev 6.4 §6.4, R4-2) | a config-sync pass selects the row under the first installation and pauses; UK→US→UK; a new bundle is published; the pass takes the guard | held: the host guard's fresh re-validation refuses its stale bundle | with `hostguard`: **violated**, INV-6, a pass of a retired installation wrote host state | 261 / 3,000 | | `orphan-order` (claim rev 6 §5.1 step 5, R3-2) | the recovery pass's cleanup pauses after its first observation; a create removes the orphan, claims and publishes; the paused cleanup resumes | **violated**: the record of the incarnation the claim names is deleted | | witness; sweep 0 / 3,000 | | `sweep-namespace` (claim rev 6 §6.2a, R3-4) | a replication PUT of incarnation 1 completes after 33 recreates dropped it from `retired_incarnations` | **violated**: INV-10, residue the sweep never finds | | 984 / 3,000 | | `live-set-region` (claim rev 6.4 §6.2a, R4-3) | a legacy host serving in UK with an earlier replica in US; discovery parks US; an old US replication PUT lands after | **violated**: INV-10, residue the US sweep exempts by key alone | | 2 / 3,000 | | `restore-overlap` (§5.1, R4-1) | a generation's snapshot ends at the push's TXID; the only continuation is a compacted file starting one below it; the disk is lost and the host wakes cold | **violated**: INV-3 since rev 7.10, the restore stops short of the certificate's `max_txid` and the box never starts (rev 7.4's `min == current + 1` discarded the file and lost four writes) | | witness; sweep 0 / 3,000 | | `restore-gap-closed` (§5.1, R4-1) | the same generation with the next file missing and the one after current | **violated**: a restore across the gap returns a database short of the acknowledged writes | | witness; sweep 0 / 3,000 | | `restore-anchor` (§5.1 step 7, R5-1) | a push installs a snapshot alone; the host wakes, writes three times, replicates, loses its disk and wakes cold | **violated**: INV-2, every write since the push is lost: replication restarted at TXID 1 below the snapshot and the planner skipped the files (facts R5-1a, b) | | 57 / 3,000 | | `root-advance` (§5.1 step 8, R5-3) | a push, then twelve replicated writes: four compactions, four snapshot rotations, retention expiring every file that continued from the pinned snapshot; the disk is lost | **violated**: INV-3, the host refuses to wake on a gap the retention made, while the newest snapshot holds every write (facts R5-3d, e) | | 17 / 3,000 | | `anchor-acked` (§5.1 step 7, R6-1) | a push installs a snapshot alone; the install's anchor PUT is held across forty attempted opens, then released; the host writes, replicates, loses its disk and wakes cold | **violated**: the host opened before its anchor was acknowledged; litestream's position would be 0 and the writes below the snapshot are lost on the cold restore (INV-2) | | 12 / 3,000 | | `return-stamp` (claim rev 6.5, R6-4) | a dirty rollback; the source's local stamp record is lost after the target's return; the return CAS writes the source installation's stamp back and the source re-initializes from it, installs under it and serves under its probe CAS's new stamp | **violated**: the source activates the target's stamp, issued for another box's installation (INV-6) | | 70 / 3,000 | | `recovery-generation` (claim rev 6.6 §6.4a, R7-1), three scenarios and a crash | writer A's upload `[N+1]` is held in flight, A dies with its disk, B wakes; (i) B replicates its own `[N+1]`, then A's lands; (ii) A's lands after B's position is initialized and before B's upload, so litestream's loop skips B's file; (iii) A's held write updated K, B durably updates K, A's lands; (iv) a clean stop drains, then the agent crashes before the certificate | **violated** in each: with the rule B recovers into a fresh generation and A's file lands in the dead one; without it B's acknowledged write is lost on the cold restore taken the moment A's file lands (INV-2), or no recovery runs after the crash (iv) | | 13 / 3,000 | | `closed-marker` (claim rev 6.6 §6.4a; a certificate since rev 7.9) | a stale certificate (`max_txid` behind the remote maximum) beside the dead writer's outstanding upload | **violated**: taken as intact, B continues in place and A's late file replaces B's (INV-2) | | witness; sweep 0 / 3,000 | | `close-waits` (claim rev 6.6 §6.4a) | a clean stop with an upload in flight | **violated**: written at once, the marker vouches for an upload that has not returned | | witness; sweep 0 / 3,000 | | `newest-l0` (§5.1 step 8, R7-3) | the local position runs ahead of the uploads across two compactions and snapshot rotations | **violated**: retention below the previous snapshot removes the newest uploaded level-0 file | | witness; sweep 0 / 3,000 | | `build-completion` (§5.1 step 2, §7.6, R7-4) | a push's build with its snapshot, then its anchor, held, the agent crashing in between each | **violated**: `generation.json` published before its snapshot landed (INV-11) | | 711 / 3,000 | | `evidence-whole-tree` (§6.1, J-D3) | an adopted legacy host runs a descendant outside the cgroup; the move cuts | **violated**: evidence blind to the escapee certifies the stop and two boxes run the host (INV-1) | | witness; sweep 0 / 3,000 | | `timer-freeze` (J-N14) | a registered timer on the source; the move holds the row; the runner's deliveries are refused; a clean rollback | **violated**: the refused deliveries advanced the runner's bookkeeping, the timer digest differs from the stop's, the source is `stuck` | | 3 / 3,000 | | `recovery-current-inputs` (§7.7, §7.8; R2-5) | a bundle is published after admission; the target writes; the rollback is dirty | **violated**: `stuck` | | 37 / 3,000 | | `writer-session` (claim rev 6.8 §6.4a, R8-1) | two sessions certify; the third dies with its disk and an upload in flight; the dead certificates still match the remote maximum | **violated**: the newest certificate readmits the box into the generation and the late upload replaces its file (INV-2) | | 6 / 3,000 | | `admit-everywhere` (claim rev 6.8 §6.4a, R8-2), two scenarios | (i) the committed move's target dies with its disk and an upload in flight before finalization; (ii) the landing's probe dies with its disk and an upload in flight in its settle window | **violated** in each: admitted for ordinary wakes alone, the target continues the generation and the late upload replaces its file (INV-2); the restarted probe reuses the install generation and the landing's restore loses its write | | 716 / 3,000 | | `exec-writer` (claim rev 6.8.1) | the admission scenario above | held alone, and under every proper subset of the four | with the three box-side fences: **violated**, INV-6 | 0 / 3,000 | | `valid-ltx` (R8-3) | a create's first wake into an empty generation (no LTX file), then a move of the never-written incarnation, whose seal is at position 0 | **violated**: the install writes `[1, 0]` and `[0, 0]`, which the encoder refuses (INV-12) | | 39 / 3,000 | | `close-evidence` (claim rev 6.8 §6.4a, R8-5) | an upload is held past the client's timeout; the retry lands; the original stays outstanding; a clean stop | **violated**: the certificate is written while the original is unresolved | | witness; sweep 0 / 3,000 | | `close-evidence` (R9-5), two more scenarios | (i) a compaction's level-1 PUT is held past the client's timeout, the next pass retries and lands, the original completes afterwards; (ii) a snapshot PUT completes on the server after the client gave up on it | **violated** in each: the certificate is written while a request of the session was retried after a timeout, or its response was lost | | witness; sweep 0 / 3,000 | | `remote-baseline` (claim rev 6.9 §6.4a, R9-1) | a session closes cleanly through N+1; the box's local database is a cached copy through N; the next wake | **violated**: the stale copy is served without the acknowledged write, and its upload for N+1 is skipped at the remote position (INV-2) | | witness; sweep 0 / 3,000 | | `writes-lost` (claim rev 6.9 §6.4a, R9-2) | a writer acknowledges a write and crashes before uploading it; the next wake is cold | **violated**: lost silently, the serving database differs from the acknowledged writes (INV-2); with the rule the admission records writes_lost and alerts | | 16 / 3,000 | | `admit-bind-op` (claim rev 6.9 §6.4a, R9-4) | a wake's admission closure computed under serving resumes after a delete began that kept session, generation and executor | held: the model's general CAS condition on `op.id` (C2) refuses it too | with `claim-cas-opid`: **violated**, INV-13, a writer admitted into a host being deleted | 0; the pair 0 / 30,000, 0 / 100,000 | | `admit-before-start` (claim rev 6.10 §6.4a, R10-1) | a background worker acknowledges a write right after the app starts; the box crashes before replication opens; admitted before the start, the acknowledgement is the session's and the next cold admission declares it | **violated**: under rev 6.8.2's order the acknowledgement is in a database the next warm admission purges silently (INV-2 local) | | witness; sweep 0 / 3,000 | | `baseline-live` (claim rev 6.10 §6.4a, R10-2) | a cold admission into a fresh generation; the box crashes before the first upload; the predecessor's retirement runs out its quiet time; the recovery | **violated**: with the chore at the CAS the predecessor is parked and the recovery restores nothing (INV-2 local) | | witness; sweep 0 / 3,000 | | `clean-return-identity` (claim rev 6.12, R11-4) | a move's target is probed and writes nothing; the seal recorded `src_covered`; a clean rollback; the source's re-admission | held: the record's predecessor is the target's writer and its coverage `clean-return`; nothing is declared | **violated** under the mutation: INV-16, the record names the target's session and box with the source's generation, a writer that never was | 13 / 3,000 | | `absent-marker-origin` (R11-1; its own rule since rev 7.13) | an adopted host's origin replica with durable writes and no `generation.json`; a cold admission crashes before its first upload; the ordinary sweep runs for three horizons; the recovery | **violated**: the sweep, reading absence as agreement, parks the origin and the recovery restores nothing (INV-2 local) | | witness; sweep 0 / 3,000 | | `session-scoped-loss` (R12-1) | two writer sessions of one box each acknowledged an unuploaded write; a `writes_lost` record for the second is applied (a constructed-state oracle test) | held: only the second session's write is exempted | **violated** under the mutation: INV-18, the record for session 2 exempted session 1's undeclared write | witness; sweep 0 / 3,000 | | `foreign-region-uncovered` (claim rev 6.14 item 5, R12-3) | a probed landing whose target published; the chore's quiet time brought forward; the source region's runner works within the settle window; a clean rollback | held: no chore was queued into the source's region, the return restores the source's generation | **violated** under the mutation: the chore parked the rollback's baseline and the return restored an empty database (INV-2) | 4 / 3,000 | | `cover-at-park` (claim rev 6.14 item 6, R12-3), three scenarios | (i) a committed move: the commit's CAS queues the source region's park; (ii) the record flips to committed and every actor pauses past the horizon before anyone follows it; (iii) `followRecord` is called again after the commit | held: in each the target's record is covered in the CAS that queued the park, one park is queued, the replay is refused | **violated** under the mutation: INV-17 as the source region's park completes, the target's record uncovered | 30 / 3,000 | | `probe-window` (R12-2) | the landing probe's launch environment is ready exactly 30 steps after the attempt's first app stage | held: the attempt fails before the start, the next admission's probe is accepted | **violated** under the mutation: the expired attempt starts and is accepted (INV-19) | witness; sweep 0 / 3,000 | | `persisted-park-scope` (claim rev 6.23 §2, R13-4, R15-1), two scenarios | (i) a committed move's source-region park, scoped at the commit with the cut's stop evidence; the source region's agent restarts 20 steps before the deadline; (ii) the move's staging lane, scoped with its seal bound; the target region's cleanup first runs 40 steps past the deadline, under a restarted agent | held: (i) acknowledged within the persisted deadline and slack; (ii) acknowledged at once, never before the deadline | **violated** under the mutation: the chores are unscoped and the runner binds the cleanup's own clock, so the horizon starts again at the restart (i) and the staging lane waits a fresh `H` past its deadline (ii) (INV-9) | witness; sweep 0 / 3,000 | | `holderless-park` (claim rev 6.15 §6.2a, R13-4) | a deleted host's region keeps late residue; the free host's sweep and a create requested at once | held: the sweep runs as a holderless parking op, the create waits for `FreeAfterPark`, then serves | **violated** under the mutation: bare chores on the free claim, the create claims the host over the outstanding park (INV-20) | 562 / 3,000 | | `scope-membership` (claim rev 6.24 §2, R16-2) | incarnation 1 is deleted and incarnation 2 serves; a late object lands under `instances//1/g-old/` | held: the sweep queues a chore scoped to `instances//1` with archive incarnation 1 and parks within it | **violated** under the mutation: rev 7.16's whole-prefix parks, scoped `instances/`, archive the child incarnation's object from outside their scope (INV-23) | 1,252 / 3,000 | | `covered-baseline-refused` (claim rev 6.22 §6.2a, R13-1) | an unfinished run g2 → g1 whose record is covered and whose g1 was retired in the region | held: the start is refused and alerts, the host does not run | **violated** under the mutation: the walk passes over the empty g1 and restores the marker's retired g1 as an empty database (INV-2) | witness; sweep 0 / 3,000 | | `restore-selected-baseline` (R14-1) | the marker names S; R's first upload is in flight when R crashes; R2 is admitted over the empty R and crashes before uploading; R's upload lands | held: the next start selects R from R2 → R → S and restores it rooted at `[1, 1]` | **violated** under the mutation: the start restores R through the marker, which names S, refuses it, and the valid recovery stalls (INV-3) | witness; sweep 0 / 3,000 | | `unique-declarations` (R14-2), two scenarios | (i) the target is destroyed after the flip and before its first admission, `src_covered` false, the source's cut session uncertified; (ii) the same after the target's probe was admitted | held: the source's cut session is declared once, in `writes_lost` (i) or in the list (ii); nothing twice | **violated** under the mutation: the cut session is declared twice, through `writes_lost` and the separate path (INV-21) | 3 / 3,000 | | `seal-return` (rev 7.11 §7.8) | the source's last upload is in flight when the cut stops it; the landing is rolled back after its build; the return must carry the seal | **violated**: treated as clean, the source's cold admission declares a write the landing's built generation holds (INV-15) | | witness; sweep 0 / 3,000 | | `close-evidence` (R10-7), two more scenarios | (i) a fresh generation's first upload is held past the client's timeout and its retry lands; (ii) a warm admission's anchor is held and its retry lands; then a clean stop | **violated** in each: the certificate is written while a request of the session was retried after a timeout | | witness; sweep 0 / 3,000 | **The purge fence, the ETag condition and C5a.** `purge-claim-fence` (CC21) is caught alone by the sweep: 28 / 3,000, 240 / 30,000. `fence-etag` (CC22): 28 / 3,000, 214 / 30,000. `claim-never-deleted` (C5a): 3 / 3,000, 44 / 30,000. `registration-check` is not caught alone by the sweep at 3,000 seeds, and its witness holds alone; the re-read alone is shown insufficient (`purge-claim-fence` + `fence-etag`, 343 / 30,000); its two pairs 243 and 263 / 30,000. **Rev 6's unexercised three, now caught (the biased schedule, §14.2.2).** | rule | result | what the model finds | |---|---|---| | `location-per-incarnation` (claim C3a) | **caught alone**: 31 / 3,000, 385 / 30,000 | with the shared location, the planner takes the previous incarnation's files (highest `MaxTXID` wins), and the cold restore equals neither incarnation's acknowledged writes; the C3a and door oracles see the rest | | `door-reread` (claim §6.5 O2) | **rev 7.4 to 7.7, historical**: with `location-per-incarnation` 432 / 30,000, of which 196 were door violations, and its witness violated. **Since rev 7.8**: witness retired; the pair's sweep count is the partner's own (below) | a listing authorized for the old incarnation selected the new incarnation's object, when a recreated incarnation reused the generation | | `door-by-version` (claim §6.5) | **rev 7.4 to 7.7, historical**: with `location-per-incarnation` 385 / 30,000, of which 60 were door violations, and its witness violated with C3a alone. **Since rev 7.8**: witness retired; the pair's sweep count is the partner's own (below) | a handle signed before the delete opened the key's current object after the recreate, when a recreated incarnation reused the generation | The three-rule combination `door-by-version` + `door-reread` + `location-per-incarnation` that rev 7.4 also reported is **not a minimal combination** (R4 §3): each door rule's two-rule pair with C3a already fails, so the three-rule row is a wider set and inherits no claim that its proper subsets hold. It is no longer run. **Caught alone: 93 of 107** (71 by the sweep at 3,000; 22 by their witness alone, marked ‡; the sweeps at 30,000 and 100,000 were not run for rev 7.17 (evidence policy; Paul 2026-10-06), so the one rare rule the sweep reaches only there, `hostguard`, has no verdict this round). | rule | seeds failing | | rule | seeds failing | |---|---|---|---|---| | `abort-target-owner` | 961 / 3,000 | | `absent-marker-origin` ‡ | witness; sweep 0 / 3,000 | | `admission-registration` | 48 / 3,000 | | `admit-before-start` ‡ | witness; sweep 0 / 3,000 | | `admit-everywhere` | 716 / 3,000 | | `anchor-acked` | 12 / 3,000 | | `baseline-live` ‡ | witness; sweep 0 / 3,000 | | `build-completion` | 711 / 3,000 | | `cancelled-creates` | 376 / 3,000 | | `claim-never-deleted` | 5 / 3,000 | | `claim-op-exclusive` | 120 / 3,000 | | `clean-return-identity` | 13 / 3,000 | | `close-evidence` ‡ | witness; sweep 0 / 3,000 | | `close-waits` ‡ | witness; sweep 0 / 3,000 | | `closed-marker` ‡ | witness; sweep 0 / 3,000 | | `commit-before-open` | 3 / 3,000 | | `cover-at-park` | 30 / 3,000 | | `covered-baseline-refused` ‡ | witness; sweep 0 / 3,000 | | `cut-deadline` | 41 / 3,000 | | `declare-lost-compute` ‡ | witness; sweep 0 / 3,000 | | `dirty-return` | 1 / 3,000 | | `evidence-from-state` | 2,632 / 3,000 | | `evidence-whole-tree` ‡ | witness; sweep 0 / 3,000 | | `fence-by-installation` | 170 / 3,000 | | `fence-etag` | 22 / 3,000 | | `fence-stamp` | 261 / 3,000 | | `files-carried` | 26 / 3,000 | | `floor-boot-recovery` | 2,563 / 3,000 | | `foreign-region-uncovered` | 4 / 3,000 | | `fresh-admission` | 11 / 3,000 | | `fresh-incarnation` | 53 / 3,000 | | `generation-conditional` ‡ | witness; sweep 0 / 3,000 | | `holderless-park` | 562 / 3,000 | | `legacy-discovery` | 367 / 3,000 | | `live-set-region` | 2 / 3,000 | | `location-per-incarnation` | 29 / 3,000 | | `lost-quarantine` ‡ | witness; sweep 0 / 3,000 | | `lost-return-park` | 1 / 3,000 | | `markers-last` ‡ | witness; sweep 0 / 3,000 | | `newest-l0` ‡ | witness; sweep 0 / 3,000 | | `obligations-block-ops` | 21 / 3,000 | | `orphan-no-deadline` | 44 / 3,000 | | `orphan-order` ‡ | witness; sweep 0 / 3,000 | | `park-all-versions` | 586 / 3,000 | | `park-by-version` | 2,539 / 3,000 | | `park-skip-lock` | 2,733 / 3,000 | | `participant-rows` | 3 / 3,000 | | `persisted-park-scope` ‡ | witness; sweep 0 / 3,000 | | `probe-window` ‡ | witness; sweep 0 / 3,000 | | `purge-adopts` | 7 / 3,000 | | `purge-claim-fence` | 28 / 3,000 | | `purge-journal` | 1,897 / 3,000 | | `purge-reserve-first` | 2,142 / 3,000 | | `quiet-before-ack` | 152 / 3,000 | | `recovery-current-inputs` | 37 / 3,000 | | `recovery-generation` | 13 / 3,000 | | `recovery-pass` | 2,432 / 3,000 | | `registration-recovery` | 2,011 / 3,000 | | `remote-baseline` ‡ | witness; sweep 0 / 3,000 | | `restore-anchor` | 57 / 3,000 | | `restore-by-generation` | 126 / 3,000 | | `restore-gap-closed` ‡ | witness; sweep 0 / 3,000 | | `restore-overlap` ‡ | witness; sweep 0 / 3,000 | | `restore-selected-baseline` ‡ | witness; sweep 0 / 3,000 | | `retirement-ids` | 461 / 3,000 | | `return-stamp` | 70 / 3,000 | | `revalidate-before-effect` | 14 / 3,000 | | `revert-journal` | 13 / 3,000 | | `rollback-journal` | 1 / 3,000 | | `root-advance` | 17 / 3,000 | | `scope-membership` | 1,252 / 3,000 | | `sdk-no-resign` | 12 / 3,000 | | `seal-by-version` | 2 / 3,000 | | `seal-return` ‡ | witness; sweep 0 / 3,000 | | `session-scoped-loss` ‡ | witness; sweep 0 / 3,000 | | `sign-at-check` | 16 / 3,000 | | `sign-before-safeuntil` | 19 / 3,000 | | `source-readiness` | 12 / 3,000 | | `staging-quiet` | 1 / 3,000 | | `stale-hold-release` | 1 / 3,000 | | `start-check` | 17 / 3,000 | | `start-check-claim-gen` | 15 / 3,000 | | `straggler-sweep` | 1,192 / 3,000 | | `strict-spawn` | 13 / 3,000 | | `sweep-namespace` | 984 / 3,000 | | `sweep-staging` | 38 / 3,000 | | `target-park` | 16 / 3,000 | | `timer-canonical` | 2 / 3,000 | | `timer-freeze` | 3 / 3,000 | | `unique-declarations` | 3 / 3,000 | | `valid-ltx` | 39 / 3,000 | | `writer-session` | 6 / 3,000 | | `writes-lost` | 16 / 3,000 | | | | **Caught only in a combination: 9 rules**, every live combination rerun this round; `admit-bind-op` joins with `claim-cas-opid` (R9-4); the `start-check` four is historical; `door-reread` and `door-by-version` are subsumed (above) and their rows below are **historical**, kept only to record what the pairs once showed. `fence-stamp`'s pair with `hostguard` is still reported because the witness needs it. Each partner's own count, on the same seeds, shows what the combination adds; the witness column is the rule's witness under the combination, having held under every proper subset in the gate. | combination | sweep | partners alone (sweep) | witness | what the model finds | |---|---|---|---|---| | `purge-claim-fence` + `fence-etag` (**the re-read alone**, rev 7's CC21 as proposed) | not run for rev 7.17 (evidence policy; Paul 2026-10-06) | `purge-claim-fence` 28 / 3,000, `fence-etag` 22 / 3,000 | (the two `registration-check` rows below) | a store destroyed under a creating or serving host | | `registration-check` + `purge-claim-fence` | not run for rev 7.17 (evidence policy; Paul 2026-10-06) | `registration-check` 0 / 3,000, `purge-claim-fence` 28 / 3,000 | violated | the same | | `registration-check` + `fence-etag` | not run for rev 7.17 (evidence policy; Paul 2026-10-06) | `registration-check` 0 / 3,000, `fence-etag` 22 / 3,000 | violated | the same | | `exec-admission` + `hold-op-open` + `start-check-claim-gen` + `exec-writer` | not run for rev 7.17 (evidence policy; Paul 2026-10-06) | `exec-admission` 0 / 3,000, `hold-op-open` 0 / 3,000, `start-check-claim-gen` 15 / 3,000, `exec-writer` 0 / 3,000 | violated, INV-1 | the non-exec source admitted cold beside the target, Astra's J2-R2-8 trace under writer sessions | | `exec-admission` + `hold-op-open` + `start-check` + `exec-writer` (**historical**, rev 7.9: 7 / 30,000, violated INV-6) | not rerun: the `start-check` witness is retired, subsumed by regional restores | | retired | the stale source cannot restore the target's generation from its own region | | `admit-bind-op` + `claim-cas-opid` | not run for rev 7.17 (evidence policy; Paul 2026-10-06) | `admit-bind-op` 0 / 3,000, `claim-cas-opid` 0 / 3,000 | violated, INV-13 | a stale serving-time closure admits a writer into a host being deleted | | `door-reread` + `location-per-incarnation` (**historical**, rev 7.7: 432 / 30,000, 196 door violations, witness violated) | not rerun: the witness is retired and the rule is subsumed | `location-per-incarnation` 28 / 3,000, 360 / 30,000, 1,179 / 100,000 this round (rev 7.10's row said 336 / 30,000 against its own result of 357, R10-9) | retired | under writer epochs the pair adds nothing the partner's own count does not show; no current witness evidence is claimed | | `door-by-version` + `location-per-incarnation` (**historical**, rev 7.7: 385 / 30,000, 60 door violations, witness violated with C3a alone) | not rerun: the witness is retired and the rule is subsumed | `location-per-incarnation` 28 / 3,000, 360 / 30,000, 1,179 / 100,000 this round | retired | the same | | `claim-cas-opid` + `revalidate-before-effect` | not run for rev 7.17 (evidence policy; Paul 2026-10-06) | `claim-cas-opid` 0 / 3,000, `revalidate-before-effect` 14 / 3,000 | violated | a stale step destroys a store | | `strict-spawn` + `evidence-whole-tree` | not run for rev 7.17 (evidence policy; Paul 2026-10-06) | `strict-spawn` 13 / 3,000, `evidence-whole-tree` witness; sweep 0 / 3,000 | violated | two writers | | `touched-incomplete` + `legacy-discovery` | not run for rev 7.17 (evidence policy; Paul 2026-10-06) | `touched-incomplete` 0 / 3,000, `legacy-discovery` 367 / 3,000 | violated | a freed host's residue | | `declare-lost-compute` + `watchdog-independent` | not run for rev 7.17 (evidence policy; Paul 2026-10-06) | `declare-lost-compute` witness; sweep 0 / 3,000, `watchdog-independent` 0 / 3,000 | violated | two writers after a paused or frozen target resumes | | `fence-stamp` + `hostguard` (no longer combination-only) | not run for rev 7.17 (evidence policy; Paul 2026-10-06) | `fence-stamp` 261 / 3,000, `hostguard` 0 / 3,000 | violated | a stale pass writes host state for a retired installation (INV-6); the witness is what shows the stamp load-bearing against the guard's own check | Every live combination was rerun on the rev 7.9 model; the gate runs every proper subset of the admission four with the witness, which holds under each. The door rows are historical (R8-6): rev 7.8 reported the partner's count in the pair's column and still said "witness violated"; the pairs have not been rerun since the witnesses were retired, and the rows claim no current evidence beyond the partner's own count. **Liveness only: 2.** `readiness-key` (T2) and `config-before-probe`. They have no witness: neither can violate an invariant. **Unreachable: `readiness-quiet` (T12).** Every retirement acknowledges only after its own horizon, and CC11 holds. No witness: there is no interleaving to force. **Seeds:** 300 in the gate, with every witness under every subset (about a second; 52 witnesses, 176 runs); the 30,000- and 100,000-seed baselines, this sweep and the witnesses are the evidence for rev 7.10. #### 14.2.6 What the model does not prove - **One box per region (region HA L18; rev 7.19).** The model keeps one box per region plus replacement boxes (a destroyed box's successor). Region HA asks for two boxes per region with create spread, relocation by drain, termination mid-serve and in every relocation phase, nested failover, a stale edges file, an unenrolled reference, a volume without `DeleteOnTermination`, a crash mid-install, late file uploads and late PUTs after termination. That is a model of relocation, an operation this spec does not define, over a world whose `box.Region` and per-region bookkeeping assume one box; it is not a change this revision can make within the CPU policy (the gate plus one niced 3000 sweep per revision) or its scope. **Stated as a limit (rev 7.20, the coordinator's scope ruling):** the model stays at one box per region; Phase 1 refuses multi-box regions at the door (§7.1), so the limit matches the scope. Nothing in §14.2.3's results speaks for two boxes in one region or for `choose()`'s spread. Rev 7.20 adds the one fence case Phase 1 meets: rule `fence-converts-moving` and witness `wFenceConvertsMoving`, a source terminated before the flip in a one-box region, converted to `relocating` at `pinning`, holding without churn and with one alert until a replacement box enrolls (the model's replacement mechanism), then completing on it. The model collapses the relocation's install into the cold admission's restore of the pinned generation from the region's remote, and stands in for region HA's files publisher by taking the fenced source's files directory as published (L17) and installing it at the installation CAS, before any admission, which is the prepared manifest's moment (§7.6, R18-1); it encodes no publisher timing beyond that and does not model `choose()`, two replacements, or `stuck` after them. The two-box extension belongs to a joint round with the region HA session, on its own branch, with its own sweeps. - **It is a model.** It checks the protocol, not the code that will implement it. Its S3 is a versioned map with the documented semantics, every one checked against the real service (§15 step 8). Its litestream is the behaviour `internal/sim/litestreamfacts/` observed in the pinned library (§14.2.1), not the library itself. - **Granularity.** One data-plane host, three regions, one box per region, and a second hostname for creates and purges only. One step is atomic, except where rev 6 splits it (§14.2.2). - **J2 fidelity points not taken in rev 6:** reads, `Head` and version listings are not checked against each principal's permissions (writes are); listings are complete, never paginated or truncated; one global clock with a fixed skew, not separate service, wall and monotonic clocks; an old executable is one Boolean, not distinct console, agent and boot-updater versions with enrollment; a spawn and its readiness answer are one event; two workers of one operation share a lease identity (only one box writes staging, and a restarted agent's goroutines die with it). Each could hide a fault; each is a next step, in that order of likely yield. - **Occupancy is by current objects (R14).** The model's run walk and oracles judge a generation as holding an object by its **current** versions in a region's bucket; a key whose every version was deleted (a parked generation) counts as empty for selection and as "had objects" only for INV-17 and the sweep's retired set. The selector F also counts tombstoned or history-bearing generations as nonempty and fails closed on them. The two agree on every case in F's table; a generation with delete markers and no current version is where they would differ, and the model's covered-local refusal (§5.1 step 9) reaches such a generation before its emptiness is consulted in every schedule the sweeps produced. Exact equivalence is not claimed. - **The sweep protects more than the run walk selects (R16).** The sweep's `deadGeneration` protects every generation of an unfinished run in the authoritative region, without the run walk's region or coverage filtering (§5.1 step 9), which main applies when it selects a baseline. The sweep's set is the wider, conservative one: it never parks what the walk could select, and may keep a covered or foreign link's objects until a publication's chore retires them. - **Admission trimming is E's rule, mirrored (R14).** Covered entries are dropped from the front only while more than eight entries remain, and an uncovered head stops the trimming; the model counted covered entries rather than entries until rev 7.15. The publication's precondition is that any entry names the published generation, as A and E read it. - **Timing is scaled, not real.** `H` (60 steps, the signature-age allowance 45 plus the request bound 15) is shorter than the cut budget (320), the reverse of production (over 15 minutes against 120 s). Results about orderings carry over; results about which wait fits inside which window do not (J2 §2). - **Restore fidelity (R3-1, R4-1, R5-1 to R5-3).** The model's LTX files have levels, ranges in replication numbering and final page images, with compaction, snapshot rotation and retention as the pinned library does them, and its restore is litestream's planner with the root rule, the anchor and the position read of §5.1, so a compacted continuation, a gap, a snapshot-only install and a rotated-away root are all representable and each has a witness. What the model's pages are not is bytes: a key's image is one integer, there is no checksum of any kind (the pinned files carry a per-file CRC-64 and no database checksum, fact R5-2), and page size, WAL salts and the lock page are absent. The claim it supports is that the planner as specified restores exactly the acknowledged transactions up to the TXID it reaches and refuses otherwise. What the real library does with real files is established by `internal/sim/litestreamfacts/` (§14.2.1): rev 7.5 pointed here at §15 step 8, which verified S3 behaviours and never ran litestream (R5 review). Not modelled: bytes and so the cost of a transfer (the budget test prices it, §7.3); the staged replica as several files with one sealed version each (the model's seal names one object holding the stopped source's state, which is what the target's build yields from the staged files); litestream's time-based level-0 retention (the model expires level 0 with level 1, by TXID, keeping the newest); `integrity_check`; a writer bound to its generation is modelled as keys under `db.Gen` and never as a client that refuses other keys; conditional PUTs are not used, so the review's list of what a create-only PUT would still need (differing bytes, the shortcut's identity, multipart, delete markers) stays a list. - **What the file client cannot show (R6 and R7 reviews).** `litestreamfacts` drives the pinned library through its file replica client. S3 lists LTX files in filename order and ages them for retention by `LastModified`, the file client by the encoded LTX timestamp kept as the file's mtime, so a copied or late-landing object can age differently for retention on S3; an upload outstanding across a writer replacement, the R7-1 hazard, is the model's, not the file client's; S3 keeps versions and delete markers where the file client replaces and deletes outright; S3 requests complete late. Those are the model's (versions, markers, late completion, §14.2.2) and §15 step 8's (the S3 behaviours). The repository has no S3 fake litestream's client can talk to (the lease tests' `fakeMirror` is a `MirrorStore` seam), so no S3-fake test was added; the production experiment of §15 step 7's rehearsals is where the two meet. - **Observation boundaries not taken in rev 7.4 or 7.5** (R3 review): the target's cleanup is queued at its release, not after the source's `recovered_at` (§7.8); a create refuses while a record PUT is pending on the server, which a real console cannot see; the release check runs the cold-restore oracle only, not the local and timer acknowledgement check, so a later delete could erase evidence of an earlier loss; two workers of one operation share a lease identity; the passes carry an installation stamp now but no release version. - **Adversarial scheduling is partial.** Paused opens, door reads, staging signers and an earlier incarnation's replica writes wake at the worst moment the model knows of; everything else wakes at random. The admission triple, C3a and the door rules depend on those targeted wake-ups and on the biased recreate. - **Not modelled:** placement (`placing`), sidecars, nested-to-flat path translation beyond the files directory, the db door's multi-console drain, app moves (F5), and `open_budget` (the model's deadline is the commit's). ### 14.3 Cross-version The door refuses below the floor; a pre-floor binary cannot be installed on a participant; v0.22 consoles refuse abort on every new state (P0's allowlist, `provision.go:1027-1035`). ### 14.4 Rehearsal (M12) `moveprobe`, a test app in this repo, shaped like ips: `instance.key` written at first boot and verified every boot; a required store binding and sealed secret; a declared schedule and a once-timer; a background loop writing rows and **a forked worker that outlives the main process**; a switch to fail its readiness path in a named region, before or after writing. Runs: dev IE to uk, done; dev forced clean rollback; dev forced dirty rollback; dev declare-lost; then the same on the flagship with scoped approval; then a disposable ips instance in `bdn`; then kpmg with Paul's approval. A run passes only if the key verifies, the object reads back, the secret is present, every timer ran exactly once around the cut, no background row is missing, and no copy remains in a live namespace of the region that lost. ## 15. Rollout and gates 1. **Ops (done, 2026-10-03):** `CARLOS_SECRETS_KEY` on every flagship box; config sync armed in every region. 2. **Release with `secrets_key_fp`.** Verify every box against the console. 3. **The host lifecycle claim** (claim spec §10) ships and every box and console honours it: claim rev 5.2 (CC22 adopted, §19.4). 4. **Phase 1 release, dormant.** Consoles first, then every box. 5. **Infra (M17, approved):** every region's `sibling_staging_arns` lists every other region's data bucket. Applied per deployment as scoped work. **Plus** the versioned-object grants claim §6.2 and §6.5 need (`s3:ListBucketVersions`, `s3:GetObjectVersion`, `s3:DeleteObjectVersion` on each box's own bucket; `s3:GetObjectVersion` for the console). 6. **Write `control/moves-protocol.json`** (§11) **before any protected production rehearsal** (joint N12). Rev 3 had this after the rehearsals. 7. **Rehearsals** in §14.4's order; each gates the next. 8. **Done (2026-10-02, before planning):** the AWS behaviours the model assumes were checked against the real service, on a versioned general-purpose bucket (§14.2.6). Every assumption holds: - **copy by version:** `CopyObject` from `versionId=V1` returns V1's body after V2 exists; - **delete by version:** `DeleteObject --version-id V1` removes only V1; - **delete markers:** a bare delete leaves a marker, and V2 is still copyable by its `versionId` (why bare deletes are forbidden on host prefixes, and why a copy by version ignores a stale marker); - **create-only writes:** `If-None-Match: *` gets 412 on an existing key and succeeds on a new one (evidence objects, manifests, `move.json`); - **conditional writes:** `If-Match` with a wrong ETag gets 412 on both PUT and DELETE, so conditional delete works on general-purpose versioned buckets (the claim CAS, lock release); - **the signature-age limit behind `H`:** a SigV4 request signed 14 minutes earlier returns 200; one signed 16 minutes earlier returns 403 `RequestTimeTooSkewed`; - **copy receipts:** a multipart ETag becomes a single-part ETag on copy; `CopyObjectResult.ETag` equals the ETag the listing shows; a `CopySourceIfMatch` mismatch gets 412. ## 16. Follow-on specs No design here, only what each must satisfy. | spec | scope | must meet | |---|---|---| | **F1 Owner surface** | owner self-serve; person auto-cut (M6); holding route for custom domains | owners only (M13); move-capable service authority explicitly owner-delegated, including the legacy mint path (`apiinstances.go:1554`), with a stated treatment of existing member-minted tokens (codex 11); Tito's router credentials (M18) | | **F2 Moves into home** | Home as a target is **delivered** by `docs/superpowers/plans/2026-10-15-moves-home-region-part2.md` (rev 7.24, #236). `CARLOS_HOME_PLACEMENT` is superseded by Plan 13's signed fleet home designation, which names home once for every reader; no environment variable is added. `carlos move back` stays a follow-on (Ruling R3) | §5's handoff unchanged | | **F3 Residency cleanup** | delete `superseded//`, `move-snapshots/`, `.parked/superseded-*`, receipts after the hold (M7) | no deletion of a live location or its ancestor, selection excluded atomically through the claim (codex 8); the complete residue inventory including noncurrent versions and delete markers (`infra/modules/deployment-buckets/main.tf:27-30`) with scoped version-delete authority and verified removal (codex 9); a warrant binding the exact scope from an immutable console ledger (codex 10) | | **F4 Stores** | private and app stores (M3, M9) | a joint instance and store transaction with binding versions and compensation (codex 12); store record transitions with config (`internal/objstore/walk.go:427`, `:448`), lifecycle operations serialized against the move (codex 17) | | **F5 App moves** | sequencer, pause (M8), sidecar refusal (M10) | durable child move ids before dispatch; an app and store cut barrier covering every consumer and concurrent create (codex 18) | | **F6 Graceful cuts** | freeze, thaw, verify hooks; sidecar moves | hooks never replace stop evidence | | **F7 `carlos db push`** | push on §5's handoff, source and target the same box | the four core invariants; the claim's `pushing` op | ## 17. Interface points with #199 and the claim spec The claim spec supersedes #199 rev 3 from its §2 onward. Against claim rev 3 (each point held in rev 2 too): | id | point | status | |---|---|---| | I-1 | **Incarnation**, including the zero-CreatedAt legacy identity | agreed: one `provision.Incarnation` (claim §2); fresh `CreatedAt` on create and `retired_incarnations` (claim §5.1, R3-12). The model reuses an identity when the rule is dropped and finds hosts that can never be admitted again | | I-2 | **Where an incarnation's data lives** | agreed: `touched`, appended before the first write (claim §2); every move write site adds first (K12) | | I-3 | **Parking `moves//`** | agreed: parks keep `instances/` and `moves/` as separate subtrees (claim §6.2) and a delete cannot run during a move | | I-4 | **Park destinations** | agreed: one destination and manifest per retirement event (claim §6.2, joint D5); `superseded//…` for move retirements | | I-5 | **Recreate** | agreed: create requires the claim free or absent (claim C2) | | I-6 | **Participant rows** | agreed: retained while an operation lists them (claim §2 obligations, K9) | | I-7 | **Flat files** | agreed: claim §6.3 retires the flat `//`, the sibling env files and markers (joint I-7) | | I-8 | **`generation.json`, `install-.json`, `lock.json`** | agreed: the parker's own renewed lock is excluded everywhere (CC3, claim rev 3 §6.2) | | I-9 | **Destructive steps** | agreed: exact versions plus the staging-work lease (claim C3), with the horizon for the replication exception (CC1, claim rev 3 §6.2a). Open: CC14, the staging writer's `SafeUntil` check | | I-10 | **Push** | agreed: later, on §5's handoff and the claim's `pushing` op, with every probe rendering held config (CC13, claim rev 3 §6.1). The model found the push's open must also accept a row already open (model 16) | ## 18. Questions for Paul None. Rev 2's question became ruling M16, and round 2's verdicts on the former questions match M4 to M15. Where this spec and the claim spec differ (§19.3), the difference is for the two specs' authors and reviewer to settle, not a product choice. ## 19. How moves use the host lifecycle claim ### 19.1 Op kinds and admission, per phase (claim spec §4 assigns these here) One op id (the move id) runs through every kind below. The kind changes by CAS; the record's state is the move's own detailed journal. `op.participants` is `{source box, target box}` from the start CAS, the target chosen by `choose()` (§7.1, L15). **Every admission in every row below is cold while P7 stands** (§0.5): "warm or cold as for `serving`" reads "cold" until #234 lands. | claim `op.kind` | record states | source may spawn | target may spawn | |---|---|---|---| | `moving` | `preparing`, `ready`, `cutting` | while `cut_phase` is before `stopping` (serving as usual, strict spawns) | never | | `reverting` | `reverting`, then cleared (`rolled_back`, by `MoveRevertClose`) | the recovery probe and then serving, once it holds the execution right with its own evidence | never | | `landing` | `landing`, `committed` | never | the probe from `landing_phase = probing`; serving from claim phase `committed` | | `restoring` | `rolling_back`, `stuck` | the recovery probe and then serving, from `source_phase = claimed` | never; it only releases and cleans | | back to `serving` | (move cleared) | if it is the execution holder named at release | likewise | The db door refuses on every move kind (claim spec §8). ### 19.2 Rules K1 to K13 | id | rule | |---|---| | **K1 Execution holder** | The move keeps `op.exec` = a box label or `none`, and `op.participants` = {source box, target box}. Admission (claim spec C4) for a move kind is the table in §19.1 **and** `exec` naming this box. | | **K2 Acquire** | The request door CASes `serving` → `moving` only when the holder incarnation equals the record's and no other op holds the claim; it sets op id, participants and `exec` = source box. | | **K3 Execution to none** | A participant that has written stop evidence for this op id (§5.3) may CAS `exec` from itself to `none`. Nobody moves `exec` from one box to another directly. | | **K4 Execution handoff** | From `none` to a participant, only in the claim CAS that mirrors a record transition (§7.5) or a recovery CAS that cites evidence, never directly from one box to another: at the flip (to the target, citing the source's `stopped-source.json`), at source recovery (to the source, citing the target's `stopped-target.json`), and at `reverting` (the source, citing its own evidence). Who writes the CAS does not matter, because the record and the cited evidence fully determine it (J2-D7). | | **K5 Revalidation** | Before each destructive step the actor re-reads the claim and requires op id, kind, phase and `exec` to be what the step expects (claim spec C3); it holds the lease of every location it writes, acting before `SafeUntil`, with the exception in §19.3 D4. A staging writer checks `SafeUntil` immediately before it signs each request, after any pause (§7.3, model 14). | | **K6 Wake admission** | `doWake` reads the claim fresh (claim spec §8) and applies K1. A box that reads a claim whose `exec` does not name it kills that host's process tree (§5.3 step 1) before anything else. | | **K7 Other operations** | Delete, purge, park and push start only from `serving` (claim spec §4); while a move op holds the claim they are refused. Create requires free or absent. | | **K8 Release** | Only the move releases to `serving`, naming `exec` (target after `done`; source after `rolled_back` or `aborted`), and only when every obligation this op created is gone (§7.8, §7.9). The release follows the record's close: the closing actor checks that nothing is owed under the op, CAS-clears `move` into `last_move`, then releases; the console's recovery pass finishes the release from `last_move` after a crash (§7.5; rev 7.23, Paul's ruling M4). | | **K9 Other passes while a move op holds the claim** | Reconcile mints participant rows held and never `ready`, and never removes or orphan-retires a participant row; the schedule converger neither upserts nor prunes the host's timers; `SyncExec` and release adoption skip the held rows (they already skip non-`ready`). | | **K10 Version floor** | The claim spec's floor (§9 there) and `moves_protocol` (§11) are one object and one check. | | **K11 Declare-lost** | §7.10; §19.3 D6. | | **K12 Touched** | Every move write site CAS-adds its region or box to `touched` before its first write: the source adds the target region before the first staging round; the target adds its box at `checking`; a dirty return adds nothing new (the source region and box are already there). | | **K13 Evidence durability** | Evidence objects are create-only per (op id, role) in the deployment bucket, recomputable from system state after a restart (§5.3), and referenced from the claim journal, never inlined. | ### 19.3 Rev 3's requests, against claim rev 2 and rev 3 Each status below holds in claim rev 3 as well, except D6, which rev 3 completes. | id | status in claim rev 2 | here | |---|---|---| | D1 admission per box and phase | adopted (C4, joint text) | K1, §19.1 | | D2 no per-handoff counter | adopted (`gen` is a revision) | §5.1 | | D3 strict spawn, whole-tree evidence | adopted (§6.1, joint text) | §5.3 | | D4 cross-region staging exclusion | replaced, as the review ruled: a shared staging-work lease (C3); sealing and draining are this spec's | §7.3 | | D5 per-event retirement destinations | adopted (§6.2) | §6.4 | | D6 declare-lost | adopted with an execution-lease branch; this spec dropped that branch (model 5), and **claim rev 3 drops it too** (CC5): compute fencing only, the lease for liveness | §7.10, CC5 | | D7 move-kind transitions | adopted (§4, joint text) | §19.1 | | D8 who waits for obligations | adopted (§2) | K8 | | I-7 flat local inventory | adopted (§6.3) | §6.1 | ### 19.4 Claim changes the model requires (for platform-1e) This spec does not edit the claim spec. Each change below is what the model needed for its invariants to hold; the mutation column says what the model finds when the rule is dropped (§14.2.5). **CC1 to CC13 (rev 4, against claim rev 2): all adopted in claim rev 3** (8b5bccb2), which maps them in its §12: CC1 to C3, §2 `quiet_after` and §6.2a; CC2 and CC3 to §6.2; CC4 and CC8 to §9; CC5, CC9 and CC12 to §7; CC6 and CC7 to §5.3; CC10 to §8; CC11 to §4; CC13 to §6.1. Rev 5's model encodes rev 3's text; §14.2.5 gives each rule's mutation result, CC11's included now (model 15). | id | change to claim rev 2 | why (model) | |---|---|---| | **CC1** | **A late-completion horizon on every retirement.** C3's "late completion is harmless" holds only for version-bound calls. Ordinary replication, the C3 exception, is plain PUTs to reused LTX names. A park acknowledges only after a verification listing taken later than `evidence time + H` (H = S3's 15-minute signature-age limit plus the longest request), and parks anything found then as a further retirement event. Persist a per-region `quiet_after` in the claim (kept across `free`, like `retired_incarnations`) and make any install into that region's namespace wait for it | model 1: residue in a freed host's namespace | | **CC2** | **Park progress accumulates.** §6.2 step 7 restarts at step 2 when no manifest exists. Key each copy's destination by source key and `versionId`, skip copies that already landed, and write the manifest once every copy exists | model 2: parks that never finish | | **CC3** | **The parker's own lock (J-N6) is renewed while it works**, so "excluded from the snapshot" must also mean excluded from the emptiness check and the straggler re-listing | the park never finishes when the lock is counted | | **CC4** | **Discovery at adoption.** Adopting a legacy host (§9) adds a `park-replica` obligation for every region other than the record's (that data cannot be the live lineage), and `touched.complete` becomes true once they acknowledge. §5.2 step 3's "every region at delete" then remains as a fallback | model 10: legacy residue for as long as the host serves | | **CC5** | **Declare-lost requires compute fencing.** Drop §7.2's lease-expiry branch: a whole-VM pause stops the self-fencing watchdog too, and on resume the app can write before it fires. Keep the lease for liveness. Cover two more states: a target that gave evidence and released dirty but never returned its writes; and a destroyed box that owes only `park-local` (legal after commit) | model 5, model 6 | | **CC6** | **Purge adopts first.** Before deciding which hosts the app holds, run the one adoption routine for every unclaimed record of the app | model 7 | | **CC7** | **The purge reservation is the purge's journal.** It records the purge's phase; the recovery pass drives those phases in order; only the purge's completion removes it, never `free` | model 8 | | **CC8** | **One adoption routine for every adopting actor**, discovery included | model 11 | | **CC9** | **A fresh read with no operation ends a self-fence** (§7.1): a box clears its local execution-lease file when the claim shows it is not a participant | model 9 | | **CC10** | **Reconcile releases a stale hold**: a held row whose `hold_op` is not the claim's operation, on the box the claim names as executor of a `serving` host, is opened | model 12 | | **CC11** | **No operation starts over a pending chore in its locations**: the start CAS from `serving` fails if an obligation names a region or box the operation will write | model 15 (rev 5): a move landed into a region whose park was still running, and the park retired the landing's new versions: acknowledged writes lost on cold restore, or a landing that never converges | | **CC12** | **A quarantined box retires its own state.** §7.2 says a box on `lost` is refused "until its old state has been retired"; name the actor: the returning box itself, under the host guard, before anything else | the returning box otherwise keeps stale local state | | **CC13** | **Every probe renders held config** (`RenderHeld`, `MaterializeHeld`) before spawning, including a future push's | model 4 | **Rev 5's request, against claim rev 3:** | id | change to claim rev 3 | why (model) | |---|---|---| | **CC14** | **Staging residency rests on `SafeUntil`.** C3's closing paragraph says correctness no longer depends on the clock and pause assumption, because every destructive request is version-bound. Cross-region staging is the exception: a staging PUT creates a version, and the moves spec's staging horizon (`seal time + staging TTL + H`, §7.3) bounds only requests signed before the lease's `SafeUntil`. State that every request under the staging-work lease checks `SafeUntil` immediately before it is signed, after any pause, and that §3.1's margin must cover clock skew plus the longest pause between that check and the signature | model 14: staging residue in a region the instance left, or in the serving region (`sign-before-safeuntil`, §14.2.5) | **Rev 6's requests, against claim rev 4 (a3c86fa5).** The stricter model (§14.2) needs each of these; the mutation column is in §14.2.5. | id | change to claim rev 4 | why (model) | |---|---|---| | **CC15** | **An owner for stragglers that complete after `H`.** C3a says a straggler "is found and parked by the next retirement listing of that location". A freed incarnation's location never has another retirement, and neither does a dead generation inside a serving location. Name the owner: each region's box sweeps the host namespace in its own bucket while the host is at rest (free, or serving with no operation and no obligation), parking everything outside the live set as a retirement event of its own, and abandoning the sweep the moment the host leaves rest (moves §6.4) | model 17: with requests that complete up to 11 `H` late, residue under a freed host and in dead generations that nothing ever retires (`straggler-sweep`) | | **CC16** | **A create registration is a lease, and purge's drain knows when it is dead.** §5.1 step 0 removes a registration only at `serving` or cancel. A console that crashes between registering and the claim CAS leaves a registration nothing removes, and every later purge of the app waits on it forever. Give each registration an expiry; a create checks it immediately before its claim CAS (the §3.1 margin applies); purge's drain removes a registration only once it has expired **and** that host's claim does not name the create. Expiry alone is not enough: a create that claimed before the expiry is still in flight after it | model 19: purges that never finish (`registration-recovery`, 894 / 3,000); and, with expiry alone, a second host serving after purge tore down its store | | **CC17** | **An orphan record has no deadline.** §5.1 removes a cancelled create's late record only until `cancelled_creates.until`. A record PUT has no completion bound either (C3a's own lesson), and a late record blocks every recreate of the host. Treat a record whose incarnation is in `retired_incarnations` as an orphan whenever it lands, and delete it with `DeleteIfMatch`; `cancelled_creates` then only needs to name the record key | model 20: a record landing after `until` (`orphan-no-deadline`) | | **CC18** | **Name `holder.generation`.** C3a says restore reads "the moves spec's generation within it, taken from the claim". Add the field to §2's schema: written by the commit's claim CAS of a move, by a push's probe CAS, and by a dirty rollback's source probe; origin is `""`. The start check (moves §5.1) compares markers against it | J2-R2-8: the third admission fence exists only if the claim names the generation | | **CC19** | **CC14, amended: sign at the check.** "Check `SafeUntil` immediately before every request signature" still leaves the pause between the signature and the send, and a whole-VM freeze can stretch that without bound. Compute the signature at the check, with the check's time as `X-Amz-Date`, never recompute it, and set `SafeUntil = Until − skew − 15 min`: S3 then refuses the request once the pause outlives the signature | model 18: a box frozen between a passed check and the send lands staging after the cleanup (`sign-at-check`, 1 / 100,000) | **CC15 to CC19: all adopted in claim rev 5** (4a458a9c), which maps them in its §12: CC15 to §6.2a's at-rest sweep; CC16 to §5.1 step 0 and §5.3 step 1b; CC17 to §5.1 step 5; CC18 to §2 `holder.generation`; CC19 to C3's staging-work lease exception. Rev 7's model encodes that text. **Rev 7's requests, against claim rev 5 (4a458a9c): both adopted in claim rev 5.1 (64eec93b),** which maps them in its §12: CC20 to §6.2a's at-rest sweep, as written; CC21 to §5.3 step 1b, differently. S3 cannot make the creator's claim write conditional on the registration object, so purge fences the **host claim** instead: before removing an expired registration whose host claim does not name the create, it CASes a free or absent claim to `{op: {kind: "purging", id: }}` with no holder, and releases the fence to free at `done`. The creator's re-read of its registration stays as a cheap extra, not sufficient on its own. Rev 7.1's model encodes that text (§14.2.2), and finds the fence needs CC22. | id | change to claim rev 5 | why (model) | |---|---|---| | **CC20** | **The at-rest sweep also lists `moves//`.** §6.2a's sweep lists retired incarnations' locations and dead generations, both under `instances//`. A staging request accepted before its lease ended can complete after the staging cleanup, and after the move releases nothing lists `moves///` again. Add `moves//` to the sweep's list for a host with no operation running | model 25: staging residue that nothing ever retires (`sweep-staging`) | | **CC21** | **A create checks its registration immediately before its claim CAS.** Renewal keeps a live create's registration from expiring, but a console that pauses between renewing and its claim CAS can resume after the drain removed the expired, unclaimed registration and purge fixed its host set; its claim CAS then succeeds, and the host serves after its store was torn down. Re-read the registration (still present, renewed within the lease) as the last step before the claim CAS, with the §3.1 margin, and abandon the create if it is gone | model 26: a host serving after its app's store was torn down (`registration-check`) | **Rev 7.1's request, against claim rev 5.1 (64eec93b):** | id | change to claim rev 5.1 | why (model) | |---|---|---| | **CC22** | **The creator's claim CAS is conditional on the claim ETag it read when it registered** (`If-Match` that ETag, or `If-None-Match: *` if the claim was absent), not on a read taken just before the CAS. The fence is released at the purge's `done`; a creator paused across the whole purge resumes against a free claim again, and a CAS "from free" conditioned on a fresh read succeeds, for an app whose store is gone. Any fence placed after the creator registered, even one since released, changes the ETag, so the stale condition fails with 412 and the create is refused before any record exists. Two corollaries. The fence on an absent claim must be **released by writing a free claim, never by deleting the object**, or the paused creator's `If-None-Match: *` succeeds again. And a 412 for any reason (a sweep acknowledgement, another create) means the create starts again at §5.1 step 0, with a new registration and a new read; it never re-reads and retries the CAS alone. With this condition the creator's re-read of its registration adds nothing the model can see; it stays as a cheap extra | model 28: a host creating on a torn-down store after a fence was released (`fence-etag`); the re-read alone is shown insufficient (`purge-claim-fence` + `fence-etag`, §14.2.5); the deletion corollary is its own rule in rev 7.2 (`claim-never-deleted`, 7 / 3,000) | **CC22: adopted in claim rev 5.2** (465fee96), which maps it in its §12: §5.1 step 0 reads the host's claim and keeps its ETag, or notes that it is absent, before registering; step 2's CAS is `If-Match` that ETag or `If-None-Match: *`, and a 412 restarts the create from step 0 with a new registration; §5.3 releases the fence at `done` by writing a free claim, never by deleting the object; and a new **C5a** states the corollary as a rule: a claim object is never deleted, "free" is a written state, and only the very first claim for a host is created create-only. Rev 7.2's model encodes that text (§14.2.2) and gives C5a its own mutation (§14.2.5). No further claim change is requested. **Claim rev 6 (3ea3f90c), after joint round 3: no request from this spec.** Rev 6 answers R3-2 (orphan cleanup in a fixed order: record, then the claim fresh, then `DeleteIfMatch`), R3-3 (the installation-scoped retired marker) and R3-4 (the sweep discovers locations from the namespace; `retired_incarnations` is anti-reuse only). Rev 7.4 encoded all three and asked platform-1e three questions. **Claim rev 6.2 (30d572c4), after joint round 4: no request from this spec.** Rev 6.1 (79a5243c) answered the three questions: operations are ordered by claim `gen`, op ids opaque; a record with no claim object is a legacy host, never an orphan; the claim owns the sweep-side rule for the live set and this spec owns generation-specific live-object selection. Rev 6.2 took joint round 4's claim side: **installation stamps** (§6.4 there: the stamp is the claim `gen` captured at the CAS that made the installation current, a writer is admitted only at the active stamp, retirement sets `none`, a reinstall sets its own stamp under the guard), the **region-qualified live set** (§6.2a there: `{authoritative region, holder.location, holder.generation}` plus the versions this spec supplies, protected in the authoritative region only; the authoritative region is `op.exec`'s while serving, or what this spec's phase table names, §5.1), and **retirement under purge** (§5.3 step 4 there: the record version in `op.intent`, `DeleteIfMatch`, `purging → parking` only with the record gone and the journal at `torn-down`; a purged host never returns to `serving`). Rev 7.5 encoded all of it (§5.1, §6.4, §14.2.2) and raised no new claim question. **Claim rev 6.3 (8faa89a4) and rev 6.4 (697f0f9f), after joint round 5: no request from this spec.** Rev 6.3 takes R5-4's claim side: **`holder.stamp`** in the claim schema (§2 there), written by the CAS that makes an installation current, so the claim is the durable record and the recovery source; **initialization** of a box's local record only from a fresh read under the guard that names the box `op.exec` or the landing participant (`active_stamp = holder.stamp`: a create's first wake, adoption, a lost record on restart); **no local record: refuse**; **two retirements**, `retire-artifacts(S)` (parks the artifacts tagged `S`, adds `S` to the retired set, leaves `active_stamp`) and `retire-installation` (moves the active stamp into the retired set, sets `none`, parks everything), with every helper naming which it is; and a dirty rollback that never keeps its predecessor's stamp. Rev 6.4 answers rev 7.6's three notes (`moves-claim-questions.md`, rev 7.6 heading): the stamp is captured from the box's local record at work selection and compared with `active_stamp` under the guard, the fresh claim read still serving the incarnation, operation and config and release identity, so `fence-stamp` and its witness stand; `retire-artifacts(S)` only parks the files tagged `S` and never changes `active_stamp` or `retired_stamps`, activation (old active into retired, then the new one) and `retire-installation` being the only moves of a stamp, so a dirty rollback parks its pre-move database with `S` still active and `S` retires when the probe CAS's new stamp activates, issuance staying with `holder.generation` in one write; and §6.2a now says that a child prefix that is not an incarnation id (64 hex) belongs to the host prefix's own location. Rev 7.6 encodes all of it (§6.4, §7.6, §7.8, §14.2.2): every installing CAS in the model writes `holder.stamp`, `initRecord` is the only writer of a box's record, `retireArtifacts` moves no stamp and `install` retires the old active stamp, and the landing's `checking` installs from `holder.stamp`. This spec's phase table (§5.1) is what the claim's "landing participant" defers to. **Claim rev 6.9 (f6180ae4), with rev 6.8.2's answers, after joint round 9: no change requested, seven readings.** Rev 6.9 rewrites §6.4a under Paul's ruling to simplify: no local database reuse, warm and cold only, `holder.writer_box`, `writes_lost`, admission after readiness and before any upload, bound to the observed `op.id` and `op.phase`, and the `park-generation {region, generation}` chore; rev 6.8.2 lets any `op.exec` box continue on a certificate and requires the disk-lost source's admission to record `writes_lost`. Rev 7.10 encodes all of it (§5.1 step 9, §7.14, §14.2.2) and mirrors the library's names (`AdmitWriter(Admission{ObservedOpID, ObservedPhase, ObservedSession, ObservedGeneration, Box, Mode, NewGeneration, PredecessorCertified, RemoteMaxTXID})`, `Holder.WriterBox`, `Obligation.Generation`). The readings for platform-1e (`moves-claim-questions.md`, rev 7.10 heading): the predecessor to restore from is `holder.generation` unless it holds no file, when the nearest ancestor with files, the marker's generation, is the baseline; an operation's prepared generation is opened once by its probe's cold admission from its pinned snapshot in the box's own region and never re-entered; the landing's prepared generation keeps the flip's stamp while a push's and a dirty return's take a new one; a clean rollback's return CAS names the source's generation again; a move's landing records `writes_lost` for the source's uncertified session although the seal carries every write; the admitted writer's start (first upload, marker, anchor) is local bookkeeping driven by its own agent loop; and the model's C2 already binds every CAS to `op.id`, so the `admit-bind-op` witness needs both rules off. **Claim rev 6.26 (98b45332), after joint round 17: R17-2 fixed on the claim's side.** `moves/` is used by a delete's park of the staging namespace (§5.2) and by an at-rest sweep, each while no operation stages there; §6.4 here says the same. No notes. **Claim rev 6.24 (0b821896), after joint round 16, and rev 6.25 (413b12cb), which answers the four notes below: note 1 agreed and now §5.2 (a delete's `park-replica` per touched region is scoped to `holder.location`, the incarnation location or the legacy `instances/` excluding child incarnations, plus one scoped to `moves/`, never the whole host prefix; E's `BeginDelete` queues them unscoped, bound to `holder.location` at the first step, and the console's delete wiring queues the `moves/` chore); note 2 agreed and fixed in code, main 67547f5 (E had recorded a boxless `writes_lost` on a created incarnation's first admission); notes 3 and 4 noted, no claim change.** Rev 6.24 sets the legacy identity (session 0, generation `""` read as @root, `holder.writer_box` the pre-release process's box, valid in both loss fields), the scope namespaces (a scope covers only its own namespace; an `instances/…` scope never touches `moves/`; the legacy `instances/` scope excludes child incarnations; staging is parked only under an explicit `moves/…` scope) and the narrowed born-covered (an unscoped `park-replica` or a park scoped exactly to `holder.location`). Rev 7.17 encodes all of it (§5.1 step 9, §6.4, §7.14 items 2, 5 and 9, §14.2.2). Notes for platform-1e (`moves-claim-questions.md`, rev 7.17 heading): **(1)** the model's delete parks the deleted incarnation's own location and the staging namespace, one chore each, never the whole host prefix; an older incarnation's residue is the at-rest sweep's, as R3-4 intended; if the claim's delete (§5.2) is read to park the whole host prefix, say so and the model follows; **(2)** a created incarnation's first admission has a predecessor with no writer (`holder.writer_box` empty, session 0 in the model, generation `""`): since every declaration names a box, it records no `writes_lost`, nothing having been acknowledged; the model's INV-24 enforces it; **(3)** the model's witness fixture for a legacy host writes a certificate for session 0 so that a warm first wake can be exercised; a real pre-release writer leaves no certificate and a real first wake is cold; the fixture is disclosed in the model, not a reading of the claim; **(4)** the sweep protects every generation of an unfinished run in the authoritative region without the run walk's region and coverage filtering, a wider, conservative set (§14.2.6). **Claim rev 6.23 (876d6908), after joint round 15: no change requested, two notes.** Rev 6.23's §2 gives the park scope its grammar (`instances/`, `instances//`, `moves//`, `moves/`), the archive incarnation (the location's own, the staging op's holder's, the newest retired for a free host's `moves/` sweep), `evidence_at` always the original evidence or fencing bound (a staging lane's seal bound, persisted at seal or fence time, never cleanup's start) and one deadline for every scope, a listing later than `evidence_at + H`; it also states that a second region's sweep joins the running op and that every sweep acknowledgement waits the horizon. Rev 7.16 mirrored its grammar, bounds and deadline (§6.4, §7.14 item 9, §14.2.2); R16-2 found the sweep's scope grouping short of it, and rev 7.17 corrects that under rev 6.24. Notes for platform-1e (`moves-claim-questions.md`, rev 7.16 heading): **(1)** a delete's and an adoption's `park-replica` are queued by a CAS that has no evidence time yet (the stop evidence is proven by the parking box later), so the model binds their scope in a CAS before the first step, with the evidence time at which the stop evidence was first in hand, once and never again; the claim's "a chore without a scope is bound to one before its first step runs" is read to mean exactly that, and no other chore is bound late; **(2)** the model's incarnation ids are decimal where the claim's are 64 hex, and its op ids are its own; the grammar oracle (INV-23) checks the four forms with those alphabets. **Platform-1e's answers to the rev 7.14 notes, and joint round 14: no change requested, three notes.** All six notes agreed (joining preferred; the sweep waits the full `H`; recovery replays only `FreeAfterPark`; a create waits on the parking op; the covered-baseline refusal; a warm record's region). Notes for platform-1e (`moves-claim-questions.md`, rev 7.15 heading): **(1)** a staging park (`park-staging`, CC20, this spec's kind) binds a persisted scope before its first step like a whole-location park but keeps the seal's own quiet time, bounded by the staging lease, rather than `evidence_at + H`; **(2)** a generation chore's scope takes `evidence_at` from the publication that queues it, the marker's publication being the evidence that the predecessor is dead, so its horizon is the publication's time plus `H`, equal to the chore's queued quiet time; **(3)** the model decomposes `H` into a signature-age allowance and the request bound (45 + 15 steps) so the scope's `request_limit` has the claim's meaning; nothing in the claim need change. **Claim rev 6.15 to 6.22 (0bf2f49f), after joint round 13: no change requested, six notes.** Rev 6.15 and 6.16 added the persisted park scope and the holderless sweep; rev 6.21 adopted rev 7.13's seven notes; rev 6.22 answers R13-1 to R13-3 (the run walk across regions, coverage in exactly four ways with warm entries born covered and `CoverParkedPredecessors`, the `additional_writes_lost` representation with one alert per declaration) and keeps E's warm-region check with note 4's reading. Rev 7.14 encodes all of it (§5.1 step 9, §6.4, §7.14's operative list, §14.2.2). Notes for platform-1e (`moves-claim-questions.md`, rev 7.14 heading): **(1)** another region's sweep while the holderless parking op runs **joins** it, adding its own scoped chores under the same op id, rather than waiting for `FreeAfterPark`; the claim's "one CAS queues only scoped remote chores owned by that op" is read to allow later CASes adding to the op; **(2)** the sweep's chore is acknowledged, with its straggler pass, only after a verification listing later than `evidence_at + H`, `H` the signature-age limit plus `request_limit` (rev 7.14 wrote `evidence_at + request_limit` here and in §6.4, dropping the signature-age allowance, R14-3; platform-1e's answer confirmed the full `H`), so a free host's successor create waits up to the horizon after the sweep found residue; rev 7.13's sweep acknowledged at once; **(3)** the console's recovery pass leaves a holderless sweep to the obligation runners and replays only `FreeAfterPark`, for an op whose chores are all acknowledged (a crash between the last acknowledgement's CAS and nothing: the CAS is one, so this is belt and braces); **(4)** INV-20 is the model's statement of "blocks `free`'s successor create": a create's claim CAS never finds an obligation outstanding; the create **waits** on the parking op rather than being refused, and `userCreate` does not even plan one while it runs; **(5)** the covered-baseline refusal also fires where rev 7.13 restored nothing: an origin swept under an unfinished admission by rev 7.11's sweep (the `absent-marker-origin` mutation) is a covered local baseline now, and the recovery refuses with an alert instead of starting on an empty database; both are INV-2 for the witness, the refusal is the better failure; **(6)** a warm record's `region` is the admitted box's, which equals its predecessor record's region in every reachable warm admission, as 6.22 states; the model records the box's. **Claim rev 6.13 (b3825827) and 6.14 (dc146058), after joint round 12: no change requested, seven notes.** Rev 6.13 adopted rev 7.12's five notes (the lost-target return is `""`; a probe never marks the host failed; chore ids are opaque; `clean-return` speaks for two sessions; coverage is per region, with `region` and `predecessor_region` on the record, the publication walk and `CoverRegionPredecessors`). Rev 6.14 answers R12-4 and R12-6: §6.4a has one operative subsection, the retry limit is a serving wake's, retention is conservative with no compaction. Rev 7.13 encodes all of it (§5.1 step 9, §7.7, §7.8, §7.14's operative list, §14.2.2). Notes for platform-1e (`moves-claim-questions.md`, rev 7.13 heading): **(1)** item 6 names the committed move's source cleanup; the model composes `CoverRegionPredecessors` into **three** CASes, the commit's, the target's release and the console's lost-target release, since the last two queue the target region's whole-location park and a restarted probe's record names the dead probe's generation there (seed 1036 / 3,000); **(2)** a rollback's return admission comes after the release queued the target region's park, so no `CoverRegionPredecessors` can see its record: the model reads such a record as **born covered** (the admitting CAS finds the park already queued for its predecessor's region), which keeps INV-17 as the park completes; the claim should say so, or have the park's acknowledgement CAS cover too; **(3)** the at-rest sweep retires a region's dead generations wholesale: the model covers the records whose predecessors it parks in the CASes that queue and acknowledge its `park-sweep`; **(4)** item 1's "a warm admission keeps its predecessor's region" is read as the admitted box's region, where the warm writer's generation keeps its objects: a clean return's warm re-admission is on the source, in the source's region, while the last record is the target's; **(5)** the probe attempt's deadline is checked before each start and before readiness is accepted (R12-2), and a recovery probe's budget is nine attempts, three times a landing's (R12-5), if the claim wants either stated; **(6)** item 5's first bullet, "predecessor is the active generation: covered, no chore", is read as **left uncovered**, no chore: the generation is live, its retirement comes with the publication that supersedes it, and a record covered now would leave it unretired then, since the superseding admission's predecessor is the last record rather than the generation it restored from (seed 41914 / 100,000: a republished lagging marker covered the target's record while the source's generation was live again, and the generation was residue); **(7)** item 2's lost-target return records `writes_lost` for **every** session the target was admitted into since the cut, certified or not, not only the last (seed 33373 / 100,000: a probe session that certified left its writes in the lost region, and the record for the warm continuation did not name them). **Claim rev 6.11 (63255a3c) and 6.12 (b202ea4e), after joint round 11: no change requested, three notes.** Rev 6.11 answered rev 7.11's eight notes (the retry limit confirmed; the chain generalized to a run of admissions; `holder.admissions` in the holder; `PublishGeneration` replacing `QueueGenerationRetire`; the clean return only with `src_covered`; `BaselineCoversPredecessor` only for a `from_db` seal; a new agent stops a running app; the lost box's timers, the source's cut session and adoption's `writer_box`). Rev 6.12 answers R11-2 to R11-5: retirement only through `PublishGeneration` and only of predecessors in `holder.admissions`, `""` and `"@root"` one origin; adoption sets `writer_box`; the predecessor is always the last record and admission carries `Coverage` `""` | `sealed` | `clean-return`; `PublishGeneration(observedOpID, observedPhase, observedSession, publishedGeneration, region, newID)` bound to the operation and phase, legal in serving and in move kinds. Rev 7.12 encodes all of it (§5.1 step 9, §7.8, §7.14, §14.2.2). Notes for platform-1e (`moves-claim-questions.md`, rev 7.12 heading): **(1)** a return made clean only by a declared-lost target is **not** `clean-return`: the target may have written and its session is uncovered, so `writes_lost` is recorded for it (and the source's cut session when `src_covered` is false); **(2)** a probe attempt's app restarts share the attempt's 30 s readiness window, the fifth failure or the window ends the attempt, and a probe never marks the host failed (the retry policy's failed-host marking is a serving wake's); **(3)** when `PublishGeneration` queues several chores for one `newID`, the model names the second and later ones `-`; E's naming is the one to mirror. **Claim rev 6.9.1 and 6.10 (3f916b05), after joint round 10: no change requested, eight notes.** Rev 6.9.1 answered rev 7.10's seven readings (confirmed: the ancestor restore from a complete listing, the prepared generation opened once, the return CAS naming the source's generation, `writer_box` as a baseline choice only, C2 and the delete witness; refined: `KeepStamp` only when `holder.stamp > holder.session`; changed: no `writes_lost` behind a seal that covers the stopped predecessor, compared against the seal's own position). Rev 6.10 answers R10-1, R10-2, R10-4 and R10-5 (§6.4a: the CAS before the app start, a session per agent process, the retry policy with this spec's limit, unfinished generations with live baselines and chores after the marker, the five refused kinds, warm with an empty `NewGeneration`, the ancestor restore only for the latest journalled unfinished admission, `session:` chores, `"@root"`). Rev 7.11 encodes all of it (§5.1 step 9, §7.14, §14.2.2). Notes for platform-1e (`moves-claim-questions.md`, rev 7.11 heading): **(1)** the retry limit is five attempts under one session with a doubling backoff from one step, then the host is marked failed and gets no session until an operator clears it; **(2)** "the marker names the journalled predecessor of the latest admission" is read to include a **chain**: a marker naming the predecessor of a run of admissions each of which died before its marker (the second's purge failed after its CAS) is the same normal state, every generation of the chain is live, and the baseline is the nearest that holds an object; **(3)** `op.journal.writers` must **survive the release** to `serving` and the start of the next operation, or the release must wait for an unfinished start: an abort released over a source admitted a tick earlier left a lagging marker nothing explained; **(4)** a writer that dies after its marker landed and before `QueueGenerationRetire` leaves its predecessor unretired: the next admission's marker retires the predecessor of every earlier cold entry the journal does not mark covered, so the journal needs that mark (or E's `QueueGenerationRetire` may be given every uncovered predecessor); **(5)** a rollback is clean only when the seal recorded the source's own replica as covering its stopped database (`src_covered`); otherwise the return carries the seal whether or not the target wrote, and a target that never restored a database returns the staged seal by version (rule `seal-return`); the clean/dirty verdict is this spec's, but the claim's "clean rollback restores the source from its own generation" should say "when its replica covers the stop"; **(6)** `BaselineCoversPredecessor` is true only for a seal built from the stopped database itself; a seal built from the remote because the disk was gone stops at the remote's position, and the prepared generation's admission records `writes_lost` for the writes beyond it; **(7)** "a new agent process is a new session" is read to mean the restarted agent **stops a running app** of the dead session and admits anew, where today's activator reattaches to the unit; **(8)** a declared-lost box's registered timers are declared by the console's record and alerted, since no return lane carries them; and `holder.writer_box` has no box before the first writer (the model uses `-1`; E's string names have no such ambiguity). **Claim rev 6.8 (5462d293) and 6.8.1 (a963d98d), after joint round 8: no change requested, three notes.** Rev 6.8's §6.4a answers R8-1 to R8-4: `AdmitWriter` as the one CAS before any upload, `holder.session`, the writers journal with its predecessor, cold mode as the installing CAS that never replaces the operation, per-session certificates never deleted, the three provenance conditions, the empty bootstrap; rev 6.8.1 adds that only `op.exec` may open replication, in every op kind. Rev 7.9 encodes all of it (§5.1 step 9, §7.14, §14.2.2) and owns the data path: the phase table of which box is admitted in which phase and the CAS that handed it `op.exec` (§7.14), the evidence rule for the certificate (every request of the session's lifetime, a sticky uncertainty, R8-5), the integrity check's content (the WAL in place), the build at position 0 (no files; `snapshot_txid = 0`), the stand-in for "a database that holds transactions but never replicated" (a snapshot at TXID 1, the first sync's TXID). Notes for platform-1e (`moves-claim-questions.md`, rev 7.9 heading): the warm rule's "or the warm baseline rules below allow otherwise" is read here as nothing beyond the three conditions, so a certificate of another box never admits warm; the admission CAS is made as the process starts, after the start and health checks, so a start that fails admits no session (the claim's "before the writer issues any upload" allows either order, and the earlier one stormed); and a source that lost its disk during a revert or rollback skips the digest check and is admitted at its probe, since it has nothing to verify (§7.14). **Claim rev 6.6 (b10547e1), after joint round 7: no change requested, three notes.** Rev 6.6's §6.4a, writer epochs, answers R7-1: a clean close drains then writes `closed.json` once; a writer continues a generation only with an intact marker; everything else is a cold recovery, an installing CAS into a fresh generation with a new stamp; writers are bound to their generation. Rev 7.8 encodes it (§5.1 step 9, §7.14, §14.2.2) and owns the data path. Notes for platform-1e (`moves-claim-questions.md`, rev 7.8 heading): the marker is **consumed by the writer as it opens**, which the text implies ("written once") but does not state, and without which a continued writer that crashes leaves the earlier marker in place with a `max_txid` that may still equal the remote maximum (its own uploads in flight) and so look intact; the first wake of a **created** incarnation and of an adopted legacy host is a cold recovery under the literal rule, because no writer has closed the origin generation and the console cannot write the marker into the data bucket, which costs one recovery generation per create; and the recovery's baseline is the **warm database** when it is the old generation's, since it holds acknowledged writes the old generation may not, which the claim's "restores from the old generation" should be read to allow. **Claim rev 6.5 (d36adcfa), after joint round 6: no request from this spec.** Rev 6.5 answers R6-4 with one invariant: `holder.stamp` always names the installation `op.exec` should run, so any CAS that changes `op.exec` writes `holder.stamp` in the same write, the new gen when it makes a new installation current, the installation's existing stamp from the op journal when it makes an existing one current again. A dirty rollback's return to source is the second case: it writes the source's pre-move stamp `S`, not the target's `F`; the source's initialization then changes nothing, the installed database keeps `S`, the probe CAS issues `R`, and activating `R` retires `S`; `F` was only ever active on the target and retires there. Rev 7.7 encodes it (§6.4, §7.8, §14.2.2): the flip journals `S`, the return CAS writes it back, a lost local record re-initializes from it, and the oracle refuses an activation of another box's stamp. No claim question this round. **Not a change, a note for platform-1e (model, rev 5).** Under rev 3 the install wait on `quiet_after` never bites: every retirement acknowledges only after its own `evidence time + H`, so the `quiet_after` it writes has already passed; CC11 keeps operations off regions with pending chores; and `free` needs every obligation gone. The model drops this spec's T12 (which reads `quiet_after`) and no cut ever waits. Keeping the field costs one map in the claim and guards any future path that acknowledges before its horizon; removing it is the claim authors' call. ## 20. Copy (pending copy review) Drafts for the operator CLI. None is approved; every string goes through the copy-review gate. No em dashes. | id | string | |---|---| | M-C1 | Moving from to . | | M-C2 | Copying data. keeps serving while this runs. | | M-C3 | Checking that can run . | | M-C4 | cannot take this move yet: . | | M-C5 | is ready. Starting the cut. will be offline for about . | | M-C6 | Stopped in . Starting in . | | M-C7 | Moved. is served from . It was offline for seconds. | | M-C8 | could not start : . Moving it back to . | | M-C9 | is back in and serving. Nothing was lost. | | M-C10 | Waiting for in to confirm it has stopped . is offline. | | M-C11 | could not start in or in . It is offline. Run carlos move resolve --host --retry once the cause is fixed. | | M-C12 | Ready to cut. Run carlos move cut --host when you are ready. | | M-C13 | This move has started its cut. It will finish or move back on its own. | | M-C14 | keeps a file this move cannot copy: . Only regular files and folders move. | | M-C15 | keeps more than 256 MiB of files beside its database. Keep large files in object storage. | | M-C16 | does not answer its readiness path. Declare moves.ready_path for this app. | | M-C17 | uses private storage. Instances with private storage cannot move yet. | | M-C18 | This declares lost. Anything wrote on after