# Edge asset cache, plan 2: the disk store Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** A self-contained package, `internal/assetcache`, that stores cached asset responses on a box's disk under a budget, with worst-case reservations, a free-space floor, per-box and per-app fill bounds, fair-share eviction, expiry and a sweep, and that nothing calls yet (plan 3 does).

**Architecture:** One `Store` per process over one directory. Each entry is a `<id>.meta` file (JSON) naming its versioned body files, `<id>.<v>.body` (identity) and optional `<id>.<v>.gz`, where `<id>` is the hex sha256 of the key and `<v>` a random version drawn per commit. A commit writes the new versioned bodies, then renames a temporary over `<id>.meta`, then deletes the previous version's bodies, so a crash at any point leaves either the old entry or the new one, never new bytes under old metadata (astra P1-9). An in-memory index is rebuilt from the `.meta` files at `Open`. A fill is a `Fill` holding a reservation; it writes temporaries in the same directory and renames them on `Commit`. Free space comes from an `FS` interface whose Linux implementation is `statfs`. The package knows nothing about HTTP semantics beyond storing headers it is given: admission, freshness rules and serving belong to plan 3. The design is spec §7, written so the media roadmap's stage 1 can reuse it with its own directory and budget (spec §11).

**Tech Stack:** Go 1.26 standard library (`os`, `compress/gzip`, `crypto/sha256`, `encoding/json`), `golang.org/x/sys/unix` for `Statfs` on Linux.

**Spec:** `docs/superpowers/specs/2026-10-24-edge-asset-cache-design.md` (rev 8). Read §5.3 ("Coalescing", "Every fill has a deadline"), §5.4 and §7 before starting.

**Branch:** `edge-asset-cache-2`, cut from amadan `main` after the spec branch has merged. Independent of plan 1.

## Global Constraints

Every task's requirements include this section.

- **Real files.** Every test runs on a real directory from `t.TempDir()`. A fake `FS` (free-space source) is allowed and is the only stand-in; tests using it say "not acceptance" in the doc comment. Task 2's `TestStatfsReportsTheTempDir` is the acceptance test for the real `FS` and runs on Linux only.
- **Targeted local runs; CI runs the full suite.** `go test ./internal/assetcache/ -count=1` is allowed in full (new, small package). `go build ./...`, `go vet ./...`, `gofmt -l .` run locally.
- **Cross-compile gate.** `internal/edge` is linked into the CLI, and plan 3 imports this package from it. `statfs_linux.go` carries `//go:build linux`; `statfs_other.go` carries `//go:build !linux` and returns `ErrUnsupported`. Before every push: `GOOS=darwin GOARCH=arm64 go build ./...` and `GOOS=windows GOARCH=amd64 go build ./...`.
- **No user-facing strings.** Every error in this package is an internal fragment, so no copy rows.
- **No tenant names.** Keys in tests use account `abc`, apps `shop` and `blog`, host `shop.example.com`.
- **Go cache.** Leave `GOCACHE`, `GOMODCACHE` and `GOTMPDIR` at their defaults.
- **Census overlay per branch, loaded LAST.** `plan14-root-reads-edge-asset-cache-2.tsv`, `rev3=PROPOSED-edge-asset-cache-2`, after whatever loads last at the merge (plan 1's overlay if it has merged), previous seal checked first.
- **Merge main first; amadan tasks; scratch under `t.TempDir()`, removed by name.** As in plan 1.
- **Spec numbers, verbatim.** Default budget: the smaller of 4 GiB and 10% of the filesystem's size (`CARLOS_ASSET_CACHE_MAX` overrides, read by plan 3). Free-space floor: no fill admitted unless free space minus outstanding reservations minus this one stays at or above 15% of the filesystem's size. Reservation: `Content-Length` if known, else 32 MiB, doubled for the gzip copy, plus 64 KiB of metadata. Body cap: 32 MiB. Fill bounds: 16 per box, 4 per app. Fair share: budget divided by the number of apps with entries, never more than 25% of the budget for one app.

## Review Focus

1. **A crash between writing `.body` and `.meta`.** A person expects a restart to leave no half entry and no leaked disk. Test: Task 1, `TestOpenRemovesOrphanBodiesAndTemporaries`.
2. **A `.meta` file from a newer binary** with fields this one does not know. A person expects it to load (unknown JSON keys ignored), not to be deleted. Test: Task 1, `TestOpenKeepsMetaWithUnknownFields`; a `.meta` that does not parse is deleted with its files.
3. **A filesystem that fills from outside the cache** (logs, a database) while fills are in flight. A person expects in-flight writes to stop at their reservation and new fills to be refused, never the cache pushing past the floor. Test: Task 3, `TestReserveRefusesWhenSomethingElseFillsTheDisk`.
4. **One app over its share when a second app arrives.** A person expects the second app's fill to evict the first app's oldest entries, not its own and not to be refused. Test: Task 5, `TestNewAppEvictsTheAppOverItsShare`.
5. **Clock moving backwards.** An entry stored "in the future" after a clock step. A person expects it to stay valid until its stored expiry and never to report a negative age. Test: Task 4, `TestAgeNeverNegative`.

## Plan review

Astra reviewed the three plans together over eight rounds (`docs/superpowers/specs/CODEX-FINDINGS-ASSET-CACHE-PLANS.md`, then `-PLANS-R2.md` to `-PLANS-R8.md`; brief `CODEX-REVIEW-BRIEF-ASSET-CACHE-PLANS.md`) and accepted them at `4cca5784`. Findings were 16, 6, 6, 3, 1, 1, 2, then none. Every finding was accepted after its citation was checked; none was declined. Findings that changed a task are marked **(astra Pn-m)** where they landed.

## Census impact

- New package `internal/assetcache`: its file-reading declarations (`Open`'s scan, `Entry.Open`) are new plain rows of class `legacy`, task `accepted-legacy`, basis `accepted-legacy;rev3=PROPOSED-edge-asset-cache-2;asset cache files under $CARLOS_DATA_DIR/asset-cache, written and read only by this package`, the form the loader requires (`plan14_root_reads_test.go:478-485`); there is no `cache` class (astra P1-3). Its envelope is pinned.
- Notes: `docs/superpowers/notes/census-proposal-edge-asset-cache-2.md`, ruling asked on #280.

## File structure

| File | Responsibility |
|---|---|
| `internal/assetcache/key.go` | `Key`, its id. |
| `internal/assetcache/meta.go` | `Meta`, its file encoding. |
| `internal/assetcache/store.go` | `Config`, `Open`, the index, `Get`, `Retain`, `Stats`. |
| `internal/assetcache/fill.go` | `Reserve`, `Fill`, `Commit`, `Abort`. |
| `internal/assetcache/evict.go` | Fair-share eviction. |
| `internal/assetcache/fs.go`, `statfs_linux.go`, `statfs_other.go` | Free space. |
| `internal/assetcache/*_test.go` | Tests. |

---

### Task 1: Keys, metadata and `Open`

**Files:**
- Create: `internal/assetcache/key.go`, `meta.go`, `store.go`, `fs.go` (the interface only), `store_test.go`

**Interfaces:**
- Produces:
  - `type Key struct { Account, App, Gen, Host, Path, RawQuery string }`; `func (k Key) ID() string` (hex sha256 over the six fields, each length-prefixed so no two keys collide); `func (k Key) AppScope() string` (`Account + "/" + App`)
  - `type Meta struct { Key Key; Version string; Header http.Header; Lifetime time.Duration; Expires time.Time; StoredAt time.Time; Size int64; GzipSize int64 }` (`GzipSize == 0` means no gzip copy; `Version` names the body files)
  - `type FS interface { Space(dir string) (total, free uint64, err error) }` in `fs.go` (declared here so this task builds; Task 2 adds the implementations) (astra P1-1)
  - `type Config struct { Dir string; Budget int64; FS FS; Now func() time.Time }`
  - `func Open(cfg Config) (*Store, error)`: creates `Dir` (0700), deletes every `*.tmp-*` file, deletes every `.body`/`.gz` file no parsed `.meta` names (orphans and superseded versions), deletes a `.meta` that does not parse together with its files, and indexes the rest.
  - `type Stats struct { Entries int; Bytes int64; Reserved int64; Apps int }`; `func (s *Store) Stats() Stats`

- [ ] **Step 1: Write the failing tests.** `TestKeyIDIsStableAndDistinct` (same key same id; moving one character between `Path` and `RawQuery` changes it); `TestOpenCreatesTheDirectory`; `TestOpenRemovesOrphanBodiesAndTemporaries`; `TestOpenKeepsMetaWithUnknownFields`; `TestOpenDropsUnparseableMeta`; `TestOpenIndexesExistingEntries` (write three entries' files by hand in the documented layout, `Open`, `Stats` counts them and their bytes); `TestOpenDeletesSupersededVersions` (a `.meta` naming version `b` beside bodies for `a` and `b`: `a` is removed).
- [ ] **Step 2:** Run `go test ./internal/assetcache/ -count=1`. Expected: FAIL (package missing).
- [ ] **Step 3:** Implement.
- [ ] **Step 4:** Run the same command. Expected: PASS.
- [ ] **Step 5:** Commit.

### Task 2: Free space

**Files:**
- Create: `internal/assetcache/statfs_linux.go`, `statfs_other.go`, `fs_test.go`, `fs_linux_test.go`

**Interfaces:**
- Consumes: `FS` (Task 1).
- Produces:
  - `var Statfs FS` (Linux: `unix.Statfs`, `free` from `Bavail * Bsize`; other systems: returns `ErrUnsupported`)
  - `var ErrUnsupported = errors.New("assetcache: free space unknown on this system")`
  - `func DefaultBudget(total uint64) int64` (the smaller of 4 GiB and `total/10`)

- [ ] **Step 1: Write the failing tests.** `TestDefaultBudget` (20 GiB gives 2 GiB, 60 GiB gives 4 GiB, 0 gives 0). `TestStatfsReportsTheTempDir` (Linux only; `total > 0`, `free <= total`).
- [ ] **Step 2:** Run `go test ./internal/assetcache/ -run 'TestDefaultBudget|TestStatfs' -count=1`. Expected: FAIL.
- [ ] **Step 3:** Implement.
- [ ] **Step 4:** Run the same command, then both cross-compiles. Expected: PASS, clean builds.
- [ ] **Step 5:** Commit.

### Task 3: Reservations, the floor and fill bounds

**Files:**
- Create: `internal/assetcache/fill.go`, `fill_test.go`

**Interfaces:**
- Consumes: `FS`, `Store` (Tasks 1 and 2).
- Produces:
  - `const MaxBody = 32 << 20`, `const metaReserve = 64 << 10`, `const maxFillsPerBox = 16`, `const maxFillsPerApp = 4`
  - `var ErrNoRoom`, `ErrBelowFloor`, `ErrBusy`, `ErrTooLarge` (sentinels)
  - `func (s *Store) Reserve(k Key, contentLength int64) (*Fill, error)`: `contentLength < 0` means unknown; reserves `2*n + metaReserve` where `n` is `contentLength` or `MaxBody`; `ErrTooLarge` when `contentLength > MaxBody`; `ErrBusy` over either fill bound; evicts (Task 5's `makeRoom`, a no-op until then) then `ErrNoRoom` if `used + reserved + r > Budget`; `ErrBelowFloor` if `free - reserved - r < total*15/100`; an `FS` error refuses with that error.
  - `type Fill struct` implementing `io.Writer` for the identity body; a write past the reservation's identity half or past `MaxBody` returns `ErrTooLarge` and marks the fill failed.
  - `func (f *Fill) Abort()`: deletes temporaries, releases the reservation and the fill slot; safe to call twice and after `Commit`.

- [ ] **Step 1: Write the failing tests** (a fake `FS` with settable total and free; "not acceptance").
  - `TestReserveWorstCaseWithUnknownLength` (`Stats().Reserved == 2*MaxBody + 64<<10`), `TestReserveUsesContentLength`.
  - `TestReserveRefusesBelowTheFloor`: total 8 GiB, free 15% plus one reservation fits once, the second concurrent reservation is `ErrBelowFloor`. This is spec §7's sixteen-fill case reduced to its rule.
  - `TestReserveRefusesWhenSomethingElseFillsTheDisk` (free drops between two calls).
  - `TestReserveBoundsFillsPerBoxAndPerApp` (17th fill and 5th per app are `ErrBusy`).
  - `TestWritePastTheReservationFails`, `TestAbortReleasesEverything` (reserved bytes and slots back to zero, no files left), `TestAbortTwiceIsSafe`.
- [ ] **Step 2:** Run `go test ./internal/assetcache/ -run 'TestReserve|TestWritePast|TestAbort' -count=1`. Expected: FAIL.
- [ ] **Step 3:** Implement. The index lock is held only to update counters, never across file I/O.
- [ ] **Step 4:** Run the same command. Expected: PASS.
- [ ] **Step 5:** Commit.

### Task 4: Commit, lookup and expiry

**Files:**
- Modify: `internal/assetcache/fill.go`, `store.go`
- Test: `internal/assetcache/entry_test.go`

**Interfaces:**
- Produces:
  - `const MaxMeta = metaReserve` (64 KiB)
  - `func (f *Fill) Commit(m Meta, gzipCopy bool) error`, in this order: (1) gzip the identity body into a temporary when `gzipCopy`, abandoning the copy (`GzipSize = 0`) if it would exceed the identity size or the reservation's gzip half; (2) draw a fresh `Version`; (3) set every meta field, `Version`, `Size`, `GzipSize` and `StoredAt = Now()`; (4) encode the **final** meta and return `ErrTooLarge` (behaving as `Abort`) when it exceeds `MaxMeta`, so no write passes the reservation (astra P1-8, P3-1); (5) fsync and rename the bodies to their versioned names; (6) write and fsync the meta temporary and rename it over `<id>.meta`; (7) delete the previous version's bodies (subject to leases); then count the meta's bytes in the store's used bytes and release the reservation. Any error behaves as `Abort`. A `Commit` for a key that already has an entry replaces it. A reader holding the old files keeps reading them (unlinked files stay readable on Linux).
  - `type Entry struct { Meta Meta }`; `func (e *Entry) Open(gzip bool) (*Reader, error)`; `type Reader struct` wrapping the `*os.File` (`Read`, `Seek`, `ReadAt`, `Stat`, so `http.ServeContent` takes it) whose `Close` releases a **lease** on the body version (astra P2-2). A version that is replaced, evicted, expired or swept while leased is unlinked at once but its bytes stay counted in the store's used bytes until its last lease closes, because an unlinked open file still holds its disk space. `Stats` gains `Retired int64` (bytes held only by open readers).
  - `func (s *Store) Get(k Key) (*Entry, bool)`: false and the entry deleted when `Now()` is at or past `Meta.Expires`; marks the entry recently used.
  - `func (m Meta) Age(now time.Time) time.Duration` (`Lifetime - Expires.Sub(now)`, never below zero)

- [ ] **Step 1: Write the failing tests.** `TestCommitThenGetRoundTrips` (header and body bytes equal); `TestCommitThenReopenThenRead` (commit with a gzip copy, close, `Open` a new `Store` on the same directory, read both identity and gzip bytes back: the persisted meta names the version it was committed with); `TestCommitWritesMetaLast` (a crash simulated by stopping before the meta rename leaves `Open` with no entry); `TestReplacementCrashKeepsTheOldEntry` (an existing entry, a replacement stopped after its bodies are renamed but before the meta: `Open` serves the old bytes with the old headers and removes the new bodies); `TestOversizedMetaIsRefused` (a 70 KiB header value and a 70 KiB query each give `ErrTooLarge`, nothing left on disk, reservation released); `TestEvictedBodyStaysCountedWhileAReaderHoldsIt` and `TestReplacedBodyStaysCountedWhileAReaderHoldsIt` (`Stats().Retired` is the old body's size until the reader closes, and a fill that would only fit by counting those bytes as free is refused); `TestGzipCopyIsAbandonedWhenLarger` (random bytes); `TestGzipCopyIsKept` (CSS text; reading `.gz` through `gzip.NewReader` gives the identity bytes); `TestGetExpires` (fake `Now` past `Expires`: false, files gone); `TestCommitReplacesAnEntry`; `TestAgeNeverNegative`.
- [ ] **Step 2:** Run `go test ./internal/assetcache/ -run 'TestCommit|TestCommitThenReopen|TestReplacementCrash|TestOversizedMeta|TestEvictedBody|TestReplacedBody|TestGzipCopy|TestGetExpires|TestAge' -count=1`. Expected: FAIL.
- [ ] **Step 3:** Implement.
- [ ] **Step 4:** Run the same command. Expected: PASS.
- [ ] **Step 5:** Commit.

### Task 5: Fair-share eviction and the sweep

**Files:**
- Create: `internal/assetcache/evict.go`, `evict_test.go`
- Modify: `fill.go` (`Reserve` calls `makeRoom`), `store.go`

**Interfaces:**
- Produces:
  - `func (s *Store) makeRoom(need int64)` (internal): while `used + reserved + need > Budget`, evict the least recently used entry of the app with the largest `bytes - share`, `share = min(Budget/apps, Budget/4)`; stops when nothing is left to evict.
  - `func (s *Store) Retain(keep func(Key) bool) int`: deletes every entry whose key `keep` rejects, returns how many. Plan 3 calls it with "this key's generation and owner are the host's current ones".

- [ ] **Step 1: Write the failing tests.** `TestEvictionTakesFromTheAppFurthestOverItsShare` (app `shop` holds 80% of the budget, `blog` 10%; a `blog` fill evicts `shop`'s oldest); `TestOneAppWalkingUniqueQueriesEvictsItself` (only `shop` has entries, it fills past the budget, `blog`'s single entry survives); `TestNewAppEvictsTheAppOverItsShare`; `TestShareNeverExceedsAQuarter` (one app alone is still evicted first once it passes 25%); `TestRetainDeletesRejectedKeys`.
- [ ] **Step 2:** Run `go test ./internal/assetcache/ -run 'TestEviction|TestOneApp|TestNewApp|TestShare|TestRetain' -count=1`. Expected: FAIL.
- [ ] **Step 3:** Implement; per-app LRU lists inside the index.
- [ ] **Step 4:** Run `go test ./internal/assetcache/ -count=1 -race`. Expected: PASS.
- [ ] **Step 5:** Commit.

### Task 6: Census, merge and CI

- [ ] **Step 1:** `git fetch origin && git merge origin/main`; re-read the last overlay.
- [ ] **Step 2:** Run `go test ./internal/provision/ -run TestPlan14RootReadCensus -count=1`. Expected: FAIL naming the new package's units and envelope.
- [ ] **Step 3:** Write `plan14-root-reads-edge-asset-cache-2.tsv` and the notes file; post on #280.
- [ ] **Step 4:** Census test PASS; cross-compiles and `gofmt -l .` clean.
- [ ] **Step 5:** Commit, push, CI green on the tip, merge per plan 1's Global Constraints with this branch's name.