# A store per environment: object storage an instance can own **Date:** 2026-09-19 **Status:** approved in direction (James, 2026-09-19). Two **(ruling)**s were taken in that conversation; everything else is a **(decision)** and can be overturned cheaply. **Discussion:** amadan `carlos/platform` #159. **Extends:** `2026-09-02-object-storage-self-serve-design.md` (the account entitlement) and the object-storage primitive it builds on. ## 1. The problem, as found `store.json` is per app. `MintedBucket` is `prefix + acct + "-" + app` and `UserName` is `"carlos-store-" + acct + "-" + app` (`internal/objstore/walk.go:62,69`), and `writeEnv` delivers the coordinates through `appconfig.Apply` (`walk.go:689`) — the app's **default** bundle. So every instance of an app shares one bucket and one credential, and there is no way to give one instance its own. That is correct for an app whose instances are one deployment. It is the blocker for a managed, multi-tenant product, where one app has one instance per customer: the only options are one bucket with per-tenant key prefixes, where isolation becomes app logic and one path bug crosses tenants, or the app bringing its own S3 and re-implementing what the platform already does. ### What is already right, and must stay right - **The entitlement is per ACCOUNT, not per app** (`store/entitlements/.json`). Approval is already orthogonal to whatever unit owns a store, so nothing here adds an approval step. - **The walks are idempotent and resumable**, and their lost-CAS retries are guarded on the key the walk started from — the one shape the walk comments call "NOT survivable" (`walk.go:258-289`). - **The record's three null-or-present blocks** are what make "declared", "granted" and "provisioned" distinguishable. - **`release.PurgeApp` hand-parses `grant`/`provisioned`** out of the store record and refuses an app's auto-purge while either is present (`internal/release/trash.go:202-221`). The field names are frozen. - **The per-app walk lock exists** (`internal/console/storelock.go`). ## 2. The shape, in one paragraph A store may belong to a **named config environment** instead of to the app as a whole. `carlos store create --app a --environment e` declares one; the walk mints a bucket and an IAM user named for `(account, app, environment)` and delivers the coordinates into **that environment's bundle**, which is the bundle the instances bound to it already read. An instance gets its own bucket by being the only instance bound to its environment. Nothing on a box changes. ## 3. Rulings - **(ruling, James 2026-09-19) A store is keyed on `(account, app, environment)`, not on a route host.** Config delivery already resolves per `(account, app, environment)` (`appconfig.LoadMerged`, `syncexec.go`'s `groupKey`); there is no per-host config source, and adding one means a new precedence layer inside the box-side sync that every route's config passes through. Keying on the environment reuses the delivery axis exactly as it is. - **(ruling, James 2026-09-19) Teardown stays an explicit verb.** Deleting an instance or abandoning an environment never destroys a bucket. `provision.DeleteRecord` already "deliberately leaves the parked database and blobs untouched", and `objstore.Teardown` is only ever called from a door a person drove. Nothing in this change becomes the platform's first automatic destroyer of tenant data. §8 covers what stops an abandoned store from billing silently. ## 4. Wire shapes **(decision) The default store does not move.** With no `--environment`, every key, name and delivery target is byte-for-byte what it is today: `apps///store.json`, bucket `-`, user `carlos-store--`, delivered to `config.json`. Existing records, buckets and credentials are untouched, and every existing test stays green unchanged. **(decision) An environment store is a sibling collection.** apps///store.json the app's own store (unchanged) apps///stores/.json one per named environment Nothing in the bucket nests keys under an instance today; the nearest precedent is `apps///sidecars/.json`, a flat sibling collection, and this follows it. **(decision) The record shape is `objstore.Record`, unchanged**, with one additive field: ```go // Environment is the named config bundle this store belongs to. "" is the // app's own store — the only shape that existed before 2026-09-19 — and is // omitted from the JSON, so every record written before this field existed // parses as the app store it is. Environment string `json:"environment,omitempty"` ``` Like `App`, it is stamped by `Put` from the key it is written to and never trusted from a caller: a record that lies about its own identity is refused, not guessed at. **(decision) Names gain the environment as a third component.** MintedBucket: -- UserName: carlos-store--- > **Implementation gate.** Both are bounded before this lands. S3 bucket > names are at most 63 characters and IAM user names at most 64; account, > app and environment names are each already bounded by their own > validators. The walk must refuse a combination that cannot be named, > **before** any backend call, with a sentence naming which part is too > long — never a half-provisioned estate discovered at `CreateBucket`. **(decision) Delivery targets the environment's bundle.** `writeEnv` calls `appconfig.ApplyEnvironment(..., env, ...)` when `Environment != ""`, and `appconfig.Apply` otherwise. The variable names are unchanged (`_BUCKET`, `_REGION`, `_ENDPOINT`, `_ACCESS_KEY`, `_SECRET_KEY`), and the two key halves are still sealed before they are written. Because a named bundle overlays the default **per key**, an app with both an app store and an environment store delivers the environment's coordinates to instances bound to it and the app's to everyone else. That is the overlay doing exactly what it already does. ## 5. What a member sees ``` carlos store create --app a --environment e [--env-prefix P] carlos store status --app a [--environment e] carlos store grant --app a --environment e (operator, unchanged gates) carlos store rotate --app a --environment e [--finish] ``` **(ruling, James 2026-09-21) There is no `carlos store destroy`.** An earlier draft of this section listed one. Destroying a store stays what it already was: a console action behind the heaviest ceremony in the platform — owner role, CSRF, the bucket name typed exactly, and fresh passkey auth. A CLI verb cannot ask for a passkey, so adding one would have quietly traded that ceremony for a bearer token on the one action with no trash and no undo. The console's destroy door takes the environment instead (§8), and the settings page grows one card per environment store, each with its own confirm-and-destroy form. **(decision) `status` with no `--environment` lists every store the app has**, the app's own first, then one line per environment: its bucket, its state, and which instances read it. An app with only the default store prints what it prints today. **(decision) The doors refuse what cannot be delivered.** A store for an environment name that fails `appconfig.ValidEnvironmentName` is refused. An environment bundle that does not exist yet is **not** refused: delivery creates it, matching "an environment does not require a default to exist". **(decision) Sharing is visible, not refused.** Two instances bound to one environment share its store, exactly as they share its config. `status` says which instances read each store, so a shared bucket is something you can see rather than something you discover. ## 6. Authority Unchanged in every respect. `apiAccountApp` first, then the operator allowlist for `grant`, then the write budget, then the backend gates. The account entitlement remains the single approval: **in an approved account, creating an environment store provisions it with no new approval**, which is what lets a signup path mint one without an operator in the loop. **(decision) The walk lock key gains the environment** — `acct + "/" + app + "/" + env`. Two stores of one app share no estate: their buckets, users and delivered bundles are all distinct, so serialising across them would cost concurrency and buy nothing. That is the same argument `storelock.go` already makes for two apps in one account. ## 7. The purge guard `release.PurgeApp` today does one `Get` of `store.json` and refuses the purge while `grant` or `provisioned` is present. With environment stores it must also **list `apps///stores/`** and apply the same test to every record there. **This is the one change in this spec that is a safety property rather than a feature.** A miss means an app is auto-purged while a bucket, an IAM user and a live access key survive with nothing naming them. It lands in the first slice, with the test that proves it, and `release` keeps hand-parsing the two frozen field names rather than importing `objstore`. ## 8. Metering, and the abandoned store **(decision) v1 sums every store of an app into the app's existing usage key.** `storagePass` gets one `Get` per app today and writes `day.Apps[acct+"/"+app]`. It will instead read the app store plus every record under `stores/`, size each, and sum. The `StorageDay` wire shape does not change, so nothing downstream of usage has to move, and an account's bill is the same arithmetic over more buckets. Per-store attribution — what discussion #140 asks for — is a later, additive change to the usage shape, and this spec deliberately does not make it. **Because teardown is explicit (§3), an abandoned environment store keeps billing.** Three things make that visible rather than silent: - `store status` lists every store of the app, including ones no instance reads. An orphan is on the screen. - A store whose environment no bundle and no instance references is named in `status` as **unused**, in those words. - The purge guard (§7) refuses to auto-purge the app while any store is provisioned, so the estate cannot be forgotten by being trashed. ## 9. Scanning **(decision) v1 scans the app's own store only.** `Sweep` is `(acct, app)`-scoped and keeps one cursor per app (`internal/objstore/sweep.go`), and the scan-result namespace is the same pair. Extending both is real work with no bearing on whether a tenant can have its own bucket. What this spec will not do is let the surface lie about it: `store status` on an environment store says scanning is not available for it yet, and the scan doors refuse `--environment` with that sentence rather than silently scanning the wrong bucket or claiming a clean result. ## 10. Slices 1. **The primitive.** ✅ `Environment` on the record, the key layout, the name builders and their length refusal, `writeEnv`'s delivery target, and the walks threaded through. Plus §7's purge guard and its test. 2. **The doors.** ✅ `?environment=` on create/status/grant/rotate/finish, the lock key, `storeReply` carrying the environment, and the refusals of §5. 3. **The CLI and the docs.** ✅ `--environment` on create/status/grant/rotate, `status` listing every store, and `docs/site/cli.md`. 4. **Teardown.** ✅ The account-teardown path covering every store, and the console destroy door taking an environment — with one card and one confirm-and-destroy form per environment store on the settings page. No CLI verb, per §5's ruling. 5. **Metering.** ✅ §8's sum, over the app's own store and every environment store, into the one existing usage key. **Not built, and named here so it is not mistaken for shipped: §8's "unused" line.** `store status` lists every environment that has a store, which is where an abandoned one becomes visible — but it does not say whether any INSTANCE is bound to that environment, which is what "unused" claims. Doing it properly means reading the app's instance records and joining on their environment binding, and a line that said "unused" from anything less would be a guess about a live bucket. The listing is the honest half; the join is follow-up work. ## 11. Non-goals - Per-host stores. §3's ruling. - Automatic teardown of anything. §3's ruling. - Per-store usage attribution (#140). - Scanning environment stores (§9). - Any change to the account entitlement, its approval queue or its notifications.