// Package certwarm is the wire shape of the cert warm's durable outcome // records (#299 §3e): control/cert-warm/.json, written by whichever // box holds the issuer lease, read by the console. // // The record exists so that "the platform tried to get this hostname a // certificate and it is not working" can be SAID. Without it the warm's // attempt counts live only in the issuer's process memory: they reset on // every fleet roll and on every failover, so a domain that has been failing // for a week is indistinguishable from one attached ninety seconds ago, and // the console has no honest way to show anything but "issuing-cert" forever. // // One record per host, and ONLY while that host is failing. A successful // warm deletes the record, so absence is the healthy case and the prefix // holds nothing at all in a deployment where every domain is fine. Two // things follow, both load-bearing: // // - The console's read is cheap by construction. Listing this prefix costs // one object per BROKEN host, not one per domain, so folding it into the // domains list does not reintroduce the per-claim cost the per-host door // (#328) exists to avoid. // - "No record" is never evidence of success on its own. It is also what a // host that has never been warmed looks like, and what a wedged issuer // leaves behind. Readers must treat absence as "nothing to report", and // must age-bound what they do find — see Fresh. // // Unlike internal/edgesfile, this package owns the shape for BOTH sides // rather than mirroring a writer that lives elsewhere. The reasoning there // was that internal/host's edges-file writer was under heavy development and // re-plumbing it onto a shared type would have fought it for no gain. Here // writer and reader are being introduced together, in one change, so there // is no incumbent to fight and no reason to maintain two decoders of one // shape. package certwarm import ( "encoding/json" "fmt" "strings" "time" ) // Prefix is the flat prefix every record lives under. Inside control/, which // the fleet door's aclRead allowlist denies by default — see // TestDoorNeverServesCertMaterial, which pins this prefix with the cert // material it sits beside. const Prefix = "control/cert-warm/" // V is the wire version this build writes. A reader refuses V<1 (a record // with no version is not one of ours) and accepts anything higher, on the // same additive-wire rule the edges files use: a newer issuer may add // fields, and an older console must degrade to "no verdict" rather than // fail the whole fold. const V = 1 // Failure classes. Stable machine words — a client may branch on them, so // they are part of the wire and are never re-spelled. const ( // ClassRateLimited: the CA refused because the account's budget is // spent. Materially different from every other failure because it is // nobody's mistake and it will clear on its own, and because the wait is // long enough (12h) that a member deserves to be told rather than shown // a spinner. ClassRateLimited = "rate-limited" // ClassOther: everything else — a failed authorization, a CA outage, a // network error. Deliberately NOT subdivided: the CA's own text is // carried in Error, and inventing a taxonomy over strings we do not // control would go stale the first time the CA rewords a problem // document. ClassOther = "other" ) // Key is the bucket key for host's record. // // The host is used verbatim. Callers hand it the same normalized form the // registry and the edges files store — lowercased and punycoded — because a // record written under one spelling and read under another is a record that // silently does not exist. func Key(host string) string { return Prefix + host + ".json" } // HostFromKey is Key's inverse, for walking a listing. It returns "" for a // key outside the prefix or without the .json suffix, so a stray object // under the prefix is skipped rather than decoded as a hostname. func HostFromKey(key string) string { rest, ok := strings.CutPrefix(key, Prefix) if !ok { return "" } host, ok := strings.CutSuffix(rest, ".json") if !ok || host == "" || strings.Contains(host, "/") { return "" } return host } // Record is one host's warm outcome. Every timestamp is unix seconds. type Record struct { V int `json:"v"` Host string `json:"host"` // Attempts is how many consecutive warm attempts have failed since the // last success. It is the field the whole package exists for: it is // carried in the bucket rather than in memory precisely so that a // failover does not reset it to one and restart the clock on a domain // that has been broken for days. Attempts int `json:"attempts"` // FirstFailedAt is when the current run of failures began — preserved // across writes, so "failing since" survives an issuer moving. FirstFailedAt int64 `json:"first_failed_at"` // LastFailedAt is the most recent failure. LastFailedAt int64 `json:"last_failed_at"` // NextAttemptAt is the backoff floor: when the issuer will try again. // Carried because it is the difference between "we have given up" and // "we are waiting out a rate limit", which is the whole of what a member // wants to know, and because Fresh needs it (see there). NextAttemptAt int64 `json:"next_attempt_at"` // Class is one of the Class* constants. Class string `json:"class"` // Error is the CA's or the client's own text, trimmed to ErrorMax. Shown // to operators, never parsed. Error string `json:"error,omitempty"` // WrittenAt is when this record was written. A stamp without its age // cannot be judged — see Fresh. WrittenAt int64 `json:"written_at"` } // ErrorMax bounds the carried error text. A CA problem document can be long, // and an unbounded string from a remote party landing in a bucket object // that the console renders is exactly the shape that turns one bad response // into a large object written every hour. const ErrorMax = 512 // Grace is how long past its own next-attempt time a record stays // authoritative. // // The age bound exists because a wedged or stopped issuer keeps its records // sitting in the bucket, frozen: without a bound the console would keep // reporting "we tried 40 times and stopped" long after nothing is trying at // all, which is a worse lie than saying nothing. Judged against // NextAttemptAt rather than WrittenAt because the two differ by design — a // rate-limited host is deliberately not retried for twelve hours, and a // bound on write time alone would call that healthy silence "stale" every // time. // // The limit this accepts, stated plainly: an issuer that wedges immediately // after writing a rate-limited record is believed for up to fourteen hours. // Closing that would need a per-pass heartbeat object, which is a second // key and a second writer-table row to buy a sharper edge on an already // rare case. const Grace = 2 * time.Hour // Fresh reports whether this record still describes what the issuer is // doing. A stale record is not deleted and not an error — it is simply not // used as evidence, and the reader falls back to whatever it would have said // with no record at all. func (r Record) Fresh(now time.Time) bool { deadline := r.WrittenAt if r.NextAttemptAt > deadline { deadline = r.NextAttemptAt } if deadline == 0 { return false } return now.Before(time.Unix(deadline, 0).Add(Grace)) } // Parse decodes one record. func Parse(body []byte) (Record, error) { var r Record if err := json.Unmarshal(body, &r); err != nil { return Record{}, fmt.Errorf("cert-warm record: %w", err) } if r.V < V { return Record{}, fmt.Errorf("cert-warm record: unsupported version %d", r.V) } return r, nil } // Encode serializes a record, stamping the version and the written-at time // so no caller can forget either. func (r Record) Encode(now time.Time) ([]byte, error) { r.V = V r.WrittenAt = now.Unix() if len(r.Error) > ErrorMax { r.Error = r.Error[:ErrorMax] } if r.Class == "" { r.Class = ClassOther } return json.Marshal(r) }