# 🤖 The public-web fetch service (`lchat fetch run`)

A shared, identity-free service that fronts **public web content** for
clients: OpenGraph unfurls and a camo-style signed image proxy for the cards'
images. One binary subcommand (`fetch.go`), one sqlite file, fronted by the
instance router like links/push/home. Born in the unfurl-through-Finland
discussion (PRs #1237/#1240 → this): if a service is going to see posted
URLs at all, it should be a service that knows *nothing else*.

## Why it exists

Unfurling is the messenger's one documented E2E exception: to build a link
card, *some* server must see the URL. Today that server is the member's own
instance. Moving the fetch to the shared service buys, in rough order of value:

1. **The instance can stop seeing URLs entirely** (once clients call the service
   directly — the phase-2 flip below). The instance server then stores and
   relays only ciphertext, full stop; the unfurl exception moves to a service
   that holds no members, threads, tokens, or instance list.
2. **Viewers stop leaking their IPs to card-image hosts.** The web client
   sets `img.src` to the raw `og:image` URL today, so every *viewer* of a
   card fetches from a third-party host; `http:`-only images die silently as
   mixed content. Its `/img` proxy fixes both.
3. **Self-hosters stop exposing their egress IP.** A member can post a link
   to a server they control and learn the instance's IP — for a home
   self-hoster, their home IP. Pointing unfurl at the shared service hides it; running
   their own keeps full autonomy.
4. **A shared warm cache.** Popular links unfurl once per language across
   every space that uses the service, and the poster/recipient in-flight window
   mostly disappears because the cache is usually already warm.

## Trust model

- **Sees:** posted URLs + an `Accept-Language` tag, transiently, attached to
  nothing — no token, no instance id, no member id. Behind the router it
  doesn't even see client IPs (nothing forwards them, deliberately).
- **Stores:** the sealed preview cache — same convergent-encryption scheme as
  the instance's (`unfurl.go`): row key = salted hash of language+URL, blob
  opens only with a key derived from language+URL, service-wide salt (minted
  on first boot, lives only in the service's DB) defeats precomputed
  dictionaries. The cache at rest reveals nothing a reader doesn't already
  know the URL for. No request logs; the only log line is startup.
- **Never:** member content. Member photos/files are encrypted client-side
  (`processImage` + `encryptBytes`) and are categorically out of scope —
  its remit is content already on the public web. Any future endpoint that
  would touch member bytes is a design error; say no.
- **Not an open proxy:** `/img` serves only URLs the service itself HMAC-signed
  while building a card (GitHub-camo shape), so third parties can't drive it
  as a generic fetcher. Signing is at response time; the cache stores raw
  URLs, so the key could rotate without invalidating cached cards.
- **SSRF:** every outbound dial (unfurl page + image, every redirect hop)
  re-validates the resolved IP via `guardDialControl` — same guard as the
  instance, links, and push.

## Abuse posture — hashcash admission (shipped, off by default)

Set `LCHAT_FETCH_POW_BITS` (0/unset = off) and `/api/unfurl` + `/api/gif`
demand proof of work — home's space-creation scheme (`fetchpow.go`, kept in
step with `home/pow.go`): a 401 hands out a sealed single-use challenge,
the client grinds SHA-256 to that many leading zero bits ONCE, the solved
request is served and answered with a sealed ~1h **ticket**, and every call
after rides (and is rate-bucketed by) the ticket — one solve per client per
hour, never per unfurl. Admission travels in query params so cross-origin
callers pay no CORS preflight; `/img` needs none (its HMAC signature is the
admission). Both web (`fetchServiceGET`, shell.js) and native
(`LChatAPI.fetchServiceGET` + the `FetchService` registry) solve and hold
tickets. Ungated, the per-IP buckets remain — which behind the router
collapse into global buckets (no forwarded IPs — a privacy feature): arm
the gate before the shared service is load-bearing for the hosted fleet.

## Deployment

Standard sibling-service shape: `lchat fetch run` behind the router on a unix
socket (`-socket /run/lchat/fetch.sock`), DB at `/data/fetch.db`
(`LCHAT_FETCH_DB`), service route + `/etc/hosts` pin like cloud/home. Not
deployed anywhere yet; when it ships, the planet-scale guide gains its row.
No secrets in env — the salt and signing key mint themselves into the DB on
first boot.

## The client flip (shipped, env-gated)

`LCHAT_FETCH_URL` on an instance routes its clients here: the chat page CSP
grows the origin in `connect-src` + `window.__fetchURL` injects it (web), and
`fetch_url` rides `/api/group`'s member-gated read (native — `FeatureGates`,
latched so a failed read never reroutes). Both platforms prefer the service
with instance-unfurl fallback on transport failure. Self-host default stays
*local fetch* — a self-hosted instance silently shipping its members' URLs to
our shared service would betray the self-host story; pointing at the shared
service (or their own) is opt-in config.

## Resize (shipped)

`/img` re-encodes oversized JPEG/PNG card images down to card width
(`shrinkCardImage`: >1600px wide → 1200px, CatmullRom, same format) — an
og:image is routinely thousands of px for a card rendering at a few hundred,
and every viewer pays that transfer. GIFs pass through (resizing freezes the
animation), as does anything that fails to decode, claims bomb-scale pixel
dims, or comes out bigger "shrunk" — polish never breaks a card. Public web
images only; member photos keep their client-side pipeline
(`processImage`), always.

## Giphy (`GET /api/gif`)

The authenticated-upstream sibling: search/trending fronted with the
**platform's** Giphy key (`LCHAT_GIPHY_KEY` in the unit's env file — the one
env secret this service has), so a space needs no key of its own and the
get-an-API-key wall disappears. The response is Giphy's own page shape with
every giphy-hosted media URL rewritten to the signed `/img` proxy, so the
browser never talks to Giphy: previews and the posted GIF's bytes all ride
this service's origin, then encrypt client-side and post as ordinary
attachments exactly as before.

Honest trade vs the old path: direct client→Giphy meant *we* saw nothing and
Giphy saw the member's IP + query; via the service, the service sees search
terms (identity-free, unlogged) and Giphy sees only the service. No key
configured → `/api/gif` 404s and the client falls back to the space's
bring-your-own-key path (docs/app-credentials.md), which remains the
self-host/override story.