Skip to content

feat(map): satellite and hybrid imagery — open sources, per-cell archives in R2, the style hook (nothing published) - #200

Merged
NateIsern merged 7 commits into
mainfrom
feat/satellite-imagery
Oct 13, 2026
Merged

NateIsern merged 7 commits into
mainfrom
feat/satellite-imagery

Conversation

@NateIsern

@NateIsern NateIsern commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

Satellite and hybrid maps for GoWay, built only from imagery whose licence allows hosting, re-serving and displaying publicly and commercially. Each source is credited while it is on screen. Nothing is published: the Worker route is off until MAP_SATELLITE_MANIFEST is set, and no R2 object or Cloudflare token was created. Full write-up: docs/MAP_SATELLITE.md.

Phase 1: sources (licences quoted with URLs in the doc)

  • Global base (chosen):
    • z8–13: ESA WorldCover 2021, the Sentinel-2 yearly median at 10 m, CC BY 4.0. It sits on AWS Open Data (9.18 TB, public).
    • z0–7: NASA Blue Marble NG, August 2004, public domain.
  • Rejected for the base:
    • EOX cloudless 2018–2025 is CC BY-NC-SA. The 2016 edition is CC BY but has no bulk download.
    • Landsat is 30 m.
    • The upgrade path is CDSE quarterly mosaics or our own Sentinel-2 composite built in us-west-2, at about $300–600 a year.
  • National and regional, usable:
    • Spain PNOA (COG per sheet), Catalonia ICGC (yearly), France BD ORTHO (Etalab).
    • Netherlands, Switzerland, Austria, 15 German states, Czechia, Poland, Denmark, Wallonia, Brussels, Luxembourg.
    • US NAIP (public domain), New Zealand LINZ (S3), Massachusetts, North Carolina, NYC, DC, Québec, Hong Kong, São Paulo.
  • Ask the publisher first: Japan GSI (no bulk download), Flanders, Portugal OrtoSat, Ontario, Uruguay.
  • No: Italy (AGEA), Ireland, Norway, UK national, Mecklenburg-Vorpommern, most Australian states, Google, Apple, Bing, Esri, Mapbox, Maxar, Planet.

Phase 2: pipeline and serving, proven on Catalonia

  • Tiers: base z0–13, detail z12–18, fine z19–20. Each tier is one raster layer, and tiles are 512 px WebP with alpha.
  • Cells: each set is cut into cells, one dated PMTiles archive per cell plus a receipt. The manifest lists each set's cells as a coverage tree, so the Worker computes the archive for a tile instead of looking it up.
  • Worker /map/satellite/*:
    • Per tile, the first set in priority order that has the tile answers. A 404 means the tier below shows through.
    • It reads through tiles.js, the shared directory memo from Worker: send tiles gzipped as stored, one storage read per tile #192.
    • WebP goes out exactly as stored, with no Content-Encoding. check:worker-wire now proves this in workerd: four clients, cache miss and hit.
    • The parsed manifest has its own 1 MiB line in budget.js, and the parse refuses a manifest larger than that.
  • Encoding, measured: WebP q80 gives 63 KB a tile at 36.0 dB. AVIF is 9–17% smaller but 64% slower to encode, and it doesn't decode on Android before 12 or iOS before 16. WebP chosen.
  • Styles, in feat(map): one style engine — Standard, Streets, Light and Dark, in day, dawn, dusk and night, with a layers picker #197's one style engine: satellite and hybrid are catalogue entries with one fixed day light and their own themes.ts recipes. buildGowayMapStyle applies applyImageryStyle, and the imagery rasters join the published layer ids.
    • Hybrid keeps roads (thinned), boundaries and labels with dark halos.
    • No 3D buildings over imagery: the orthophotos are not true orthophotos. 3D terrain drapes the imagery, with no hillshade.
    • Picker thumbnails are rendered from the Catalonia proof. The two styles are labelled in all 12 languages.
    • Both carry offer: 'development': the picker and /frame?style= offer them only in development builds until the imagery is published. Publishing is then a one-line removal.
    • There is no parallel prop, registry or URL. Light, dark and every other document are unchanged except the id list in their metadata.
  • Water: WorldCover's water is made transparent with a VRT mask band (NDWI > 0 and NIR < 0.08). Coasts then meet the style's ocean without the stepped edge the composite's own water mask drew. A NIR-only threshold left speckles at sea, and NDWI alone punched holes in blue and white port roofs; the combined rule avoids both.
  • Credit: useImageryCredits reads /map/satellite/coverage.json. The compact ⓘ credit names every source on screen, and /attribution#imagery quotes each notice in full. A pitched view whose bounds can't be read over-credits rather than under-credits.
  • Workflow: build-satellite-tiles.yml dry-runs by default. With publish, it uploads dated keys through city-r2.sh, which is now prefix-generic.

Local proof:

Set Result
Blue Marble z0–7 104 MB
WorldCover z8–13 over Catalonia 9 cells, 176 MB
ICGC 2025 z13–18 over central Barcelona 13 cells, 15,253 tiles, 774 MB

Each cell is verified byte for byte. It was served by serve-map-build.ts --satellite running the Worker's own module. Screenshots satellite-01…09 are in goway-videos/.

Costs

  • Base for the whole planet: about 0.86 TB, about $13 a month on R2. Builds run on free runners, about 1,300 runner-hours.
  • ICGC for all of Catalonia: about 106–257 GB, $1.6–3.9 a month, about 54 runner-hours.
  • Base + Spain + France: about 4.6–10 TB, $69–150 a month.
  • All 16 ranked sources: about 18–42 TB, $260–630 a month.
  • R2 egress is free. Publish commands are in the doc.

Decisions for the owner

  1. Approve publishing the base set.
  2. Pick the upload path: wrangler's 300 MiB per object, or an R2 access key for multipart. I created no key.
  3. Pick Catalonia's source. ICGC's server stopped answering this machine after a parallel read burst. Recommended: PNOA's per-sheet COGs for Spain now, and ICGC's yearly set from a whole-file download later.
  4. Decide on z19 for the 5–12 cm sources (4× the bytes).
  5. Licences to ask the publishers about: GSI, Flanders, OrtoSat, Ontario, Uruguay.
  6. 3D buildings over imagery: off for now; decide whether to add them.
  7. After publishing, remove offer: 'development' from the two catalogue entries.

Checks

Check Result
Frontend tests 1,748 pass
check:worker-wire pass
tsc / bun check pass
Biome / ESLint pass
map:style:check pass
validate:agents-md pass

frameBudget's terrain case is flaky under machine load. It has also failed on clean main.

🤖 Generated with Claude Code

NateIsern and others added 7 commits October 13, 2026 02:42
…ifest

Imagery is published as sets (one source at one vintage) cut into cells, one
dated PMTiles archive per cell; a manifest lists each set's cells as a
coverage tree, in priority order, so the archive for a tile is computed.

- lib/map/imagerySources.ts: the licence registry (ESA WorldCover 2021,
  NASA Blue Marble, ICGC). Only imagery GoWay may host, re-serve and show
  commercially; the build refuses anything else.
- scripts/build-satellite-set.ts: per cell, warp in parallel strips
  (retried whole), gdal raster tile (512 px WebP q80), pack (streaming
  PMTiles writer sharing the city writer's directories), verify every tile
  with the Worker's reader, receipt. --plan prints the workflow matrix.
- scripts/build-satellite-manifest.ts: merges entries; a rebuilt cell moves
  to the new stamp and the old keys are listed as retired.
- worker/satellite.js: /map/satellite/{base,detail,fine}/z/x/y.webp,
  coverage.json, manifest.json, receipts. Reads through tiles.js (shared
  directory memo); WebP stored as is goes out with no Content-Encoding;
  1 MiB line in budget.js; check:worker-wire now covers it. Off until
  MAP_SATELLITE_MANIFEST is set (commented out).
- .github/workflows/build-satellite-tiles.yml: dry run by default; publish
  uploads dated keys through city-r2.sh (now prefix-generic).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…while on screen

- lib/map/style/imagery.ts: buildImageryStyle derives goway-satellite.json
  (imagery alone) and goway-hybrid.json (imagery under GoWay's roads and
  labels, fills dropped, roads thinned, dark halos) from the dark document;
  map:style:check keeps them in step; light/dark byte-identical.
- No 3D buildings over imagery (territorial orthophotos are not true
  orthophotos); the relief drapes the imagery on 3D terrain, no hillshade.
- MapCanvasProps.imagery ('satellite' | 'hybrid' | null), both canvases;
  resolveMapStyleUrl(appearance, imagery); dev-only ?imagery=.
- useImageryCredits: each source's credit in the compact ⓘ credit while a
  set of it is on screen; /attribution#imagery quotes every notice.
- An imagery tier's 404 (nothing finer published) never raises "Map data
  unavailable".
- serve-map-build.ts --satellite=<dir>: the Worker's own satellite.js over
  a local directory, for a local proof.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…planet plan

docs/MAP_SATELLITE.md: the open imagery source table (licences quoted with
URLs: global, Europe, rest of the world, and what is excluded), the tiers,
cells and manifest, WebP measured against AVIF, the style hook and the
credit, the local Catalonia proof, publish commands and costs, the ranked
planet rollout and the owner's decisions. AGENTS.md: the imagery rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ean cleanly

The composite keeps near-shore water (a grey-blue once stretched) and masks
the open sea in blocks, which drew a stepped coastline against the style's
ocean. A VRT mask band from the composite's own green and NIR bands —
NDWI > 0 and NIR < 0.08 — makes every sea, lake and wide river transparent,
so it is the one ocean colour at every zoom. NDWI alone also caught blue and
white port roofs; the NIR ceiling keeps them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ts, never under-credits

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e style engine

#197's catalogue takes imagery as two styles, so the parallel hook goes:
no `imagery` prop, no `goway-{satellite,hybrid}.json`, no second style URL.

- catalogue.ts: `satellite`, `hybrid` (one fixed `day` light, `imagery`,
  `offer: 'development'` until the imagery is published; picker and embed
  ask `isMapStyleOffered`).
- themes.ts / options.ts: their recipes, Standard's options plus `imagery`
  over the night palette; `buildGowayMapStyle` hands them to
  `applyImageryStyle`. The imagery rasters join the published layer ids.
- Labels in every language; thumbnails rendered from the Catalonia proof
  (`map:thumbs --only=`, so the others are untouched).
- Canvases read the style's `imagery` from the catalogue for the credit;
  relief and the native hillshade keep both Outdoors' and imagery's rules.
- check:worker-wire: main's bundled Worker plus the satellite clients.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#202 moved the catalogue to packages/contracts (the ids the SDK and the
embed share). Satellite and hybrid join it there with `imagery` and
`offer: 'development'`; `/map/styles/index.json` lists only offered styles
until the imagery is published, and the app's picker and embed ask
`isMapStyleOffered`. Navigation and Monochrome kept as main has them; the
imagery documents are byte-identical, so their thumbnails stand. The SDK's
"unknown style" fixture uses an id that is still unknown.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@NateIsern
NateIsern merged commit ba8db99 into main Oct 13, 2026
13 checks passed
@NateIsern
NateIsern deleted the feat/satellite-imagery branch October 13, 2026 01:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant