From ea3860a60c80e6b21f5862e2af90b6fc6964e56d Mon Sep 17 00:00:00 2001 From: Zwitschi Date: Tue, 3 Nov 2026 16:31:42 +0100 Subject: [PATCH] Defer Gaussian splatting and re-plan maintenance --- TODO.md | 227 +++++++++++++++++++++++++++++++------------------------- 1 file changed, 127 insertions(+), 100 deletions(-) diff --git a/TODO.md b/TODO.md index 460d5de..8ab41a1 100644 --- a/TODO.md +++ b/TODO.md @@ -450,96 +450,6 @@ facade evidence is still the 2009 CC BY frame. The 8 downloaded candidates are i for shape and the generated board only informs colour, mood and detail density. Human review stays the acceptance gate, and AI output is never treated as licensed map data. -### Gaussian splatting evaluation: photo intake and provenance - -Gaussian splatting is evaluated as a photo-driven pipeline: licensed or permissioned photos -become a reference, texture source, mesh proxy or distant backdrop. It is an unversioned local -track with no runtime dependency or release gate until the ADR closes it, and it extends the -reference-image licence and privacy gates above rather than replacing them. - -- [ ] Define the accepted photo channels: project/team captures, volunteer or user-supplied - photos with written permission, and downloaded images from clearly licensed archives - (Wikimedia Commons, Openverse, Mapillary and similar). Record which channel produced - each reconstruction. -- [ ] Write a one-page capture brief for user-supplied photos: overlapping arc or orbit around - the facade, 60–80% view overlap, 12 MP+ originals, fixed exposure/white balance, HDR and - filters off, no motion blur, no people or plates in frame where avoidable, overcast or - even light, and the facade height/doors/steps included in several frames. -- [ ] Build an intake checklist and provenance sidecar per photo set: contributor, consent or - property permission, licence and attribution text, source URL, capture date, camera/lens, - location, faces/plates/interiors present, redactions applied, file hashes and whether the - set is shipped or development-only. -- [ ] Set the licence gate: own photos need an explicit contributor licence; CC0/CC BY are - preferred; CC BY-SA triggers a share-alike review for derived textures and meshes; reject - Street View, social-media scrapes, unlicensed commercial imagery and AI-upscaled copies of - unknown originals. Attribution must be feasible in `CREDITS.md` before use. -- [ ] Set the privacy gate: blur or remove faces, minors, licence plates, screens, private - interiors and anything behind a closed door; obtain property permission where required; - keep a redacted copy as the reconstruction input and never commit the originals. -- [ ] Treat downloaded photo sets as sparse and of unknown intrinsics. Use them for reference, - palette and texture study, or as extra training views; do not expect a usable splat from - 10–30 casual web photos of a facade. -- [ ] Keep photos, masks, COLMAP databases, checkpoints and splats outside git; commit only - derived, licensed, provenance-stamped assets that pass the review gate. - -### Gaussian splatting: reconstruction and alignment - -- [ ] Pin a reproducible offline toolchain (for example COLMAP or GLOMAP for camera poses plus - gsplat, OpenSplat, Nerfstudio or Brush for training), with exact versions, GPU/VRAM/disk - needs and known failure modes recorded in the sidecar. -- [ ] Reconstruct two quality tiers: a dense 100–400-photo arc/orbit capture and a sparse - 10–30-photo downloaded set. Record camera-registration rate, training time, splat count, - file size and held-out PSNR/SSIM/LPIPS for both. -- [ ] Mask sky, people, vehicles and moving foliage before training; verify the masks do not - erase facade edges, signage or window detail, and keep the redacted input variant. -- [ ] Align the splat to game coordinates from the OSM footprint plus surveyed points (door - width, column spacing, step rise, storefront height) and record alignment residuals in a - Blender scene containing both the splat and the current venue geometry (`scene-assembly`). -- [ ] Judge quality from held-out views and a render from the game's day/night camera poses - (`camera-cinematography`, `lookdev`, `qa-review`), not from training views alone; reject - splats that only look correct from the capture path. -- [ ] Maintain a coverage map per block: which facades, corners and rooflines have usable views, - and which areas must fall back to the facade kit or stylized geometry. - -### Gaussian splatting: map enhancement paths - -- [ ] **Path A — Reference and QA (lowest risk):** render the splat from game camera poses and - compare it with the realism captures; derive dimensions, signage placement, facade rhythm - and material colours. No splat data reaches the runtime. -- [ ] **Path B — Texture derivation:** project splat renders **or reference photos** onto the - facade kit to produce albedo, normal, roughness and AO candidates; de-light and bake them - (`texture-workflow`), then verify UV scale, tiling and readability under the Compatibility - renderer. Deriving directly from a committed reference photo requires its licence to allow - derived works and a `CREDITS.md` entry; CC BY-SA sources stay out unless reviewed. -- [ ] **Path C — Mesh proxy:** extract a mesh from the splat or dense reconstruction, retopo and - dress it in Blender (`retopology`, `prop-artist`), assign LODs (`lod-pipeline`), and keep it - inside the venue triangle/material budgets with mesh-based colliders (`collision-proxy`). -- [ ] **Path D — Distant backdrop:** test a baked panorama, impostor or billboard generated from - the splat for skyline and opposite-street depth where the player cannot walk; measure file - size, loading and visible pop at the venue boundary. -- [ ] **Path E — User-photo supplement:** let a small supplied photo set improve one facade or - landmark (a signage patch, colour reference or hero detail) without rebuilding the venue; - define the minimum evidence and the same licence/privacy/review gate as a full capture. -- [ ] For every path record quality gain versus effort, licence compatibility of the derived - output, and whether the result may ship or stays development-only. Prefer the cheapest - path that closes the visible gap. - -### Gaussian splatting: runtime evaluation and decision - -- [ ] Probe Godot 4.7.2 Compatibility and Web support for splat rendering in a throwaway scene: - addon or importer availability, shader/sort cost, memory, load time and fallback behaviour. - Do not add a splat dependency to the release project during evaluation. -- [ ] Set hard limits before any runtime experiment: maximum per-splat file size and total - memory, allowed frame-time delta on the 24-NPC budget scene, acceptable Web load time, and - no compute-shader requirement for the Compatibility/Web path. -- [ ] Keep collision and navigation mesh-based even if a splat is displayed; a splat must never - become the playable floor or the access/occlusion authority. -- [ ] Write the ADR with a per-path go/no-go, the accepted licence/provenance rules, whether each - result ships or stays development-only, rejected alternatives and the evidence bundle. -- [ ] Exit: one licensed photo capture measurably improves a venue through a documented path and - passes visual, performance and provenance review; or the approach is rejected with - reproducible findings and removed from the release path. - ### QA and exit - [ ] Compare the data-backed block against the existing screenshots and map documentation; keep @@ -562,27 +472,47 @@ reference-image licence and privacy gates above rather than replacing them. Unversioned housekeeping that keeps the planning sources trustworthy. Do these when they block a task, not on a schedule. +**Status (2026-11-03):** v0.10.0-rc.1 source; three venues; 276 headless + 143 Python tests; the +Hollywood and Joke Shop facade-realism passes landed (`limestone_ashlar`, `dark_stucco`, banner +night emissive, marquee emissive test); reference boards (`hollywood`, `joke-shop`) with sidecars; +three-up boards; `tools/refboard.py` + reference-anchored generation (`--reference`, `anchor`) + +`tools/three_up.py`. Gaussian splatting is deferred to the end of this file. + ### Memory and status accuracy -- [ ] Trim `MEMORY.md` from ~970 lines to the durable set: drop the pre-0.8 host/disk/NSIS/release +- [ ] Trim `MEMORY.md` from ~1088 lines to the durable set: drop the pre-0.8 host/disk/NSIS/release anecdotes, superseded v0.2.x handoff status and duplicated release notes, keep the gotchas - that still bite (async/event wiring, import order, colliders, RNG, contracts), and keep the - easter-egg note. + that still bite (async/event wiring, import order, colliders, RNG, contracts, the Godot + `--import` cache gotcha, the image-recitation filter), and keep the easter-egg note. - [ ] Re-check the remaining memory claims against code before deleting them; `MAX_LIVE_NPCS` is still 16 for gameplay while the separate budget smoke spawns 24, so do not "fix" that. -- [ ] Fix stale status claims as they are found (the step-collider and flashlight-beam claims were - corrected on 2026-11-03); add a documentation check or grep note only if the same class of - drift returns. +- [ ] Fix stale status claims as they are found; add a documentation check or grep note only if + the same class of drift returns. +- [ ] Keep the Gap Analysis baseline and the README status line in sync when a test count, version + or feature milestone changes (the baseline was bumped to 276/143 on 2026-11-03). -### Reference and generated imagery upkeep +### Imagery and provenance upkeep -- [ ] Keep `docs/screens/refs/` sidecars, concept-board records and `CREDITS.md` in sync whenever an - image is added, replaced or dropped, including the model/prompt for generated boards. +- [ ] Keep `docs/screens/refs/` sidecars, concept-board records and `CREDITS.md` in sync whenever + an image is added, replaced or dropped; photo-anchored concept frames must keep their + `.refgen.json` provenance sidecar (source photo, licence, page, model) next to the frame. - [ ] Run `python3 tools/check_docs.py` after any gallery or reference-board edit so every tracked image stays linked from its folder README. +- [ ] Use `tools/refboard.py search/fetch/rows` for new Commons research instead of ad-hoc API + scripts; downloads stay under `tmp/` and only licence-cleared rows (commit/review gate) + reach a board after a sidecar entry. - [ ] Delete or ignore development-only photos and raw generations once a board is committed; the repository carries the picks and their provenance, not the working set. +### Godot import-cache hygiene + +- [ ] When a new committed texture (`assets/textures/...`, `picks/`) fails to import under the + existing `.godot/` cache, use the scratch-copy import (`rsync --exclude .godot ...` then + `godot --headless --import` and copy back the `.import` + `.godot/imported/*.ctex/.md5`); + never hand-edit the `uid=` in a copied `.import`. Record any new instance in MEMORY. +- [ ] Prefer `tools/scene_concepts.py promote` (or a manual move into `game_ready/v2/` plus a + `PICKS` entry) so new picks follow the same path as `limestone_ashlar` and `dark_stucco`. + ### Blender skills upkeep - [ ] Re-check the installed Blender pack (`version.json`) when updating it and record the version @@ -594,7 +524,7 @@ task, not on a schedule. Baseline: **v0.9.6** (2026-11-03). Three playable venues, tutorial/campaign/checkpoint resume, difficulty/help, audio/accessibility settings, a 24-NPC budget baseline, a complete -escalation/access/radio loop, shift review, the Blender asset library, 272 headless + 108 Python +escalation/access/radio loop, shift review, the Blender asset library, 276 headless + 143 Python tests, a 21/21 clean screen-design judge and an optional Forward+ Desktop HQ build. This is the baseline; 0.10.0-rc.1 is the current candidate source, with release approvals still open. @@ -826,3 +756,100 @@ order. Only a release-blocking defect from those tracks may enter this milestone - [ ] Swap the primitive prop builder for an image-to-mesh model API behind the same `assets/models/generated/.glb` contract. - [ ] Rare easter eggs/cameos (e.g. Bob Dobbs) only after core content and QA are stable. + +## Gaussian splatting evaluation (deferred) + +Postponed for further development on 2026-11-03. The local-map track ships the facade kit and +reference-anchored material passes first; this photo-driven pipeline stays an unversioned, +offline study with no runtime or release dependency until an ADR closes it. All licence and +privacy gates below remain in force whenever the track resumes. + +### Photo intake and provenance + +Gaussian splatting is evaluated as a photo-driven pipeline: licensed or permissioned photos +become a reference, texture source, mesh proxy or distant backdrop. It is an unversioned local +track with no runtime dependency or release gate until the ADR closes it, and it extends the +reference-image licence and privacy gates above rather than replacing them. + +- [ ] Define the accepted photo channels: project/team captures, volunteer or user-supplied + photos with written permission, and downloaded images from clearly licensed archives + (Wikimedia Commons, Openverse, Mapillary and similar). Record which channel produced + each reconstruction. +- [ ] Write a one-page capture brief for user-supplied photos: overlapping arc or orbit around + the facade, 60–80% view overlap, 12 MP+ originals, fixed exposure/white balance, HDR and + filters off, no motion blur, no people or plates in frame where avoidable, overcast or + even light, and the facade height/doors/steps included in several frames. +- [ ] Build an intake checklist and provenance sidecar per photo set: contributor, consent or + property permission, licence and attribution text, source URL, capture date, camera/lens, + location, faces/plates/interiors present, redactions applied, file hashes and whether the + set is shipped or development-only. +- [ ] Set the licence gate: own photos need an explicit contributor licence; CC0/CC BY are + preferred; CC BY-SA triggers a share-alike review for derived textures and meshes; reject + Street View, social-media scrapes, unlicensed commercial imagery and AI-upscaled copies of + unknown originals. Attribution must be feasible in `CREDITS.md` before use. +- [ ] Set the privacy gate: blur or remove faces, minors, licence plates, screens, private + interiors and anything behind a closed door; obtain property permission where required; + keep a redacted copy as the reconstruction input and never commit the originals. +- [ ] Treat downloaded photo sets as sparse and of unknown intrinsics. Use them for reference, + palette and texture study, or as extra training views; do not expect a usable splat from + 10–30 casual web photos of a facade. +- [ ] Keep photos, masks, COLMAP databases, checkpoints and splats outside git; commit only + derived, licensed, provenance-stamped assets that pass the review gate. + +### Reconstruction and alignment + +- [ ] Pin a reproducible offline toolchain (for example COLMAP or GLOMAP for camera poses plus + gsplat, OpenSplat, Nerfstudio or Brush for training), with exact versions, GPU/VRAM/disk + needs and known failure modes recorded in the sidecar. +- [ ] Reconstruct two quality tiers: a dense 100–400-photo arc/orbit capture and a sparse + 10–30-photo downloaded set. Record camera-registration rate, training time, splat count, + file size and held-out PSNR/SSIM/LPIPS for both. +- [ ] Mask sky, people, vehicles and moving foliage before training; verify the masks do not + erase facade edges, signage or window detail, and keep the redacted input variant. +- [ ] Align the splat to game coordinates from the OSM footprint plus surveyed points (door + width, column spacing, step rise, storefront height) and record alignment residuals in a + Blender scene containing both the splat and the current venue geometry (`scene-assembly`). +- [ ] Judge quality from held-out views and a render from the game's day/night camera poses + (`camera-cinematography`, `lookdev`, `qa-review`), not from training views alone; reject + splats that only look correct from the capture path. +- [ ] Maintain a coverage map per block: which facades, corners and rooflines have usable views, + and which areas must fall back to the facade kit or stylized geometry. + +### Map enhancement paths + +- [ ] **Path A — Reference and QA (lowest risk):** render the splat from game camera poses and + compare it with the realism captures; derive dimensions, signage placement, facade rhythm + and material colours. No splat data reaches the runtime. +- [ ] **Path B — Texture derivation:** project splat renders **or reference photos** onto the + facade kit to produce albedo, normal, roughness and AO candidates; de-light and bake them + (`texture-workflow`), then verify UV scale, tiling and readability under the Compatibility + renderer. Deriving directly from a committed reference photo requires its licence to allow + derived works and a `CREDITS.md` entry; CC BY-SA sources stay out unless reviewed. +- [ ] **Path C — Mesh proxy:** extract a mesh from the splat or dense reconstruction, retopo and + dress it in Blender (`retopology`, `prop-artist`), assign LODs (`lod-pipeline`), and keep it + inside the venue triangle/material budgets with mesh-based colliders (`collision-proxy`). +- [ ] **Path D — Distant backdrop:** test a baked panorama, impostor or billboard generated from + the splat for skyline and opposite-street depth where the player cannot walk; measure file + size, loading and visible pop at the venue boundary. +- [ ] **Path E — User-photo supplement:** let a small supplied photo set improve one facade or + landmark (a signage patch, colour reference or hero detail) without rebuilding the venue; + define the minimum evidence and the same licence/privacy/review gate as a full capture. +- [ ] For every path record quality gain versus effort, licence compatibility of the derived + output, and whether the result may ship or stays development-only. Prefer the cheapest + path that closes the visible gap. + +### Runtime evaluation and decision + +- [ ] Probe Godot 4.7.2 Compatibility and Web support for splat rendering in a throwaway scene: + addon or importer availability, shader/sort cost, memory, load time and fallback behaviour. + Do not add a splat dependency to the release project during evaluation. +- [ ] Set hard limits before any runtime experiment: maximum per-splat file size and total + memory, allowed frame-time delta on the 24-NPC budget scene, acceptable Web load time, and + no compute-shader requirement for the Compatibility/Web path. +- [ ] Keep collision and navigation mesh-based even if a splat is displayed; a splat must never + become the playable floor or the access/occlusion authority. +- [ ] Write the ADR with a per-path go/no-go, the accepted licence/provenance rules, whether each + result ships or stays development-only, rejected alternatives and the evidence bundle. +- [ ] Exit: one licensed photo capture measurably improves a venue through a documented path and + passes visual, performance and provenance review; or the approach is rejected with + reproducible findings and removed from the release path.