feat(qa): reference-vs-render iteration loop; screenshots are demo-only
Tools/visual_loop.py:
- --verify gates every capture against its expectation in Tools/visual_targets.json
(missing, blank, tone-less or wrong-size shots fail), so 'each screenshot shows
what was expected' is checked rather than assumed.
- --compare measures luminance, RMS contrast, edge density, palette distance and a
3x3 luma grid against the reference photograph, with a 0-1 similarity score.
- --iteration N writes Art/reference/iterations/N/{metrics.json,feedback.md}; the
feedback names the skill to consult (blender-modeler, archviz, materials, lookdev,
camera-cinematography), so the next modelling pass starts from a measurement.
- Iterations are gitignored until the owner confirms a shot; only confirmed shots
are promoted into Screenshots/.
Screenshots/ holds demo-map captures only now: render_screenshots.sh renders just
DemoMap, collects the shots and runs the verification gate (--verify); the review
staging map is no longer rendered for review and its PNGs are removed. The README
is rewritten around the four demo shots with per-shot expected content and a
verified column, and no longer claims the review staging set.
Tests: 67 pass (13 new) covering metrics on synthetic images, comparison scoring,
the feedback rules and the target spec (including that every target is a demo shot
and that iterations are gitignored).
This commit is contained in:
1 parent
ad5bb25009
commit
bd1f8a9c2e
25 files changed
+688
-121
No files matched your search
+3
-1
@@ -56,7 +56,9 @@ Geographic tools live in [`Geo/tools/`](../Geo/README.md) and the Blender-side b
|
||||
| Tool | Purpose |
|
||||
| --- | --- |
|
||||
| `render_screenshots.py` / `render_screenshots.sh` | Rebuilds the review map, runs the game, captures the shot list into `Screenshots/` |
|
||||
| `judge_screenshot.py` | Vision-model pre-check: judges a screenshot against an expected-scene description (`--image` + `--expect`, or `--shots` for every shot in the README). Reads `OPENROUTER_API_KEY` from `.env` (ignored). Optional helper — a human must still approve. |
|
||||
| `judge_screenshot.py` | Vision-model pre-check: judges a screenshot against an expected-scene description (`--image` + `--expect`, or `--shots` for every shot in the README). Reads `OPENROUTER_API_KEY` from `.env` (ignored). Optional, advisory only — the model is unreliable on this content and never replaces a human verdict. |
|
||||
| `visual_loop.py` | The reference-vs-render loop. `--verify` gates each capture against its expectation in `visual_targets.json` (blank/short/wrong-size/tone-less shots fail); `--compare` measures luminance, contrast, edge density, palette and 3×3 composition against the reference photograph and turns the deltas into modelling actions naming the skill to consult (`blender-modeler`, `archviz`, `materials`, `lookdev`, `camera-cinematography`); `--iteration N` writes the artefacts under `Art/reference/iterations/N/`, which is **gitignored until the owner confirms a shot**. |
|
||||
| `visual_targets.json` | Per-shot expectation and reference photograph for `visual_loop.py`. Every entry must be a demo-map shot. |
|
||||
|
||||
## Gameplay QA
|
||||
|
||||
|
||||
Reference in new issue
Block a user