feat(tools): every image judged by two models, every generated texture drawn by two checkpoints
A single vision model has been wrong in both directions on this project - it missed real defects and disputed frames that were correct - so agreement between different model families is worth more than one confident opinion. judge_screenshot.py now runs two models by default (openai/gpt-4o-mini + qwen/qwen3.7-flash, both served on OpenRouter; claude-3.5-haiku is gone, which the first attempt discovered with a 404). combine() reconciles them: agreement carries the verdict with the mean confidence, disagreement reports 'uncertain' and keeps both reasons rather than letting the more confident model win. --model still forces a single judgement, and every line prints which model said what. comfyui_generate.py generates the same prompt and seed with two checkpoints, scores both by high-frequency detail (mean absolute Laplacian, which catches blurry or flat results), keeps the loser as <out>__<model>.png with its sidecar, and moves the winner onto --out. --alt-model restores the single run. Tests cover the reconciliation - agreement, disagreement, unknown verdicts, and no verdicts at all - and the suite is at 103.
This commit is contained in:
1 parent
bf7dfe1052
commit
abde390045
4 files changed
+158
-10
No files matched your search
+2
-2
@@ -69,7 +69,7 @@ Geographic tools live in [`Geo/tools/`](../Geo/README.md) and the Blender-side b
|
||||
| Tool | Purpose |
|
||||
| --- | --- |
|
||||
| `render_screenshots.py` / `render_screenshots.sh` | Rebuilds the review map, runs the game, captures the shot list into `Screenshots/` |
|
||||
| `judge_screenshot.py` | Vision-model pre-check: judges a screenshot against an expected-scene description (`--image` + `--expect`, or `--shots` for every shot in the README). Reads `OPENROUTER_API_KEY` from `.env` (ignored). Optional, advisory only — the model is unreliable on this content and never replaces a human verdict. |
|
||||
| `judge_screenshot.py` | Vision-model pre-check: judges a screenshot against an expected-scene description (`--image` + `--expect`, or `--shots` for every shot in the README). Reads `OPENROUTER_API_KEY` from `.env` (ignored). Judged by **two models by default** (`--models a,b`), reconciled: agreement carries the verdict, disagreement reports `uncertain` with both reasons. Optional, advisory only — never replaces a human verdict. | `--image` + `--expect`; `--model` for a single judgement |
|
||||
| `visual_loop.py` | The reference-vs-render loop. `--verify` gates each capture against its expectation in `visual_targets.json` (blank/short/wrong-size/tone-less shots fail); `--compare` measures luminance, contrast, edge density, palette and 3×3 composition against the reference photograph and turns the deltas into modelling actions naming the skill to consult (`blender-modeler`, `archviz`, `materials`, `lookdev`, `camera-cinematography`); `--iteration N` writes the artefacts under `Art/reference/iterations/N/`, which is **gitignored until the owner confirms a shot**. |
|
||||
| `visual_targets.json` | Per-shot expectation and reference photograph for `visual_loop.py`. Every entry must be a demo-map shot. |
|
||||
|
||||
@@ -94,7 +94,7 @@ Geographic tools live in [`Geo/tools/`](../Geo/README.md) and the Blender-side b
|
||||
| --- | --- |
|
||||
| `input_emulation.py` | Pure-stdlib `ctypes`/`fcntl` driver that creates a virtual keyboard+mouse on `/dev/uinput`. The desktop treats it as real hardware, so it proves whether OS input actually reaches the game. Needs `/dev/uinput` writable (ACL on this host). |
|
||||
| `input_test.py` | Injects a scripted sequence into the focused window: `--sequence move:2,look:3,jump:1,reset:1,pause:1,click`. |
|
||||
| `comfyui_generate.py` | Local ComfyUI driver: list checkpoints, run txt2img through the HTTP API, and write a provenance sidecar (model, prompt, seed, full graph). Local, human-triggered. | `models`; `txt2img --prompt ... --out <png>` | `python3 Tools/comfyui_generate.py models` |
|
||||
| `comfyui_generate.py` | Local ComfyUI driver: list checkpoints, run txt2img through the HTTP API, and write a provenance sidecar (model, prompt, seed, full graph). **Two checkpoints by default**: both are generated from the same prompt and seed, scored by high-frequency detail, and the better one takes `--out` while the loser is kept beside it. Local, human-triggered. | `models`; `txt2img --prompt ... --out <png> [--alt-model ...]` | `python3 Tools/comfyui_generate.py models` |
|
||||
| `comfyui_character_blueprint.py` | Generates a character blueprint sheet - front, side, back and head views plus a contact sheet - from one description, with fixed seeds and a JSON record. | `--desc`, `--out`, `--model`, `--size` | `python3 Tools/comfyui_character_blueprint.py --desc "lean pedestrian, navy tee, jeans" --out Art/characters/blueprints` |
|
||||
| `character_match.py` | Compares a rendered character against a blueprint: figure aspect, fill and coverage deltas, mean CIE76 colour distance, dominant-palette distance, and a side-by-side montage. | `--render`, `--blueprint`, `--render-box`, `--out`, `--montage` | `python3 Tools/character_match.py --render Screenshots/demo/review-demo-third-person-front.png --blueprint Art/characters/blueprints/front.png --render-box 0.42,0.46,0.56,0.97 --montage /tmp/gigline/match.png` |
|
||||
| `generate_character_textures.py` | Procedural skin-pore and hair-strand textures with sidecars; no model involved. | `--out` | `python3 Tools/generate_character_textures.py --out Art/textures/generated` |
|
||||
|
||||
Reference in new issue
Block a user