Make the vision judge discover cost-effective OpenRouter models
Full build / Import + headless tests (push) Successful in 40s
Full build / Full desktop build + optional release (push) Successful in 4m6s

Add openrouter_judge.py to rank live image+JSON models by advertised price,
with ordered fallbacks, auth-error handling and auto selection. Wire both judges
to --model auto, record the model used, and add fake-catalogue/mocked-call
tests. Re-run the judges with the valid key: 13/13 screen designs match, 0 diff;
the Forward+ A/B report records the selected model and ~$0.006 spend. Headless
231/231, Python 94/94.
This commit is contained in:
zwitschi committed 2026-11-01 19:55:43 +01:00
1 parent 28f2a48811
commit 659b0e35b2
15 files changed
+535 -65

No files matched your search

+1
View File
@@ -44,6 +44,7 @@ Session memory for durable decisions, state, and gotchas. Keep concise, remove o
- Shift length is the scenario's `shift_duration_s` (900 s in all three venues). Tests and the screenshot tool override elapsed time; they do not change the default.
- RNG: NPC reactions and event scheduling are seedable for deterministic tests.
- **Vision judge (v0.7.2):** `tools/openrouter_judge.py` reads `/api/v1/models`, filters image-input models with `response_format`, and ranks by advertised prompt/image/completion prices. `--model auto` (default) uses the cheapest (`z-ai/glm-5.3-flash` on 2026-11-01) with fallbacks; explicit `--model`/`GIG_LINE_JUDGE_MODEL` wins. Auth errors abort, provider/model errors fall through. Full screen judge 13/13 match, 0 diff; A/B judge ~$0.006.
- **Desktop HQ preset (v0.7.1):** `packaging/desktop_hq/override.cfg` switches a
copied executable to Forward+; `Materials.forward_plus_active()` gates
volumetric fog/SDFGI and `light_volumetric_fog_energy`, and