Make the vision judge discover cost-effective OpenRouter models
Full build / Import + headless tests (push) Successful in 40s
Full build / Full desktop build + optional release (push) Successful in 4m6s

Add openrouter_judge.py to rank live image+JSON models by advertised price,
with ordered fallbacks, auth-error handling and auto selection. Wire both judges
to --model auto, record the model used, and add fake-catalogue/mocked-call
tests. Re-run the judges with the valid key: 13/13 screen designs match, 0 diff;
the Forward+ A/B report records the selected model and ~$0.006 spend. Headless
231/231, Python 94/94.
This commit is contained in:
zwitschi committed 2026-11-01 19:55:43 +01:00
1 parent 28f2a48811
commit 659b0e35b2
15 files changed
+535 -65

No files matched your search

+2 -3
View File
@@ -5,9 +5,8 @@ see `AGENTS.md` for the workflow and `MEMORY.md` for the decisions behind these
## Realism follow-ups (deferred)
- [ ] Re-run the screen-design judge for `12-joke-shop`, `13-studio-arrival` and the
Forward+ A/B when `OPENROUTER_API_KEY` is valid; the 2026-11-01 attempt failed with an
invalid-key HTTP 400.
- [x] Re-run the screen-design judge for `12-joke-shop`, `13-studio-arrival` and the
Forward+ A/B with the valid key: **13/13 match, 0 diff**; the HQ A/B is in the report.
- [ ] Capture a hooded/winter NPC close-up once a scenario spawns types 5-7 (the explicit
`--portrait-type` override is already in `capture_realism.gd`).
- [ ] Human-authored tree/hedge silhouettes: the whole scene now reads well except the foliage,