Make the vision judge discover cost-effective OpenRouter models
Add openrouter_judge.py to rank live image+JSON models by advertised price, with ordered fallbacks, auth-error handling and auto selection. Wire both judges to --model auto, record the model used, and add fake-catalogue/mocked-call tests. Re-run the judges with the valid key: 13/13 screen designs match, 0 diff; the Forward+ A/B report records the selected model and ~$0.006 spend. Headless 231/231, Python 94/94.
This commit is contained in:
1 parent
28f2a48811
commit
659b0e35b2
15 files changed
+535
-65
No files matched your search
@@ -4,6 +4,25 @@ All notable user-facing changes to Gig Line. The format follows
|
||||
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and versions use
|
||||
Semantic Versioning.
|
||||
|
||||
## [0.7.2] — 2026-11-01
|
||||
|
||||
### Added
|
||||
|
||||
- Automatic OpenRouter model selection for the screenshot judges: the cheapest
|
||||
currently available image+JSON-capable model is used, with ordered fallbacks
|
||||
and a pinned `GIG_LINE_JUDGE_MODEL`/`--model` override.
|
||||
|
||||
### Changed
|
||||
|
||||
- The judge reports the model that produced each verdict and fails clearly on an
|
||||
invalid API key instead of retrying other models.
|
||||
- The screen-design and realism reports now include the fresh judge results.
|
||||
|
||||
### Fixed
|
||||
|
||||
- The blocked screenshot judge now runs with the valid `OPENROUTER_API_KEY`
|
||||
(`13/13` screen designs match, `0` diff).
|
||||
|
||||
## [0.7.1] — 2026-11-01
|
||||
|
||||
### Added
|
||||
|
||||
Reference in new issue
Block a user