Commit Graph
1 Commits
Author SHA1 Message Date
zwitschi abde390045 feat(tools): every image judged by two models, every generated texture drawn by two checkpoints
A single vision model has been wrong in both directions on this project - it missed real defects and
disputed frames that were correct - so agreement between different model families is worth more than one
confident opinion.

judge_screenshot.py now runs two models by default (openai/gpt-4o-mini + qwen/qwen3.7-flash, both served
on OpenRouter; claude-3.5-haiku is gone, which the first attempt discovered with a 404). combine()
reconciles them: agreement carries the verdict with the mean confidence, disagreement reports 'uncertain'
and keeps both reasons rather than letting the more confident model win. --model still forces a single
judgement, and every line prints which model said what.

comfyui_generate.py generates the same prompt and seed with two checkpoints, scores both by
high-frequency detail (mean absolute Laplacian, which catches blurry or flat results), keeps the loser as
<out>__<model>.png with its sidecar, and moves the winner onto --out. --alt-model  restores the single
run.

Tests cover the reconciliation - agreement, disagreement, unknown verdicts, and no verdicts at all - and the
suite is at 103.
2026-11-10 09:12:26 +01:00