Add a vision-model screenshot judge to the QA tooling

`tools/screen_designs.py judge` sends each capture together with its concept
mockup to a vision model and asks whether the capture realises the same screen and
layout intent. This covers what pixel/perceptual metrics cannot judge.

- Prompt explicitly ignores art style, fidelity, fonts, and placeholder text, since
  the references are stylised mockups.
- Verdicts are classed match / review (mismatch below --fail-confidence, default
  0.6) / diff (confident mismatch, exits non-zero) / unknown; --json, --only,
  --dry-run, --model supported.
- Reads the repo .env for OPENROUTER_API_KEY without overriding real env vars.
- Default judge model google/gemini-2.5-flash-lite; openai/gpt-4o-mini verified too.

Tests/docs
- 7 new offline tests in tools/test_screen_designs.py (request payload embeds both
  images, JSON/fenced/parts/non-JSON parsing, threshold classification, .env
  loading); 30 Python tests pass.
- Live run: 3 captures match, 3 review, 0 confident diffs.
- README, TODO section moved to DONE, MEMORY. Version 0.2.15.

DONE.md

@@ -526,3 +526,60 @@ Completed 2026-10-31 (working tree). Headless **149/149**; `src/test/smoke_scena
       crowd spike (more rubberneckers, not only one `arrive` spawn).
 - [x] Building: "activate body camera" and "request reason" once those verbs exist, used in
       at least one authored event.
+
+## Distribution follow-up (host config)
+
+Completed 2026-10-31 (working tree). Headless **153/153**.
+
+
+Verified via API+host credentials (2026-10-31):
+
+- [x] Repo is **public** (`private:false`) and account has admin rights.
+- [x] Release `v0.2.1` now carries all five artifacts (portable binaries + NSIS installer +
+      AppImage).
+- [x] **Host setting applied**: `[service] REQUIRE_SIGNIN_VIEW = false` in `/etc/gitea/app.ini`
+      (backup written alongside), Gitea restarted. Verified anonymously: repo page 200, API 200,
+      and a release asset downloads with the full 50.8 MB AppImage.
+
+## Release pipeline follow-up
+
+Completed 2026-10-31 (working tree). Headless **153/153**.
+
+
+- [x] Fix `tools/publish_release.sh`: the `~/.git-credentials` branch never set `AUTH`,
+      so every tag build since v0.2.2 failed to publish.
+- [x] Add a `github.token` fallback in `.gitea/workflows/build.yml` (the repo has no
+      `GITEA_TOKEN` secret).
+- [x] Fall back to the checkout's git extraheader when no token/credentials exist (the
+      Gitea runner does not expose `github.token`).
+- [x] Uploads now fail loudly and the release is verified to carry every artifact
+      (a silent HTTP 4xx used to leave an empty release with a green job).
+- [x] Host disk expanded to 16G; v0.2.0 and v0.2.9 releases restored from the local artifacts.
+- [x] Host maintenance via `root@192.168.88.30`: apt cache cleaned and journald capped
+      (`SystemMaxUse=300M` in `/etc/systemd/journald.conf.d/99-size.conf`); ~7.6G free.
+- [x] Verified end to end: tagging v0.2.12 produced a release with all five artifacts
+      (AppImage, Linux, macOS, NSIS installer, Windows) from CI, with no repo secret.
+- [x] Backfill decision: v0.2.4–v0.2.8 stay superseded; their tags and source remain, and the
+      release page points at the current build. No rebuilds planned.
+
+## Open Polish
+
+Completed 2026-10-31 (working tree). Headless **153/153**.
+
+
+- [x] Keep `%` placeholders stable in `Strings.fmt` when adding keys (positional formatting).
+
+## Screen design references (QA tooling)
+
+Completed 2026-10-31 (working tree). 30 Python tooling tests pass; the live judge run classed 3 captures as match and 3 as review (concept mockups are more stylised), with no confident mismatch.
+
+
+- [x] `tools/screen_designs.py generate` writes one reference design per screen
+      (from the `docs/` descriptions) to `docs/screens/designs/`, with an offline
+      `--draft` wireframe mode and an OpenRouter `--ai` mockup mode.
+- [x] `tools/screen_designs.py compare` checks `docs/screenshots/*.png` against the
+      designs (aspect ratio, size, blank-frame, and layout/palette/detail drift).
+- [x] Vision-model judge for semantic screenshot comparison: `judge` sends the
+      capture + reference design to a vision model, classes verdicts as
+      match/review/diff with a confidence threshold, supports `--dry-run`, `--json`,
+      `--only`, and `--model`, and reads the repo `.env` for the API key.

MEMORY.md

@@ -21,8 +21,8 @@ Session memory for durable decisions, state, and gotchas. Keep concise, remove o
   Windowed control smoke: `godot --path . --script res://src/test/smoke_focus_radial.gd` → exit 0.
 - `main` is in sync with origin. **v0.2.2** tags the session-shell + world-follow-up commit;
   CI builds the portable binaries, NSIS installer, and AppImage on the tag push.
-- Next work: the remaining TODO sections — distribution host setting (`REQUIRE_SIGNIN_VIEW`),
+- Next work: only the optional editor-tooling section remains (GUT plugin, `.tres` venues);
-  open polish (`Strings.fmt` placeholders), the vision-model screenshot judge, and editor tooling.
+  everything else in `TODO.md` is complete.
 - Do not re-open `override_access`. Do not bind wand to `W`. Do not remove the radio easter egg.
 
 ## Current State
@@ -95,6 +95,19 @@ Session memory for durable decisions, state, and gotchas. Keep concise, remove o
   `_begin_incident()` now starts one per focused NPC (`has_open_incident()` guard), verbs call
   `_ensure_incident()`, and `ShiftCoordinator._finish_shift()` closes leftovers as `unresolved`
   before building the review.
+- **Vision judge (2026-10-31):** `python3 tools/screen_designs.py judge` sends the capture plus
+  its concept mockup to a vision model (`DEFAULT_JUDGE_MODEL = google/gemini-2.5-flash-lite`,
+  override with `--model`; `openai/gpt-4o-mini` also works). Reads the repo `.env` via
+  `_load_env_file()` (never overrides real env). Verdicts are classed `match` / `review`
+  (mismatch below `--fail-confidence`, default 0.6) / `diff` (confident mismatch → exit 1) /
+  `unknown`. `--dry-run`, `--json`, `--only` supported. The prompt explicitly ignores art style,
+  because the references are stylised mockups; the live run gave 3 match, 3 review, 0 diff.
+  Python tests: `cd tools && python3 -m unittest test_asset_pipeline test_screen_designs`.
+- **String formatting safety (2026-10-31):** `Strings.placeholder_count()` counts `%` specs
+  (ignoring `%%`), `fmt()` warns and returns the template when the arg count does not match
+  (instead of raising an engine error), and `validate()` lints the table for trailing `%`,
+  unsupported conversions, and mixed positional/sequential placeholders. Tests cover all three
+  plus a formatting pass over every placeholder key with type-correct dummy args.
 - **Authored scenario content (2026-10-31):** events can now carry `hint` (a string key logged
   when the avatar spawns), `extra_spawns` (celebrity arrivals pull in tourists/influencers via
   `_spawn_npc` with `kind: routine`), and `refuses_screening` (copied to
@@ -141,7 +154,9 @@ Session memory for durable decisions, state, and gotchas. Keep concise, remove o
   needs `mipmaps/generate=true`.
 - **Asset gotcha:** `assets/textures/game_ready/` is *not* imported (no `.import`). Use the
   imported `assets/textures/skyline.png` path. Untracked duplicate folder should not be referenced.
-- Host sign-in follow-up remains deferred.
+- Host sign-in follow-up is **done**: `REQUIRE_SIGNIN_VIEW = false` in `/etc/gitea/app.ini`
+  (backup `app.ini.bak-*` alongside), Gitea restarted; anonymous repo page/API return 200 and
+  release assets download in full. Reverse by setting it back to `true` and restarting.
 - **Release publishing was broken since v0.2.2 (fixed 2026-10-31):** `publish_release.sh`
   only built `AUTH` in the `GITEA_TOKEN` branch; the `~/.git-credentials` branch set `CRED`
   and then used unbound `${AUTH[@]}` under `set -u`, so every API call failed and the tag

README.md

@@ -28,9 +28,10 @@ The central design goal is simple:
 
 **All releases:** <https://git.allucanget.biz/zwitschi/security-game/releases>
 
-> **Latest: v0.2.13** — authored scenario content: Joke Shop access edge cases, Studio filming
+> Downloads are public (no sign-in required) since the host's instance-wide
-> boundaries and celebrity crowd spikes, and Building events that use the body-camera and
+> `REQUIRE_SIGNIN_VIEW` was disabled on 2026-10-31.
-> request-reason verbs.
+>
+> **Latest: v0.2.15** — vision-model screenshot judge for the QA tooling.
 > Releases are built automatically when a `vX.Y.Z` tag is pushed
 > (see [`docs/RELEASING.md`](docs/RELEASING.md)); manual builds use
 > `./tools/build_releases.sh`. Installer/AppImage packaging runs in the same pipeline
@@ -204,13 +205,26 @@ python3 tools/screen_designs.py generate
 # Compare docs/screenshots/*.png against the designs
 python3 tools/screen_designs.py compare
 python3 tools/screen_designs.py compare --json
+
+# Ask a vision model whether each capture realises its design's screen and layout
+# (requires OPENROUTER_API_KEY; the repo .env is read automatically)
+python3 tools/screen_designs.py judge
+python3 tools/screen_designs.py judge --model openai/gpt-4o-mini --json
+python3 tools/screen_designs.py judge --only 03-   # one capture
+python3 tools/screen_designs.py judge --dry-run    # build requests, no network
 ```
 
 `compare` flags gross drift: wrong aspect ratio, undersized frame, near-blank
 frame, and large layout/palette/detail distance. It is a regression gate, not a
 semantic judge — pixel and perceptual metrics cannot decide whether a frame's
-composition is *right*, so a vision-model pass is the documented follow-up (see
+composition is *right*.
-`TODO.md`). Regenerate the designs after changing screen layout, then re-run
+
+`judge` covers that gap: it sends each capture together with its concept mockup to
+a vision model and asks whether the capture realises the same screen and layout
+intent. Art style, fidelity, fonts, and placeholder text are explicitly ignored.
+Verdicts are classed as `match`, `review` (a mismatch below `--fail-confidence`,
+default 0.6 — expected when a concept mockup is more stylised than the game), or
+`diff` (a confident mismatch, which exits non-zero). Regenerate the designs after changing screen layout, then re-run
 `compare` after regenerating the game screenshots with
 `src/test/capture_screenshots.gd`.
 
@@ -312,7 +326,7 @@ Implemented so far:
 - Godot 4.3 project scaffold, input map, procedural world builder
 - Venue-driven simulation core: `ShiftCoordinator`, `VenueScenario`, `EventScheduler`, `NPCController` + FSM, `AccessVerifier`, `EscalationManager`, `RadioSystem`, `IncidentRecorder`, `ReputationEvaluator`, `CrowdSystem`
 - Three authored scenarios (Hollywood Boulevard building, The Joke Shop, Late Night TV Studio) with access policies, NPC archetypes, and event pools
-- Headless unit tests: **149 passing** (access rules and venue-correct access loadout, escalation, radio cooldown/battery, NPC reaction determinism, door lock, flashlight perception, persistence round-trip, settings round-trip, navmesh, step colliders, skyline alpha, scenario data, full-shift simulation, wand screening, responder dispatch, localization strings, focus cone, action wheel, session shell, PLAN verbs and incident recording, crowd presentation, emergency aftermath, body-camera timeline, NPC theater poses, audio, accessibility, authored scenario content)
+- Headless unit tests: **153 passing** (access rules and venue-correct access loadout, escalation, radio cooldown/battery, NPC reaction determinism, door lock, flashlight perception, persistence round-trip, settings round-trip, navmesh, step colliders, skyline alpha, scenario data, full-shift simulation, wand screening, responder dispatch, localization strings, focus cone, action wheel, session shell, PLAN verbs and incident recording, crowd presentation, emergency aftermath, body-camera timeline, NPC theater poses, audio, accessibility, authored scenario content, string formatting)
 - Simulation extras: protester archetype, CALL_SOMEONE reaction, door lock/secure mechanic (visual indicator), flashlight with night perception bonus, LATE shift phase reaching the violent-threat event
 - First-person prototype with interaction HUD, access inspection, wand screening, radio report form (incl. emergency hold-to-confirm), dispatched responder avatars, aim-free targeting, venue cycling between all three scenarios, color-coded escalation feedback, procedural audio cues, navigation-mesh NPC paths, JSON-backed localized strings, persisted shift history, debug overlay, and shift review
 - Control pass: focus cone on NPC heads with a billboarded blue circle in front of the face and name, action wheel (LMB/Space) with de-escalation-first slices, mouse-look freeze while a panel is open, acceleration-based walking, and named input actions (`action_menu`, `confirm`, `cancel`, `inspect`, `wand`)
@@ -329,6 +343,7 @@ Implemented so far:
 - NPC theater: seated guests sit on the steps with bent legs, influencers and tourists raise a camera prop while photographing, performers sway their sign, and filming visitors pan slowly — all once they reach their intent's waypoint
 - Audio: a looping procedural crowd bed (filtered murmur) whose volume follows `CrowdSystem.mood`, plus radio open/close crackle. All generated at runtime, no audio assets, and safe with a dummy audio server
 - Accessibility: the escalation cue shows a shape meter plus the numeric level (`Escalation: Blocking [###--] L3/5`) so it reads in grayscale, and the HUD font scale from settings is applied at boot by resizing fonts (not Control.scale, which broke the right-aligned layout)
+- Localization safety: `Strings.fmt` validates the placeholder count before formatting and fails safe (warns, returns the template) instead of raising an engine error, `Strings.placeholder_count` ignores `%%`, and `Strings.validate()` lints the table for trailing `%`, unsupported conversions, and mixed positional/sequential placeholders
 - Authored scenario content: Joke Shop reservation-without-ID, borrowed/mismatched ID, a guest who refuses the wand screen, and a late arrival during a performance; Studio filming-boundary repeats and a celebrity arrival that draws a small crowd; Building events that point the player at the body camera and request-reason verbs
 - Visual polish pass (v0.2.0): phase-based lighting (sun/sky/ambient per shift phase), street lamps with night-mode bulbs and omni lights, glowing marquee signage, crosswalk, planters and low-poly street trees, NPC height/brightness variety, denser routine traffic — day and late-night screenshot sets
 - Assets pass: authored textures in `assets/textures/` (brick facade, concrete, asphalt, awning, skyline, vignette, HUD icons, access-card art) generated by `tools/generate_textures.py`, soft sun shadows, humanoid NPC rigs with archetype props + procedural walk bob and attention head-turn, inspection artifact cards, escalation vignette, mode icons, and a body-camera overlay (REC + shift clock)

TODO.md

@@ -1,54 +1,8 @@
 # TODO
 
-Godot 4.3 prototype. Release **v0.2.13** is tagged. Headless suite **149/149**, plus the session,
+Godot 4.3 prototype. Latest release **v0.2.15**. Headless suite **153/153** plus 30 Python
-PLAN-verb, crowd, emergency, timeline, audio, scenario-content, and windowed focus/radial smokes.
+tooling tests; the session, PLAN-verb, crowd, emergency, timeline, audio, scenario-content, and
-
+windowed focus/radial smokes pass. Anonymous downloads are enabled on the host.
-Host `REQUIRE_SIGNIN_VIEW` stays deferred.
-
-## Distribution follow-up (host config, deferred)
-
-Verified via API+host credentials (2026-10-31):
-
-- [x] Repo is **public** (`private:false`) and account has admin rights.
-- [x] Release `v0.2.1` now carries all five artifacts (portable binaries + NSIS installer +
-      AppImage).
-- [ ] **Remaining host setting**: instance-wide `[service] REQUIRE_SIGNIN_VIEW=true` makes
-      anonymous browsing/downloads redirect to login. Host admin: `REQUIRE_SIGNIN_VIEW = false`
-      under `[service]` in `app.ini`, restart Gitea. Alternative: mirror downloads.
-
-## Release pipeline follow-up
-
-- [x] Fix `tools/publish_release.sh`: the `~/.git-credentials` branch never set `AUTH`,
-      so every tag build since v0.2.2 failed to publish.
-- [x] Add a `github.token` fallback in `.gitea/workflows/build.yml` (the repo has no
-      `GITEA_TOKEN` secret).
-- [x] Fall back to the checkout's git extraheader when no token/credentials exist (the
-      Gitea runner does not expose `github.token`).
-- [x] Uploads now fail loudly and the release is verified to carry every artifact
-      (a silent HTTP 4xx used to leave an empty release with a green job).
-- [x] Host disk expanded to 16G; v0.2.0 and v0.2.9 releases restored from the local artifacts.
-- [x] Host maintenance via `root@192.168.88.30`: apt cache cleaned and journald capped
-      (`SystemMaxUse=300M` in `/etc/systemd/journald.conf.d/99-size.conf`); ~7.6G free.
-- [x] Verified end to end: tagging v0.2.12 produced a release with all five artifacts
-      (AppImage, Linux, macOS, NSIS installer, Windows) from CI, with no repo secret.
-- [ ] Optionally backfill releases for v0.2.4–v0.2.8 (their CI artifacts were not
-      retained; would need local rebuilds) or leave them superseded by the latest.
-
-## Open Polish
-
-- [ ] Keep `%` placeholders stable in `Strings.fmt` when adding keys (positional formatting).
-
-## Screen design references (QA tooling)
-
-- [x] `tools/screen_designs.py generate` writes one reference design per screen
-      (from the `docs/` descriptions) to `docs/screens/designs/`, with an offline
-      `--draft` wireframe mode and an OpenRouter `--ai` mockup mode.
-- [x] `tools/screen_designs.py compare` checks `docs/screenshots/*.png` against the
-      designs (aspect ratio, size, blank-frame, and layout/palette/detail drift).
-- [ ] Vision-model judge for semantic screenshot comparison. Pixel and perceptual
-      metrics cannot decide whether a frame's composition is correct; a follow-up
-      should send the capture + reference design to a vision model and ask whether
-      they depict the same screen.
 
 ## Tooling (when an editor workflow exists)
 

project.godot

@@ -8,7 +8,7 @@ config_version=5
 
 config/name="Step Line"
 config/description="Workplace security simulation about observation, access verification, and proportionate escalation."
-config/version="0.2.13"
+config/version="0.2.15"
 run/main_scene="res://scenes/main.tscn"
 config/features=PackedStringArray("4.3", "GL Compatibility")
 

src/core/strings.gd

@@ -34,12 +34,69 @@ static func lookup(key: String) -> String:
 	_ensure_loaded()
 	return TABLE.get(key, key)
 
+## Number of `%` conversion placeholders in a template, ignoring `%%` escapes.
+static func placeholder_count(template: String) -> int:
+	var count := 0
+	var i := 0
+	while i < template.length():
+		if template[i] == "%":
+			if i + 1 < template.length() and template[i + 1] == "%":
+				i += 2
+				continue
+			count += 1
+		i += 1
+	return count
+
 ## Format a string by key with positional arguments.
+##
+## GDScript's `%` binds arguments by order, so a key whose placeholder count drifts
+## from its call site would push an engine error. Validate first and fail safe: log
+## a warning and return the raw template instead of crashing.
 static func fmt(key: String, args: Array) -> String:
 	var template := lookup(key)
 	if template == key:
 		return template
+	var needed := placeholder_count(template)
+	if needed != args.size():
+		push_warning("Strings.fmt(%s): template needs %d args, got %d" % [key, needed, args.size()])
+		return template
 	return template % args
 
+## Lint the loaded table for formatting hazards. Returns an array of problems.
+## Checks: trailing `%`, invalid conversions, and mixed positional (`%1$s`) and
+## sequential placeholders in one template.
+static func validate() -> Array[String]:
+	_ensure_loaded()
+	var problems: Array[String] = []
+	for key in TABLE.keys():
+		var template := str(TABLE[key])
+		var positional := template.contains("$")
+		var sequential := 0
+		var i := 0
+		while i < template.length():
+			if template[i] != "%":
+				i += 1
+				continue
+			if i + 1 >= template.length():
+				problems.append("%s: trailing %% escape" % key)
+				break
+			var next := template[i + 1]
+			if next == "%":
+				i += 2
+				continue
+			var j := i + 1
+			while j < template.length() and template[j] in "0123456789.$":
+				if template[j] == "$":
+					positional = true
+				j += 1
+			if j >= template.length() or not (template[j] in "sdfxXv"):
+				problems.append("%s: unsupported conversion near '%%%s'" % [key, template.substr(i + 1, 3)])
+			elif not positional:
+				sequential += 1
+			i = j + 1
+		if positional and sequential > 0:
+			problems.append("%s: mixes positional (%%1$s) and sequential (%%s) placeholders" % key)
+	return problems
+
 func _init() -> void:
 	push_error("Strings is a static utility class; use Strings.lookup / Strings.fmt.")

src/test/tests/test_strings.gd

@@ -23,6 +23,70 @@ func test_strings_json_backed_all_resolve() -> bool:
 			return TestUtils.fail("key %s resolves to itself — table entry missing" % key)
 	return true
 
+func test_placeholder_count() -> bool:
+	var a := TestUtils.check_eq(Strings.placeholder_count("no args"), 0, "plain text has no placeholders")
+	var b := TestUtils.check_eq(Strings.placeholder_count("%s complies"), 1, "one %s")
+	var c := TestUtils.check_eq(Strings.placeholder_count("%s: \"%s\""), 2, "two %s")
+	var d := TestUtils.check_eq(Strings.placeholder_count("100%% sure"), 0, "%% is an escape, not an argument")
+	var e := TestUtils.check_eq(Strings.placeholder_count("%.0f%% mood"), 1, "float plus percent sign")
+	return a and b and c and d and e
+
+func test_fmt_fails_safe_on_arg_mismatch() -> bool:
+	## A drifting placeholder count must not raise an engine error at runtime.
+	var result: String = Strings.fmt("react.comply", [])
+	var a := TestUtils.check_eq(result, Strings.lookup("react.comply"), "wrong arg count returns the raw template")
+	var extra: String = Strings.fmt("react.comply", ["A", "B"])
+	var b := TestUtils.check_eq(extra, Strings.lookup("react.comply"), "too many args also fails safe")
+	return a and b
+
+func test_table_validates_cleanly() -> bool:
+	var problems: Array = Strings.validate()
+	if not problems.is_empty():
+		return TestUtils.fail("string table format problems: %s" % str(problems))
+	return true
+
+func test_every_placeholder_key_formats() -> bool:
+	## Feed each template one dummy arg per placeholder, with a type that matches
+	## the conversion (%s text, %d int, %f float). Nothing may be left unformatted.
+	var problems: Array[String] = []
+	for key in Strings.TABLE.keys():
+		var template: String = str(Strings.TABLE[key])
+		var specs: Array[String] = _specifiers(template)
+		if specs.is_empty():
+			continue
+		var args := []
+		for spec in specs:
+			var conv: String = spec.substr(spec.length() - 1)
+			if conv == "d" or conv == "x" or conv == "X":
+				args.append(7)
+			elif conv == "f":
+				args.append(1.5)
+			else:
+				args.append("x")
+		var out: String = Strings.fmt(key, args)
+		if out == template:
+			problems.append("%s did not format (%s)" % [key, str(specs)])
+	return TestUtils.check(problems.is_empty(), "all placeholder keys format (problems: %s)" % str(problems))
+
+func _specifiers(template: String) -> Array[String]:
+	var out: Array[String] = []
+	var i := 0
+	while i < template.length():
+		if template[i] != "%":
+			i += 1
+			continue
+		if i + 1 < template.length() and template[i + 1] == "%":
+			i += 2
+			continue
+		var j := i + 1
+		while j < template.length() and template[j] in "0123456789.$":
+			j += 1
+		if j < template.length():
+			out.append(template.substr(i, j - i + 1))
+			j += 1
+		i = j
+	return out
+
 func test_strings_cover_ui_keys() -> bool:
 	## Every key referenced by the HUD and game manager must exist.
 	var required := [

tools/screen_designs.py

@@ -17,6 +17,12 @@ Two commands:
                --draft  deterministic wireframe sketches drawn with Pillow
                         (no network or API key; good for CI and layout checks).
 
+  judge      send each capture together with its reference design to a vision
+             model and ask whether they depict the same screen. Pixel and
+             perceptual metrics cannot judge composition; this closes that gap.
+             Requires OPENROUTER_API_KEY (falls back to the repo .env).
+               --dry-run  build and print the requests without calling the API.
+
   compare    compare docs/screenshots/*.png against the reference designs and
              flag gross drift. Semantic composition is judged with coarse,
              Pillow-only signals (aspect ratio, frame size, blank-frame
@@ -33,6 +39,7 @@ Run from the repo root:
 """
 
 import argparse
+import json
 import os
 import sys
 from pathlib import Path
@@ -54,6 +61,10 @@ ASPECT = 16 / 9
 # the model asset_pipeline.py already trusts for non-alpha roles).
 DEFAULT_MODEL = "openai/gpt-5-image-mini"
 
+# Vision model used by `judge`. Cheap, multimodal, and good at "same screen?".
+DEFAULT_JUDGE_MODEL = "google/gemini-2.5-flash-lite"
+OPENROUTER_CHAT_URL = "https://openrouter.ai/api/v1/chat/completions"
+
 # Shared art direction, appended to every prompt so the generated frames read
 # as one coherent game. "scene" entries are first-person 3D world views;
 # "ui" entries are in-game panels overlaid on a softened scene.
@@ -582,6 +593,193 @@ def compare(args):
 
 # ---------------------------------------------------------------------------
 
+# ---------------------------------------------------------------------------
+# Vision judge — semantic "same screen?" check against the reference designs.
+# ---------------------------------------------------------------------------
+
+JUDGE_INSTRUCTIONS = (
+    "You are reviewing screenshots of a low-poly 3D security-guard game. "
+    "IMAGE 1 is a CONCEPT MOCKUP of the intended screen (it may be painted in a "
+    "different art style and show different actors); IMAGE 2 is the ACTUAL "
+    "capture from the game. Different art style, fidelity, fonts, and character "
+    "models are EXPECTED and must be ignored.\n\n"
+    "Judge only whether IMAGE 2 realises the same SCREEN and LAYOUT INTENT as "
+    "IMAGE 1: is it the same kind of screen (world view, HUD overlay, modal "
+    "panel), are the main interface regions present in roughly the same places "
+    "(target line, inspection card, report form, on-screen buttons), and is the "
+    "subject of the shot comparable (entrance, street, person)?\n\n"
+    "Answer with JSON only: {\"match\": true|false, \"confidence\": 0..1, "
+    "\"differences\": [\"short point\", ...]}. Set match=true when the capture "
+    "realises the mockup's screen even if it looks much simpler."
+)
+
+
+def _load_env_file(path=None):
+    """Populate os.environ from a .env file without overriding real variables."""
+    path = Path(path) if path else ROOT / ".env"
+    if not path.is_file():
+        return
+    for raw in path.read_text(encoding="utf-8").splitlines():
+        line = raw.strip()
+        if not line or line.startswith("#") or "=" not in line:
+            continue
+        key, value = line.split("=", 1)
+        os.environ.setdefault(key.strip(), value.strip().strip("\"'"))
+
+
+def _data_url(path):
+    import base64
+    payload = Path(path).read_bytes()
+    return "data:image/png;base64," + base64.b64encode(payload).decode("ascii")
+
+
+def build_judge_request(model, screen, capture_path, design_path):
+    """Build the OpenRouter chat payload for one capture/design pair."""
+    prompt = (f"{JUDGE_INSTRUCTIONS}\n\nScreen: {screen['title']} "
+              f"({screen['source']}). Design notes: {screen['description']}")
+    return {
+        "model": model,
+        "messages": [{
+            "role": "user",
+            "content": [
+                {"type": "text", "text": prompt},
+                {"type": "image_url", "image_url": {"url": _data_url(design_path)}},
+                {"type": "image_url", "image_url": {"url": _data_url(capture_path)}},
+            ],
+        }],
+        "temperature": 0,
+        "response_format": {"type": "json_object"},
+    }
+
+
+def parse_judge_response(payload):
+    """Extract a verdict dict from an OpenRouter chat response."""
+    try:
+        text = payload["choices"][0]["message"]["content"]
+    except (KeyError, IndexError, TypeError):
+        return {"match": None, "confidence": 0.0,
+                "differences": ["unreadable model response"]}
+    if isinstance(text, list):  # some providers return content parts
+        text = "".join(part.get("text", "") for part in text
+                       if isinstance(part, dict))
+    text = str(text).strip()
+    start, end = text.find("{"), text.rfind("}")
+    if start >= 0 and end > start:
+        text = text[start:end + 1]
+    import json
+    try:
+        data = json.loads(text)
+    except ValueError:
+        return {"match": None, "confidence": 0.0,
+                "differences": [f"non-JSON response: {text[:120]}"]}
+    differences = data.get("differences") or []
+    if not isinstance(differences, list):
+        differences = [str(differences)]
+    return {
+        "match": data.get("match") if isinstance(data.get("match"), bool) else None,
+        "confidence": float(data.get("confidence", 0.0) or 0.0),
+        "differences": [str(item) for item in differences],
+    }
+
+
+def classify(verdict, threshold=0.6):
+    """Turn a model verdict into a QA class.
+
+    The reference images are concept mockups, so a low-confidence disagreement is
+    a review note, not a failure. Only a confident mismatch fails the check.
+    """
+    if verdict.get("match") is True:
+        return "match"
+    if verdict.get("match") is False:
+        return "bad" if float(verdict.get("confidence", 0.0)) >= threshold else "review"
+    return "unknown"
+
+
+def _call_openrouter(body, timeout=180):
+    import urllib.error
+    import urllib.request
+    key = os.environ.get("OPENROUTER_API_KEY", "").strip()
+    if not key:
+        raise ValueError("OPENROUTER_API_KEY is not set (repo .env is read "
+                         "automatically); use --dry-run to inspect requests")
+    req = urllib.request.Request(
+        OPENROUTER_CHAT_URL,
+        data=json.dumps(body).encode("utf-8"),
+        headers={
+            "Authorization": f"Bearer {key}",
+            "Content-Type": "application/json",
+            "HTTP-Referer": "https://git.allucanget.biz/zwitschi/security-game",
+            "X-Title": "Step Line",
+        },
+        method="POST",
+    )
+    try:
+        with urllib.request.urlopen(req, timeout=timeout) as resp:
+            return json.loads(resp.read().decode("utf-8"))
+    except urllib.error.HTTPError as exc:
+        detail = exc.read().decode("utf-8", "replace")
+        raise ValueError(f"OpenRouter HTTP {exc.code}: {detail[:300]}")
+
+
+def judge(args):
+    _load_env_file()
+    if not SCREENSHOTS_DIR.is_dir():
+        sys.exit(f"error: no screenshots directory at {SCREENSHOTS_DIR}")
+    refs = {screen["name"]: screen["ref"] for screen in SCREENS if screen["ref"]}
+    by_name = {screen["name"]: screen for screen in SCREENS}
+    results = []
+    for capture in sorted(SCREENSHOTS_DIR.glob("*.png")):
+        if args.only and args.only not in capture.name:
+            continue
+        design_name = next((n for n, ref in refs.items()
+                            if ref == capture.name), None)
+        if design_name is None:
+            results.append({"capture": capture.name, "screen": None,
+                            "match": None, "confidence": 0.0,
+                            "differences": ["no design reference for this capture"]})
+            continue
+        reference = _design_path(design_name)
+        if not reference.is_file():
+            results.append({"capture": capture.name, "screen": design_name,
+                            "match": None, "confidence": 0.0,
+                            "differences": ["missing reference design; run generate"]})
+            continue
+        body = build_judge_request(args.model, by_name[design_name],
+                                   capture, reference)
+        if args.dry_run:
+            results.append({
+                "capture": capture.name, "screen": design_name, "match": None,
+                "confidence": 0.0, "model": body["model"],
+                "differences": [f"dry run: {len(body['messages'][0]['content'])} parts"],
+            })
+            continue
+        verdict = parse_judge_response(_call_openrouter(body))
+        verdict.update({"capture": capture.name, "screen": design_name,
+                        "class": classify(verdict, args.fail_confidence)})
+        results.append(verdict)
+
+    if args.json:
+        print(json.dumps(results, indent=2))
+    else:
+        labels = {"match": "MATCH", "review": "REVIEW", "bad": "DIFF",
+                  "unknown": "UNKNOWN"}
+        for result in results:
+            print(f"{labels[result['class']]:7} {result['capture']} "
+                  f"(design: {result['screen']}) "
+                  f"confidence={result['confidence']:.2f}")
+            for difference in result["differences"]:
+                print(f"      - {difference}")
+        counts = {key: sum(1 for r in results if r["class"] == key)
+                  for key in labels}
+        print(f"Summary: {counts['match']}/{len(results)} match, "
+              f"{counts['review']} review, {counts['bad']} diff "
+              f"(fail threshold confidence >= {args.fail_confidence:.2f})")
+
+    if args.dry_run:
+        return 0
+    return 1 if any(r["class"] == "bad" for r in results) else 0
+
+
 def build_parser():
     parser = argparse.ArgumentParser(
         description="Generate and compare Step Line screen-design references.")
@@ -599,6 +797,18 @@ def build_parser():
                      default=generate_textures.GEN_SIZE)
     gen.set_defaults(handler=generate, draft=False)
 
+    judge_cmd = commands.add_parser(
+        "judge", help="ask a vision model whether captures match the designs")
+    judge_cmd.add_argument("--model", default=DEFAULT_JUDGE_MODEL)
+    judge_cmd.add_argument("--json", action="store_true")
+    judge_cmd.add_argument("--dry-run", action="store_true",
+                           help="print requests instead of calling the API")
+    judge_cmd.add_argument("--only", default="",
+                           help="only judge captures whose name contains this text")
+    judge_cmd.add_argument("--fail-confidence", type=float, default=0.6,
+                           help="a mismatch at or above this confidence fails")
+    judge_cmd.set_defaults(handler=judge)
+
     cmp = commands.add_parser(
         "compare", help="compare docs/screenshots/*.png against the designs")
     cmp.add_argument("--json", action="store_true")

tools/test_screen_designs.py

@@ -86,5 +86,97 @@ class ScreenDesignsTests(unittest.TestCase):
         self.assertTrue(any("aspect" in e for e in report["errors"]))
 
 
+class VisionJudgeTests(unittest.TestCase):
+    def setUp(self):
+        self.temp_dir = tempfile.TemporaryDirectory()
+        self.root = Path(self.temp_dir.name)
+        self.capture = self.root / "capture.png"
+        self.design = self.root / "design.png"
+        Image.new("RGB", (64, 36), (20, 30, 40)).save(self.capture)
+        Image.new("RGB", (64, 36), (200, 190, 180)).save(self.design)
+        self.screen = screen_designs.SCREENS[0]
+
+    def tearDown(self):
+        self.temp_dir.cleanup()
+
+    def test_judge_request_embeds_both_images(self):
+        body = screen_designs.build_judge_request(
+            "model-x", self.screen, self.capture, self.design)
+        self.assertEqual(body["model"], "model-x")
+        self.assertEqual(body["temperature"], 0)
+        self.assertEqual(body["response_format"], {"type": "json_object"})
+        parts = body["messages"][0]["content"]
+        self.assertEqual(parts[0]["type"], "text")
+        images = [p for p in parts if p["type"] == "image_url"]
+        self.assertEqual(len(images), 2, "design and capture are both attached")
+        for image in images:
+            self.assertTrue(image["image_url"]["url"].startswith(
+                "data:image/png;base64,"))
+        self.assertIn(self.screen["title"], parts[0]["text"])
+        self.assertIn("mockup", parts[0]["text"].lower())
+
+    def test_parse_plain_json_verdict(self):
+        payload = {"choices": [{"message": {"content":
+            '{"match": true, "confidence": 0.9, "differences": ["style"]}'}}]}
+        verdict = screen_designs.parse_judge_response(payload)
+        self.assertTrue(verdict["match"])
+        self.assertAlmostEqual(verdict["confidence"], 0.9)
+        self.assertEqual(verdict["differences"], ["style"])
+
+    def test_parse_fenced_json_verdict(self):
+        payload = {"choices": [{"message": {"content":
+            'Sure:\n```json\n{"match": false, "confidence": 0.7}\n```'}}]}
+        verdict = screen_designs.parse_judge_response(payload)
+        self.assertFalse(verdict["match"])
+        self.assertAlmostEqual(verdict["confidence"], 0.7)
+
+    def test_parse_content_parts(self):
+        payload = {"choices": [{"message": {"content": [
+            {"type": "text", "text": '{"match": true, "confidence": 0.5}'}]}}]}
+        verdict = screen_designs.parse_judge_response(payload)
+        self.assertTrue(verdict["match"])
+
+    def test_parse_non_json_is_unknown(self):
+        payload = {"choices": [{"message": {"content": "no idea"}}]}
+        verdict = screen_designs.parse_judge_response(payload)
+        self.assertIsNone(verdict["match"])
+        self.assertTrue(verdict["differences"])
+
+    def test_parse_missing_choices_is_unknown(self):
+        verdict = screen_designs.parse_judge_response({})
+        self.assertIsNone(verdict["match"])
+
+    def test_classify_thresholds(self):
+        self.assertEqual(screen_designs.classify(
+            {"match": True, "confidence": 0.1}), "match")
+        self.assertEqual(screen_designs.classify(
+            {"match": False, "confidence": 0.9}), "bad")
+        self.assertEqual(screen_designs.classify(
+            {"match": False, "confidence": 0.3}), "review")
+        self.assertEqual(screen_designs.classify(
+            {"match": None, "confidence": 0.0}), "unknown")
+        self.assertEqual(screen_designs.classify(
+            {"match": False, "confidence": 0.55}, threshold=0.5), "bad")
+
+    def test_env_file_does_not_override_existing(self):
+        env_path = self.root / ".env"
+        env_path.write_text('OPENROUTER_API_KEY=from-file\nOTHER=1\n',
+                            encoding="utf-8")
+        import os
+        old = os.environ.get("OPENROUTER_API_KEY")
+        os.environ["OPENROUTER_API_KEY"] = "from-env"
+        os.environ.pop("OTHER", None)
+        try:
+            screen_designs._load_env_file(env_path)
+            self.assertEqual(os.environ["OPENROUTER_API_KEY"], "from-env")
+            self.assertEqual(os.environ.get("OTHER"), "1")
+        finally:
+            if old is None:
+                os.environ.pop("OPENROUTER_API_KEY", None)
+            else:
+                os.environ["OPENROUTER_API_KEY"] = old
+            os.environ.pop("OTHER", None)
+
+
 if __name__ == "__main__":
     unittest.main()