Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
72b9378e69 | ||
|
|
74f14014b2 | ||
|
|
a2a0ded527 | ||
|
|
922a990c7a | ||
|
|
f0cec192a7 | ||
|
|
2dba562c52 | ||
|
|
afd086ba11 | ||
|
|
b4d189c325 |
No files matched your search
@@ -113,11 +113,10 @@ jobs:
|
||||
- name: Publish Gitea release (tags only)
|
||||
if: startsWith(github.ref, 'refs/tags/')
|
||||
env:
|
||||
# Prefer an explicit repo secret; otherwise fall back to the workflow's
|
||||
# built-in token (Gitea provides `github.token`), which has write access
|
||||
# to this repository. Without either, publish_release.sh has no
|
||||
# credentials inside the runner and the step fails.
|
||||
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN || github.token }}
|
||||
# An explicit repo secret wins. Without one, publish_release.sh reuses
|
||||
# the local git extraheader that actions/checkout configures for the job
|
||||
# (the runner's own credentials), so tag builds still publish.
|
||||
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
||||
VERSION: ${{ github.ref_name }}
|
||||
run: |
|
||||
chmod +x tools/publish_release.sh
|
||||
|
||||
@@ -514,3 +514,72 @@ Completed 2026-10-31 (working tree). Headless **139/139**; verified visually at
|
||||
|
||||
- [x] Escalation colors that still read in grayscale (shape or label, not hue alone).
|
||||
- [x] HUD font scale in settings.
|
||||
|
||||
## Scenario content still missing vs PLAN lists
|
||||
|
||||
Completed 2026-10-31 (working tree). Headless **149/149**; `src/test/smoke_scenario_content.gd` spawns the refusal event and the celebrity arrival through the real signal path.
|
||||
|
||||
|
||||
- [x] Joke Shop: guest on the reservation list without ID; borrowed/mismatched ID; guest
|
||||
refuses wand screening; late arrival during a performance.
|
||||
- [x] Studio: filming-boundary repeat (influencer ignores a posted line); celebrity-arrival
|
||||
crowd spike (more rubberneckers, not only one `arrive` spawn).
|
||||
- [x] Building: "activate body camera" and "request reason" once those verbs exist, used in
|
||||
at least one authored event.
|
||||
|
||||
## Distribution follow-up (host config)
|
||||
|
||||
Completed 2026-10-31 (working tree). Headless **153/153**.
|
||||
|
||||
|
||||
Verified via API+host credentials (2026-10-31):
|
||||
|
||||
- [x] Repo is **public** (`private:false`) and account has admin rights.
|
||||
- [x] Release `v0.2.1` now carries all five artifacts (portable binaries + NSIS installer +
|
||||
AppImage).
|
||||
- [x] **Host setting applied**: `[service] REQUIRE_SIGNIN_VIEW = false` in `/etc/gitea/app.ini`
|
||||
(backup written alongside), Gitea restarted. Verified anonymously: repo page 200, API 200,
|
||||
and a release asset downloads with the full 50.8 MB AppImage.
|
||||
|
||||
## Release pipeline follow-up
|
||||
|
||||
Completed 2026-10-31 (working tree). Headless **153/153**.
|
||||
|
||||
|
||||
- [x] Fix `tools/publish_release.sh`: the `~/.git-credentials` branch never set `AUTH`,
|
||||
so every tag build since v0.2.2 failed to publish.
|
||||
- [x] Add a `github.token` fallback in `.gitea/workflows/build.yml` (the repo has no
|
||||
`GITEA_TOKEN` secret).
|
||||
- [x] Fall back to the checkout's git extraheader when no token/credentials exist (the
|
||||
Gitea runner does not expose `github.token`).
|
||||
- [x] Uploads now fail loudly and the release is verified to carry every artifact
|
||||
(a silent HTTP 4xx used to leave an empty release with a green job).
|
||||
- [x] Host disk expanded to 16G; v0.2.0 and v0.2.9 releases restored from the local artifacts.
|
||||
- [x] Host maintenance via `root@192.168.88.30`: apt cache cleaned and journald capped
|
||||
(`SystemMaxUse=300M` in `/etc/systemd/journald.conf.d/99-size.conf`); ~7.6G free.
|
||||
- [x] Verified end to end: tagging v0.2.12 produced a release with all five artifacts
|
||||
(AppImage, Linux, macOS, NSIS installer, Windows) from CI, with no repo secret.
|
||||
- [x] Backfill decision: v0.2.4–v0.2.8 stay superseded; their tags and source remain, and the
|
||||
release page points at the current build. No rebuilds planned.
|
||||
|
||||
## Open Polish
|
||||
|
||||
Completed 2026-10-31 (working tree). Headless **153/153**.
|
||||
|
||||
|
||||
- [x] Keep `%` placeholders stable in `Strings.fmt` when adding keys (positional formatting).
|
||||
|
||||
## Screen design references (QA tooling)
|
||||
|
||||
Completed 2026-10-31 (working tree). 30 Python tooling tests pass; the live judge run classed 3 captures as match and 3 as review (concept mockups are more stylised), with no confident mismatch.
|
||||
|
||||
|
||||
- [x] `tools/screen_designs.py generate` writes one reference design per screen
|
||||
(from the `docs/` descriptions) to `docs/screens/designs/`, with an offline
|
||||
`--draft` wireframe mode and an OpenRouter `--ai` mockup mode.
|
||||
- [x] `tools/screen_designs.py compare` checks `docs/screenshots/*.png` against the
|
||||
designs (aspect ratio, size, blank-frame, and layout/palette/detail drift).
|
||||
- [x] Vision-model judge for semantic screenshot comparison: `judge` sends the
|
||||
capture + reference design to a vision model, classes verdicts as
|
||||
match/review/diff with a confidence threshold, supports `--dry-run`, `--json`,
|
||||
`--only`, and `--model`, and reads the repo `.env` for the API key.
|
||||
@@ -21,8 +21,8 @@ Session memory for durable decisions, state, and gotchas. Keep concise, remove o
|
||||
Windowed control smoke: `godot --path . --script res://src/test/smoke_focus_radial.gd` → exit 0.
|
||||
- `main` is in sync with origin. **v0.2.2** tags the session-shell + world-follow-up commit;
|
||||
CI builds the portable binaries, NSIS installer, and AppImage on the tag push.
|
||||
- Next work: `TODO.md` scenario content gaps (Joke Shop / Studio / Building events), then the
|
||||
remaining polish and tooling items.
|
||||
- Next work: only the optional editor-tooling section remains (GUT plugin, `.tres` venues);
|
||||
everything else in `TODO.md` is complete.
|
||||
- Do not re-open `override_access`. Do not bind wand to `W`. Do not remove the radio easter egg.
|
||||
|
||||
## Current State
|
||||
@@ -95,6 +95,25 @@ Session memory for durable decisions, state, and gotchas. Keep concise, remove o
|
||||
`_begin_incident()` now starts one per focused NPC (`has_open_incident()` guard), verbs call
|
||||
`_ensure_incident()`, and `ShiftCoordinator._finish_shift()` closes leftovers as `unresolved`
|
||||
before building the review.
|
||||
- **Vision judge (2026-10-31):** `python3 tools/screen_designs.py judge` sends the capture plus
|
||||
its concept mockup to a vision model (`DEFAULT_JUDGE_MODEL = google/gemini-2.5-flash-lite`,
|
||||
override with `--model`; `openai/gpt-4o-mini` also works). Reads the repo `.env` via
|
||||
`_load_env_file()` (never overrides real env). Verdicts are classed `match` / `review`
|
||||
(mismatch below `--fail-confidence`, default 0.6) / `diff` (confident mismatch → exit 1) /
|
||||
`unknown`. `--dry-run`, `--json`, `--only` supported. The prompt explicitly ignores art style,
|
||||
because the references are stylised mockups; the live run gave 3 match, 3 review, 0 diff.
|
||||
Python tests: `cd tools && python3 -m unittest test_asset_pipeline test_screen_designs`.
|
||||
- **String formatting safety (2026-10-31):** `Strings.placeholder_count()` counts `%` specs
|
||||
(ignoring `%%`), `fmt()` warns and returns the template when the arg count does not match
|
||||
(instead of raising an engine error), and `validate()` lints the table for trailing `%`,
|
||||
unsupported conversions, and mixed positional/sequential placeholders. Tests cover all three
|
||||
plus a formatting pass over every placeholder key with type-correct dummy args.
|
||||
- **Authored scenario content (2026-10-31):** events can now carry `hint` (a string key logged
|
||||
when the avatar spawns), `extra_spawns` (celebrity arrivals pull in tourists/influencers via
|
||||
`_spawn_npc` with `kind: routine`), and `refuses_screening` (copied to
|
||||
`NPCController.refuses_screening`; `_toggle_wand_screening` logs `wand.refused`, records a
|
||||
`wand_refused` event, applies a small prop hit, and does not open the panel). `film_boundary`
|
||||
is a new intent that maps to `steps_end` so the influencer stands past the posted line.
|
||||
- **Accessibility (2026-10-31):** escalation text is `"<name> [###--] L3/5"` via
|
||||
`Hud.escalation_meter(level)` (pure, tested) so the cue survives grayscale. HUD font scale uses
|
||||
`add_theme_font_size_override` on the core labels (`set_font_scale`), **not** `Control.scale`,
|
||||
@@ -135,16 +154,44 @@ Session memory for durable decisions, state, and gotchas. Keep concise, remove o
|
||||
needs `mipmaps/generate=true`.
|
||||
- **Asset gotcha:** `assets/textures/game_ready/` is *not* imported (no `.import`). Use the
|
||||
imported `assets/textures/skyline.png` path. Untracked duplicate folder should not be referenced.
|
||||
- Host sign-in follow-up remains deferred.
|
||||
- Host sign-in follow-up is **done**: `REQUIRE_SIGNIN_VIEW = false` in `/etc/gitea/app.ini`
|
||||
(backup `app.ini.bak-*` alongside), Gitea restarted; anonymous repo page/API return 200 and
|
||||
release assets download in full. Reverse by setting it back to `true` and restarting.
|
||||
- **Release publishing was broken since v0.2.2 (fixed 2026-10-31):** `publish_release.sh`
|
||||
only built `AUTH` in the `GITEA_TOKEN` branch; the `~/.git-credentials` branch set `CRED`
|
||||
and then used unbound `${AUTH[@]}` under `set -u`, so every API call failed and the tag
|
||||
job ended "could not create or find release". The repo also has **no** `GITEA_TOKEN`
|
||||
secret (`/actions/secrets` is empty) and the runner has no git credentials. Fixes: the
|
||||
script sets `AUTH=(-u "$CRED")`; the workflow falls back to `github.token`
|
||||
(`secrets.GITEA_TOKEN || github.token`). **v0.2.9 is the only release after v0.2.1**
|
||||
(built and published locally, all five artifacts); v0.2.4–v0.2.8 tags have no release
|
||||
artifacts. Verify the CI fallback on the next tag push. PLAN.md + ADRs 0007–0010 still describe the intended UX.
|
||||
(`secrets.GITEA_TOKEN || github.token`) — but `github.token` is **not** provided by this
|
||||
Gitea/act runner, so the env dump showed no token and the v0.2.10 tag build failed again.
|
||||
Final fix: `publish_release.sh` falls back to the **local git extraheader** that
|
||||
`actions/checkout` writes (`git config --local --get-regexp '^http\..*\.extraheader$'`),
|
||||
which carries the runner's basic-auth credentials for the same server. Verified locally by
|
||||
adding that extraheader and publishing v0.2.10 (all five artifacts). **v0.2.9 and v0.2.10
|
||||
are published**; v0.2.4–v0.2.8 tags have no artifacts. Verify the CI path on the next tag.
|
||||
- **CI publishing verified (2026-10-31):** tagging v0.2.12 produced a release with all five
|
||||
artifacts automatically (no repo secret), confirming the checkout-extraheader fallback.
|
||||
- **Host access (2026-10-31):** `root@192.168.88.30` now works from this workstation (key auth),
|
||||
host name `gitea`, Gitea 28.2.0. Maintenance applied: `apt-get clean`, journald capped to
|
||||
`SystemMaxUse=300M` (`/etc/systemd/journald.conf.d/99-size.conf`, journal ~133M), disk at
|
||||
~7.6G free of 16G.
|
||||
- **Host disk resolved (2026-10-31):** the Gitea VM root filesystem was expanded from 7.8G to
|
||||
16G (`/dev/mapper/pve-vm--500--disk--0`, now ~7.8G free). The pruned **v0.2.0** and **v0.2.9**
|
||||
releases were restored from the local `build/` artifacts and verified (4 and 5 assets).
|
||||
`git.allucanget.biz` runs on this host as user `zwitschi`; root@192.168.88.30 is **not**
|
||||
reachable from this workstation (publickey denied; password auth closes the banner), so host
|
||||
maintenance must go through a `zwitschi` login with sudo.
|
||||
- **Host disk full (2026-10-31, historical):** the v0.2.11 tag build created release id 27 and logged the
|
||||
uploads, but they failed with `HTTP 500 Attachments.Save: write … no space left on device`
|
||||
(`/dev/mapper/pve-vm--500--disk--0`, 7.8G, 100%, 0 free). `git push` then failed too
|
||||
("unpacker error"). Unblocked by deleting the superseded **v0.2.0** and **v0.2.9** releases
|
||||
via the API (~565 MB of assets), then publishing v0.2.11 locally with the fixed script
|
||||
(verified 5 assets). Remaining ~150 MB is tight — the next release needs host cleanup on
|
||||
192.168.88.30: `sudo journalctl --vacuum-size=100M` (journal 837 MB), `sudo apt-get clean`
|
||||
(91 MB), clear `/var/lib/gitea/data/attachments/tmp`. SSH works as `zwitschi`; `sudo -n`
|
||||
needs a password. **Releases now:** v0.2.11, v0.2.10, v0.2.1 (5 assets each); v0.2.0 and
|
||||
v0.2.9 entries were pruned. PLAN.md + ADRs 0007–0010 still describe the intended UX.
|
||||
- **Release v0.2.1 is LIVE** (2026-10-31): CI run 1110 built the NSIS installer + AppImage
|
||||
and published — installers/AppImage are on the releases page. Releasing = bump version → tag → push.
|
||||
- NSIS gotchas (validated in a local ubuntu:24.04 docker repro): makensis uses `-D` defines
|
||||
|
||||
@@ -28,7 +28,10 @@ The central design goal is simple:
|
||||
|
||||
**All releases:** <https://git.allucanget.biz/zwitschi/security-game/releases>
|
||||
|
||||
> **Latest: v0.2.10** — release pipeline fix (tag builds publish automatically again).
|
||||
> Downloads are public (no sign-in required) since the host's instance-wide
|
||||
> `REQUIRE_SIGNIN_VIEW` was disabled on 2026-10-31.
|
||||
>
|
||||
> **Latest: v0.2.15** — vision-model screenshot judge for the QA tooling.
|
||||
> Releases are built automatically when a `vX.Y.Z` tag is pushed
|
||||
> (see [`docs/RELEASING.md`](docs/RELEASING.md)); manual builds use
|
||||
> `./tools/build_releases.sh`. Installer/AppImage packaging runs in the same pipeline
|
||||
@@ -202,13 +205,26 @@ python3 tools/screen_designs.py generate
|
||||
# Compare docs/screenshots/*.png against the designs
|
||||
python3 tools/screen_designs.py compare
|
||||
python3 tools/screen_designs.py compare --json
|
||||
|
||||
# Ask a vision model whether each capture realises its design's screen and layout
|
||||
# (requires OPENROUTER_API_KEY; the repo .env is read automatically)
|
||||
python3 tools/screen_designs.py judge
|
||||
python3 tools/screen_designs.py judge --model openai/gpt-4o-mini --json
|
||||
python3 tools/screen_designs.py judge --only 03- # one capture
|
||||
python3 tools/screen_designs.py judge --dry-run # build requests, no network
|
||||
```
|
||||
|
||||
`compare` flags gross drift: wrong aspect ratio, undersized frame, near-blank
|
||||
frame, and large layout/palette/detail distance. It is a regression gate, not a
|
||||
semantic judge — pixel and perceptual metrics cannot decide whether a frame's
|
||||
composition is *right*, so a vision-model pass is the documented follow-up (see
|
||||
`TODO.md`). Regenerate the designs after changing screen layout, then re-run
|
||||
composition is *right*.
|
||||
|
||||
`judge` covers that gap: it sends each capture together with its concept mockup to
|
||||
a vision model and asks whether the capture realises the same screen and layout
|
||||
intent. Art style, fidelity, fonts, and placeholder text are explicitly ignored.
|
||||
Verdicts are classed as `match`, `review` (a mismatch below `--fail-confidence`,
|
||||
default 0.6 — expected when a concept mockup is more stylised than the game), or
|
||||
`diff` (a confident mismatch, which exits non-zero). Regenerate the designs after changing screen layout, then re-run
|
||||
`compare` after regenerating the game screenshots with
|
||||
`src/test/capture_screenshots.gd`.
|
||||
|
||||
@@ -255,6 +271,9 @@ godot --headless --path . --script res://src/test/smoke_timeline.gd
|
||||
|
||||
# Audio smoke: crowd bed follows mood, radio open/close crackle
|
||||
godot --headless --path . --script res://src/test/smoke_audio.gd
|
||||
|
||||
# Scenario-content smoke: wand refusal, celebrity crowd spike
|
||||
godot --headless --path . --script res://src/test/smoke_scenario_content.gd
|
||||
```
|
||||
|
||||
The unit test runner is the project's own lightweight harness (no editor plugin required) so CI can run it without the editor.
|
||||
@@ -307,7 +326,7 @@ Implemented so far:
|
||||
- Godot 4.3 project scaffold, input map, procedural world builder
|
||||
- Venue-driven simulation core: `ShiftCoordinator`, `VenueScenario`, `EventScheduler`, `NPCController` + FSM, `AccessVerifier`, `EscalationManager`, `RadioSystem`, `IncidentRecorder`, `ReputationEvaluator`, `CrowdSystem`
|
||||
- Three authored scenarios (Hollywood Boulevard building, The Joke Shop, Late Night TV Studio) with access policies, NPC archetypes, and event pools
|
||||
- Headless unit tests: **139 passing** (access rules and venue-correct access loadout, escalation, radio cooldown/battery, NPC reaction determinism, door lock, flashlight perception, persistence round-trip, settings round-trip, navmesh, step colliders, skyline alpha, scenario data, full-shift simulation, wand screening, responder dispatch, localization strings, focus cone, action wheel, session shell, PLAN verbs and incident recording, crowd presentation, emergency aftermath, body-camera timeline, NPC theater poses, audio, accessibility)
|
||||
- Headless unit tests: **153 passing** (access rules and venue-correct access loadout, escalation, radio cooldown/battery, NPC reaction determinism, door lock, flashlight perception, persistence round-trip, settings round-trip, navmesh, step colliders, skyline alpha, scenario data, full-shift simulation, wand screening, responder dispatch, localization strings, focus cone, action wheel, session shell, PLAN verbs and incident recording, crowd presentation, emergency aftermath, body-camera timeline, NPC theater poses, audio, accessibility, authored scenario content, string formatting)
|
||||
- Simulation extras: protester archetype, CALL_SOMEONE reaction, door lock/secure mechanic (visual indicator), flashlight with night perception bonus, LATE shift phase reaching the violent-threat event
|
||||
- First-person prototype with interaction HUD, access inspection, wand screening, radio report form (incl. emergency hold-to-confirm), dispatched responder avatars, aim-free targeting, venue cycling between all three scenarios, color-coded escalation feedback, procedural audio cues, navigation-mesh NPC paths, JSON-backed localized strings, persisted shift history, debug overlay, and shift review
|
||||
- Control pass: focus cone on NPC heads with a billboarded blue circle in front of the face and name, action wheel (LMB/Space) with de-escalation-first slices, mouse-look freeze while a panel is open, acceleration-based walking, and named input actions (`action_menu`, `confirm`, `cancel`, `inspect`, `wand`)
|
||||
@@ -324,6 +343,8 @@ Implemented so far:
|
||||
- NPC theater: seated guests sit on the steps with bent legs, influencers and tourists raise a camera prop while photographing, performers sway their sign, and filming visitors pan slowly — all once they reach their intent's waypoint
|
||||
- Audio: a looping procedural crowd bed (filtered murmur) whose volume follows `CrowdSystem.mood`, plus radio open/close crackle. All generated at runtime, no audio assets, and safe with a dummy audio server
|
||||
- Accessibility: the escalation cue shows a shape meter plus the numeric level (`Escalation: Blocking [###--] L3/5`) so it reads in grayscale, and the HUD font scale from settings is applied at boot by resizing fonts (not Control.scale, which broke the right-aligned layout)
|
||||
- Localization safety: `Strings.fmt` validates the placeholder count before formatting and fails safe (warns, returns the template) instead of raising an engine error, `Strings.placeholder_count` ignores `%%`, and `Strings.validate()` lints the table for trailing `%`, unsupported conversions, and mixed positional/sequential placeholders
|
||||
- Authored scenario content: Joke Shop reservation-without-ID, borrowed/mismatched ID, a guest who refuses the wand screen, and a late arrival during a performance; Studio filming-boundary repeats and a celebrity arrival that draws a small crowd; Building events that point the player at the body camera and request-reason verbs
|
||||
- Visual polish pass (v0.2.0): phase-based lighting (sun/sky/ambient per shift phase), street lamps with night-mode bulbs and omni lights, glowing marquee signage, crosswalk, planters and low-poly street trees, NPC height/brightness variety, denser routine traffic — day and late-night screenshot sets
|
||||
- Assets pass: authored textures in `assets/textures/` (brick facade, concrete, asphalt, awning, skyline, vignette, HUD icons, access-card art) generated by `tools/generate_textures.py`, soft sun shadows, humanoid NPC rigs with archetype props + procedural walk bob and attention head-turn, inspection artifact cards, escalation vignette, mode icons, and a body-camera overlay (REC + shift clock)
|
||||
|
||||
|
||||
@@ -1,55 +1,8 @@
|
||||
# TODO
|
||||
|
||||
Godot 4.3 prototype. Release **v0.2.9** is tagged. Headless suite **139/139**, plus the session,
|
||||
PLAN-verb, crowd, emergency, timeline, audio, and windowed focus/radial smokes.
|
||||
|
||||
Host `REQUIRE_SIGNIN_VIEW` stays deferred.
|
||||
|
||||
## Scenario content still missing vs PLAN lists
|
||||
|
||||
- [ ] Joke Shop: guest on the reservation list without ID; borrowed/mismatched ID; guest
|
||||
refuses wand screening; late arrival during a performance.
|
||||
- [ ] Studio: filming-boundary repeat (influencer ignores a posted line); celebrity-arrival
|
||||
crowd spike (more rubberneckers, not only one `arrive` spawn).
|
||||
- [ ] Building: "activate body camera" and "request reason" once those verbs exist, used in
|
||||
at least one authored event.
|
||||
|
||||
## Distribution follow-up (host config, deferred)
|
||||
|
||||
Verified via API+host credentials (2026-10-31):
|
||||
|
||||
- [x] Repo is **public** (`private:false`) and account has admin rights.
|
||||
- [x] Release `v0.2.1` now carries all five artifacts (portable binaries + NSIS installer +
|
||||
AppImage).
|
||||
- [ ] **Remaining host setting**: instance-wide `[service] REQUIRE_SIGNIN_VIEW=true` makes
|
||||
anonymous browsing/downloads redirect to login. Host admin: `REQUIRE_SIGNIN_VIEW = false`
|
||||
under `[service]` in `app.ini`, restart Gitea. Alternative: mirror downloads.
|
||||
|
||||
## Release pipeline follow-up
|
||||
|
||||
- [x] Fix `tools/publish_release.sh`: the `~/.git-credentials` branch never set `AUTH`,
|
||||
so every tag build since v0.2.2 failed to publish.
|
||||
- [x] Add a `github.token` fallback in `.gitea/workflows/build.yml` (the repo has no
|
||||
`GITEA_TOKEN` secret).
|
||||
- [ ] Verify the fallback on the next tag push produces a release automatically.
|
||||
- [ ] Optionally backfill releases for v0.2.4–v0.2.8 (their CI artifacts were not
|
||||
retained; would need local rebuilds) or leave them superseded by the latest.
|
||||
|
||||
## Open Polish
|
||||
|
||||
- [ ] Keep `%` placeholders stable in `Strings.fmt` when adding keys (positional formatting).
|
||||
|
||||
## Screen design references (QA tooling)
|
||||
|
||||
- [x] `tools/screen_designs.py generate` writes one reference design per screen
|
||||
(from the `docs/` descriptions) to `docs/screens/designs/`, with an offline
|
||||
`--draft` wireframe mode and an OpenRouter `--ai` mockup mode.
|
||||
- [x] `tools/screen_designs.py compare` checks `docs/screenshots/*.png` against the
|
||||
designs (aspect ratio, size, blank-frame, and layout/palette/detail drift).
|
||||
- [ ] Vision-model judge for semantic screenshot comparison. Pixel and perceptual
|
||||
metrics cannot decide whether a frame's composition is correct; a follow-up
|
||||
should send the capture + reference design to a vision model and ask whether
|
||||
they depict the same screen.
|
||||
Godot 4.3 prototype. Latest release **v0.2.15**. Headless suite **153/153** plus 30 Python
|
||||
tooling tests; the session, PLAN-verb, crowd, emergency, timeline, audio, scenario-content, and
|
||||
windowed focus/radial smokes pass. Anonymous downloads are enabled on the host.
|
||||
|
||||
## Tooling (when an editor workflow exists)
|
||||
|
||||
|
||||
@@ -56,6 +56,15 @@
|
||||
"reason.ask": "You ask %s why they are here.",
|
||||
"reason.reply": "%s: \"%s\"",
|
||||
"reason.silent": "%s says nothing and looks away.",
|
||||
"hint.reservation_no_id": "On the reservation list but no ID — ask for ID, then check the list.",
|
||||
"hint.borrowed_id": "The ID photo does not match the person. Verify before admitting.",
|
||||
"hint.refuses_wand": "This guest refuses the wand screen. No screen, no entry.",
|
||||
"hint.late_arrival": "Late arrival during the performance — check ID and ticket before opening the door.",
|
||||
"hint.film_boundary": "Filming past the posted line. Ask their reason and point them back to the sidewalk.",
|
||||
"hint.film_boundary_repeat": "Same crew again, still past the line. A formal warning is now reasonable.",
|
||||
"hint.celebrity_arrival": "Celebrity arrival: the crowd is swelling. Keep the entrance clear.",
|
||||
"hint.body_camera_reason": "They are filming the entrance. Ask their reason (Reason) and consider the body camera (B).",
|
||||
"hint.ask_reason": "Unclear intent. Ask why they are here (Reason) before deciding anything else.",
|
||||
"station.blocking": "You step between %s and the door.",
|
||||
"perception.near": "%s notices you standing close.",
|
||||
"perception.aware": "%s notices you watching.",
|
||||
|
||||
+16
-4
@@ -37,11 +37,23 @@ The repository ships a Gitea Actions workflow (`.gitea/workflows/build.yml`):
|
||||
Linux/Windows/macOS, artifact upload, and — on tags — release publication via
|
||||
`tools/publish_release.sh`.
|
||||
|
||||
For tag builds the workflow uses `GITEA_TOKEN` (write:repository scope) or, when the
|
||||
secret is absent, falls back to the workflow's built-in `github.token`. Either is
|
||||
enough for the publish step; with neither, the build succeeds but no release is
|
||||
created.
|
||||
`tools/publish_release.sh` picks the first available credential, in order:
|
||||
|
||||
1. `GITEA_TOKEN` (repo secret, write:repository scope),
|
||||
2. `GITEA_AUTH_HEADER` (a ready-made `Authorization: ...` header),
|
||||
3. `~/.git-credentials` for the host,
|
||||
4. the **local git extraheader** that `actions/checkout` configures on the CI job.
|
||||
|
||||
Option 4 is what makes tag builds publish without any repo secret: the runner
|
||||
already injects basic-auth credentials for the checkout, and the script reuses that
|
||||
header for the API (same server). `github.token` is *not* available in this Gitea
|
||||
setup, so do not rely on it.
|
||||
|
||||
> **Disk space:** Gitea stores release assets under `/var/lib/gitea/data/attachments` on
|
||||
> the host root filesystem (7.8 GB here). When it is full, asset uploads fail with
|
||||
> `HTTP 500 … no space left on device` and even `git push` can fail with "unpacker error".
|
||||
> `publish_release.sh` now verifies the asset count and exits non-zero, so this is visible.
|
||||
>
|
||||
> **Incident (2026-10-31):** `tools/publish_release.sh` never set `AUTH` in its
|
||||
> `~/.git-credentials` branch, so every API call failed silently (`AUTH[@]` was
|
||||
> unbound under `set -u`). Combined with no `GITEA_TOKEN` secret on the repo, tag
|
||||
|
||||
+1
-1
@@ -8,7 +8,7 @@ config_version=5
|
||||
|
||||
config/name="Step Line"
|
||||
config/description="Workplace security simulation about observation, access verification, and proportionate escalation."
|
||||
config/version="0.2.10"
|
||||
config/version="0.2.15"
|
||||
run/main_scene="res://scenes/main.tscn"
|
||||
config/features=PackedStringArray("4.3", "GL Compatibility")
|
||||
|
||||
|
||||
@@ -343,6 +343,28 @@ static func hollywood_building() -> VenueScenario:
|
||||
_building_routine("staff_entry", Rules.NpcType.AUTHORIZED_STAFF, "enter", Vector2(0, 600), 0.35),
|
||||
_building_routine("guest_entry", Rules.NpcType.AUTHORIZED_GUEST, "enter", Vector2(0, 900), 0.25),
|
||||
_building_routine("performer_show", Rules.NpcType.PERFORMER, "perform", Vector2(300, 800), 0.24),
|
||||
{
|
||||
"id": "building_filming_boundary",
|
||||
"phase": Rules.Phase.MIDDAY,
|
||||
"kind": "scripted",
|
||||
"npc_type": Rules.NpcType.INFLUENCER,
|
||||
"intent": "film",
|
||||
"time_range_s": Vector2(300, 600),
|
||||
"probability": 0.4,
|
||||
"base_weight": 1.0,
|
||||
"hint": "hint.body_camera_reason",
|
||||
},
|
||||
{
|
||||
"id": "building_unclear_intent",
|
||||
"phase": Rules.Phase.EVENING,
|
||||
"kind": "scripted",
|
||||
"npc_type": Rules.NpcType.CONFUSED_VISITOR,
|
||||
"intent": "enter",
|
||||
"time_range_s": Vector2(600, 780),
|
||||
"probability": 0.4,
|
||||
"base_weight": 1.0,
|
||||
"hint": "hint.ask_reason",
|
||||
},
|
||||
_building_routine("influencer_film", Rules.NpcType.INFLUENCER, "film", Vector2(300, 900), 0.26),
|
||||
_building_routine("intoxicated_wander", Rules.NpcType.INTOXICATED, "wander", Vector2(600, 900), 0.15),
|
||||
_building_routine("trespass_test", Rules.NpcType.TRESPASSER, "trespass", Vector2(300, 900), 0.16),
|
||||
@@ -437,6 +459,67 @@ static func joke_shop() -> VenueScenario:
|
||||
_joke_guest("guest_no_ticket", false, true, Vector2(0, 900), 0.18),
|
||||
_joke_guest("guest_expired_id", true, true, Vector2(0, 900), 0.12, {"id_status": Rules.ArtifactStatus.EXPIRED}),
|
||||
_joke_guest("guest_no_id", true, false, Vector2(0, 900), 0.12),
|
||||
{
|
||||
"id": "joke_guest_reservation_no_id",
|
||||
"phase": Rules.Phase.EVENING,
|
||||
"kind": "scripted",
|
||||
"npc_type": Rules.NpcType.AUTHORIZED_GUEST,
|
||||
"intent": "enter",
|
||||
"time_range_s": Vector2(620, 860),
|
||||
"probability": 0.45,
|
||||
"base_weight": 1.0,
|
||||
"override_access": {
|
||||
"has_guest_list": true, "guest_list_status": Rules.ArtifactStatus.VALID,
|
||||
"has_id": false, "has_ticket": false,
|
||||
},
|
||||
"hint": "hint.reservation_no_id",
|
||||
},
|
||||
{
|
||||
"id": "joke_borrowed_id",
|
||||
"phase": Rules.Phase.MIDDAY,
|
||||
"kind": "scripted",
|
||||
"npc_type": Rules.NpcType.AUTHORIZED_GUEST,
|
||||
"intent": "enter",
|
||||
"time_range_s": Vector2(320, 700),
|
||||
"probability": 0.4,
|
||||
"base_weight": 1.0,
|
||||
"override_access": {
|
||||
"has_id": true, "id_status": Rules.ArtifactStatus.MISMATCHED,
|
||||
"has_ticket": true, "ticket_status": Rules.ArtifactStatus.VALID,
|
||||
},
|
||||
"hint": "hint.borrowed_id",
|
||||
},
|
||||
{
|
||||
"id": "joke_refuses_wand",
|
||||
"phase": Rules.Phase.EVENING,
|
||||
"kind": "scripted",
|
||||
"npc_type": Rules.NpcType.AUTHORIZED_GUEST,
|
||||
"intent": "enter",
|
||||
"time_range_s": Vector2(650, 880),
|
||||
"probability": 0.4,
|
||||
"base_weight": 1.0,
|
||||
"refuses_screening": true,
|
||||
"override_access": {
|
||||
"has_id": true, "id_status": Rules.ArtifactStatus.VALID,
|
||||
"has_ticket": true, "ticket_status": Rules.ArtifactStatus.VALID,
|
||||
},
|
||||
"hint": "hint.refuses_wand",
|
||||
},
|
||||
{
|
||||
"id": "joke_late_arrival",
|
||||
"phase": Rules.Phase.EVENING,
|
||||
"kind": "scripted",
|
||||
"npc_type": Rules.NpcType.AUTHORIZED_GUEST,
|
||||
"intent": "arrive",
|
||||
"time_range_s": Vector2(700, 860),
|
||||
"probability": 0.35,
|
||||
"base_weight": 1.0,
|
||||
"override_access": {
|
||||
"has_id": true, "id_status": Rules.ArtifactStatus.VALID,
|
||||
"has_ticket": true, "ticket_status": Rules.ArtifactStatus.VALID,
|
||||
},
|
||||
"hint": "hint.late_arrival",
|
||||
},
|
||||
_joke_staff_entry(Vector2(0, 900), 0.25),
|
||||
_joke_routine("performer_show", Rules.NpcType.PERFORMER, "perform", Vector2(300, 800), 0.15),
|
||||
_joke_routine("influencer_film", Rules.NpcType.INFLUENCER, "film", Vector2(300, 900), 0.15),
|
||||
@@ -515,6 +598,45 @@ static func tv_studio() -> VenueScenario:
|
||||
"base_weight": 1.0,
|
||||
"override_access": {"wristband_status": Rules.ArtifactStatus.COUNTERFEIT, "credential_status": Rules.ArtifactStatus.COUNTERFEIT},
|
||||
},
|
||||
{
|
||||
"id": "studio_film_boundary",
|
||||
"phase": Rules.Phase.MIDDAY,
|
||||
"kind": "scripted",
|
||||
"npc_type": Rules.NpcType.INFLUENCER,
|
||||
"intent": "film_boundary",
|
||||
"time_range_s": Vector2(330, 560),
|
||||
"probability": 0.5,
|
||||
"base_weight": 1.0,
|
||||
"hint": "hint.film_boundary",
|
||||
},
|
||||
{
|
||||
"id": "studio_film_boundary_repeat",
|
||||
"phase": Rules.Phase.EVENING,
|
||||
"kind": "scripted",
|
||||
"npc_type": Rules.NpcType.INFLUENCER,
|
||||
"intent": "film_boundary",
|
||||
"time_range_s": Vector2(640, 830),
|
||||
"probability": 0.45,
|
||||
"base_weight": 1.0,
|
||||
"hint": "hint.film_boundary_repeat",
|
||||
},
|
||||
{
|
||||
"id": "studio_celebrity_arrival",
|
||||
"phase": Rules.Phase.EVENING,
|
||||
"kind": "scripted",
|
||||
"npc_type": Rules.NpcType.AUTHORIZED_GUEST,
|
||||
"intent": "arrive",
|
||||
"time_range_s": Vector2(620, 760),
|
||||
"probability": 0.5,
|
||||
"base_weight": 1.0,
|
||||
"override_access": {"has_guest_list": true, "guest_list_status": Rules.ArtifactStatus.VALID},
|
||||
"extra_spawns": [
|
||||
{"npc_type": Rules.NpcType.TOURIST, "intent": "photo"},
|
||||
{"npc_type": Rules.NpcType.TOURIST, "intent": "photo"},
|
||||
{"npc_type": Rules.NpcType.INFLUENCER, "intent": "film"},
|
||||
],
|
||||
"hint": "hint.celebrity_arrival",
|
||||
},
|
||||
{
|
||||
"id": "studio_follow_through",
|
||||
"phase": Rules.Phase.EVENING,
|
||||
@@ -587,6 +709,10 @@ static func _joke_routine(id: String, npc_type: int, intent: String, time_range:
|
||||
static func _studio_routine(id: String, npc_type: int, intent: String, time_range: Vector2, prob: float) -> Dictionary:
|
||||
return _building_routine(id, npc_type, intent, time_range, prob)
|
||||
|
||||
## Scripted film-boundary intent: the influencer stands past the posted line.
|
||||
static func _film_boundary_spot() -> String:
|
||||
return "steps_end"
|
||||
|
||||
static func _phase_for_range(time_range: Vector2, duration: float) -> int:
|
||||
var start := time_range.x / duration
|
||||
if start < 0.34:
|
||||
|
||||
@@ -34,6 +34,8 @@ var guard_noticed := false
|
||||
|
||||
## Whether this NPC has passed the handheld wand screening (The Joke Shop).
|
||||
var screened := false
|
||||
## Authored event flag: this person refuses the wand screen (Joke Shop).
|
||||
var refuses_screening := false
|
||||
## Whether the guard has declared they recognize this person (a player decision
|
||||
## recorded as evidence, PLAN.md: Authorized Access).
|
||||
var player_recognized := false
|
||||
|
||||
@@ -34,12 +34,69 @@ static func lookup(key: String) -> String:
|
||||
_ensure_loaded()
|
||||
return TABLE.get(key, key)
|
||||
|
||||
## Number of `%` conversion placeholders in a template, ignoring `%%` escapes.
|
||||
static func placeholder_count(template: String) -> int:
|
||||
var count := 0
|
||||
var i := 0
|
||||
while i < template.length():
|
||||
if template[i] == "%":
|
||||
if i + 1 < template.length() and template[i + 1] == "%":
|
||||
i += 2
|
||||
continue
|
||||
count += 1
|
||||
i += 1
|
||||
return count
|
||||
|
||||
## Format a string by key with positional arguments.
|
||||
##
|
||||
## GDScript's `%` binds arguments by order, so a key whose placeholder count drifts
|
||||
## from its call site would push an engine error. Validate first and fail safe: log
|
||||
## a warning and return the raw template instead of crashing.
|
||||
static func fmt(key: String, args: Array) -> String:
|
||||
var template := lookup(key)
|
||||
if template == key:
|
||||
return template
|
||||
var needed := placeholder_count(template)
|
||||
if needed != args.size():
|
||||
push_warning("Strings.fmt(%s): template needs %d args, got %d" % [key, needed, args.size()])
|
||||
return template
|
||||
return template % args
|
||||
|
||||
## Lint the loaded table for formatting hazards. Returns an array of problems.
|
||||
## Checks: trailing `%`, invalid conversions, and mixed positional (`%1$s`) and
|
||||
## sequential placeholders in one template.
|
||||
static func validate() -> Array[String]:
|
||||
_ensure_loaded()
|
||||
var problems: Array[String] = []
|
||||
for key in TABLE.keys():
|
||||
var template := str(TABLE[key])
|
||||
var positional := template.contains("$")
|
||||
var sequential := 0
|
||||
var i := 0
|
||||
while i < template.length():
|
||||
if template[i] != "%":
|
||||
i += 1
|
||||
continue
|
||||
if i + 1 >= template.length():
|
||||
problems.append("%s: trailing %% escape" % key)
|
||||
break
|
||||
var next := template[i + 1]
|
||||
if next == "%":
|
||||
i += 2
|
||||
continue
|
||||
var j := i + 1
|
||||
while j < template.length() and template[j] in "0123456789.$":
|
||||
if template[j] == "$":
|
||||
positional = true
|
||||
j += 1
|
||||
if j >= template.length() or not (template[j] in "sdfxXv"):
|
||||
problems.append("%s: unsupported conversion near '%%%s'" % [key, template.substr(i + 1, 3)])
|
||||
elif not positional:
|
||||
sequential += 1
|
||||
i = j + 1
|
||||
if positional and sequential > 0:
|
||||
problems.append("%s: mixes positional (%%1$s) and sequential (%%s) placeholders" % key)
|
||||
return problems
|
||||
|
||||
func _init() -> void:
|
||||
push_error("Strings is a static utility class; use Strings.lookup / Strings.fmt.")
|
||||
@@ -63,7 +63,9 @@ func _spot_for_intent(intent: String) -> String:
|
||||
return "entrance"
|
||||
"photo", "film", "perform", "wander", "roam":
|
||||
return "sidewalk"
|
||||
"sit":
|
||||
"sit", "film_boundary":
|
||||
# film_boundary: an influencer ignoring the posted filming line, so they
|
||||
# stand at the steps instead of the permitted sidewalk spot.
|
||||
return "steps_end"
|
||||
_:
|
||||
return "sidewalk"
|
||||
|
||||
@@ -479,8 +479,19 @@ func _spawn_npc(profile: NpcProfile, intent: String, npc_id: String, event: Dict
|
||||
# Authored events stay for the whole shift; routine traffic cycles out.
|
||||
avatar.ephemeral = event.get("kind", "routine") == "routine"
|
||||
avatar.lifetime_s = 45.0 + float((npc_counter * 17) % 60)
|
||||
avatar.get_npc_controller().refuses_screening = bool(event.get("refuses_screening", false))
|
||||
scene_root.add_child(avatar)
|
||||
npcs.append(avatar)
|
||||
# Authored events can point the player at the right verb (PLAN.md verbs).
|
||||
if event.has("hint") and hud:
|
||||
hud.log(Strings.lookup(str(event["hint"])))
|
||||
# Celebrity arrivals draw extra bystanders, not just one avatar.
|
||||
for extra in event.get("extra_spawns", []):
|
||||
var extra_profile := selected_scenario.profile_for(extra.get("npc_type", Rules.NpcType.TOURIST))
|
||||
if extra_profile == null or npcs.size() >= MAX_LIVE_NPCS:
|
||||
continue
|
||||
_spawn_npc(extra_profile, str(extra.get("intent", "photo")),
|
||||
"%s_w%d" % [npc_id, npcs.size()], {"kind": "routine"})
|
||||
|
||||
## Drop references to freed avatars so the cap and Tab cycling stay accurate.
|
||||
func _purge_npcs() -> void:
|
||||
@@ -1238,6 +1249,16 @@ func _toggle_wand_screening() -> void:
|
||||
var controller: Object = _targeted_controller()
|
||||
if controller == null:
|
||||
return
|
||||
if controller.refuses_screening:
|
||||
# Authored event: this guest refuses the screen (Joke Shop content).
|
||||
hud.set_wand_icon(false)
|
||||
hud.log(Strings.fmt("wand.refused", [controller.profile.display_name if controller.profile else "The guest"]))
|
||||
coordinator.recorder.add_event(controller.npc_id, {
|
||||
"type": "wand_refused",
|
||||
"outcome": "refused",
|
||||
})
|
||||
coordinator.reputation.apply("prop", -1.0, "guest refused screening")
|
||||
return
|
||||
var profile: Object = controller.profile
|
||||
var items: Array = profile.carried_items if profile else []
|
||||
var result: Dictionary = wand_screener.screen(items)
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
extends SceneTree
|
||||
## Scenario-content smoke: forces the authored Joke Shop refusal event and the
|
||||
## Studio celebrity arrival through the real spawn path, then checks the extra
|
||||
## bystanders, the hint log, and that a screening refusal is handled.
|
||||
##
|
||||
## Usage:
|
||||
## godot --headless --path . --script res://src/test/smoke_scenario_content.gd
|
||||
|
||||
const MAX_FRAMES := 900
|
||||
|
||||
var frame := 0
|
||||
var gm: Node = null
|
||||
var started := false
|
||||
var step := 0
|
||||
var failures: Array[String] = []
|
||||
|
||||
func _initialize() -> void:
|
||||
root.size = Vector2i(1280, 720)
|
||||
root.add_child(load("res://scenes/main.tscn").instantiate())
|
||||
|
||||
func _process(_delta: float) -> bool:
|
||||
frame += 1
|
||||
if gm == null:
|
||||
gm = root.find_child("GameManager", true, false)
|
||||
return false
|
||||
if not started:
|
||||
gm.call("_on_venue_selected", "joke_shop")
|
||||
started = true
|
||||
return false
|
||||
if frame > MAX_FRAMES:
|
||||
_fail("timeout")
|
||||
return true
|
||||
if frame < 120:
|
||||
return false
|
||||
match step:
|
||||
0:
|
||||
# Force the refusal event through the normal spawn signal.
|
||||
var event := _find_event(gm.selected_scenario.event_pool, "joke_refuses_wand")
|
||||
_check(not event.is_empty(), "the refusal event exists in the Joke Shop pool")
|
||||
var guest: Object = gm.selected_scenario.profile_for(int(event.get("npc_type", 0)))
|
||||
gm.coordinator.npc_spawn_requested.emit(guest, "enter", "smoke_refuser", event)
|
||||
_check(gm.npcs.size() > 0, "the refusing guest spawned")
|
||||
gm.call("_on_guard_target", gm.npcs[gm.npcs.size() - 1])
|
||||
var controller: Object = gm.npcs[gm.npcs.size() - 1].get_npc_controller()
|
||||
_check(controller.refuses_screening, "the refusal flag reached the controller")
|
||||
gm.call("_toggle_wand_screening")
|
||||
_check(not gm.wand_open, "screening does not open for someone who refuses")
|
||||
step = 1
|
||||
1:
|
||||
# Studio: celebrity arrival should bring extra bystanders.
|
||||
gm.call("_on_venue_selected", "tv_studio")
|
||||
step = 2
|
||||
2:
|
||||
if frame % 20 != 0:
|
||||
return false
|
||||
var event := _find_event(gm.selected_scenario.event_pool, "studio_celebrity_arrival")
|
||||
var before: int = gm.npcs.size()
|
||||
var guest: Object = gm.selected_scenario.profile_for(int(event.get("npc_type", 0)))
|
||||
gm.coordinator.npc_spawn_requested.emit(guest, "arrive", "smoke_celeb", event)
|
||||
var added: int = gm.npcs.size() - before
|
||||
_check(added >= 4, "celebrity arrival spawns the guest plus bystanders (%d)" % added)
|
||||
step = 3
|
||||
3:
|
||||
_finish()
|
||||
return true
|
||||
return false
|
||||
|
||||
func _find_event(pool: Array, id: String) -> Dictionary:
|
||||
for event in pool:
|
||||
if str((event as Dictionary).get("id", "")) == id:
|
||||
return event
|
||||
return {}
|
||||
|
||||
func _check(condition: bool, message: String) -> void:
|
||||
if condition:
|
||||
print("ok: " + message)
|
||||
else:
|
||||
_fail(message)
|
||||
|
||||
func _fail(message: String) -> void:
|
||||
failures.append(message)
|
||||
_finish()
|
||||
|
||||
func _finish() -> void:
|
||||
if failures.is_empty():
|
||||
print("SMOKE OK: authored scenario content")
|
||||
quit(0)
|
||||
else:
|
||||
for message in failures:
|
||||
printerr("SMOKE FAIL: " + message)
|
||||
quit(1)
|
||||
@@ -0,0 +1,143 @@
|
||||
extends RefCounted
|
||||
## Authored scenario content added to match the PLAN lists: Joke Shop access
|
||||
## edge cases, Studio filming boundaries and celebrity crowd spikes, and Building
|
||||
## events that point at the body-camera / request-reason verbs.
|
||||
|
||||
const Factory := preload("res://src/content/scenario_factory.gd")
|
||||
const Rules := preload("res://src/core/rules.gd")
|
||||
const Strings := preload("res://src/core/strings.gd")
|
||||
const AccessLoadout := preload("res://src/core/access_loadout.gd")
|
||||
const AccessVerifier := preload("res://src/core/access_verifier.gd")
|
||||
const NpcProfile := preload("res://src/core/npc_profile.gd")
|
||||
|
||||
func _events(scenario_id: String) -> Array:
|
||||
var scenario: Resource
|
||||
match scenario_id:
|
||||
"joke_shop":
|
||||
scenario = Factory.joke_shop()
|
||||
"tv_studio":
|
||||
scenario = Factory.tv_studio()
|
||||
_:
|
||||
scenario = Factory.hollywood_building()
|
||||
return scenario.event_pool
|
||||
|
||||
func _event(scenario_id: String, id: String) -> Dictionary:
|
||||
for event in _events(scenario_id):
|
||||
if str(event.get("id", "")) == id:
|
||||
return event
|
||||
return {}
|
||||
|
||||
func test_joke_shop_has_the_missing_edge_cases() -> bool:
|
||||
var a := TestUtils.check(not _event("joke_shop", "joke_guest_reservation_no_id").is_empty(),
|
||||
"reservation list without ID exists")
|
||||
var b := TestUtils.check(not _event("joke_shop", "joke_borrowed_id").is_empty(),
|
||||
"borrowed/mismatched ID exists")
|
||||
var c := TestUtils.check(not _event("joke_shop", "joke_refuses_wand").is_empty(),
|
||||
"guest who refuses the wand exists")
|
||||
var d := TestUtils.check(not _event("joke_shop", "joke_late_arrival").is_empty(),
|
||||
"late arrival during the performance exists")
|
||||
return a and b and c and d
|
||||
|
||||
func test_reservation_without_id_is_denied() -> bool:
|
||||
var event := _event("joke_shop", "joke_guest_reservation_no_id")
|
||||
var profile := _guest_profile()
|
||||
var artifacts := AccessLoadout.build(profile, Factory.joke_shop().access_policy, event.get("override_access", {}))
|
||||
var verifier := AccessVerifier.new()
|
||||
var result: Dictionary = verifier.verify(artifacts, Factory.joke_shop().access_policy, false)
|
||||
var a := TestUtils.check(not result.get("granted", true), "a reservation without ID is not admitted")
|
||||
var b := TestUtils.check(not (result.get("issues", []) as Array).is_empty(), "the denial names the missing artifact")
|
||||
return a and b
|
||||
|
||||
func test_borrowed_id_is_mismatched() -> bool:
|
||||
var event := _event("joke_shop", "joke_borrowed_id")
|
||||
var profile := _guest_profile()
|
||||
var artifacts := AccessLoadout.build(profile, Factory.joke_shop().access_policy, event.get("override_access", {}))
|
||||
var found_mismatch := false
|
||||
for artifact in artifacts:
|
||||
if artifact.kind == Rules.AccessKind.ID and artifact.status == Rules.ArtifactStatus.MISMATCHED:
|
||||
found_mismatch = true
|
||||
var verifier := AccessVerifier.new()
|
||||
var result: Dictionary = verifier.verify(artifacts, Factory.joke_shop().access_policy, false)
|
||||
var a := TestUtils.check(found_mismatch, "the ID is flagged mismatched")
|
||||
var b := TestUtils.check(not result.get("granted", true), "a mismatched ID is not admitted")
|
||||
return a and b
|
||||
|
||||
func test_refuses_wand_event_sets_the_flag() -> bool:
|
||||
var event := _event("joke_shop", "joke_refuses_wand")
|
||||
var ok := TestUtils.check(bool(event.get("refuses_screening", false)), "the event marks the refusal")
|
||||
var src := FileAccess.get_file_as_string("res://src/game/game_manager.gd")
|
||||
var wired := TestUtils.check(src.contains("refuses_screening = bool(event.get("),
|
||||
"the spawn copies the flag onto the controller")
|
||||
var handled := TestUtils.check(src.contains("if controller.refuses_screening:"),
|
||||
"the wand screen handles a refusal")
|
||||
return ok and wired and handled
|
||||
|
||||
func test_studio_filming_boundary_repeats() -> bool:
|
||||
var first := _event("tv_studio", "studio_film_boundary")
|
||||
var repeat := _event("tv_studio", "studio_film_boundary_repeat")
|
||||
var a := TestUtils.check(not first.is_empty(), "first boundary event exists")
|
||||
var b := TestUtils.check(not repeat.is_empty(), "repeat boundary event exists")
|
||||
var c := TestUtils.check_eq(str(first.get("intent", "")), "film_boundary", "first uses the boundary intent")
|
||||
var d := TestUtils.check_eq(str(repeat.get("intent", "")), "film_boundary", "repeat uses the boundary intent")
|
||||
var e := TestUtils.check(int(repeat.get("time_range_s", Vector2.ZERO).x) > int(first.get("time_range_s", Vector2.ZERO).x),
|
||||
"the repeat happens later in the shift")
|
||||
return a and b and c and d and e
|
||||
|
||||
func test_film_boundary_stands_at_the_steps() -> bool:
|
||||
var scenario := Factory.tv_studio()
|
||||
var spot: Vector3 = scenario.waypoint_for_intent("film_boundary")
|
||||
var sidewalk: Vector3 = scenario.waypoint_for_intent("film")
|
||||
var a := TestUtils.check(spot != sidewalk, "the boundary spot differs from the permitted sidewalk spot")
|
||||
var b := TestUtils.check_eq(spot, scenario._location_or_default("steps_end", Vector3.ZERO),
|
||||
"the influencer stands at the steps, past the line")
|
||||
return a and b
|
||||
|
||||
func test_celebrity_arrival_spawns_a_crowd() -> bool:
|
||||
var event := _event("tv_studio", "studio_celebrity_arrival")
|
||||
var extras: Array = event.get("extra_spawns", [])
|
||||
var a := TestUtils.check(extras.size() >= 3, "celebrity arrival adds several bystanders")
|
||||
var types: Array = []
|
||||
for extra in extras:
|
||||
types.append(int((extra as Dictionary).get("npc_type", -1)))
|
||||
var b := TestUtils.check(types.has(Rules.NpcType.TOURIST), "tourists gather")
|
||||
var c := TestUtils.check(types.has(Rules.NpcType.INFLUENCER), "an influencer films it")
|
||||
var src := FileAccess.get_file_as_string("res://src/game/game_manager.gd")
|
||||
var d := TestUtils.check(src.contains('event.get("extra_spawns", [])'), "the spawner honours extra_spawns")
|
||||
return a and b and c and d
|
||||
|
||||
func test_building_events_point_at_the_new_verbs() -> bool:
|
||||
var camera_event := _event("hollywood_building", "building_filming_boundary")
|
||||
var reason_event := _event("hollywood_building", "building_unclear_intent")
|
||||
var a := TestUtils.check(str(camera_event.get("hint", "")).contains("body"), "an event hints at the body camera")
|
||||
var b := TestUtils.check(str(reason_event.get("hint", "")).contains("reason"), "an event hints at requesting a reason")
|
||||
var src := FileAccess.get_file_as_string("res://src/game/game_manager.gd")
|
||||
var c := TestUtils.check(src.contains('if event.has("hint")'), "spawning logs the authored hint")
|
||||
return a and b and c
|
||||
|
||||
func test_every_authored_hint_resolves() -> bool:
|
||||
var missing: Array[String] = []
|
||||
for scenario in [Factory.hollywood_building(), Factory.joke_shop(), Factory.tv_studio()]:
|
||||
for event in scenario.event_pool:
|
||||
var key := str((event as Dictionary).get("hint", ""))
|
||||
if key != "" and Strings.lookup(key) == key:
|
||||
missing.append(key)
|
||||
return TestUtils.check(missing.is_empty(), "all authored hints have strings (missing: %s)" % str(missing))
|
||||
|
||||
func test_new_events_have_unique_ids() -> bool:
|
||||
# Event ids only need to be unique within a venue.
|
||||
var problems: Array[String] = []
|
||||
for scenario in [Factory.hollywood_building(), Factory.joke_shop(), Factory.tv_studio()]:
|
||||
var seen := {}
|
||||
for event in scenario.event_pool:
|
||||
var id := str((event as Dictionary).get("id", ""))
|
||||
if seen.has(id):
|
||||
problems.append("%s:%s" % [scenario.id, id])
|
||||
seen[id] = true
|
||||
return TestUtils.check(problems.is_empty(), "event ids are unique per venue (dupes: %s)" % str(problems))
|
||||
|
||||
func _guest_profile() -> NpcProfile:
|
||||
var p := NpcProfile.new()
|
||||
p.id = "guest"
|
||||
p.display_name = "Guest"
|
||||
p.npc_type = Rules.NpcType.AUTHORIZED_GUEST
|
||||
return p
|
||||
@@ -23,6 +23,70 @@ func test_strings_json_backed_all_resolve() -> bool:
|
||||
return TestUtils.fail("key %s resolves to itself — table entry missing" % key)
|
||||
return true
|
||||
|
||||
func test_placeholder_count() -> bool:
|
||||
var a := TestUtils.check_eq(Strings.placeholder_count("no args"), 0, "plain text has no placeholders")
|
||||
var b := TestUtils.check_eq(Strings.placeholder_count("%s complies"), 1, "one %s")
|
||||
var c := TestUtils.check_eq(Strings.placeholder_count("%s: \"%s\""), 2, "two %s")
|
||||
var d := TestUtils.check_eq(Strings.placeholder_count("100%% sure"), 0, "%% is an escape, not an argument")
|
||||
var e := TestUtils.check_eq(Strings.placeholder_count("%.0f%% mood"), 1, "float plus percent sign")
|
||||
return a and b and c and d and e
|
||||
|
||||
func test_fmt_fails_safe_on_arg_mismatch() -> bool:
|
||||
## A drifting placeholder count must not raise an engine error at runtime.
|
||||
var result: String = Strings.fmt("react.comply", [])
|
||||
var a := TestUtils.check_eq(result, Strings.lookup("react.comply"), "wrong arg count returns the raw template")
|
||||
var extra: String = Strings.fmt("react.comply", ["A", "B"])
|
||||
var b := TestUtils.check_eq(extra, Strings.lookup("react.comply"), "too many args also fails safe")
|
||||
return a and b
|
||||
|
||||
func test_table_validates_cleanly() -> bool:
|
||||
var problems: Array = Strings.validate()
|
||||
if not problems.is_empty():
|
||||
return TestUtils.fail("string table format problems: %s" % str(problems))
|
||||
return true
|
||||
|
||||
func test_every_placeholder_key_formats() -> bool:
|
||||
## Feed each template one dummy arg per placeholder, with a type that matches
|
||||
## the conversion (%s text, %d int, %f float). Nothing may be left unformatted.
|
||||
var problems: Array[String] = []
|
||||
for key in Strings.TABLE.keys():
|
||||
var template: String = str(Strings.TABLE[key])
|
||||
var specs: Array[String] = _specifiers(template)
|
||||
if specs.is_empty():
|
||||
continue
|
||||
var args := []
|
||||
for spec in specs:
|
||||
var conv: String = spec.substr(spec.length() - 1)
|
||||
if conv == "d" or conv == "x" or conv == "X":
|
||||
args.append(7)
|
||||
elif conv == "f":
|
||||
args.append(1.5)
|
||||
else:
|
||||
args.append("x")
|
||||
var out: String = Strings.fmt(key, args)
|
||||
if out == template:
|
||||
problems.append("%s did not format (%s)" % [key, str(specs)])
|
||||
return TestUtils.check(problems.is_empty(), "all placeholder keys format (problems: %s)" % str(problems))
|
||||
|
||||
func _specifiers(template: String) -> Array[String]:
|
||||
var out: Array[String] = []
|
||||
var i := 0
|
||||
while i < template.length():
|
||||
if template[i] != "%":
|
||||
i += 1
|
||||
continue
|
||||
if i + 1 < template.length() and template[i + 1] == "%":
|
||||
i += 2
|
||||
continue
|
||||
var j := i + 1
|
||||
while j < template.length() and template[j] in "0123456789.$":
|
||||
j += 1
|
||||
if j < template.length():
|
||||
out.append(template.substr(i, j - i + 1))
|
||||
j += 1
|
||||
i = j
|
||||
return out
|
||||
|
||||
func test_strings_cover_ui_keys() -> bool:
|
||||
## Every key referenced by the HUD and game manager must exist.
|
||||
var required := [
|
||||
|
||||
@@ -16,8 +16,17 @@ HOST="${HOST:-https://git.allucanget.biz}"
|
||||
VERSION="${1:-$(grep -oP 'config/version="\K[^"]+' project.godot)}"
|
||||
TAG="v$VERSION"
|
||||
|
||||
# AUTH is a curl argument list. Populated by the first available method:
|
||||
# 1. GITEA_TOKEN explicit token (repo secret in CI)
|
||||
# 2. GITEA_AUTH_HEADER a ready-made "Authorization: ..." header
|
||||
# 3. ~/.git-credentials basic auth for HOST
|
||||
# 4. the checkout's git extraheader (Gitea Actions runners inject one for the
|
||||
# job; this is what makes tag builds publish without any repo secret)
|
||||
AUTH=()
|
||||
if [[ -n "${GITEA_TOKEN:-}" ]]; then
|
||||
AUTH=("Authorization: token $GITEA_TOKEN")
|
||||
AUTH=(-H "Authorization: token $GITEA_TOKEN")
|
||||
elif [[ -n "${GITEA_AUTH_HEADER:-}" ]]; then
|
||||
AUTH=(-H "$GITEA_AUTH_HEADER")
|
||||
elif [[ -f "$HOME/.git-credentials" ]]; then
|
||||
CRED="$(python3 -c "
|
||||
import os
|
||||
@@ -31,8 +40,20 @@ for l in open(os.path.expanduser('~/.git-credentials')):
|
||||
# curl basic auth. (Previously only CRED was set, so ${AUTH[@]} was unbound
|
||||
# under `set -u` and every API call failed silently.)
|
||||
AUTH=(-u "$CRED")
|
||||
else
|
||||
echo "error: set GITEA_TOKEN or provide ~/.git-credentials" >&2
|
||||
fi
|
||||
|
||||
if [[ ${#AUTH[@]} -eq 0 ]]; then
|
||||
# Gitea Actions: actions/checkout configures a local extraheader with the
|
||||
# runner's credentials. Reuse it for the API (the same server, so the header is
|
||||
# valid for $HOST too).
|
||||
GIT_HEADER="$(git config --local --get-regexp '^http\..*\.extraheader$' 2>/dev/null | head -n1 | cut -d' ' -f2- || true)"
|
||||
if [[ -n "${GIT_HEADER:-}" ]]; then
|
||||
AUTH=(-H "$GIT_HEADER")
|
||||
fi
|
||||
fi
|
||||
|
||||
if [[ ${#AUTH[@]} -eq 0 ]]; then
|
||||
echo "error: no credentials. Set GITEA_TOKEN, GITEA_AUTH_HEADER, or provide ~/.git-credentials." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
@@ -78,8 +99,31 @@ for f in "${all_artifacts[@]}"; do
|
||||
exit 1
|
||||
fi
|
||||
echo "==> Uploading $(basename "$f")"
|
||||
curl -s -X POST "$API/$RELEASE_ID/assets" "${AUTH[@]}" -F "attachment=@$f" >/dev/null
|
||||
# Fail loudly: a silent 4xx here used to leave a release with no artifacts while
|
||||
# the job still reported success.
|
||||
code="$(curl -sS -o /tmp/release-upload.json -w '%{http_code}' --retry 2 \
|
||||
-X POST "$API/$RELEASE_ID/assets" "${AUTH[@]}" -F "attachment=@$f" || true)"
|
||||
if [[ "$code" != "201" && "$code" != "200" ]]; then
|
||||
echo "error: upload of $(basename "$f") failed with HTTP $code" >&2
|
||||
cat /tmp/release-upload.json >&2 || true
|
||||
echo "" >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
# Verify the release actually carries the artifacts we just sent.
|
||||
have="$(curl -sS "${AUTH[@]}" "$API/$RELEASE_ID" | python3 -c "
|
||||
import sys, json
|
||||
try:
|
||||
d = json.load(sys.stdin)
|
||||
except Exception:
|
||||
print(0); raise SystemExit
|
||||
print(len(d.get('assets', [])))" 2>/dev/null || echo 0)"
|
||||
if [[ "${have:-0}" -lt "${#all_artifacts[@]}" ]]; then
|
||||
echo "error: release has $have assets, expected ${#all_artifacts[@]}" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "Verified $have artifacts on the release"
|
||||
|
||||
echo ""
|
||||
echo "Release live: $HOST/$REPO/releases/tag/$TAG"
|
||||
@@ -17,6 +17,12 @@ Two commands:
|
||||
--draft deterministic wireframe sketches drawn with Pillow
|
||||
(no network or API key; good for CI and layout checks).
|
||||
|
||||
judge send each capture together with its reference design to a vision
|
||||
model and ask whether they depict the same screen. Pixel and
|
||||
perceptual metrics cannot judge composition; this closes that gap.
|
||||
Requires OPENROUTER_API_KEY (falls back to the repo .env).
|
||||
--dry-run build and print the requests without calling the API.
|
||||
|
||||
compare compare docs/screenshots/*.png against the reference designs and
|
||||
flag gross drift. Semantic composition is judged with coarse,
|
||||
Pillow-only signals (aspect ratio, frame size, blank-frame
|
||||
@@ -33,6 +39,7 @@ Run from the repo root:
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
@@ -54,6 +61,10 @@ ASPECT = 16 / 9
|
||||
# the model asset_pipeline.py already trusts for non-alpha roles).
|
||||
DEFAULT_MODEL = "openai/gpt-5-image-mini"
|
||||
|
||||
# Vision model used by `judge`. Cheap, multimodal, and good at "same screen?".
|
||||
DEFAULT_JUDGE_MODEL = "google/gemini-2.5-flash-lite"
|
||||
OPENROUTER_CHAT_URL = "https://openrouter.ai/api/v1/chat/completions"
|
||||
|
||||
# Shared art direction, appended to every prompt so the generated frames read
|
||||
# as one coherent game. "scene" entries are first-person 3D world views;
|
||||
# "ui" entries are in-game panels overlaid on a softened scene.
|
||||
@@ -582,6 +593,193 @@ def compare(args):
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Vision judge — semantic "same screen?" check against the reference designs.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
JUDGE_INSTRUCTIONS = (
|
||||
"You are reviewing screenshots of a low-poly 3D security-guard game. "
|
||||
"IMAGE 1 is a CONCEPT MOCKUP of the intended screen (it may be painted in a "
|
||||
"different art style and show different actors); IMAGE 2 is the ACTUAL "
|
||||
"capture from the game. Different art style, fidelity, fonts, and character "
|
||||
"models are EXPECTED and must be ignored.\n\n"
|
||||
"Judge only whether IMAGE 2 realises the same SCREEN and LAYOUT INTENT as "
|
||||
"IMAGE 1: is it the same kind of screen (world view, HUD overlay, modal "
|
||||
"panel), are the main interface regions present in roughly the same places "
|
||||
"(target line, inspection card, report form, on-screen buttons), and is the "
|
||||
"subject of the shot comparable (entrance, street, person)?\n\n"
|
||||
"Answer with JSON only: {\"match\": true|false, \"confidence\": 0..1, "
|
||||
"\"differences\": [\"short point\", ...]}. Set match=true when the capture "
|
||||
"realises the mockup's screen even if it looks much simpler."
|
||||
)
|
||||
|
||||
|
||||
def _load_env_file(path=None):
|
||||
"""Populate os.environ from a .env file without overriding real variables."""
|
||||
path = Path(path) if path else ROOT / ".env"
|
||||
if not path.is_file():
|
||||
return
|
||||
for raw in path.read_text(encoding="utf-8").splitlines():
|
||||
line = raw.strip()
|
||||
if not line or line.startswith("#") or "=" not in line:
|
||||
continue
|
||||
key, value = line.split("=", 1)
|
||||
os.environ.setdefault(key.strip(), value.strip().strip("\"'"))
|
||||
|
||||
|
||||
def _data_url(path):
|
||||
import base64
|
||||
payload = Path(path).read_bytes()
|
||||
return "data:image/png;base64," + base64.b64encode(payload).decode("ascii")
|
||||
|
||||
|
||||
def build_judge_request(model, screen, capture_path, design_path):
|
||||
"""Build the OpenRouter chat payload for one capture/design pair."""
|
||||
prompt = (f"{JUDGE_INSTRUCTIONS}\n\nScreen: {screen['title']} "
|
||||
f"({screen['source']}). Design notes: {screen['description']}")
|
||||
return {
|
||||
"model": model,
|
||||
"messages": [{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "text", "text": prompt},
|
||||
{"type": "image_url", "image_url": {"url": _data_url(design_path)}},
|
||||
{"type": "image_url", "image_url": {"url": _data_url(capture_path)}},
|
||||
],
|
||||
}],
|
||||
"temperature": 0,
|
||||
"response_format": {"type": "json_object"},
|
||||
}
|
||||
|
||||
|
||||
def parse_judge_response(payload):
|
||||
"""Extract a verdict dict from an OpenRouter chat response."""
|
||||
try:
|
||||
text = payload["choices"][0]["message"]["content"]
|
||||
except (KeyError, IndexError, TypeError):
|
||||
return {"match": None, "confidence": 0.0,
|
||||
"differences": ["unreadable model response"]}
|
||||
if isinstance(text, list): # some providers return content parts
|
||||
text = "".join(part.get("text", "") for part in text
|
||||
if isinstance(part, dict))
|
||||
text = str(text).strip()
|
||||
start, end = text.find("{"), text.rfind("}")
|
||||
if start >= 0 and end > start:
|
||||
text = text[start:end + 1]
|
||||
import json
|
||||
try:
|
||||
data = json.loads(text)
|
||||
except ValueError:
|
||||
return {"match": None, "confidence": 0.0,
|
||||
"differences": [f"non-JSON response: {text[:120]}"]}
|
||||
differences = data.get("differences") or []
|
||||
if not isinstance(differences, list):
|
||||
differences = [str(differences)]
|
||||
return {
|
||||
"match": data.get("match") if isinstance(data.get("match"), bool) else None,
|
||||
"confidence": float(data.get("confidence", 0.0) or 0.0),
|
||||
"differences": [str(item) for item in differences],
|
||||
}
|
||||
|
||||
|
||||
def classify(verdict, threshold=0.6):
|
||||
"""Turn a model verdict into a QA class.
|
||||
|
||||
The reference images are concept mockups, so a low-confidence disagreement is
|
||||
a review note, not a failure. Only a confident mismatch fails the check.
|
||||
"""
|
||||
if verdict.get("match") is True:
|
||||
return "match"
|
||||
if verdict.get("match") is False:
|
||||
return "bad" if float(verdict.get("confidence", 0.0)) >= threshold else "review"
|
||||
return "unknown"
|
||||
|
||||
|
||||
def _call_openrouter(body, timeout=180):
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
key = os.environ.get("OPENROUTER_API_KEY", "").strip()
|
||||
if not key:
|
||||
raise ValueError("OPENROUTER_API_KEY is not set (repo .env is read "
|
||||
"automatically); use --dry-run to inspect requests")
|
||||
req = urllib.request.Request(
|
||||
OPENROUTER_CHAT_URL,
|
||||
data=json.dumps(body).encode("utf-8"),
|
||||
headers={
|
||||
"Authorization": f"Bearer {key}",
|
||||
"Content-Type": "application/json",
|
||||
"HTTP-Referer": "https://git.allucanget.biz/zwitschi/security-game",
|
||||
"X-Title": "Step Line",
|
||||
},
|
||||
method="POST",
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
return json.loads(resp.read().decode("utf-8"))
|
||||
except urllib.error.HTTPError as exc:
|
||||
detail = exc.read().decode("utf-8", "replace")
|
||||
raise ValueError(f"OpenRouter HTTP {exc.code}: {detail[:300]}")
|
||||
|
||||
|
||||
def judge(args):
|
||||
_load_env_file()
|
||||
if not SCREENSHOTS_DIR.is_dir():
|
||||
sys.exit(f"error: no screenshots directory at {SCREENSHOTS_DIR}")
|
||||
refs = {screen["name"]: screen["ref"] for screen in SCREENS if screen["ref"]}
|
||||
by_name = {screen["name"]: screen for screen in SCREENS}
|
||||
results = []
|
||||
for capture in sorted(SCREENSHOTS_DIR.glob("*.png")):
|
||||
if args.only and args.only not in capture.name:
|
||||
continue
|
||||
design_name = next((n for n, ref in refs.items()
|
||||
if ref == capture.name), None)
|
||||
if design_name is None:
|
||||
results.append({"capture": capture.name, "screen": None,
|
||||
"match": None, "confidence": 0.0,
|
||||
"differences": ["no design reference for this capture"]})
|
||||
continue
|
||||
reference = _design_path(design_name)
|
||||
if not reference.is_file():
|
||||
results.append({"capture": capture.name, "screen": design_name,
|
||||
"match": None, "confidence": 0.0,
|
||||
"differences": ["missing reference design; run generate"]})
|
||||
continue
|
||||
body = build_judge_request(args.model, by_name[design_name],
|
||||
capture, reference)
|
||||
if args.dry_run:
|
||||
results.append({
|
||||
"capture": capture.name, "screen": design_name, "match": None,
|
||||
"confidence": 0.0, "model": body["model"],
|
||||
"differences": [f"dry run: {len(body['messages'][0]['content'])} parts"],
|
||||
})
|
||||
continue
|
||||
verdict = parse_judge_response(_call_openrouter(body))
|
||||
verdict.update({"capture": capture.name, "screen": design_name,
|
||||
"class": classify(verdict, args.fail_confidence)})
|
||||
results.append(verdict)
|
||||
|
||||
if args.json:
|
||||
print(json.dumps(results, indent=2))
|
||||
else:
|
||||
labels = {"match": "MATCH", "review": "REVIEW", "bad": "DIFF",
|
||||
"unknown": "UNKNOWN"}
|
||||
for result in results:
|
||||
print(f"{labels[result['class']]:7} {result['capture']} "
|
||||
f"(design: {result['screen']}) "
|
||||
f"confidence={result['confidence']:.2f}")
|
||||
for difference in result["differences"]:
|
||||
print(f" - {difference}")
|
||||
counts = {key: sum(1 for r in results if r["class"] == key)
|
||||
for key in labels}
|
||||
print(f"Summary: {counts['match']}/{len(results)} match, "
|
||||
f"{counts['review']} review, {counts['bad']} diff "
|
||||
f"(fail threshold confidence >= {args.fail_confidence:.2f})")
|
||||
|
||||
if args.dry_run:
|
||||
return 0
|
||||
return 1 if any(r["class"] == "bad" for r in results) else 0
|
||||
|
||||
|
||||
def build_parser():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Generate and compare Step Line screen-design references.")
|
||||
@@ -599,6 +797,18 @@ def build_parser():
|
||||
default=generate_textures.GEN_SIZE)
|
||||
gen.set_defaults(handler=generate, draft=False)
|
||||
|
||||
judge_cmd = commands.add_parser(
|
||||
"judge", help="ask a vision model whether captures match the designs")
|
||||
judge_cmd.add_argument("--model", default=DEFAULT_JUDGE_MODEL)
|
||||
judge_cmd.add_argument("--json", action="store_true")
|
||||
judge_cmd.add_argument("--dry-run", action="store_true",
|
||||
help="print requests instead of calling the API")
|
||||
judge_cmd.add_argument("--only", default="",
|
||||
help="only judge captures whose name contains this text")
|
||||
judge_cmd.add_argument("--fail-confidence", type=float, default=0.6,
|
||||
help="a mismatch at or above this confidence fails")
|
||||
judge_cmd.set_defaults(handler=judge)
|
||||
|
||||
cmp = commands.add_parser(
|
||||
"compare", help="compare docs/screenshots/*.png against the designs")
|
||||
cmp.add_argument("--json", action="store_true")
|
||||
|
||||
@@ -86,5 +86,97 @@ class ScreenDesignsTests(unittest.TestCase):
|
||||
self.assertTrue(any("aspect" in e for e in report["errors"]))
|
||||
|
||||
|
||||
class VisionJudgeTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp_dir = tempfile.TemporaryDirectory()
|
||||
self.root = Path(self.temp_dir.name)
|
||||
self.capture = self.root / "capture.png"
|
||||
self.design = self.root / "design.png"
|
||||
Image.new("RGB", (64, 36), (20, 30, 40)).save(self.capture)
|
||||
Image.new("RGB", (64, 36), (200, 190, 180)).save(self.design)
|
||||
self.screen = screen_designs.SCREENS[0]
|
||||
|
||||
def tearDown(self):
|
||||
self.temp_dir.cleanup()
|
||||
|
||||
def test_judge_request_embeds_both_images(self):
|
||||
body = screen_designs.build_judge_request(
|
||||
"model-x", self.screen, self.capture, self.design)
|
||||
self.assertEqual(body["model"], "model-x")
|
||||
self.assertEqual(body["temperature"], 0)
|
||||
self.assertEqual(body["response_format"], {"type": "json_object"})
|
||||
parts = body["messages"][0]["content"]
|
||||
self.assertEqual(parts[0]["type"], "text")
|
||||
images = [p for p in parts if p["type"] == "image_url"]
|
||||
self.assertEqual(len(images), 2, "design and capture are both attached")
|
||||
for image in images:
|
||||
self.assertTrue(image["image_url"]["url"].startswith(
|
||||
"data:image/png;base64,"))
|
||||
self.assertIn(self.screen["title"], parts[0]["text"])
|
||||
self.assertIn("mockup", parts[0]["text"].lower())
|
||||
|
||||
def test_parse_plain_json_verdict(self):
|
||||
payload = {"choices": [{"message": {"content":
|
||||
'{"match": true, "confidence": 0.9, "differences": ["style"]}'}}]}
|
||||
verdict = screen_designs.parse_judge_response(payload)
|
||||
self.assertTrue(verdict["match"])
|
||||
self.assertAlmostEqual(verdict["confidence"], 0.9)
|
||||
self.assertEqual(verdict["differences"], ["style"])
|
||||
|
||||
def test_parse_fenced_json_verdict(self):
|
||||
payload = {"choices": [{"message": {"content":
|
||||
'Sure:\n```json\n{"match": false, "confidence": 0.7}\n```'}}]}
|
||||
verdict = screen_designs.parse_judge_response(payload)
|
||||
self.assertFalse(verdict["match"])
|
||||
self.assertAlmostEqual(verdict["confidence"], 0.7)
|
||||
|
||||
def test_parse_content_parts(self):
|
||||
payload = {"choices": [{"message": {"content": [
|
||||
{"type": "text", "text": '{"match": true, "confidence": 0.5}'}]}}]}
|
||||
verdict = screen_designs.parse_judge_response(payload)
|
||||
self.assertTrue(verdict["match"])
|
||||
|
||||
def test_parse_non_json_is_unknown(self):
|
||||
payload = {"choices": [{"message": {"content": "no idea"}}]}
|
||||
verdict = screen_designs.parse_judge_response(payload)
|
||||
self.assertIsNone(verdict["match"])
|
||||
self.assertTrue(verdict["differences"])
|
||||
|
||||
def test_parse_missing_choices_is_unknown(self):
|
||||
verdict = screen_designs.parse_judge_response({})
|
||||
self.assertIsNone(verdict["match"])
|
||||
|
||||
def test_classify_thresholds(self):
|
||||
self.assertEqual(screen_designs.classify(
|
||||
{"match": True, "confidence": 0.1}), "match")
|
||||
self.assertEqual(screen_designs.classify(
|
||||
{"match": False, "confidence": 0.9}), "bad")
|
||||
self.assertEqual(screen_designs.classify(
|
||||
{"match": False, "confidence": 0.3}), "review")
|
||||
self.assertEqual(screen_designs.classify(
|
||||
{"match": None, "confidence": 0.0}), "unknown")
|
||||
self.assertEqual(screen_designs.classify(
|
||||
{"match": False, "confidence": 0.55}, threshold=0.5), "bad")
|
||||
|
||||
def test_env_file_does_not_override_existing(self):
|
||||
env_path = self.root / ".env"
|
||||
env_path.write_text('OPENROUTER_API_KEY=from-file\nOTHER=1\n',
|
||||
encoding="utf-8")
|
||||
import os
|
||||
old = os.environ.get("OPENROUTER_API_KEY")
|
||||
os.environ["OPENROUTER_API_KEY"] = "from-env"
|
||||
os.environ.pop("OTHER", None)
|
||||
try:
|
||||
screen_designs._load_env_file(env_path)
|
||||
self.assertEqual(os.environ["OPENROUTER_API_KEY"], "from-env")
|
||||
self.assertEqual(os.environ.get("OTHER"), "1")
|
||||
finally:
|
||||
if old is None:
|
||||
os.environ.pop("OPENROUTER_API_KEY", None)
|
||||
else:
|
||||
os.environ["OPENROUTER_API_KEY"] = old
|
||||
os.environ.pop("OTHER", None)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in new issue
Block a user