security-game/docs/process/release-qa.md

Release QA and Balance

Manual verification and the 0.10.0 balance protocol. The executable command matrix is in the release runbook/skill; this page owns the human smoke checks and the measurable balance targets. Architecture owns the gates themselves; gameplay owns the mechanics.

Manual smoke checklist

1. godot --path . starts the game on the venue menu without script errors.
2. Choosing a venue (click or ↑/↓ + Enter) starts that venue's shift and hides the menu.
3. The guard moves and looks around; the building, steps, and sidewalk are visible.
4. The guard's camera sits at eye height and the guard stands on the landing, not inside the building.
5. Walking into the steps carries the guard up onto the landing (steps have colliders).
6. The skyline backdrop shows silhouettes against the sky, not an opaque black slab.
7. The GIG LINE marquee is readable from the sidewalk.
8. NPCs spawn and approach the entrance during the first minutes.
9. Looking at an NPC inside the focus cone shows a blue focus circle in front of the face, the person's name, and the HUD target line.
10. Space or left click opens the action wheel; A/D highlight a slice, Space confirms it, Esc/right click cancels without a log entry.
11. E/C open the inspection/wand panels from the wheel and freeze movement until they close; walking never fires the wand.
12. Pressing Q produces a reaction log entry and reputation feedback.
13. E opens access inspection showing venue-correct artifacts and policy result; A/D record a decision.
14. 6 opens the assistance form; submitting shows dispatch feedback.
15. R opens the emergency form; the button must be held ~1.6s or the call cancels.
16. G toggles the flashlight (indicator appears in the HUD); L toggles the door lock (indicator turns red when locked).
17. F toggles the debug overlay with NPC states and shift stats.
18. The wheel's Look slice is eye contact (Level 1) and logs as its own action.
19. Reason asks why the person is there and logs their dialogue_lines reply.
20. Recognize records the guard's recognition as access evidence; pressing it twice says so.
21. Warn turns the body camera on (REC overlay appears); B toggles it manually.
22. Block moves the guard between the person and the door.
23. The shift review reports the incident count, including interactions left unresolved.
24. Escalating on someone (or a responder arriving) makes idle bystanders stop and face the incident.
25. Repeated alarms panic the crowd and nearby people clear away from the door instead of standing around.
26. Esc with nothing open pauses the shift; resume / restart / settings / quit-to-menu all work, and settings changes persist across a restart.
27. NPCs route around the building using the navigation mesh (no stuck-at-wall behavior during a smoke run).
28. An unjustified emergency call (calm person, no weapon) logs an investigation and costs reputation.
29. After an evacuation outcome the door is locked and NPCs are leaving the entrance.
30. The shift review's "Body camera timeline" button steps through the recorded events (Left/Right or the buttons) and Esc returns to the review.
31. The escalation label shows a [###--] L3/5 shape meter as well as the level name, and stays readable in grayscale.
32. Changing "HUD font scale" in Settings resizes the HUD text and persists across a restart.
33. When the shift timer reaches 0, the shift review screen shows grade, policy, incident summary, responder outcomes, investigations, and persisted shift history; campaign Continue starts the next venue.
34. On a fresh profile the tutorial starts on Hollywood, advances through its eight beats, and F1 skips it; Settings Replay tutorial restarts it.
35. Pausing saves a checkpoint: returning to the menu and choosing Continue resumes elapsed time, NPCs, incidents, door/flashlight state and campaign position.
36. Settings difficulty changes spawn pacing/compliance/report strictness while escalation rules stay the same; Help filters controls and shows the venue policy.
37. The F debug overlay lists the next eligible events; recovery events and recently played scripted events behave as authored.
38. Master/SFX/ambience volume settings clamp at 0/1 and persist; Reduce motion disables head bob; the captions toggle persists.
39. The 24-NPC budget smoke stays inside its frame budget; coats/dresses do not show obvious leg clipping in a normal walk.
40. Help remains readable with the largest HUD font scale, and CI/local verification use the same Godot 4.7.2 baseline.
41. UI copy: inspection/wand use "A = admit · D = refuse entry", review returns with "Return to menu", and application exit reads "Quit game".
42. Button states remain readable on hover/focus/pressed; settings Apply/Cancel and radio normal-send/emergency-hold behave as labelled.
43. The venue menu offers Hollywood Boulevard — Day and — Night; either starts the same map, and Night starts in the late phase with a full shift clock.
44. The Joke Shop opens on the Sunset Strip club: the black hall, signature canopy, circular THE JOKE SHOP sign, red curtains, west entrance, palms, dry hillside with houses, neighbouring frontage and the opposite side of Sunset all read from the door camera. Day lighting and late lighting both look correct; the guard can walk the sidewalk without leaving the nav bounds.
45. Esc and right-click close only the top panel (Help, Settings, Pause, Review, Radio, timeline, inspection/wand, action wheel); Settings opened from the venue menu returns to the menu, Help no longer opens Pause behind it, and closing a modal recaptures the mouse without leaving the guard frozen.
46. At 0.75× and 1.75× HUD font scale, long target names stay inside the target chip, the chip grows/caps correctly, and long button labels ellipsize instead of pushing panels off-screen.
47. Reset campaign and Replay tutorial both ask for confirmation; a second click reuses the open dialog, Cancel does nothing, and OK fires once. Emergency send requires the full hold, sends exactly once, and does not send again on release.
48. UI copy/button sweep: every panel uses the canonical admit/refuse, warning/blocking, assistance/emergency and body-camera terms; the start menu, pause, settings, Help, inspection, wand, radio, wheel, timeline and shift review all show readable hover/focus/pressed/disabled states; Tab/arrow focus wraps without landing on hidden or disabled actions; Esc/right-click closes only the top panel; and destructive reset/replay plus emergency send still require their confirmation or hold.

Balance acceptance

These are the acceptance targets for the 0.10.0 balance pass, not measured results. No reaction tables, event weights or scoring rules were changed to set them. The roadmap tracks the sweeps and external playtests still needed. Gameplay owns the mechanics; this page owns their balance-test protocol.

Test matrix and evidence

Run Hollywood, Joke Shop and TV Studio on easy, normal and hard, in Early, Midday, Evening and Late: 36 cells. Use seeds 1–100 for each cell and retain those seeds for before/after comparisons. Exercise each eligible authored event at least once as a separate coverage pass; a random schedule can miss rare events. Record scheduled event counts separately from forced coverage cases so the latter do not distort event-frequency estimates.

For reaction measurements, use a fresh controller at the entrance for each sample. Routine cases are tourists, confused visitors and delivery workers obstructing the entrance without threatening behavior. Do not include authorized entry requests in this redirect cohort. For peaceful-resolution measurements, send one polite direction (L1), then one warning (L2) only if still unresolved. Do not reroll until success. Observe return behavior for 60 simulated seconds before closing the case. Do not issue emergency calls as part of this policy.

For access cases, follow the venue's required inspection and screening steps before allow/deny. Include valid, expired, counterfeit, mismatched, missing and ambiguous artifacts, known recognition, age boundaries, screening refusal, benign wand alerts and prohibited items. Recognition must be independently justified by the fixture; calling a stranger recognized is not evidence of correct admission. An unresolved hold counts as unresolved, not as a correct refusal.

Evidence must identify the commit, engine, venue, difficulty, phase, seed, event ID, cohort, visible evidence, policy action, reaction/state, return, access truth and decision, emergency truth and call, report fields, reputation modifiers, final grade and unresolved cases. Save raw rows plus numerator/denominator summaries for every cell. Use at least 100 eligible cases per metric per cell; if the schedule provides fewer, add a separately labelled forced-case sample. Report zero eligible cases as N/A, never as 100% success. Keep scheduled and forced results separate.

Targets

Percentages below apply to each matrix cell, not only a pooled average. Scripted access/report cases use complete evidence and are held to exact correctness; first-time-player results measure whether that evidence is understandable.

Metric	Numerator / denominator	Acceptance target
First-direction routine compliance	First L1 reaction is COMPLY or LEAVE / eligible routine redirect cases	Easy ≥80%; normal ≥75%; hard ≥65%
Routine peaceful resolution	Routine cases leaving after at most L1 then L2, with no return during 60 s and no L3–L5 / eligible routine redirect cases	Easy ≥90%; normal ≥85%; hard ≥80%
Access accuracy	Correct final allow/deny decisions / all eligible access cases (including unresolved cases)	Scripted 100%; first-time players ≥90% pooled, with each error reviewed
Access false positive	Unauthorized people allowed / unauthorized access cases	Scripted 0%; first-time players ≤5% pooled
Access false negative	Authorized people denied / authorized access cases	Scripted 0%; first-time players ≤10% pooled
Emergency precision	Calls supported by immediate-danger evidence / all emergency calls	Scripted 100%; first-time players ≥95% pooled; zero calls is N/A
Emergency false positive	Routine/no-immediate-danger cases receiving an emergency call / all routine/no-immediate-danger cases	Scripted 0%; first-time players ≤5% pooled
Emergency false negative	Immediate-danger cases without an accepted emergency call within 15 simulated seconds of visible danger / all immediate-danger cases with radio initially available	Scripted 0%; first-time players ≤10% pooled
Report accuracy	Reports with correct weapon and danger fields / all submitted reports	Scripted 100%; first-time players ≥90% pooled

For access, authorized means entitled under the venue policy after required screening, without a prohibited item or an independent immediate-danger reason to refuse. Compare to a hand-labelled fixture expectation, not only to AccessVerifier.verify(): using the verifier as both decision and oracle would hide policy bugs. Report unresolved access cases separately; they cannot improve either error rate by being omitted from accuracy.

Immediate danger means an observed weapon, assault, active forced entry or a credible immediate threat to occupants. Argument, filming, intoxication, a crisis, a refused screen or a benign wand alert alone is not immediate danger. A mental health label or clothing must never be used as the emergency oracle. The 15 s window is a test-response target, not the responder-arrival time (currently 15 s after an accepted emergency call). Separately test depleted battery and active cooldown cases for safe containment/assistance; do not silently drop them from the evidence or claim the initially-available-radio target covers them.

Scoring and emergency dominance

Compare the calm policy with an emergency-first policy on the same routine events and seeds. Each comparison starts with fresh radio, crowd and reputation state. An accurate report about harmless behavior must not make its emergency call justified. Require an unjustified-call investigation and a strictly lower rating for emergency-first in every routine comparison. Inspect modifiers as well as the final rating: the 0–100 clamp can hide rewards/penalties. Emergency-first must not improve the campaign outcome or final grade. Grade boundaries remain A ≥90, B ≥75, C ≥60, D ≥40, otherwise F; these are existing rules, not new tuning.

In genuine-danger cases, verify that the call is accepted, responders arrive, the incident remains answerable and the debrief explains the outcome without an unjustified-call investigation. Also compare assistance/containment routes where appropriate; emergency need not be mandatory for every difficult interaction.

Implementation cautions and sign-off

• NPCController.roll_reaction() samples per-level weights. Difficulty adds +12/0/−8 to the COMPLY weight before normalization; base_compliance alone is not the observed compliance percentage. Measure COMPLY plus LEAVE.
• GameManager._read_npc_truth() currently derives weapon/danger truth from the NPC type (violent threat: weapon/3; everyone else: no weapon/1), rather than observed behavior. This is a balance-review risk: test nonviolent profiles that threaten or force entry, and violent profiles before visible danger. Do not treat agreement with this helper as proof of fair emergency judgment.
• RadioSystem.credibility_hits() checks weapon and danger fields only; location, description and behavior still need human review. Difficulty scales report penalties, not the oracle.
• A terminal peaceful reaction is not sufficient evidence of an incident staying resolved: returning NPCs and unresolved shift-review entries must be retained.

Do not tune to a pooled average that hides a failing venue, phase or difficulty. Record each miss, its severity, the raw evidence and any approved exception before changing weights. Re-run the affected cells and tutorial/campaign smokes after a fix. Five first-time-player sessions must cover the tutorial and a complete venue shift; report pooled counts and per-session errors, not statistical confidence from five players. Scripted sweeps do not substitute for those sessions.

Status: targets defined; sweeps, emergency fairness review, grade/campaign verification and external-player acceptance remain pending. No balance sign-off or release approval is implied by this document.