zwitschi
2ba40f3578
Group the top-level docs into docs/project, docs/design, docs/assets and docs/process, each with its own README index. Delete the superseded docs/screens/head-review page and sheets. Rewrite every markdown link, absolute docs path and tool/test reference to the new locations, fix the pre-existing broken anchors and remove the dead Joke Shop photo-board link. check_docs OK, Python tooling 167/167.
174 lines
14 KiB
Markdown
174 lines
14 KiB
Markdown
# Release QA and Balance
|
||
|
||
Manual verification and the 0.10.0 balance protocol. The executable command
|
||
matrix is in the release runbook/skill; this page owns the human smoke checks and
|
||
the measurable balance targets. Architecture owns the gates themselves; gameplay
|
||
owns the mechanics.
|
||
|
||
## Manual smoke checklist
|
||
|
||
1. `godot --path .` starts the game on the venue menu without script errors.
|
||
2. Choosing a venue (click or ↑/↓ + Enter) starts that venue's shift and hides the menu.
|
||
3. The guard moves and looks around; the building, steps, and sidewalk are visible.
|
||
4. The guard's camera sits at eye height and the guard stands on the landing, not inside the building.
|
||
5. Walking into the steps carries the guard up onto the landing (steps have colliders).
|
||
6. The skyline backdrop shows silhouettes against the sky, not an opaque black slab.
|
||
7. The `GIG LINE` marquee is readable from the sidewalk.
|
||
8. NPCs spawn and approach the entrance during the first minutes.
|
||
9. Looking at an NPC inside the focus cone shows a blue focus circle in front of the face, the person's name, and the HUD target line.
|
||
10. `Space` or left click opens the action wheel; `A`/`D` highlight a slice, `Space` confirms it, `Esc`/right click cancels without a log entry.
|
||
11. `E`/`C` open the inspection/wand panels from the wheel and freeze movement until they close; walking never fires the wand.
|
||
12. Pressing `Q` produces a reaction log entry and reputation feedback.
|
||
13. `E` opens access inspection showing venue-correct artifacts and policy result; `A`/`D` record a decision.
|
||
14. `6` opens the assistance form; submitting shows dispatch feedback.
|
||
15. `R` opens the emergency form; the button must be held ~1.6s or the call cancels.
|
||
16. `G` toggles the flashlight (indicator appears in the HUD); `L` toggles the door lock (indicator turns red when locked).
|
||
17. `F` toggles the debug overlay with NPC states and shift stats.
|
||
18. The wheel's `Look` slice is eye contact (Level 1) and logs as its own action.
|
||
19. `Reason` asks why the person is there and logs their `dialogue_lines` reply.
|
||
20. `Recognize` records the guard's recognition as access evidence; pressing it twice says so.
|
||
21. `Warn` turns the body camera on (REC overlay appears); `B` toggles it manually.
|
||
22. `Block` moves the guard between the person and the door.
|
||
23. The shift review reports the incident count, including interactions left unresolved.
|
||
24. Escalating on someone (or a responder arriving) makes idle bystanders stop and face the incident.
|
||
25. Repeated alarms panic the crowd and nearby people clear away from the door instead of standing around.
|
||
26. `Esc` with nothing open pauses the shift; resume / restart / settings / quit-to-menu all work, and settings changes persist across a restart.
|
||
27. NPCs route around the building using the navigation mesh (no stuck-at-wall behavior during a smoke run).
|
||
28. An unjustified emergency call (calm person, no weapon) logs an investigation and costs reputation.
|
||
29. After an evacuation outcome the door is locked and NPCs are leaving the entrance.
|
||
30. The shift review's "Body camera timeline" button steps through the recorded events (Left/Right or the buttons) and Esc returns to the review.
|
||
31. The escalation label shows a `[###--] L3/5` shape meter as well as the level name, and stays readable in grayscale.
|
||
32. Changing "HUD font scale" in Settings resizes the HUD text and persists across a restart.
|
||
33. When the shift timer reaches 0, the shift review screen shows grade, policy, incident summary, responder outcomes, investigations, and persisted shift history; campaign Continue starts the next venue.
|
||
34. On a fresh profile the tutorial starts on Hollywood, advances through its eight beats, and F1 skips it; Settings Replay tutorial restarts it.
|
||
35. Pausing saves a checkpoint: returning to the menu and choosing Continue resumes elapsed time, NPCs, incidents, door/flashlight state and campaign position.
|
||
36. Settings difficulty changes spawn pacing/compliance/report strictness while escalation rules stay the same; Help filters controls and shows the venue policy.
|
||
37. The F debug overlay lists the next eligible events; recovery events and recently played scripted events behave as authored.
|
||
38. Master/SFX/ambience volume settings clamp at 0/1 and persist; Reduce motion disables head bob; the captions toggle persists.
|
||
39. The 24-NPC budget smoke stays inside its frame budget; coats/dresses do not show obvious leg clipping in a normal walk.
|
||
40. Help remains readable with the largest HUD font scale, and CI/local verification use the same Godot 4.7.2 baseline.
|
||
41. UI copy: inspection/wand use "A = admit · D = refuse entry", review returns with "Return to menu", and application exit reads "Quit game".
|
||
42. Button states remain readable on hover/focus/pressed; settings Apply/Cancel and radio normal-send/emergency-hold behave as labelled.
|
||
43. The venue menu offers Hollywood Boulevard — Day and — Night; either starts the same map, and Night starts in the late phase with a full shift clock.
|
||
44. The Joke Shop opens on the Sunset Strip club: the black hall, signature canopy, circular **THE JOKE SHOP** sign, red curtains, west entrance, palms, dry hillside with houses, neighbouring frontage and the opposite side of Sunset all read from the door camera. Day lighting and late lighting both look correct; the guard can walk the sidewalk without leaving the nav bounds.
|
||
45. Esc and right-click close only the top panel (Help, Settings, Pause, Review, Radio, timeline, inspection/wand, action wheel); Settings opened from the venue menu returns to the menu, Help no longer opens Pause behind it, and closing a modal recaptures the mouse without leaving the guard frozen.
|
||
46. At 0.75× and 1.75× HUD font scale, long target names stay inside the target chip, the chip grows/caps correctly, and long button labels ellipsize instead of pushing panels off-screen.
|
||
47. Reset campaign and Replay tutorial both ask for confirmation; a second click reuses the open dialog, Cancel does nothing, and OK fires once. Emergency send requires the full hold, sends exactly once, and does not send again on release.
|
||
48. UI copy/button sweep: every panel uses the canonical admit/refuse, warning/blocking, assistance/emergency and body-camera terms; the start menu, pause, settings, Help, inspection, wand, radio, wheel, timeline and shift review all show readable hover/focus/pressed/disabled states; Tab/arrow focus wraps without landing on hidden or disabled actions; Esc/right-click closes only the top panel; and destructive reset/replay plus emergency send still require their confirmation or hold.
|
||
|
||
## Balance acceptance
|
||
|
||
These are the acceptance targets for the 0.10.0 balance pass, not measured results.
|
||
No reaction tables, event weights or scoring rules were changed to set them. The
|
||
[roadmap](../../TODO.md) tracks the sweeps and external playtests still needed.
|
||
[Gameplay](../design/gameplay.md) owns the mechanics; this page owns their balance-test protocol.
|
||
|
||
### Test matrix and evidence
|
||
|
||
Run Hollywood, Joke Shop and TV Studio on easy, normal and hard, in Early,
|
||
Midday, Evening and Late: **36 cells**. Use seeds 1–100 for each cell and retain
|
||
those seeds for before/after comparisons. Exercise each eligible authored event
|
||
at least once as a separate coverage pass; a random schedule can miss rare events.
|
||
Record scheduled event counts separately from forced coverage cases so the latter
|
||
do not distort event-frequency estimates.
|
||
|
||
For reaction measurements, use a fresh controller at the entrance for each sample.
|
||
Routine cases are tourists, confused visitors and delivery workers obstructing the
|
||
entrance without threatening behavior. Do not include authorized entry requests in
|
||
this redirect cohort. For peaceful-resolution measurements, send one polite
|
||
direction (L1), then one warning (L2) only if still unresolved. Do not reroll until
|
||
success. Observe return behavior for 60 simulated seconds before closing the case.
|
||
Do not issue emergency calls as part of this policy.
|
||
|
||
For access cases, follow the venue's required inspection and screening steps before
|
||
allow/deny. Include valid, expired, counterfeit, mismatched, missing and ambiguous
|
||
artifacts, known recognition, age boundaries, screening refusal, benign wand alerts
|
||
and prohibited items. Recognition must be independently justified by the fixture;
|
||
calling a stranger recognized is not evidence of correct admission. An unresolved
|
||
hold counts as unresolved, not as a correct refusal.
|
||
|
||
Evidence must identify the commit, engine, venue, difficulty, phase, seed, event ID,
|
||
cohort, visible evidence, policy action, reaction/state, return, access truth and
|
||
decision, emergency truth and call, report fields, reputation modifiers, final grade
|
||
and unresolved cases. Save raw rows plus numerator/denominator summaries for every
|
||
cell. Use at least **100 eligible cases per metric per cell**; if the schedule
|
||
provides fewer, add a separately labelled forced-case sample. Report zero eligible
|
||
cases as **N/A**, never as 100% success. Keep scheduled and forced results separate.
|
||
|
||
### Targets
|
||
|
||
Percentages below apply to each matrix cell, not only a pooled average. Scripted
|
||
access/report cases use complete evidence and are held to exact correctness;
|
||
first-time-player results measure whether that evidence is understandable.
|
||
|
||
| Metric | Numerator / denominator | Acceptance target |
|
||
| --- | --- | --- |
|
||
| First-direction routine compliance | First L1 reaction is `COMPLY` or `LEAVE` / eligible routine redirect cases | Easy ≥80%; normal ≥75%; hard ≥65% |
|
||
| Routine peaceful resolution | Routine cases leaving after at most L1 then L2, with no return during 60 s and no L3–L5 / eligible routine redirect cases | Easy ≥90%; normal ≥85%; hard ≥80% |
|
||
| Access accuracy | Correct final allow/deny decisions / all eligible access cases (including unresolved cases) | Scripted 100%; first-time players ≥90% pooled, with each error reviewed |
|
||
| Access false positive | Unauthorized people allowed / unauthorized access cases | Scripted 0%; first-time players ≤5% pooled |
|
||
| Access false negative | Authorized people denied / authorized access cases | Scripted 0%; first-time players ≤10% pooled |
|
||
| Emergency precision | Calls supported by immediate-danger evidence / all emergency calls | Scripted 100%; first-time players ≥95% pooled; zero calls is N/A |
|
||
| Emergency false positive | Routine/no-immediate-danger cases receiving an emergency call / all routine/no-immediate-danger cases | Scripted 0%; first-time players ≤5% pooled |
|
||
| Emergency false negative | Immediate-danger cases without an accepted emergency call within 15 simulated seconds of visible danger / all immediate-danger cases with radio initially available | Scripted 0%; first-time players ≤10% pooled |
|
||
| Report accuracy | Reports with correct weapon and danger fields / all submitted reports | Scripted 100%; first-time players ≥90% pooled |
|
||
|
||
For access, authorized means entitled under the venue policy **after required
|
||
screening**, without a prohibited item or an independent immediate-danger reason
|
||
to refuse. Compare to a hand-labelled fixture expectation, not only to
|
||
`AccessVerifier.verify()`: using the verifier as both decision and oracle would
|
||
hide policy bugs. Report unresolved access cases separately; they cannot improve
|
||
either error rate by being omitted from accuracy.
|
||
|
||
Immediate danger means an observed weapon, assault, active forced entry or a
|
||
credible immediate threat to occupants. Argument, filming, intoxication, a crisis,
|
||
a refused screen or a benign wand alert alone is not immediate danger. A mental
|
||
health label or clothing must never be used as the emergency oracle. The 15 s
|
||
window is a test-response target, not the responder-arrival time (currently 15 s
|
||
after an accepted emergency call). Separately test depleted battery and active
|
||
cooldown cases for safe containment/assistance; do not silently drop them from the
|
||
evidence or claim the initially-available-radio target covers them.
|
||
|
||
### Scoring and emergency dominance
|
||
|
||
Compare the calm policy with an emergency-first policy on the same routine events
|
||
and seeds. Each comparison starts with fresh radio, crowd and reputation state.
|
||
An accurate report about harmless behavior must not make its emergency call
|
||
justified. Require an unjustified-call investigation and a strictly lower rating
|
||
for emergency-first in every routine comparison. Inspect modifiers as well as the
|
||
final rating: the 0–100 clamp can hide rewards/penalties. Emergency-first must not
|
||
improve the campaign outcome or final grade. Grade boundaries remain A ≥90,
|
||
B ≥75, C ≥60, D ≥40, otherwise F; these are existing rules, not new tuning.
|
||
|
||
In genuine-danger cases, verify that the call is accepted, responders arrive,
|
||
the incident remains answerable and the debrief explains the outcome without an
|
||
unjustified-call investigation. Also compare assistance/containment routes where
|
||
appropriate; emergency need not be mandatory for every difficult interaction.
|
||
|
||
### Implementation cautions and sign-off
|
||
|
||
- `NPCController.roll_reaction()` samples per-level weights. Difficulty adds
|
||
+12/0/−8 to the `COMPLY` weight before normalization; `base_compliance` alone
|
||
is not the observed compliance percentage. Measure `COMPLY` plus `LEAVE`.
|
||
- `GameManager._read_npc_truth()` currently derives weapon/danger truth from the
|
||
NPC type (violent threat: weapon/3; everyone else: no weapon/1), rather than
|
||
observed behavior. This is a balance-review risk: test nonviolent profiles that
|
||
threaten or force entry, and violent profiles before visible danger. Do not
|
||
treat agreement with this helper as proof of fair emergency judgment.
|
||
- `RadioSystem.credibility_hits()` checks weapon and danger fields only;
|
||
location, description and behavior still need human review. Difficulty scales
|
||
report penalties, not the oracle.
|
||
- A terminal peaceful reaction is not sufficient evidence of an incident staying
|
||
resolved: returning NPCs and unresolved shift-review entries must be retained.
|
||
|
||
Do not tune to a pooled average that hides a failing venue, phase or difficulty.
|
||
Record each miss, its severity, the raw evidence and any approved exception before
|
||
changing weights. Re-run the affected cells and tutorial/campaign smokes after a
|
||
fix. Five first-time-player sessions must cover the tutorial and a complete venue
|
||
shift; report pooled counts and per-session errors, not statistical confidence
|
||
from five players. Scripted sweeps do not substitute for those sessions.
|
||
|
||
**Status:** targets defined; sweeps, emergency fairness review, grade/campaign
|
||
verification and external-player acceptance remain pending. No balance sign-off
|
||
or release approval is implied by this document.
|