security-game/docs/balance-acceptance.md

Release-candidate balance acceptance

These are the acceptance targets for the 0.10.0 balance pass, not measured results. No reaction tables, event weights or scoring rules were changed to set them. The roadmap tracks the sweeps and external playtests still needed. Gameplay owns the mechanics; this page owns their balance-test protocol.

Test matrix and evidence

Run Hollywood, Joke Shop and TV Studio on easy, normal and hard, in Early, Midday, Evening and Late: 36 cells. Use seeds 1–100 for each cell and retain those seeds for before/after comparisons. Exercise each eligible authored event at least once as a separate coverage pass; a random schedule can miss rare events. Record scheduled event counts separately from forced coverage cases so the latter do not distort event-frequency estimates.

For reaction measurements, use a fresh controller at the entrance for each sample. Routine cases are tourists, confused visitors and delivery workers obstructing the entrance without threatening behavior. Do not include authorized entry requests in this redirect cohort. For peaceful-resolution measurements, send one polite direction (L1), then one warning (L2) only if still unresolved. Do not reroll until success. Observe return behavior for 60 simulated seconds before closing the case. Do not issue emergency calls as part of this policy.

For access cases, follow the venue's required inspection and screening steps before allow/deny. Include valid, expired, counterfeit, mismatched, missing and ambiguous artifacts, known recognition, age boundaries, screening refusal, benign wand alerts and prohibited items. Recognition must be independently justified by the fixture; calling a stranger recognized is not evidence of correct admission. An unresolved hold counts as unresolved, not as a correct refusal.

Evidence must identify the commit, engine, venue, difficulty, phase, seed, event ID, cohort, visible evidence, policy action, reaction/state, return, access truth and decision, emergency truth and call, report fields, reputation modifiers, final grade and unresolved cases. Save raw rows plus numerator/denominator summaries for every cell. Use at least 100 eligible cases per metric per cell; if the schedule provides fewer, add a separately labelled forced-case sample. Report zero eligible cases as N/A, never as 100% success. Keep scheduled and forced results separate.

Targets

Percentages below apply to each matrix cell, not only a pooled average. Scripted access/report cases use complete evidence and are held to exact correctness; first-time-player results measure whether that evidence is understandable.

Metric	Numerator / denominator	Acceptance target
First-direction routine compliance	First L1 reaction is COMPLY or LEAVE / eligible routine redirect cases	Easy ≥80%; normal ≥75%; hard ≥65%
Routine peaceful resolution	Routine cases leaving after at most L1 then L2, with no return during 60 s and no L3–L5 / eligible routine redirect cases	Easy ≥90%; normal ≥85%; hard ≥80%
Access accuracy	Correct final allow/deny decisions / all eligible access cases (including unresolved cases)	Scripted 100%; first-time players ≥90% pooled, with each error reviewed
Access false positive	Unauthorized people allowed / unauthorized access cases	Scripted 0%; first-time players ≤5% pooled
Access false negative	Authorized people denied / authorized access cases	Scripted 0%; first-time players ≤10% pooled
Emergency precision	Calls supported by immediate-danger evidence / all emergency calls	Scripted 100%; first-time players ≥95% pooled; zero calls is N/A
Emergency false positive	Routine/no-immediate-danger cases receiving an emergency call / all routine/no-immediate-danger cases	Scripted 0%; first-time players ≤5% pooled
Emergency false negative	Immediate-danger cases without an accepted emergency call within 15 simulated seconds of visible danger / all immediate-danger cases with radio initially available	Scripted 0%; first-time players ≤10% pooled
Report accuracy	Reports with correct weapon and danger fields / all submitted reports	Scripted 100%; first-time players ≥90% pooled

For access, authorized means entitled under the venue policy after required screening, without a prohibited item or an independent immediate-danger reason to refuse. Compare to a hand-labelled fixture expectation, not only to AccessVerifier.verify(): using the verifier as both decision and oracle would hide policy bugs. Report unresolved access cases separately; they cannot improve either error rate by being omitted from accuracy.

Immediate danger means an observed weapon, assault, active forced entry or a credible immediate threat to occupants. Argument, filming, intoxication, a crisis, a refused screen or a benign wand alert alone is not immediate danger. A mental health label or clothing must never be used as the emergency oracle. The 15 s window is a test-response target, not the responder-arrival time (currently 15 s after an accepted emergency call). Separately test depleted battery and active cooldown cases for safe containment/assistance; do not silently drop them from the evidence or claim the initially-available-radio target covers them.

Scoring and emergency dominance

Compare the calm policy with an emergency-first policy on the same routine events and seeds. Each comparison starts with fresh radio, crowd and reputation state. An accurate report about harmless behavior must not make its emergency call justified. Require an unjustified-call investigation and a strictly lower rating for emergency-first in every routine comparison. Inspect modifiers as well as the final rating: the 0–100 clamp can hide rewards/penalties. Emergency-first must not improve the campaign outcome or final grade. Grade boundaries remain A ≥90, B ≥75, C ≥60, D ≥40, otherwise F; these are existing rules, not new tuning.

In genuine-danger cases, verify that the call is accepted, responders arrive, the incident remains answerable and the debrief explains the outcome without an unjustified-call investigation. Also compare assistance/containment routes where appropriate; emergency need not be mandatory for every difficult interaction.

Implementation cautions and sign-off

• NPCController.roll_reaction() samples per-level weights. Difficulty adds +12/0/−8 to the COMPLY weight before normalization; base_compliance alone is not the observed compliance percentage. Measure COMPLY plus LEAVE.
• GameManager._read_npc_truth() currently derives weapon/danger truth from the NPC type (violent threat: weapon/3; everyone else: no weapon/1), rather than observed behavior. This is a balance-review risk: test nonviolent profiles that threaten or force entry, and violent profiles before visible danger. Do not treat agreement with this helper as proof of fair emergency judgment.
• RadioSystem.credibility_hits() checks weapon and danger fields only; location, description and behavior still need human review. Difficulty scales report penalties, not the oracle.
• A terminal peaceful reaction is not sufficient evidence of an incident staying resolved: returning NPCs and unresolved shift-review entries must be retained.

Do not tune to a pooled average that hides a failing venue, phase or difficulty. Record each miss, its severity, the raw evidence and any approved exception before changing weights. Re-run the affected cells and tutorial/campaign smokes after a fix. Five first-time-player sessions must cover the tutorial and a complete venue shift; report pooled counts and per-session errors, not statistical confidence from five players. Scripted sweeps do not substitute for those sessions.

Status: targets defined; sweeps, emergency fairness review, grade/campaign verification and external-player acceptance remain pending. No balance sign-off or release approval is implied by this document.