security-game/docs/plan-graphics-v0.6.md

# Development Plan — Graphics Quality v0.6.0 (Blender-authored NPCs)

> Status: Proposed · Target version: **v0.6.0** (MINOR)
> Companion decision: [ADR-0013](adr/0013-blender-authored-npc-characters.md)

## Why this exists

The game is functionally complete through v0.5.0, but NPCs are still the
procedural capsule-and-box rig from [ADR-0010](adr/0010-procedural-npc-presentation.md).
They read as people at conversation distance but not at cinematic distance, and
every surface on a body is a flat color. ADR-0010 recorded that "a later
skinned-mesh pass will replace this rig and must keep the `Head` contract" —
this plan is that pass.

The goal is **better NPCs first**, then a modest environment lift, using the
existing AI tooling for art direction and quality gates, and **Blender** for the
geometry, UVs, skeleton, and baked material maps.

## Constraints that shape the plan

- **Renderer is GL Compatibility** (`project.godot` `config/features`). SSAO,
  SSIL, and volumetrics are unavailable. Quality therefore comes from better
  meshes, baked maps (AO/normal/roughness), and lighting — not from a renderer
  switch.
- **CI builds offline.** `.gitea/workflows/build.yml` runs the Godot editor only
  (no Blender, no network). Everything the game loads must be committed; the
  Blender step is a developer tool, and the game keeps a procedural fallback so a
  fresh checkout still boots and the headless suite still passes.
- **Determinism.** Seed-driven variety (`outfit_for`, `layer_for`,
  `hand_item_for`, `idle_style_for`) must keep working and keep its tests green.
  AI generation is a dev-time input, not a runtime dependency; the committed
  assets are the source of truth.
- **Frame budget.** v0.5.0 measured 16 avatars = 534 meshes ≈ 6.9 ms headless.
  A skinned rig replaces ~33 draw calls per avatar with one skin, but skinning
  costs CPU. Re-measure with `src/test/smoke_npc_budget.gd` before raising
  `MAX_LIVE_NPCS`.

## The NPC asset contract (what must not break)

From `src/game/npc_avatar.gd` and `src/game/guard_controller.gd`:

- A scene-level node named **`Rig/Head`** — `GuardController` raycasts for it
  (`get_node_or_null("Rig/Head")`), and the focus circle + name tag + reaction
  tag attach under it.
- Limb pivots currently named `HipL`/`HipR` → `Knee`, `ShoulderL`/`ShoulderR` →
  `Elbow` → `Hand`. `_animate()` and `_idle_layer()` drive them by `rotation.x`.
- Pure static helpers must stay pure and seed-deterministic: `outfit_for`,
  `layer_for`, `hand_item_for`, `idle_style_for`, `queue_slot_for`,
  `lod_tier_for_distance`, `reaction_label`, `crouch_for_pose`,
  `prop_lift_for_pose`, `limb_swing`, `sway_for_pose`.
- LOD tiers (`apply_lod`, 0 / 1 / 2 with a billboard impostor at tier 2) must
  keep working; `_detail_nodes` are hidden at tier 1.
- `set_focus_visual`, `_focus_circle`, `_name_tag`, `_reaction_tag`, the
  theater poses, and the crowd reaction (`start_rubberneck`, `start_flee`) all
  operate on the rig and must survive the swap.

## Workstreams

### Phase 0 — Toolchain spike and contract freeze (small, throwaway)

1. Confirm headless Blender works in this environment. Blender **5.2.0 LTS**
   flatpak is installed; the sandbox cwd is `/app/blender`, so `--python` and
   all input/output paths must be absolute:

   ```bash
   flatpak run org.blender.Blender --background \
     --python /var/home/zwitschi/Documents/06-games/security-game/tools/npc_pipeline.py -- --build
   ```

   Record the working invocation in `MEMORY.md` (the Im-Dornäcker notes already
   warn about the cwd gotcha).
2. Freeze the asset contract in writing (this document + ADR-0013). No
   animation clips are exported: `_animate()` and `_idle_layer()` drive the
   armature bones directly at runtime, so the GLB only needs the skinned rest
   mesh, the armature, and the material maps.
   - `assets/models/generated/npc_base.glb` — one skinned mesh (or a small fixed
     set of sub-meshes: body / head / hands) with a **named armature** and UVs.
   - Material slots rather than one atlas-per-prop: `skin`, `cloth`,
     `pants`, `accessory`, `hair` each carry a neutral albedo + normal + ORM
     (occlusion/roughness/metallic) map. `outfit_for` colors tint the slot, so
     multi-color outfits and seed variety survive.
   - Bone names mirror the current node names (`Head`, `HipL/R`, `KneeL/R`,
     `ShoulderL/R`, `ElbowL/R`, `HandL/R`) so `_animate()` can target them.
   - Budget: tier-0 base body ≤ ~6,000 triangles, ≤ ~4,000 vertices (revisit
     after the first `smoke_npc_budget.gd` run).

### Phase 1 — `tools/npc_pipeline.py`: author + export + audit

New tool, run through the Blender flatpak, mirroring the `model_pipeline.py`
shape (`list` / `build` / `audit`):

1. **Author** a low-poly base humanoid in Blender: torso, articulated arms/legs,
   head with a face front (so the focus ring and the "noticed the guard" cue
   keep a direction), hands, shoes. Keep the current silhouette language
   (heads, caps, hoods, backpacks, lanyards) so existing `outfit_for`
   accessories still attach.
2. **Rig** it with an armature whose bones are named to match the node contract.
   Godot 4.3 imports glTF skeletons as `Skeleton3D`; keep the bone rest pose
   matching the current `_animate()` assumptions (arms hang, legs straight,
   +Y up, origin at the feet on the floor).
3. **UV-unwrap** into a power-of-two layout and **bake** maps in Blender from
   node materials or a higher-detail source:
   - neutral albedo (white fabric pattern) per material slot,
   - normal map (fabric weave, denim, knit),
   - ORM map (occlusion in R, roughness in G, metallic in B — same packing
     `model_pipeline.atlas_for` already uses),
   - ambient occlusion baked into the albedo or the ORM R channel (this is the
     Compatibility renderer's only AO source).
4. **Export** `npc_base.glb` with materials referenced as Godot-importable
   textures (or export textures alongside and wire them in GDScript — decide in
   the spike).
5. **Audit** (`npc_pipeline.py audit`) enforcing: UVs in [0,1], expected bone
   names present, triangle/vertex budget, feet at y ≈ 0, no missing material
   slots. Reuse the checks and the colour-space discipline from
   `model_pipeline.py` (glTF `baseColorFactor` is linear; albedo textures are
   sRGB — do not mix them up).
6. **Commit** the `.blend` source (`assets/models/npc/npc_base.blend`) so the
   build is reproducible and editable, plus the generated `.glb` + textures so
   CI never needs Blender.

### Phase 2 — AI-assisted texturing (existing tools only)

Use the tools already in `tools/`, not new services:

1. `tools/asset_pipeline.py generate` (OpenRouter,
   `openai/gpt-5-image-mini`) for seamless clothing albedos — shirt, trousers,
   jacket, hi-vis vest, coat, dress — at the role sizes its `audit` validates.
   Run `seamless` for tile-edge blending and `audit` to reject wrong sizes.
2. `tools/model_pipeline.py`'s palette helpers (`dominant_colors`,
   `assign_roles`) or a small shared import, to derive per-role outfit palettes
   from AI concept sheets (`concept`). `outfit_for` keeps determinism: the AI
   supplies palettes and textures, never per-frame randomness.
3. Normal/roughness detail (fabric weave, denim) baked procedurally in Blender,
   with the AI albedos layered on top.
4. Close the loop with `tools/screen_designs.py judge`: regenerate the staged
   captures and confirm the new NPCs realise the same screen intents (art style
   is ignored by the judge; layout and readability are not).

Outputs land under `assets/textures/ai/game_ready/` (or a new
`assets/textures/npc/`) and are **committed**.

### Phase 3 — Godot integration (`npc_avatar.gd`, `materials.gd`)

1. Add a `_load_model()` path in `NpcAvatar` that instantiates the imported
   `npc_base.glb` `PackedScene`, exactly like `WorldBuilder._add_prop_model()`
   loads props. Keep `_build_body()` as the offline/procedural fallback; choose
   per-avatar with the same "asset or fallback" pattern (record the choice in a
   `loaded_model` / `fallback_avatar` field for tests).
2. Map `outfit_for()` onto the model's material slots (skin/cloth/pants/
   accessory/hair) so seed variety, `layer_for`, `hand_item_for`, and the
   archetype color read stay intact. Accessories (cap/hood/backpack/
   sunglasses/lanyard) keep attaching under `Rig`/`Head` as they do today.
3. Add `SurfaceMaterials` recipes for the baked maps — e.g. `npc()` /
   `skin()` / `cloth()` — so `_material()` gains a textured path and the body
   stops calling `Materials.flat()`.
4. Abstract the animation targets so `_animate()` / `_idle_layer()` write to
   either the procedural `Node3D` pivots (fallback) or `Skeleton3D` bones
   (skinned). Keep the existing `rotation.x` math; the abstraction is a thin
   `_set_limb_rotation(part, axis, radians)` wrapper.
5. Keep `Rig/Head` as a real scene node (a marker following the head bone, or
   the imported bone's Godot node) so focus, nametag, and head-turn still find
   it. Re-verify the focus cone smoke and `test_npc_character` /
   `test_npc_theater` / `test_target_cycle`.
6. Re-measure `smoke_npc_budget.gd`; adjust `LOD_TIERS` if the skinned rig is
   heavier up close. The impostor path is unchanged.

### Phase 3.5 — Blender asset library (characters, gear, props)

The library generator is `tools/blender_assets.py` (flatpak Blender, headless).
It stages outputs for integration; nothing is wired into the game until the
audit and manifest are clean. Integration notes: [`blender-assets.md`](blender-assets.md).

**Outputs and contract**

| Kind | Path | Notes |
| --- | --- | --- |
| Characters | `assets/models/blender/npc/<name>.glb` | 8 variants: `npc_casual`, `npc_jacket`, `npc_coat`, `npc_vest`, `npc_dress`, `npc_hoodie`, `npc_firefighter`, `npc_guard` |
| Gear | `assets/models/blender/npc_gear/<name>.glb` | 18 pieces; authored in attach-point space (`head`, `hand`, `back`, `chest`) |
| Props | `assets/models/blender/props/<name>.glb` | 20 pieces; the first nine replace current generated/primitive street furniture, the rest are new venue dressing |
| Manifest | `assets/models/blender/manifest.json` | path, category, triangles, vertices, bounds, materials, roles, tint slots, attach points |
| Sources | `assets/models/blender_src/<name>.blend` | committed, editable source |
| Previews | `docs/blender_previews/sheet_<category>.jpg` (committed) + per-asset `*.png` (local) | `render` / `sheet` for visual review |

All characters reuse the `npc_base` contract: mesh node `npc_base`, bones
`Head/HipL/R/KneeL/R/ShoulderL/R/ElbowL/R/HandL/R` (+ Hips/Spine/Neck),
material slots `skin/cloth/pants/accessory/hair`, front toward Blender `-Y`
(Godot `+Z`), feet at `y ~= 0`. Gear origins are the attach points (head at
`Rig/Head`, hand at the hand proxy, back at the spine, chest at the sternum).
Props are grounded except `wall_lantern` and `marquee`; both may be emissive.

**Integration steps**

1. **Audit the finished build.** `python3 tools/blender_assets.py audit` and
   `sheet`; fix assets or reconcile the budgets. **Done 2026-11-01:** the
   characters were simplified to fit the unchanged 6,000 / 4,000 contract
   (3,788–4,640 tris), and all 46 assets audit clean.
2. **Commit the library and sources.** Add `assets/models/blender/**`,
   `assets/models/blender_src/**`, the generator, manifest, previews, and
   `docs/blender-assets.md`; generated folders that CI can rebuild offline may
   stay ignored. `blender_assets.py audit` must work without Blender.
3. **Select characters by role.** Extend `NpcAvatar` to pick a variant from the
   manifest using the profile's role and `outfit_for`/`layer_for` (casual,
   jacket, coat, vest, dress, hoodie, firefighter, guard). Fall back to
   `npc_base.glb`, then to the procedural rig, and record the chosen path.
4. **Tint and anchor.** Apply `outfit_for` to every variant's five slots via
   `SurfaceMaterials.npc_slot()`; keep `Rig/Head` for the focus circle, name tag,
   reaction tag, and accessories. Keep the walk/idle/escalation bone-driving
   wrapper.
5. **Replace procedural gear.** Use the `npc_gear` catalogue for caps, hoods,
   helmets, backpacks, lanyards, hand items (phone, cup, clipboard, umbrella,
   camera, sign, flashlight, wand, radio) and the SCBA tank; attach at the
   documented points and fall back to the current primitives when an asset is
   missing.
6. **Replace and extend props.** Load the replacement set through
   `WorldBuilder` (`bollard`, `bench`, `trash_bin`, `planter_box`,
   `newspaper_box`, `fire_hydrant`, `street_lamp`, `street_tree`,
   `traffic_cone`), then place the new dressing (`palm_tree`, `round_planter`,
   `a_frame_sign`, `wall_lantern`, `crowd_barrier`, `stanchion`, `handrail`,
   `security_podium`, `double_door`, `marquee`, `sedan`) using the existing
   placement table/collider/avoidance contract. No prop may enter the
   door/steps/guest lane.
7. **Wire emissives to the phase.** Street-lamp bulbs, wall lanterns, the door
   lock LED, marquee bulbs/sign face, and sedan headlights should follow the
   existing day/night lighting rather than staying always on.
8. **Measure and gate.** Re-run `smoke_npc_budget.gd` with the new characters
   before raising `MAX_LIVE_NPCS`; add a prop triangle/draw budget check. Then
   recapture `docs/screenshots/` and run `screen_designs.py compare` + `judge`
   plus a close-up target shot for character readability.
9. **Fallbacks stay offline.** A fresh checkout with no Blender and no network
   must boot on the committed library (or `npc_base.glb`/procedural rig when an
   asset is absent); CI still runs the Godot import and headless suite.


### Phase 4 — Environment and lighting lift (after NPCs land)

1. **Baked AO on props:** use the Phase 3.5 Blender prop replacements
   (`street_lamp`, `street_tree`, `bench`, …) and bake AO into their
   albedo/ORM maps so contact shadows read without SSAO. Keep the same
   `WorldBuilder` load/fallback contract (ADR-0012's follow-up).
2. **Higher-resolution environment textures:** regenerate brick/concrete/asphalt
   through `asset_pipeline.py` with normal maps, keeping the current UV scales;
   use the same sRGB/albedo + packed ORM discipline as the library assets.
3. **Evaluate a desktop-only Forward+ preset** (SSAO/SSIL/volumetrics) while the
   Web export preset keeps `thread_support=false` and GL Compatibility. Decide
   explicitly in an ADR addendum; do not silently change the renderer.
4. Regenerate `docs/screenshots/` with `capture_screenshots.gd`, then run
   `screen_designs.py compare` and `judge` as the regression gate, plus
   `blender_assets.py sheet` for the character/gear/prop library.

### Phase 5 — Verification and release

- Headless: fallback build, `npc_pipeline.py audit` (via `test_model_pipeline`
  style Python tests if Blender is present, otherwise skip with reason), outfit
  determinism, `Rig/Head` presence, material recipe assertions.
- Windowed smokes: `smoke_npc_budget.gd`, `smoke_movement.gd`,
  `smoke_focus_radial.gd`, `smoke_props.gd`; screenshot comparison.
- Bump `config/version` to `0.6.0`, update `README.md`, `docs/architecture.md`
  (Level 2 gains `tools/npc_pipeline.py`), and publish per
  [`docs/RELEASING.md`](RELEASING.md).

## Risks and mitigations

| Risk | Mitigation |
| --- | --- |
| Blender flatpak sandbox paths (cwd `/app/blender`) | Absolute paths everywhere; record the invocation in `MEMORY.md`. |
| glTF colour space (linear factors vs sRGB textures) | Reuse `model_pipeline.srgb_to_linear`; bake textures as sRGB, keep factors linear; assert in `audit`. |
| Skeleton retargeting vs current node math | Keep the bone names + a thin limb-rotation wrapper; fallback rig keeps old math running. |
| Frame budget with skinned avatars | Measure first (`smoke_npc_budget.gd`); tune `LOD_TIERS`; impostor path unchanged. |
| No SSAO in Compatibility | Bake AO into albedo/ORM; contact-shadow tune in Phase 4. |
| AI output nondeterminism | Generation is dev-time only; commit outputs and gate with `audit`/`judge`/`compare`. |
| Blender unavailable in CI | Committed `.glb` + textures; procedural fallback; `audit` skipped with reason when Blender is absent. |

## Acceptance criteria (definition of done)

1. NPCs and street furniture are Blender-authored meshes with baked maps; the
   character variants, gear, and props read as people/objects at conversation
   distance and survive a cinematic close-up.
2. `Rig/Head`, the focus cone, nametag, theater poses, crowd reactions, and LOD
   all still work; all existing NPC tests pass unchanged or are updated with
   clear reasons.
3. The game boots and plays from a fresh checkout with **no** Blender and no
   network (committed assets + procedural fallback).
4. `npc_pipeline.py build` is deterministic and `audit` enforces the contract.
5. 212 headless + 60 Python tests stay green (Blender-dependent Python tests
   skip with a reason when Blender is absent); `npc_pipeline.py audit` and
   `blender_assets.py audit` pass; NPC-budget and movement smokes pass;
   screenshot `compare`/`judge` and the library contact sheets show no
   regressions.
6. The library is committed with its generator/manifest/previews, runtime
   selection and fallbacks are covered by tests, version bumped to v0.6.0,
   docs and ADR-0013 updated, release published.