security-game/docs/plan-graphics-v0.6.md

Development Plan — Graphics Quality v0.6.0 (Blender-authored NPCs)

Status: Proposed · Target version: v0.6.0 (MINOR) Companion decision: ADR-0013

Why this exists

The game is functionally complete through v0.5.0, but NPCs are still the procedural capsule-and-box rig from ADR-0010. They read as people at conversation distance but not at cinematic distance, and every surface on a body is a flat color. ADR-0010 recorded that "a later skinned-mesh pass will replace this rig and must keep the Head contract" — this plan is that pass.

The goal is better NPCs first, then a modest environment lift, using the existing AI tooling for art direction and quality gates, and Blender for the geometry, UVs, skeleton, and baked material maps.

Constraints that shape the plan

• Renderer is GL Compatibility (project.godot config/features). SSAO, SSIL, and volumetrics are unavailable. Quality therefore comes from better meshes, baked maps (AO/normal/roughness), and lighting — not from a renderer switch.
• CI builds offline. .gitea/workflows/build.yml runs the Godot editor only (no Blender, no network). Everything the game loads must be committed; the Blender step is a developer tool, and the game keeps a procedural fallback so a fresh checkout still boots and the headless suite still passes.
• Determinism. Seed-driven variety (outfit_for, layer_for, hand_item_for, idle_style_for) must keep working and keep its tests green. AI generation is a dev-time input, not a runtime dependency; the committed assets are the source of truth.
• Frame budget. v0.5.0 measured 16 avatars = 534 meshes ≈ 6.9 ms headless. A skinned rig replaces ~33 draw calls per avatar with one skin, but skinning costs CPU. Re-measure with src/test/smoke_npc_budget.gd before raising MAX_LIVE_NPCS.

The NPC asset contract (what must not break)

From src/game/npc_avatar.gd and src/game/guard_controller.gd:

• A scene-level node named Rig/Head — GuardController raycasts for it (get_node_or_null("Rig/Head")), and the focus circle + name tag + reaction tag attach under it.
• Limb pivots currently named HipL/HipR → Knee, ShoulderL/ShoulderR → Elbow → Hand. _animate() and _idle_layer() drive them by rotation.x.
• Pure static helpers must stay pure and seed-deterministic: outfit_for, layer_for, hand_item_for, idle_style_for, queue_slot_for, lod_tier_for_distance, reaction_label, crouch_for_pose, prop_lift_for_pose, limb_swing, sway_for_pose.
• LOD tiers (apply_lod, 0 / 1 / 2 with a billboard impostor at tier 2) must keep working; _detail_nodes are hidden at tier 1.
• set_focus_visual, _focus_circle, _name_tag, _reaction_tag, the theater poses, and the crowd reaction (start_rubberneck, start_flee) all operate on the rig and must survive the swap.

Workstreams

Phase 0 — Toolchain spike and contract freeze (small, throwaway)

1. Confirm headless Blender works in this environment. Blender 5.2.0 LTS flatpak is installed; the sandbox cwd is /app/blender, so --python and all input/output paths must be absolute:

   flatpak run org.blender.Blender --background \
     --python /var/home/zwitschi/Documents/06-games/security-game/tools/npc_pipeline.py -- --build
   

   Record the working invocation in MEMORY.md (the Im-Dornäcker notes already warn about the cwd gotcha).

2. Freeze the asset contract in writing (this document + ADR-0013):

   • assets/models/generated/npc_base.glb — one skinned mesh (or a small fixed set of sub-meshes: body / head / hands) with a named armature and UVs.
   • Material slots rather than one atlas-per-prop: skin, cloth, pants, accessory, hair each carry a neutral albedo + normal + ORM (occlusion/roughness/metallic) map. outfit_for colors tint the slot, so multi-color outfits and seed variety survive.
   • Bone names mirror the current node names (Head, HipL/R, KneeL/R, ShoulderL/R, ElbowL/R, HandL/R) so _animate() can target them.
   • Budget: tier-0 base body ≤ ~6,000 triangles, ≤ ~4,000 vertices (revisit after the first smoke_npc_budget.gd run).

Phase 1 — tools/npc_pipeline.py: author + export + audit

New tool, run through the Blender flatpak, mirroring the model_pipeline.py shape (list / build / audit):

1. Author a low-poly base humanoid in Blender: torso, articulated arms/legs, head with a face front (so the focus ring and the "noticed the guard" cue keep a direction), hands, shoes. Keep the current silhouette language (heads, caps, hoods, backpacks, lanyards) so existing outfit_for accessories still attach.
2. Rig it with an armature whose bones are named to match the node contract. Godot 4.3 imports glTF skeletons as Skeleton3D; keep the bone rest pose matching the current _animate() assumptions (arms hang, legs straight, +Y up, origin at the feet on the floor).
3. UV-unwrap into a power-of-two layout and bake maps in Blender from node materials or a higher-detail source:
   • neutral albedo (white fabric pattern) per material slot,
   • normal map (fabric weave, denim, knit),
   • ORM map (occlusion in R, roughness in G, metallic in B — same packing model_pipeline.atlas_for already uses),
   • ambient occlusion baked into the albedo or the ORM R channel (this is the Compatibility renderer's only AO source).
4. Export npc_base.glb with materials referenced as Godot-importable textures (or export textures alongside and wire them in GDScript — decide in the spike).
5. Audit (npc_pipeline.py audit) enforcing: UVs in [0,1], expected bone names present, triangle/vertex budget, feet at y ≈ 0, no missing material slots. Reuse the checks and the colour-space discipline from model_pipeline.py (glTF baseColorFactor is linear; albedo textures are sRGB — do not mix them up).
6. Commit the .blend source (assets/models/npc/npc_base.blend) so the build is reproducible and editable, plus the generated .glb + textures so CI never needs Blender.

Phase 2 — AI-assisted texturing (existing tools only)

Use the tools already in tools/, not new services:

1. tools/asset_pipeline.py generate (OpenRouter, openai/gpt-5-image-mini) for seamless clothing albedos — shirt, trousers, jacket, hi-vis vest, coat, dress — at the role sizes its audit validates. Run seamless for tile-edge blending and audit to reject wrong sizes.
2. tools/model_pipeline.py's palette helpers (dominant_colors, assign_roles) or a small shared import, to derive per-role outfit palettes from AI concept sheets (concept). outfit_for keeps determinism: the AI supplies palettes and textures, never per-frame randomness.
3. Normal/roughness detail (fabric weave, denim) baked procedurally in Blender, with the AI albedos layered on top.
4. Close the loop with tools/screen_designs.py judge: regenerate the staged captures and confirm the new NPCs realise the same screen intents (art style is ignored by the judge; layout and readability are not).

Outputs land under assets/textures/ai/game_ready/ (or a new assets/textures/npc/) and are committed.

Phase 3 — Godot integration (npc_avatar.gd, materials.gd)

1. Add a _load_model() path in NpcAvatar that instantiates the imported npc_base.glb PackedScene, exactly like WorldBuilder._add_prop_model() loads props. Keep _build_body() as the offline/procedural fallback; choose per-avatar with the same "asset or fallback" pattern (record the choice in a loaded_model / fallback_avatar field for tests).
2. Map outfit_for() onto the model's material slots (skin/cloth/pants/ accessory/hair) so seed variety, layer_for, hand_item_for, and the archetype color read stay intact. Accessories (cap/hood/backpack/ sunglasses/lanyard) keep attaching under Rig/Head as they do today.
3. Add SurfaceMaterials recipes for the baked maps — e.g. npc() / skin() / cloth() — so _material() gains a textured path and the body stops calling Materials.flat().
4. Abstract the animation targets so _animate() / _idle_layer() write to either the procedural Node3D pivots (fallback) or Skeleton3D bones (skinned). Keep the existing rotation.x math; the abstraction is a thin _set_limb_rotation(part, axis, radians) wrapper.
5. Keep Rig/Head as a real scene node (a marker following the head bone, or the imported bone's Godot node) so focus, nametag, and head-turn still find it. Re-verify the focus cone smoke and test_npc_character / test_npc_theater / test_target_cycle.
6. Re-measure smoke_npc_budget.gd; adjust LOD_TIERS if the skinned rig is heavier up close. The impostor path is unchanged.

Phase 4 — Environment and lighting lift (after NPCs land)

1. Baked AO on props: export the existing prop set through Blender (or bake AO in npc_pipeline-style tooling) so contact shadows read without SSAO. This replaces the pure box/cylinder builder with Blender-authored props behind the same assets/models/generated/<prop>.glb contract (ADR-0012's follow-up).
2. Higher-resolution environment textures: regenerate brick/concrete/asphalt through asset_pipeline.py with normal maps, keeping the current UV scales.
3. Evaluate a desktop-only Forward+ preset (SSAO/SSIL/volumetrics) while the Web export preset keeps thread_support=false and GL Compatibility. Decide explicitly in an ADR addendum; do not silently change the renderer.
4. Regenerate docs/screenshots/ with capture_screenshots.gd, then run screen_designs.py compare and judge as the regression gate.

Phase 5 — Verification and release

• Headless: fallback build, npc_pipeline.py audit (via test_model_pipeline style Python tests if Blender is present, otherwise skip with reason), outfit determinism, Rig/Head presence, material recipe assertions.
• Windowed smokes: smoke_npc_budget.gd, smoke_movement.gd, smoke_focus_radial.gd, smoke_props.gd; screenshot comparison.
• Bump config/version to 0.6.0, update README.md, docs/architecture.md (Level 2 gains tools/npc_pipeline.py), and publish per docs/RELEASING.md.

Risks and mitigations

Risk	Mitigation
Blender flatpak sandbox paths (cwd /app/blender)	Absolute paths everywhere; record the invocation in MEMORY.md.
glTF colour space (linear factors vs sRGB textures)	Reuse model_pipeline.srgb_to_linear; bake textures as sRGB, keep factors linear; assert in audit.
Skeleton retargeting vs current node math	Keep the bone names + a thin limb-rotation wrapper; fallback rig keeps old math running.
Frame budget with skinned avatars	Measure first (smoke_npc_budget.gd); tune LOD_TIERS; impostor path unchanged.
No SSAO in Compatibility	Bake AO into albedo/ORM; contact-shadow tune in Phase 4.
AI output nondeterminism	Generation is dev-time only; commit outputs and gate with audit/judge/compare.
Blender unavailable in CI	Committed .glb + textures; procedural fallback; audit skipped with reason when Blender is absent.

Acceptance criteria (definition of done)

1. NPCs are Blender-authored skinned meshes with baked maps; they read as people at conversation distance and survive a cinematic close-up.
2. Rig/Head, the focus cone, nametag, theater poses, crowd reactions, and LOD all still work; all existing NPC tests pass unchanged or are updated with clear reasons.
3. The game boots and plays from a fresh checkout with no Blender and no network (committed assets + procedural fallback).
4. npc_pipeline.py build is deterministic and audit enforces the contract.
5. 208 headless + 56 Python tests stay green (Python audit tests skip when Blender is absent); NPC-budget and movement smokes pass; screenshot compare/judge show no regressions.
6. Version bumped to v0.6.0, docs and ADR-0013 updated, release published.