security-game/docs/plan-graphics-v0.6.md

Development Plan — Graphics Quality v0.6.0 (Blender-authored NPCs)

Status: Proposed · Target version: v0.6.0 (MINOR) Companion decision: ADR-0013

Why this exists

The game is functionally complete through v0.5.0, but NPCs are still the procedural capsule-and-box rig from ADR-0010. They read as people at conversation distance but not at cinematic distance, and every surface on a body is a flat color. ADR-0010 recorded that "a later skinned-mesh pass will replace this rig and must keep the Head contract" — this plan is that pass.

The goal is better NPCs first, then a modest environment lift, using the existing AI tooling for art direction and quality gates, and Blender for the geometry, UVs, skeleton, and baked material maps.

Constraints that shape the plan

• Renderer is GL Compatibility (project.godot config/features). SSAO, SSIL, and volumetrics are unavailable. Quality therefore comes from better meshes, baked maps (AO/normal/roughness), and lighting — not from a renderer switch.
• CI builds offline. .gitea/workflows/build.yml runs the Godot editor only (no Blender, no network). Everything the game loads must be committed; the Blender step is a developer tool, and the game keeps a procedural fallback so a fresh checkout still boots and the headless suite still passes.
• Determinism. Seed-driven variety (outfit_for, layer_for, hand_item_for, idle_style_for) must keep working and keep its tests green. AI generation is a dev-time input, not a runtime dependency; the committed assets are the source of truth.
• Frame budget. v0.5.0 measured 16 avatars = 534 meshes ≈ 6.9 ms headless. A skinned rig replaces ~33 draw calls per avatar with one skin, but skinning costs CPU. Re-measure with src/test/smoke_npc_budget.gd before raising MAX_LIVE_NPCS.

The NPC asset contract (what must not break)

From src/game/npc_avatar.gd and src/game/guard_controller.gd:

• A scene-level node named Rig/Head — GuardController raycasts for it (get_node_or_null("Rig/Head")), and the focus circle + name tag + reaction tag attach under it.
• Limb pivots currently named HipL/HipR → Knee, ShoulderL/ShoulderR → Elbow → Hand. _animate() and _idle_layer() drive them by rotation.x.
• Pure static helpers must stay pure and seed-deterministic: outfit_for, layer_for, hand_item_for, idle_style_for, queue_slot_for, lod_tier_for_distance, reaction_label, crouch_for_pose, prop_lift_for_pose, limb_swing, sway_for_pose.
• LOD tiers (apply_lod, 0 / 1 / 2 with a billboard impostor at tier 2) must keep working; _detail_nodes are hidden at tier 1.
• set_focus_visual, _focus_circle, _name_tag, _reaction_tag, the theater poses, and the crowd reaction (start_rubberneck, start_flee) all operate on the rig and must survive the swap.

Workstreams

Phase 0 — Toolchain spike and contract freeze (small, throwaway)

1. Confirm headless Blender works in this environment. Blender 5.2.0 LTS flatpak is installed; the sandbox cwd is /app/blender, so --python and all input/output paths must be absolute:

   flatpak run org.blender.Blender --background \
     --python /var/home/zwitschi/Documents/06-games/security-game/tools/npc_pipeline.py -- --build
   

   Record the working invocation in MEMORY.md (the Im-Dornäcker notes already warn about the cwd gotcha).

2. Freeze the asset contract in writing (this document + ADR-0013). No animation clips are exported: _animate() and _idle_layer() drive the armature bones directly at runtime, so the GLB only needs the skinned rest mesh, the armature, and the material maps.

   • assets/models/generated/npc_base.glb — one skinned mesh (or a small fixed set of sub-meshes: body / head / hands) with a named armature and UVs.
   • Material slots rather than one atlas-per-prop: skin, cloth, pants, accessory, hair each carry a neutral albedo + normal + ORM (occlusion/roughness/metallic) map. outfit_for colors tint the slot, so multi-color outfits and seed variety survive.
   • Bone names mirror the current node names (Head, HipL/R, KneeL/R, ShoulderL/R, ElbowL/R, HandL/R) so _animate() can target them.
   • Budget: tier-0 base body ≤ ~6,000 triangles, ≤ ~4,000 vertices (revisit after the first smoke_npc_budget.gd run).

Phase 1 — tools/npc_pipeline.py: author + export + audit

New tool, run through the Blender flatpak, mirroring the model_pipeline.py shape (list / build / audit):

1. Author a low-poly base humanoid in Blender: torso, articulated arms/legs, head with a face front (so the focus ring and the "noticed the guard" cue keep a direction), hands, shoes. Keep the current silhouette language (heads, caps, hoods, backpacks, lanyards) so existing outfit_for accessories still attach.
2. Rig it with an armature whose bones are named to match the node contract. Godot 4.3 imports glTF skeletons as Skeleton3D; keep the bone rest pose matching the current _animate() assumptions (arms hang, legs straight, +Y up, origin at the feet on the floor).
3. UV-unwrap into a power-of-two layout and bake maps in Blender from node materials or a higher-detail source:
   • neutral albedo (white fabric pattern) per material slot,
   • normal map (fabric weave, denim, knit),
   • ORM map (occlusion in R, roughness in G, metallic in B — same packing model_pipeline.atlas_for already uses),
   • ambient occlusion baked into the albedo or the ORM R channel (this is the Compatibility renderer's only AO source).
4. Export npc_base.glb with materials referenced as Godot-importable textures (or export textures alongside and wire them in GDScript — decide in the spike).
5. Audit (npc_pipeline.py audit) enforcing: UVs in [0,1], expected bone names present, triangle/vertex budget, feet at y ≈ 0, no missing material slots. Reuse the checks and the colour-space discipline from model_pipeline.py (glTF baseColorFactor is linear; albedo textures are sRGB — do not mix them up).
6. Commit the .blend source (assets/models/npc/npc_base.blend) so the build is reproducible and editable, plus the generated .glb + textures so CI never needs Blender.

Phase 2 — AI-assisted texturing (existing tools only)

Use the tools already in tools/, not new services:

1. tools/asset_pipeline.py generate (OpenRouter, openai/gpt-5-image-mini) for seamless clothing albedos — shirt, trousers, jacket, hi-vis vest, coat, dress — at the role sizes its audit validates. Run seamless for tile-edge blending and audit to reject wrong sizes.
2. tools/model_pipeline.py's palette helpers (dominant_colors, assign_roles) or a small shared import, to derive per-role outfit palettes from AI concept sheets (concept). outfit_for keeps determinism: the AI supplies palettes and textures, never per-frame randomness.
3. Normal/roughness detail (fabric weave, denim) baked procedurally in Blender, with the AI albedos layered on top.
4. Close the loop with tools/screen_designs.py judge: regenerate the staged captures and confirm the new NPCs realise the same screen intents (art style is ignored by the judge; layout and readability are not).

Outputs land under assets/textures/ai/game_ready/ (or a new assets/textures/npc/) and are committed.

Phase 3 — Godot integration (npc_avatar.gd, materials.gd)

1. Add a _load_model() path in NpcAvatar that instantiates the imported npc_base.glb PackedScene, exactly like WorldBuilder._add_prop_model() loads props. Keep _build_body() as the offline/procedural fallback; choose per-avatar with the same "asset or fallback" pattern (record the choice in a loaded_model / fallback_avatar field for tests).
2. Map outfit_for() onto the model's material slots (skin/cloth/pants/ accessory/hair) so seed variety, layer_for, hand_item_for, and the archetype color read stay intact. Accessories (cap/hood/backpack/ sunglasses/lanyard) keep attaching under Rig/Head as they do today.
3. Add SurfaceMaterials recipes for the baked maps — e.g. npc() / skin() / cloth() — so _material() gains a textured path and the body stops calling Materials.flat().
4. Abstract the animation targets so _animate() / _idle_layer() write to either the procedural Node3D pivots (fallback) or Skeleton3D bones (skinned). Keep the existing rotation.x math; the abstraction is a thin _set_limb_rotation(part, axis, radians) wrapper.
5. Keep Rig/Head as a real scene node (a marker following the head bone, or the imported bone's Godot node) so focus, nametag, and head-turn still find it. Re-verify the focus cone smoke and test_npc_character / test_npc_theater / test_target_cycle.
6. Re-measure smoke_npc_budget.gd; adjust LOD_TIERS if the skinned rig is heavier up close. The impostor path is unchanged.

Phase 3.5 — Blender asset library (characters, gear, props)

The library generator is tools/blender_assets.py (flatpak Blender, headless). Status: integrated — NpcAvatar selects characters/gear and WorldBuilder loads the replacement/new props plus phase-driven emissives; the procedural rig and primitive props remain fallbacks. Integration notes: blender-assets.md.

Outputs and contract

Kind	Path	Notes
Characters	assets/models/blender/npc/<name>.glb	8 variants: npc_casual, npc_jacket, npc_coat, npc_vest, npc_dress, npc_hoodie, npc_firefighter, npc_guard
Gear	assets/models/blender/npc_gear/<name>.glb	18 pieces; authored in attach-point space (head, hand, back, chest)
Props	assets/models/blender/props/<name>.glb	20 pieces; the first nine replace current generated/primitive street furniture, the rest are new venue dressing
Manifest	assets/models/blender/manifest.json	path, category, triangles, vertices, bounds, materials, roles, tint slots, attach points
Sources	assets/models/blender_src/<name>.blend	committed, editable source
Previews	docs/blender_previews/sheet_<category>.jpg (committed) + per-asset *.png (local)	render / sheet for visual review

All characters reuse the npc_base contract: mesh node npc_base, bones Head/HipL/R/KneeL/R/ShoulderL/R/ElbowL/R/HandL/R (+ Hips/Spine/Neck), material slots skin/cloth/pants/accessory/hair, front toward Blender -Y (Godot +Z), feet at y ~= 0. Gear origins are the attach points (head at Rig/Head, hand at the hand proxy, back at the spine, chest at the sternum). Props are grounded except wall_lantern and marquee; both may be emissive.

Integration steps

1. Audit the finished build. python3 tools/blender_assets.py audit and sheet; fix assets or reconcile the budgets. Done 2026-11-01: the characters were simplified to fit the unchanged 6,000 / 4,000 contract (3,788–4,640 tris), and all 46 assets audit clean.
2. Commit the library and sources. Add assets/models/blender/**, assets/models/blender_src/**, the generator, manifest, previews, and docs/blender-assets.md; generated folders that CI can rebuild offline may stay ignored. blender_assets.py audit must work without Blender.
3. Select characters by role. Extend NpcAvatar to pick a variant from the manifest using the profile's role and outfit_for/layer_for (casual, jacket, coat, vest, dress, hoodie, firefighter, guard). Fall back to npc_base.glb, then to the procedural rig, and record the chosen path.
4. Tint and anchor. Apply outfit_for to every variant's five slots via SurfaceMaterials.npc_slot(); keep Rig/Head for the focus circle, name tag, reaction tag, and accessories. Keep the walk/idle/escalation bone-driving wrapper.
5. Replace procedural gear. Use the npc_gear catalogue for caps, hoods, helmets, backpacks, lanyards, hand items (phone, cup, clipboard, umbrella, camera, sign, flashlight, wand, radio) and the SCBA tank; attach at the documented points and fall back to the current primitives when an asset is missing.
6. Replace and extend props. Load the replacement set through WorldBuilder (bollard, bench, trash_bin, planter_box, newspaper_box, fire_hydrant, street_lamp, street_tree, traffic_cone), then place the new dressing (palm_tree, round_planter, a_frame_sign, wall_lantern, crowd_barrier, stanchion, handrail, security_podium, double_door, marquee, sedan) using the existing placement table/collider/avoidance contract. No prop may enter the door/steps/guest lane.
7. Wire emissives to the phase. Street-lamp bulbs, wall lanterns, the door lock LED, marquee bulbs/sign face, and sedan headlights should follow the existing day/night lighting rather than staying always on.
8. Measure and gate. Re-run smoke_npc_budget.gd with the new characters before raising MAX_LIVE_NPCS; add a prop triangle/draw budget check. Then recapture docs/screenshots/ and run screen_designs.py compare + judge plus a close-up target shot for character readability.
9. Fallbacks stay offline. A fresh checkout with no Blender and no network must boot on the committed library (or npc_base.glb/procedural rig when an asset is absent); CI still runs the Godot import and headless suite.

Phase 4 — Environment and lighting lift (after NPCs land) ✅

1. Baked AO on props: use the Phase 3.5 Blender prop replacements (street_lamp, street_tree, bench, …) and bake AO into their albedo/ORM maps so contact shadows read without SSAO. Keep the same WorldBuilder load/fallback contract (ADR-0012's follow-up).
2. Higher-resolution environment textures: regenerate brick/concrete/asphalt through asset_pipeline.py with normal maps, keeping the current UV scales; use the same sRGB/albedo + packed ORM discipline as the library assets.
3. Evaluate a desktop-only Forward+ preset — done: it boots on the desktop Vulkan driver, but the renderer is project-wide and the Web preset must stay GL Compatibility. v0.6.0 keeps Compatibility; a desktop-only override is deferred.
4. Regenerate docs/screenshots/ with capture_screenshots.gd, then run screen_designs.py compare and judge as the regression gate, plus blender_assets.py sheet for the character/gear/prop library.

Phase 5 — Verification and release

• Headless: fallback build, npc_pipeline.py audit (via test_model_pipeline style Python tests if Blender is present, otherwise skip with reason), outfit determinism, Rig/Head presence, material recipe assertions.
• Windowed smokes: smoke_npc_budget.gd, smoke_movement.gd, smoke_focus_radial.gd, smoke_props.gd; screenshot comparison.
• Bump config/version to 0.6.0, update README.md, docs/architecture.md (Level 2 gains tools/npc_pipeline.py), and publish per docs/RELEASING.md.

Risks and mitigations

Risk	Mitigation
Blender flatpak sandbox paths (cwd /app/blender)	Absolute paths everywhere; record the invocation in MEMORY.md.
glTF colour space (linear factors vs sRGB textures)	Reuse model_pipeline.srgb_to_linear; bake textures as sRGB, keep factors linear; assert in audit.
Skeleton retargeting vs current node math	Keep the bone names + a thin limb-rotation wrapper; fallback rig keeps old math running.
Frame budget with skinned avatars	Measure first (smoke_npc_budget.gd); tune LOD_TIERS; impostor path unchanged.
No SSAO in Compatibility	Bake AO into albedo/ORM; contact-shadow tune in Phase 4.
AI output nondeterminism	Generation is dev-time only; commit outputs and gate with audit/judge/compare.
Blender unavailable in CI	Committed .glb + textures; procedural fallback; audit skipped with reason when Blender is absent.

Acceptance criteria (definition of done)

1. NPCs and street furniture are Blender-authored meshes with baked maps; the character variants, gear, and props read as people/objects at conversation distance and survive a cinematic close-up.
2. Rig/Head, the focus cone, nametag, theater poses, crowd reactions, and LOD all still work; all existing NPC tests pass unchanged or are updated with clear reasons.
3. The game boots and plays from a fresh checkout with no Blender and no network (committed assets + procedural fallback).
4. npc_pipeline.py build is deterministic and audit enforces the contract.
5. 212 headless + 60 Python tests stay green (Blender-dependent Python tests skip with a reason when Blender is absent); npc_pipeline.py audit and blender_assets.py audit pass; NPC-budget and movement smokes pass; screenshot compare/judge and the library contact sheets show no regressions.
6. The library is committed with its generator/manifest/previews, runtime selection and fallbacks are covered by tests, version bumped to v0.6.0, docs and ADR-0013 updated, release published.