diff --git a/MEMORY.md b/MEMORY.md index a3096c0..7074fdd 100644 --- a/MEMORY.md +++ b/MEMORY.md @@ -4,6 +4,15 @@ Session memory for durable decisions, state, and gotchas. Keep concise, remove o ## Decisions +- **Pose capture is blocked (2026-11-03):** `src/test/capture_npc_pose.gd` stages idle/walk/sit + poses for the pose/weight review, but the saved frames show the rest pose while the same run + logs `rig_y=-0.339`, `leg0=-1.357` and a rotated `HipL` bone quaternion at capture time. + Verified meanwhile: one skinned mesh bound to one `Skeleton3D`; hiding it removes the body; + direct `set_bone_pose_rotation` on HipL/KneeL/Spine/ShoulderL/ElbowL/Head changes rendered + pixels; a bone *position* offset moves the head; a standalone probe running the same + `_animate()` sequence renders the crouch. A staged in-game `film` frame also showed the arms + down, so live venue animation may be affected too. Do not treat the pose sheets as evidence; + a scene test (posed frame differs from rest frame) is the next step. - **3D review baseline (2026-11-03):** `docs/3d-model-review.md` records the first skill-guided measurements: 46/46 assets clean, library 59,012 tris, characters 4,482–5,474 tris (68–69% of the 8,000 caps), `npc_base` only 1,204/2,488, 24-NPC 6.94 ms, and authored texel density diff --git a/TODO.md b/TODO.md index 488a02a..d44b9cf 100644 --- a/TODO.md +++ b/TODO.md @@ -60,6 +60,14 @@ except the rebuilt assets themselves. finger-curl rig or the current open hand plus `HAND_ITEM_FORWARD`, and record the limit. - [ ] Weights/pose: capture idle, walk, sit and wave; inspect shoulder, hip and neck collapse; fix the worst offenders using `rigging` practice without changing the 13-bone contract. + **Blocked:** the new `capture_npc_pose.gd` reports the seated/walking pose while the saved + frames still show the rest pose; direct bone writes deform the mesh, so this is a staging/ + capture discrepancy that may also affect live venue frames. See + `docs/3d-model-review.md` §"Pose evidence is blocked". +- [ ] Diagnose the pose-capture discrepancy: compare the harness staging with the working + standalone probe (SceneTree lifetime, node parents, first-frame skin bind, capture timing) + and add a scene test asserting a posed avatar renders differently from its rest frame. + Do not trust pose sheets or claim rig deformation until that test passes. - [ ] UV/texel density: measure the 256² slot maps; decide whether atlas, UDIM or a resolution change is justified. Do not change `CHAR_MAX_TRIS`/`CHAR_MAX_VERTS` without re-running the budget smoke and the Web/Compatibility path. diff --git a/docs/3d-model-review.md b/docs/3d-model-review.md index 39ad3dc..1a370ba 100644 --- a/docs/3d-model-review.md +++ b/docs/3d-model-review.md @@ -88,14 +88,15 @@ yet reviewed; this document is the place to fill it. | Character | Silhouette | Face @1.85 m | Hands/grip | Weights/pose | Cloth/hair | LOD | Verdict | | --- | --- | --- | --- | --- | --- | --- | --- | -| `npc_casual` | accept (0.9.4) | P1 (faceted planes) | accept (open palm) | — | accept | accept (two-quad impostor) | pending owner | -| `npc_jacket` | accept | P1 | accept | — | accept | accept | pending owner | -| `npc_coat` | accept | P1 | accept | — | accept (hem 0.76 stride) | accept | pending owner | -| `npc_vest` | accept | P1 | accept | — | accept | accept | pending owner | -| `npc_dress` | accept; performer props dominate | P1 | accept | — | accept | accept | pending owner | -| `npc_hoodie` | accept (hooded criterion) | accept (lower face only) | accept | — | accept | accept | pending owner | -| `npc_firefighter` | accept | P1 | accept | — | accept | accept | pending owner | -| `npc_guard` | accept | P1 | accept | — | accept | accept | pending owner | +| *(pose evidence blocked: see "Pose evidence is blocked" below)* | | | | | | | | +| `npc_casual` | accept (0.9.4) | P1 (faceted planes) | accept (open palm) | blocked | accept | accept (two-quad impostor) | pending owner | +| `npc_jacket` | accept | P1 | accept | blocked | accept | accept | pending owner | +| `npc_coat` | accept | P1 | accept | blocked | accept (hem 0.76 stride) | accept | pending owner | +| `npc_vest` | accept | P1 | accept | blocked | accept | accept | pending owner | +| `npc_dress` | accept; performer props dominate | P1 | accept | blocked | accept | accept | pending owner | +| `npc_hoodie` | accept (hooded criterion) | accept (lower face only) | accept | blocked | accept | accept | pending owner | +| `npc_firefighter` | accept | P1 | accept | blocked | accept | accept | pending owner | +| `npc_guard` | accept | P1 | accept | blocked | accept | accept | pending owner | ### Gear (18) @@ -162,6 +163,29 @@ for presence, not re-reviewed for art quality. | The Joke Shop | 22,680 tris / 454 meshes | shape-language review pending | | TV Studio | — | shape-language review pending | +## Pose evidence is blocked (2026-11-03) + +The weights/pose column cannot be filled yet. A pose harness was added +(`src/test/capture_npc_pose.gd`, staged in `tmp/model-pose*`), and its output does not match the +state it reports: + +- `_animate()` applies the seated pose (logged `rig_y=-0.339`, `leg0=-1.357`) and + `Skeleton3D.get_bone_pose_rotation()` returns the rotated quaternion at capture time, yet the + saved frame shows the rest pose. +- Ruled out: missing skeleton (one `Skeleton3D`, mesh bound to it, `same=true`), hidden mesh + (hiding `npc_base` removes the body), pose not applied (proxies and bone poses verified), + and unbound skinning: writing a single bone rotation directly (`HipL`, `KneeL`, `Spine`, + `ShoulderL`, `ElbowL`, `Head`) changes the rendered pixels, and a bone *position* offset moves + the head. A standalone probe that calls the same `_animate()` sequence renders the crouch. +- Also observed: a staged in-game `film` pose frame showed the arms down, so this may affect the + live game rather than only the harness. That is why it is a P0/P1 candidate rather than a QA + tooling nit. + +Next step is a focused diagnosis: compare the harness/venue staging with the standalone probe +(SceneTree lifetime, node parents, first-frame skin bind, capture timing) and add a scene test +that asserts a posed avatar renders differently from its rest frame. Do not mark the pose/weight +scorecard row until that test exists. + ## Ranked candidate gaps Ordered by expected player-visible gain per unit of work. None is approved for fixing yet; the diff --git a/docs/asset-tooling.md b/docs/asset-tooling.md index 0a60f8e..b6e1e17 100644 --- a/docs/asset-tooling.md +++ b/docs/asset-tooling.md @@ -307,6 +307,22 @@ height/head/shoulder/hip/torso/leg/arm/foot guide; the current results are in [`npc-proportion-audit.md`](npc-proportion-audit.md). It reads the exported GLBs and does not require Blender. +`capture_npc_pose.gd` is the pose/deformation harness for the 3D re-evaluation: idle, +walk-contact, walk-pass and seated poses of all eight variants, full-body plus shoulder +close-ups. **It is currently blocked**: the applied pose is visible in the rig and bone values +but not in the captured frames, so do not use its sheets as pose evidence. The reproduction and +what was ruled out are in [`3d-model-review.md`](3d-model-review.md) §"Pose evidence is blocked". + +The baseline sheet tool accepts a custom view list, which is also how pose sheets are tiled: + +```bash +# baseline set (six views, eight variants) +python3 tools/npc_baseline.py sheet --dir tmp/npc-baseline +# pose set produced by capture_npc_pose.gd +python3 tools/npc_baseline.py sheet --dir tmp/npc-pose \ + --views idle,walk_contact,walk_pass,sit --out tmp/npc-pose/sheet_pose.jpg +``` + `capture_npc_baseline.gd` is the 0.9.4 neutral baseline: eight characters × front/three-quarter/profile/back/full-body/crowd = 48 frames with a fixed camera, studio lighting and no venue/HUD/animation. The raw PNGs stay in diff --git a/src/test/capture_npc_pose.gd b/src/test/capture_npc_pose.gd new file mode 100644 index 0000000..c804b30 --- /dev/null +++ b/src/test/capture_npc_pose.gd @@ -0,0 +1,268 @@ +extends SceneTree +## Pose/deformation review harness for the 3D model re-evaluation. +## +## The neutral baseline (`capture_npc_baseline.gd`) deliberately disables +## animation, so it cannot show shoulder/hip/neck collapse, leg contact or how +## coats and dresses behave while walking or seated. This harness stages the same +## eight characters against the same studio and drives the real pose code: +## +## idle rest pose, animation settled +## walk_contact mid-stride with the legs at maximum split +## walk_pass half a stride later, opposite leg forward +## sit seated theater pose (crouch, knees, hem) +## +## Two views per pose: a full-body frame for silhouette/proportion and a +## shoulder close-up for deformation. The avatar's own `_process` drives the real +## animation path, and the pose settles over the recorded tick count, so a run is +## deterministic enough for a before/after comparison. +## +## STATUS: BLOCKED (2026-11-03). Captured frames still show the rest pose even +## though `_animate()` reports the seated/walking pose (`rig_y=-0.339`, +## `leg0=-1.357`) and `Skeleton3D.get_bone_pose_rotation()` returns the posed +## quaternion at capture time. A near-identical standalone probe that calls the +## same `_animate()` sequence *does* render the crouch, and writing a single +## bone rotation directly does deform the mesh, so this is a staging/capture +## discrepancy, not a rig failure. Do not use these sheets as pose evidence. +## Repro and what was ruled out: `docs/3d-model-review.md` section +## "Pose evidence is blocked". +## +## Usage (windowed; `xvfb-run -a` works for automation): +## godot --path . --script res://src/test/capture_npc_pose.gd -- \ +## --out=res://tmp/npc-pose +## +## Output is local QA art. Build sheets with the shared tool: +## python3 tools/npc_baseline.py sheet --dir tmp/npc-pose \ +## --views idle,walk_contact,walk_pass,sit --out tmp/npc-pose/sheet_pose.jpg +## python3 tools/npc_baseline.py sheet --dir tmp/npc-pose \ +## --views idle_shoulder,walk_contact_shoulder,walk_pass_shoulder,sit_shoulder \ +## --out tmp/npc-pose/sheet_shoulder.jpg + +const Rules := preload("res://src/core/rules.gd") +const NpcAvatar := preload("res://src/game/npc_avatar.gd") +const NpcProfile := preload("res://src/core/npc_profile.gd") + +const VARIANTS := [ + {"name": "npc_casual", "type": Rules.NpcType.TOURIST}, + {"name": "npc_jacket", "type": Rules.NpcType.AUTHORIZED_GUEST}, + {"name": "npc_coat", "type": Rules.NpcType.INFLUENCER}, + {"name": "npc_vest", "type": Rules.NpcType.DELIVERY}, + {"name": "npc_dress", "type": Rules.NpcType.PERFORMER}, + {"name": "npc_hoodie", "type": Rules.NpcType.VIOLENT_THREAT}, + {"name": "npc_firefighter", "type": Rules.NpcType.RESPONDER}, + {"name": "npc_guard", "type": Rules.NpcType.CONFUSED_VISITOR}, +] +## Ticks of `_animate(1/60)` per pose. The walk phase advances by +## dt * (6 + speed * 1.7) ≈ 0.148 rad/tick at 1.7 m/s, so 30 ticks lands on a +## near-maximum leg split and 52 ticks on the opposite one. +const POSES := [ + {"name": "idle", "ticks": 40, "speed": 0.0, "sit": false}, + {"name": "walk_contact", "ticks": 30, "speed": 1.7, "sit": false}, + {"name": "walk_pass", "ticks": 52, "speed": 1.7, "sit": false}, + {"name": "sit", "ticks": 45, "speed": 0.0, "sit": true}, +] +const VIEWS := [ + {"name": "full", "yaw": 0.0, "distance": 4.2, "height": 0.95, + "target_y": 0.95, "fov": 38.0, "suffix": ""}, + {"name": "shoulder", "yaw": 35.0, "distance": 1.25, "height": 1.50, + "target_y": 1.42, "fov": 32.0, "suffix": "_shoulder"}, +] +const SETTLE_FRAMES := 8 +const MAX_FRAMES := 8000 + +var out_dir := "user://screenshots/npc_pose" +var only_pose := "" +var _poses: Array = POSES +var frames := 0 +var _phase := "stage" +var _variant_i := 0 +var _pose_i := 0 +var _view_i := 0 +var _wait := 0 +var _settle_ticks := 0 +var _avatar: NpcAvatar = null +var _camera: Camera3D = null +var _stage: Node3D = null + +func _initialize() -> void: + for arg in OS.get_cmdline_user_args(): + if str(arg).begins_with("--out="): + out_dir = str(arg).substr(6) + elif str(arg).begins_with("--pose="): + only_pose = str(arg).substr(7) + if only_pose != "": + _poses = POSES.filter(func(pose): return str(pose["name"]) == only_pose) + if _poses.is_empty(): + push_error("unknown pose %s" % only_pose) + root.size = Vector2i(1280, 720) + _build_stage() + +func _process(_delta: float) -> bool: + frames += 1 + if frames > MAX_FRAMES: + push_error("capture_npc_pose timed out") + quit(1) + return true + match _phase: + "clear": + if _avatar != null and is_instance_valid(_avatar): + _avatar.queue_free() + _avatar = null + _phase = "stage" + return false + "stage": + _stage_pose() + # The skinned mesh must be bound for a frame before pose writes stick. + _wait = 2 + _phase = "bind" + "bind": + _wait -= 1 + if _wait <= 0: + _apply_pose() + _wait = SETTLE_FRAMES + _phase = "settle" + "settle": + _wait -= 1 + if _wait <= 0: + _frame_view() + _wait = 1 + _phase = "frame" + "frame": + _wait -= 1 + if _wait <= 0: + _phase = "shoot" + "shoot": + _shoot() + _view_i += 1 + if _view_i >= VIEWS.size(): + _view_i = 0 + _pose_i += 1 + if _pose_i >= _poses.size(): + _pose_i = 0 + _variant_i += 1 + if _variant_i >= VARIANTS.size(): + print("capture_npc_pose done -> %s" % out_dir) + quit(0) + return true + _phase = "clear" + else: + _wait = 1 + _phase = "settle" + return false + +func _build_stage() -> void: + _stage = Node3D.new() + _stage.name = "NpcPoseStage" + root.add_child(_stage) + + var env := Environment.new() + env.background_mode = Environment.BG_COLOR + env.background_color = Color(0.56, 0.59, 0.63) + env.ambient_light_source = Environment.AMBIENT_SOURCE_COLOR + env.ambient_light_color = Color(0.72, 0.75, 0.80) + env.ambient_light_energy = 0.55 + env.tonemap_mode = Environment.TONE_MAPPER_ACES + var world_env := WorldEnvironment.new() + world_env.environment = env + _stage.add_child(world_env) + + var sun := DirectionalLight3D.new() + sun.name = "KeyLight" + sun.rotation_degrees = Vector3(-38.0, -32.0, 0.0) + sun.light_energy = 1.35 + sun.light_color = Color(1.0, 0.97, 0.92) + sun.shadow_enabled = true + _stage.add_child(sun) + var fill := DirectionalLight3D.new() + fill.name = "FillLight" + fill.rotation_degrees = Vector3(-12.0, 145.0, 0.0) + fill.light_energy = 0.45 + fill.light_color = Color(0.75, 0.82, 1.0) + _stage.add_child(fill) + + var ground := MeshInstance3D.new() + var plane := PlaneMesh.new() + plane.size = Vector2(24.0, 24.0) + ground.mesh = plane + var mat := StandardMaterial3D.new() + mat.albedo_color = Color(0.30, 0.32, 0.35) + mat.roughness = 0.92 + ground.material_override = mat + _stage.add_child(ground) + + _camera = Camera3D.new() + _camera.name = "PoseCamera" + _camera.current = true + _stage.add_child(_camera) + +func _stage_pose() -> void: + var variant: Dictionary = VARIANTS[_variant_i] + var pose: Dictionary = _poses[_pose_i] + var profile := NpcProfile.new() + profile.id = "pose_" + str(variant["name"]) + profile.display_name = str(variant["name"]).to_upper() + profile.npc_type = int(variant["type"]) + var avatar := NpcAvatar.new() + avatar.name = "PoseSubject%d" % (_variant_i * 10 + _pose_i) + avatar.character_override = str(variant["name"]) + avatar.setup(profile, "roam", 4242, Vector3.ZERO, Vector3(0, 0, 2), + Vector3(0, 0.5, 1.2), str(profile.id)) + _stage.add_child(avatar) + avatar.global_position = Vector3.ZERO + avatar.rotation.y = 0.0 + + for node in avatar.find_children("*", "Label3D", true, false): + (node as Label3D).visible = false + for node in avatar.find_children("FocusCircle", "MeshInstance3D", true, false): + (node as MeshInstance3D).visible = false + _avatar = avatar + # Drive the real pose code deterministically: the walk branch reads + # `velocity`, the theater branch needs `arrived` + the sit intent. + + + +## Drive the real pose code deterministically. `_animate` lives in +## `_physics_process`, which stays disabled so navigation cannot overwrite the +## velocity we set for the walk poses. +func _apply_pose() -> void: + if _avatar == null or not is_instance_valid(_avatar): + return + var pose: Dictionary = _poses[_pose_i] + _avatar.velocity = Vector3(0.0, 0.0, float(pose["speed"])) + _avatar.arrived = bool(pose["sit"]) + _avatar.npc.intent = "sit" if bool(pose["sit"]) else "roam" + for _tick in int(pose["ticks"]): + _avatar._animate(1.0 / 60.0) + print("pose %s/%s rig_y=%.3f leg0=%.3f" % [ + VARIANTS[_variant_i]["name"], pose["name"], _avatar._rig.position.y, + _avatar._legs[0].rotation.x]) + + +func _frame_view() -> void: + if _avatar == null or not is_instance_valid(_avatar): + return + var view: Dictionary = VIEWS[_view_i] + var yaw := deg_to_rad(float(view["yaw"])) + var distance := float(view["distance"]) + var target := Vector3(0.0, float(view["target_y"]), 0.0) + _camera.position = target + Vector3(sin(yaw) * distance, float(view["height"]) - target.y, + cos(yaw) * distance) + _camera.fov = float(view["fov"]) + _camera.look_at(target, Vector3.UP) + +func _shoot() -> void: + var variant: Dictionary = VARIANTS[_variant_i] + var pose: Dictionary = _poses[_pose_i] + var view: Dictionary = VIEWS[_view_i] + var dir := ProjectSettings.globalize_path(out_dir) + DirAccess.make_dir_recursive_absolute(dir) + var base := "%s__%s%s" % [variant["name"], pose["name"], view["suffix"]] + var path := dir.path_join(base + ".png") + var index := 2 + while FileAccess.file_exists(path): + path = dir.path_join("%s-v%d.png" % [base, index]) + index += 1 + var hip_bone: int = int(_avatar._bone_indices.get("HipL", -1)) + var knee_bone: int = int(_avatar._bone_indices.get("KneeL", -1)) + var img: Image = root.get_texture().get_image() + var err := img.save_png(path) + print("%s -> err=%d" % [base, err]) diff --git a/tools/npc_baseline.py b/tools/npc_baseline.py index aee17b0..eb39bb0 100644 --- a/tools/npc_baseline.py +++ b/tools/npc_baseline.py @@ -52,21 +52,24 @@ def next_free_path(path: Path) -> Path: index += 1 -def build_sheet(source: Path, output: Path) -> Path: +def build_sheet(source: Path, output: Path, views: list[str] | None = None) -> Path: + """Tile one capture per (variant, view). `views` defaults to the six baseline + views; the pose harness passes pose names instead (see the module docstring).""" from PIL import Image, ImageDraw + views = views or VIEWS cell_w, cell_h = 320, 180 label_h = 22 header_h = 28 - sheet_w = cell_w * len(VIEWS) + sheet_w = cell_w * len(views) sheet_h = header_h + (cell_h + label_h) * len(VARIANTS) sheet = Image.new("RGB", (sheet_w, sheet_h), (24, 26, 30)) draw = ImageDraw.Draw(sheet) - for column, view in enumerate(VIEWS): + for column, view in enumerate(views): draw.text((column * cell_w + 8, 8), view.replace("_", " ").title(), fill=(220, 224, 230)) for row, variant in enumerate(VARIANTS): - for column, view in enumerate(VIEWS): + for column, view in enumerate(views): path = latest_for(source, f"{variant}__{view}") if path is None: raise FileNotFoundError( @@ -132,6 +135,9 @@ def main(argv: list[str] | None = None) -> int: help="directory containing the capture_npc_baseline.gd PNGs") sheet.add_argument("--out", type=Path, default=None, help="output JPEG (default: /sheet.jpg; never overwritten)") + sheet.add_argument("--views", default="", + help="comma-separated view tokens instead of the six baseline views " + "(pose harness: idle,walk_contact,walk_pass,sit)") compare = sub.add_parser("compare") compare.add_argument("--before", required=True, type=Path) compare.add_argument("--after", required=True, type=Path) @@ -143,8 +149,9 @@ def main(argv: list[str] | None = None) -> int: if not output.is_absolute(): output = ROOT / output output = next_free_path(output) + views = [v.strip() for v in args.views.split(",") if v.strip()] or None try: - written = build_sheet(source, output) + written = build_sheet(source, output, views) except FileNotFoundError as exc: print(exc) return 2 diff --git a/tools/test_npc_baseline.py b/tools/test_npc_baseline.py index e81227f..481225f 100644 --- a/tools/test_npc_baseline.py +++ b/tools/test_npc_baseline.py @@ -14,6 +14,19 @@ class NpcBaselineTests(unittest.TestCase): self.assertEqual({row[1] for row in rows}, set(npc_baseline.VIEWS)) self.assertEqual(len({row[2] for row in rows}), 48) + def test_sheet_accepts_a_custom_view_list(self): + from PIL import Image + with tempfile.TemporaryDirectory() as tmp: + folder = Path(tmp) + for variant in npc_baseline.VARIANTS: + for view in ("idle", "sit"): + Image.new("RGB", (32, 18), (40, 60, 80)).save( + folder / f"{variant}__{view}.png") + out = npc_baseline.build_sheet(folder, folder / "sheet.jpg", ["idle", "sit"]) + with Image.open(out) as sheet: + # 2 views x 320 px cells. + self.assertEqual(sheet.width, 640) + def test_latest_capture_prefers_the_full_stem(self): with tempfile.TemporaryDirectory() as tmp: folder = Path(tmp)