Add a pose review harness and record the blocked pose capture
Full build / Import + headless tests (push) Successful in 43s
Full build / Full desktop build + optional release (push) Skipped

This commit is contained in:
zwitschi committed 2026-11-03 10:59:23 +01:00
1 parent 2d67e14e52
commit 7a5b902797
7 files changed
+358 -13

No files matched your search

+9
View File
@@ -4,6 +4,15 @@ Session memory for durable decisions, state, and gotchas. Keep concise, remove o
## Decisions
- **Pose capture is blocked (2026-11-03):** `src/test/capture_npc_pose.gd` stages idle/walk/sit
poses for the pose/weight review, but the saved frames show the rest pose while the same run
logs `rig_y=-0.339`, `leg0=-1.357` and a rotated `HipL` bone quaternion at capture time.
Verified meanwhile: one skinned mesh bound to one `Skeleton3D`; hiding it removes the body;
direct `set_bone_pose_rotation` on HipL/KneeL/Spine/ShoulderL/ElbowL/Head changes rendered
pixels; a bone *position* offset moves the head; a standalone probe running the same
`_animate()` sequence renders the crouch. A staged in-game `film` frame also showed the arms
down, so live venue animation may be affected too. Do not treat the pose sheets as evidence;
a scene test (posed frame differs from rest frame) is the next step.
- **3D review baseline (2026-11-03):** `docs/3d-model-review.md` records the first skill-guided
measurements: 46/46 assets clean, library 59,012 tris, characters 4,482–5,474 tris (68–69% of
the 8,000 caps), `npc_base` only 1,204/2,488, 24-NPC 6.94 ms, and authored texel density
+8
View File
@@ -60,6 +60,14 @@ except the rebuilt assets themselves.
finger-curl rig or the current open hand plus `HAND_ITEM_FORWARD`, and record the limit.
- [ ] Weights/pose: capture idle, walk, sit and wave; inspect shoulder, hip and neck collapse;
fix the worst offenders using `rigging` practice without changing the 13-bone contract.
**Blocked:** the new `capture_npc_pose.gd` reports the seated/walking pose while the saved
frames still show the rest pose; direct bone writes deform the mesh, so this is a staging/
capture discrepancy that may also affect live venue frames. See
`docs/3d-model-review.md` §"Pose evidence is blocked".
- [ ] Diagnose the pose-capture discrepancy: compare the harness staging with the working
standalone probe (SceneTree lifetime, node parents, first-frame skin bind, capture timing)
and add a scene test asserting a posed avatar renders differently from its rest frame.
Do not trust pose sheets or claim rig deformation until that test passes.
- [ ] UV/texel density: measure the 256² slot maps; decide whether atlas, UDIM or a resolution
change is justified. Do not change `CHAR_MAX_TRIS`/`CHAR_MAX_VERTS` without re-running the
budget smoke and the Web/Compatibility path.
+32 -8
View File
@@ -88,14 +88,15 @@ yet reviewed; this document is the place to fill it.
| Character | Silhouette | Face @1.85 m | Hands/grip | Weights/pose | Cloth/hair | LOD | Verdict |
| --- | --- | --- | --- | --- | --- | --- | --- |
| `npc_casual` | accept (0.9.4) | P1 (faceted planes) | accept (open palm) | — | accept | accept (two-quad impostor) | pending owner |
| `npc_jacket` | accept | P1 | accept | — | accept | accept | pending owner |
| `npc_coat` | accept | P1 | accept | — | accept (hem 0.76 stride) | accept | pending owner |
| `npc_vest` | accept | P1 | accept | — | accept | accept | pending owner |
| `npc_dress` | accept; performer props dominate | P1 | accept | — | accept | accept | pending owner |
| `npc_hoodie` | accept (hooded criterion) | accept (lower face only) | accept | — | accept | accept | pending owner |
| `npc_firefighter` | accept | P1 | accept | — | accept | accept | pending owner |
| `npc_guard` | accept | P1 | accept | — | accept | accept | pending owner |
| *(pose evidence blocked: see "Pose evidence is blocked" below)* | | | | | | | |
| `npc_casual` | accept (0.9.4) | P1 (faceted planes) | accept (open palm) | blocked | accept | accept (two-quad impostor) | pending owner |
| `npc_jacket` | accept | P1 | accept | blocked | accept | accept | pending owner |
| `npc_coat` | accept | P1 | accept | blocked | accept (hem 0.76 stride) | accept | pending owner |
| `npc_vest` | accept | P1 | accept | blocked | accept | accept | pending owner |
| `npc_dress` | accept; performer props dominate | P1 | accept | blocked | accept | accept | pending owner |
| `npc_hoodie` | accept (hooded criterion) | accept (lower face only) | accept | blocked | accept | accept | pending owner |
| `npc_firefighter` | accept | P1 | accept | blocked | accept | accept | pending owner |
| `npc_guard` | accept | P1 | accept | blocked | accept | accept | pending owner |
### Gear (18)
@@ -162,6 +163,29 @@ for presence, not re-reviewed for art quality.
| The Joke Shop | 22,680 tris / 454 meshes | shape-language review pending |
| TV Studio | — | shape-language review pending |
## Pose evidence is blocked (2026-11-03)
The weights/pose column cannot be filled yet. A pose harness was added
(`src/test/capture_npc_pose.gd`, staged in `tmp/model-pose*`), and its output does not match the
state it reports:
- `_animate()` applies the seated pose (logged `rig_y=-0.339`, `leg0=-1.357`) and
`Skeleton3D.get_bone_pose_rotation()` returns the rotated quaternion at capture time, yet the
saved frame shows the rest pose.
- Ruled out: missing skeleton (one `Skeleton3D`, mesh bound to it, `same=true`), hidden mesh
(hiding `npc_base` removes the body), pose not applied (proxies and bone poses verified),
and unbound skinning: writing a single bone rotation directly (`HipL`, `KneeL`, `Spine`,
`ShoulderL`, `ElbowL`, `Head`) changes the rendered pixels, and a bone *position* offset moves
the head. A standalone probe that calls the same `_animate()` sequence renders the crouch.
- Also observed: a staged in-game `film` pose frame showed the arms down, so this may affect the
live game rather than only the harness. That is why it is a P0/P1 candidate rather than a QA
tooling nit.
Next step is a focused diagnosis: compare the harness/venue staging with the standalone probe
(SceneTree lifetime, node parents, first-frame skin bind, capture timing) and add a scene test
that asserts a posed avatar renders differently from its rest frame. Do not mark the pose/weight
scorecard row until that test exists.
## Ranked candidate gaps
Ordered by expected player-visible gain per unit of work. None is approved for fixing yet; the
+16
View File
@@ -307,6 +307,22 @@ height/head/shoulder/hip/torso/leg/arm/foot guide; the current results are in
[`npc-proportion-audit.md`](npc-proportion-audit.md). It reads the exported GLBs
and does not require Blender.
`capture_npc_pose.gd` is the pose/deformation harness for the 3D re-evaluation: idle,
walk-contact, walk-pass and seated poses of all eight variants, full-body plus shoulder
close-ups. **It is currently blocked**: the applied pose is visible in the rig and bone values
but not in the captured frames, so do not use its sheets as pose evidence. The reproduction and
what was ruled out are in [`3d-model-review.md`](3d-model-review.md) §"Pose evidence is blocked".
The baseline sheet tool accepts a custom view list, which is also how pose sheets are tiled:
```bash
# baseline set (six views, eight variants)
python3 tools/npc_baseline.py sheet --dir tmp/npc-baseline
# pose set produced by capture_npc_pose.gd
python3 tools/npc_baseline.py sheet --dir tmp/npc-pose \
--views idle,walk_contact,walk_pass,sit --out tmp/npc-pose/sheet_pose.jpg
```
`capture_npc_baseline.gd` is the 0.9.4 neutral baseline: eight characters ×
front/three-quarter/profile/back/full-body/crowd = 48 frames with a fixed
camera, studio lighting and no venue/HUD/animation. The raw PNGs stay in
+268
View File
@@ -0,0 +1,268 @@
extends SceneTree
## Pose/deformation review harness for the 3D model re-evaluation.
##
## The neutral baseline (`capture_npc_baseline.gd`) deliberately disables
## animation, so it cannot show shoulder/hip/neck collapse, leg contact or how
## coats and dresses behave while walking or seated. This harness stages the same
## eight characters against the same studio and drives the real pose code:
##
## idle rest pose, animation settled
## walk_contact mid-stride with the legs at maximum split
## walk_pass half a stride later, opposite leg forward
## sit seated theater pose (crouch, knees, hem)
##
## Two views per pose: a full-body frame for silhouette/proportion and a
## shoulder close-up for deformation. The avatar's own `_process` drives the real
## animation path, and the pose settles over the recorded tick count, so a run is
## deterministic enough for a before/after comparison.
##
## STATUS: BLOCKED (2026-11-03). Captured frames still show the rest pose even
## though `_animate()` reports the seated/walking pose (`rig_y=-0.339`,
## `leg0=-1.357`) and `Skeleton3D.get_bone_pose_rotation()` returns the posed
## quaternion at capture time. A near-identical standalone probe that calls the
## same `_animate()` sequence *does* render the crouch, and writing a single
## bone rotation directly does deform the mesh, so this is a staging/capture
## discrepancy, not a rig failure. Do not use these sheets as pose evidence.
## Repro and what was ruled out: `docs/3d-model-review.md` section
## "Pose evidence is blocked".
##
## Usage (windowed; `xvfb-run -a` works for automation):
## godot --path . --script res://src/test/capture_npc_pose.gd -- \
## --out=res://tmp/npc-pose
##
## Output is local QA art. Build sheets with the shared tool:
## python3 tools/npc_baseline.py sheet --dir tmp/npc-pose \
## --views idle,walk_contact,walk_pass,sit --out tmp/npc-pose/sheet_pose.jpg
## python3 tools/npc_baseline.py sheet --dir tmp/npc-pose \
## --views idle_shoulder,walk_contact_shoulder,walk_pass_shoulder,sit_shoulder \
## --out tmp/npc-pose/sheet_shoulder.jpg
const Rules := preload("res://src/core/rules.gd")
const NpcAvatar := preload("res://src/game/npc_avatar.gd")
const NpcProfile := preload("res://src/core/npc_profile.gd")
const VARIANTS := [
{"name": "npc_casual", "type": Rules.NpcType.TOURIST},
{"name": "npc_jacket", "type": Rules.NpcType.AUTHORIZED_GUEST},
{"name": "npc_coat", "type": Rules.NpcType.INFLUENCER},
{"name": "npc_vest", "type": Rules.NpcType.DELIVERY},
{"name": "npc_dress", "type": Rules.NpcType.PERFORMER},
{"name": "npc_hoodie", "type": Rules.NpcType.VIOLENT_THREAT},
{"name": "npc_firefighter", "type": Rules.NpcType.RESPONDER},
{"name": "npc_guard", "type": Rules.NpcType.CONFUSED_VISITOR},
]
## Ticks of `_animate(1/60)` per pose. The walk phase advances by
## dt * (6 + speed * 1.7) ≈ 0.148 rad/tick at 1.7 m/s, so 30 ticks lands on a
## near-maximum leg split and 52 ticks on the opposite one.
const POSES := [
{"name": "idle", "ticks": 40, "speed": 0.0, "sit": false},
{"name": "walk_contact", "ticks": 30, "speed": 1.7, "sit": false},
{"name": "walk_pass", "ticks": 52, "speed": 1.7, "sit": false},
{"name": "sit", "ticks": 45, "speed": 0.0, "sit": true},
]
const VIEWS := [
{"name": "full", "yaw": 0.0, "distance": 4.2, "height": 0.95,
"target_y": 0.95, "fov": 38.0, "suffix": ""},
{"name": "shoulder", "yaw": 35.0, "distance": 1.25, "height": 1.50,
"target_y": 1.42, "fov": 32.0, "suffix": "_shoulder"},
]
const SETTLE_FRAMES := 8
const MAX_FRAMES := 8000
var out_dir := "user://screenshots/npc_pose"
var only_pose := ""
var _poses: Array = POSES
var frames := 0
var _phase := "stage"
var _variant_i := 0
var _pose_i := 0
var _view_i := 0
var _wait := 0
var _settle_ticks := 0
var _avatar: NpcAvatar = null
var _camera: Camera3D = null
var _stage: Node3D = null
func _initialize() -> void:
for arg in OS.get_cmdline_user_args():
if str(arg).begins_with("--out="):
out_dir = str(arg).substr(6)
elif str(arg).begins_with("--pose="):
only_pose = str(arg).substr(7)
if only_pose != "":
_poses = POSES.filter(func(pose): return str(pose["name"]) == only_pose)
if _poses.is_empty():
push_error("unknown pose %s" % only_pose)
root.size = Vector2i(1280, 720)
_build_stage()
func _process(_delta: float) -> bool:
frames += 1
if frames > MAX_FRAMES:
push_error("capture_npc_pose timed out")
quit(1)
return true
match _phase:
"clear":
if _avatar != null and is_instance_valid(_avatar):
_avatar.queue_free()
_avatar = null
_phase = "stage"
return false
"stage":
_stage_pose()
# The skinned mesh must be bound for a frame before pose writes stick.
_wait = 2
_phase = "bind"
"bind":
_wait -= 1
if _wait <= 0:
_apply_pose()
_wait = SETTLE_FRAMES
_phase = "settle"
"settle":
_wait -= 1
if _wait <= 0:
_frame_view()
_wait = 1
_phase = "frame"
"frame":
_wait -= 1
if _wait <= 0:
_phase = "shoot"
"shoot":
_shoot()
_view_i += 1
if _view_i >= VIEWS.size():
_view_i = 0
_pose_i += 1
if _pose_i >= _poses.size():
_pose_i = 0
_variant_i += 1
if _variant_i >= VARIANTS.size():
print("capture_npc_pose done -> %s" % out_dir)
quit(0)
return true
_phase = "clear"
else:
_wait = 1
_phase = "settle"
return false
func _build_stage() -> void:
_stage = Node3D.new()
_stage.name = "NpcPoseStage"
root.add_child(_stage)
var env := Environment.new()
env.background_mode = Environment.BG_COLOR
env.background_color = Color(0.56, 0.59, 0.63)
env.ambient_light_source = Environment.AMBIENT_SOURCE_COLOR
env.ambient_light_color = Color(0.72, 0.75, 0.80)
env.ambient_light_energy = 0.55
env.tonemap_mode = Environment.TONE_MAPPER_ACES
var world_env := WorldEnvironment.new()
world_env.environment = env
_stage.add_child(world_env)
var sun := DirectionalLight3D.new()
sun.name = "KeyLight"
sun.rotation_degrees = Vector3(-38.0, -32.0, 0.0)
sun.light_energy = 1.35
sun.light_color = Color(1.0, 0.97, 0.92)
sun.shadow_enabled = true
_stage.add_child(sun)
var fill := DirectionalLight3D.new()
fill.name = "FillLight"
fill.rotation_degrees = Vector3(-12.0, 145.0, 0.0)
fill.light_energy = 0.45
fill.light_color = Color(0.75, 0.82, 1.0)
_stage.add_child(fill)
var ground := MeshInstance3D.new()
var plane := PlaneMesh.new()
plane.size = Vector2(24.0, 24.0)
ground.mesh = plane
var mat := StandardMaterial3D.new()
mat.albedo_color = Color(0.30, 0.32, 0.35)
mat.roughness = 0.92
ground.material_override = mat
_stage.add_child(ground)
_camera = Camera3D.new()
_camera.name = "PoseCamera"
_camera.current = true
_stage.add_child(_camera)
func _stage_pose() -> void:
var variant: Dictionary = VARIANTS[_variant_i]
var pose: Dictionary = _poses[_pose_i]
var profile := NpcProfile.new()
profile.id = "pose_" + str(variant["name"])
profile.display_name = str(variant["name"]).to_upper()
profile.npc_type = int(variant["type"])
var avatar := NpcAvatar.new()
avatar.name = "PoseSubject%d" % (_variant_i * 10 + _pose_i)
avatar.character_override = str(variant["name"])
avatar.setup(profile, "roam", 4242, Vector3.ZERO, Vector3(0, 0, 2),
Vector3(0, 0.5, 1.2), str(profile.id))
_stage.add_child(avatar)
avatar.global_position = Vector3.ZERO
avatar.rotation.y = 0.0
for node in avatar.find_children("*", "Label3D", true, false):
(node as Label3D).visible = false
for node in avatar.find_children("FocusCircle", "MeshInstance3D", true, false):
(node as MeshInstance3D).visible = false
_avatar = avatar
# Drive the real pose code deterministically: the walk branch reads
# `velocity`, the theater branch needs `arrived` + the sit intent.
## Drive the real pose code deterministically. `_animate` lives in
## `_physics_process`, which stays disabled so navigation cannot overwrite the
## velocity we set for the walk poses.
func _apply_pose() -> void:
if _avatar == null or not is_instance_valid(_avatar):
return
var pose: Dictionary = _poses[_pose_i]
_avatar.velocity = Vector3(0.0, 0.0, float(pose["speed"]))
_avatar.arrived = bool(pose["sit"])
_avatar.npc.intent = "sit" if bool(pose["sit"]) else "roam"
for _tick in int(pose["ticks"]):
_avatar._animate(1.0 / 60.0)
print("pose %s/%s rig_y=%.3f leg0=%.3f" % [
VARIANTS[_variant_i]["name"], pose["name"], _avatar._rig.position.y,
_avatar._legs[0].rotation.x])
func _frame_view() -> void:
if _avatar == null or not is_instance_valid(_avatar):
return
var view: Dictionary = VIEWS[_view_i]
var yaw := deg_to_rad(float(view["yaw"]))
var distance := float(view["distance"])
var target := Vector3(0.0, float(view["target_y"]), 0.0)
_camera.position = target + Vector3(sin(yaw) * distance, float(view["height"]) - target.y,
cos(yaw) * distance)
_camera.fov = float(view["fov"])
_camera.look_at(target, Vector3.UP)
func _shoot() -> void:
var variant: Dictionary = VARIANTS[_variant_i]
var pose: Dictionary = _poses[_pose_i]
var view: Dictionary = VIEWS[_view_i]
var dir := ProjectSettings.globalize_path(out_dir)
DirAccess.make_dir_recursive_absolute(dir)
var base := "%s__%s%s" % [variant["name"], pose["name"], view["suffix"]]
var path := dir.path_join(base + ".png")
var index := 2
while FileAccess.file_exists(path):
path = dir.path_join("%s-v%d.png" % [base, index])
index += 1
var hip_bone: int = int(_avatar._bone_indices.get("HipL", -1))
var knee_bone: int = int(_avatar._bone_indices.get("KneeL", -1))
var img: Image = root.get_texture().get_image()
var err := img.save_png(path)
print("%s -> err=%d" % [base, err])
+12 -5
View File
@@ -52,21 +52,24 @@ def next_free_path(path: Path) -> Path:
index += 1
def build_sheet(source: Path, output: Path) -> Path:
def build_sheet(source: Path, output: Path, views: list[str] | None = None) -> Path:
"""Tile one capture per (variant, view). `views` defaults to the six baseline
views; the pose harness passes pose names instead (see the module docstring)."""
from PIL import Image, ImageDraw
views = views or VIEWS
cell_w, cell_h = 320, 180
label_h = 22
header_h = 28
sheet_w = cell_w * len(VIEWS)
sheet_w = cell_w * len(views)
sheet_h = header_h + (cell_h + label_h) * len(VARIANTS)
sheet = Image.new("RGB", (sheet_w, sheet_h), (24, 26, 30))
draw = ImageDraw.Draw(sheet)
for column, view in enumerate(VIEWS):
for column, view in enumerate(views):
draw.text((column * cell_w + 8, 8), view.replace("_", " ").title(),
fill=(220, 224, 230))
for row, variant in enumerate(VARIANTS):
for column, view in enumerate(VIEWS):
for column, view in enumerate(views):
path = latest_for(source, f"{variant}__{view}")
if path is None:
raise FileNotFoundError(
@@ -132,6 +135,9 @@ def main(argv: list[str] | None = None) -> int:
help="directory containing the capture_npc_baseline.gd PNGs")
sheet.add_argument("--out", type=Path, default=None,
help="output JPEG (default: <dir>/sheet.jpg; never overwritten)")
sheet.add_argument("--views", default="",
help="comma-separated view tokens instead of the six baseline views "
"(pose harness: idle,walk_contact,walk_pass,sit)")
compare = sub.add_parser("compare")
compare.add_argument("--before", required=True, type=Path)
compare.add_argument("--after", required=True, type=Path)
@@ -143,8 +149,9 @@ def main(argv: list[str] | None = None) -> int:
if not output.is_absolute():
output = ROOT / output
output = next_free_path(output)
views = [v.strip() for v in args.views.split(",") if v.strip()] or None
try:
written = build_sheet(source, output)
written = build_sheet(source, output, views)
except FileNotFoundError as exc:
print(exc)
return 2
+13
View File
@@ -14,6 +14,19 @@ class NpcBaselineTests(unittest.TestCase):
self.assertEqual({row[1] for row in rows}, set(npc_baseline.VIEWS))
self.assertEqual(len({row[2] for row in rows}), 48)
def test_sheet_accepts_a_custom_view_list(self):
from PIL import Image
with tempfile.TemporaryDirectory() as tmp:
folder = Path(tmp)
for variant in npc_baseline.VARIANTS:
for view in ("idle", "sit"):
Image.new("RGB", (32, 18), (40, 60, 80)).save(
folder / f"{variant}__{view}.png")
out = npc_baseline.build_sheet(folder, folder / "sheet.jpg", ["idle", "sit"])
with Image.open(out) as sheet:
# 2 views x 320 px cells.
self.assertEqual(sheet.width, 640)
def test_latest_capture_prefers_the_full_stem(self):
with tempfile.TemporaryDirectory() as tmp:
folder = Path(tmp)