feat(character): ComfyUI blueprints and a render-vs-blueprint evaluation

Two tools that turn character work into something measurable.

- Tools/comfyui_character_blueprint.py generates a four-view sheet (front, side, back,
  head) plus a contact sheet from one description, with fixed seeds and a JSON record
  of prompts and prompt ids. First sheet: 'GigLine walker', lean pedestrian in a navy
  tee, jeans and sneakers.
- Tools/character_match.py compares a rendered character against a sheet: figure
  aspect, fill and coverage deltas, mean CIE76 colour distance, dominant-palette
  distance, and a side-by-side montage. The deltas stay separate on purpose - a single
  similarity score between a photoreal sheet and a low-detail body would be a number
  with no meaning.
- comfyui_generate.py grew wait_for_image() so both tools share the polling path
  instead of duplicating it.
- 13 unit tests for the colour maths, masks and palette; 95 tool tests pass.

First real result, recorded rather than smoothed over: aspect delta 1.24, fill delta
0.12, mean deltaE 31.9, palette deltaE 25.4. The montage shows the reason - the model
is a nude body and the blueprint wears a tee and jeans, so clothing dominates every
metric. Figure segmentation is explicit (--render-box): auto-detection picked backdrop
blobs on every attempt, and a background plate is the proper fix.
This commit is contained in:
zwitschi committed 2026-11-09 13:36:28 +01:00
1 parent 1fc5835327
commit 85a8437cbc
56 files changed
+841 -170

No files matched your search

+5 -2
View File
@@ -15,12 +15,15 @@ Actionable task list for this project. This file is the single, central place fo
- [x] Restore the stride capture after the pose rewrite. The walk cycle started on a **sine**, zero at both the start and the middle of the cycle, so the deterministic capture kept landing on the two passing frames where the legs are together. Starting the cycle at maximum stride (cosine) fixed it with no other change; `demo-third-person-walk` is back as shot 8 and shows a genuine stride.
- [~] Tune skin shading and hair materials on the player character. **Tooling in place 2026-11-09, hair blocked on an engine-side limit:**
- [~] Tune skin shading and hair materials on the player character. **Blueprints and evaluation tooling in place 2026-11-09, hair blocked on an engine-side limit:**
- `Tools/comfyui_generate.py` drives the local ComfyUI instance (`127.0.0.1:8188`) for textures, writing a JSON sidecar with the model, prompt, seed and full workflow so anything generated can be regenerated or rejected. Installed models include `juggernautXL_ragnarok.safetensors` and `sd-v1-5-inpainting.ckpt`.
- **Stable Diffusion is not usable for flat skin swatches.** Three attempts on two models produced an anatomical close-up twice and a leather-scale surface once; SDXL has no reliable notion of "texture without anatomy". Skin detail is therefore generated procedurally instead (`Tools/generate_character_textures.py`: tileable octave noise for pores plus a strand-sheet texture for hair).
- **Hair is now its own asset** (`Tools/character_hair_build.py` → `SM_Char_Hair` → `/Game/Test/Character/CharacterHair`), which is what the `character-artist` skill asks for: cards are a separate mesh, not geometry joined into the body. It imports with its own material slot and is attached in C++ as a second `USkeletalMeshComponent` on `SetLeaderPoseComponent`, so it follows the head bone without a second animation. 321 verts, 192 quads, 404 tris.
- **The card layout now hugs the skull.** Each card is oriented by the scalp normal at its anchor — the length runs along the downward tangent, the width across it, and a small lift at the tip gives the cut volume. Three overlapping rows (7/9/10 cards, 26 total, 468 tris) replaced the single outward-pointing ring that read as a spiked crown. Overlap is what stops cards reading as separate spikes.
- **Skin still needs work:** it renders darker than the baked diffuse suggests — possibly because the crashed (or half-run) material tool rebuilt the graph — and the generated pore/strand textures are still unwired.
- **Skin still needs work.** `M_Character_Skin` (constant tone 0.62/0.42/0.33, roughness 0.58) is built and the pawn calls `SetMaterial(0, ...)`, but the body still renders neutral grey in review shots, so the assignment is not taking. The baked `T_Character_Diffuse` cannot be used either: sampling it renders the body black, so the MPFB mesh's UVs do not map onto that bake. Next: verify the slot index after `SetSkeletalMesh`, or assign in `BeginPlay` instead of the constructor.
- **Character blueprint and evaluation tools exist.** `Tools/comfyui_character_blueprint.py` generates a four-view sheet (front/side/back/head) from one description with fixed seeds; `Tools/character_match.py` compares a render against it and reports aspect, fill, coverage, mean CIE76 and palette distance plus a side-by-side montage. First result against `Screenshots/demo/review-demo-third-person-front.png`: **aspect delta 1.24, fill delta 0.12, mean ΔE 31.9, palette ΔE 25.4** — and the montage shows why: the model is a **nude body**, the blueprint is clothed (navy tee, jeans, sneakers).
- **Clothing is the largest visible gap** and is not in this patch's scope: the body carries no garments, so every colour and silhouette metric is dominated by the missing clothes rather than by skin or hair tuning.
- **Figure segmentation is explicit, not automatic.** Auto-detection failed on the review shots: the backdrop (sky band, pale wall, dark pavement) is larger and more saturated than the subject, so every size-based rule picked a backdrop blob. `--render-box` takes the figure box as fractions of the frame. A background plate rendered without the character would make this automatic.
- `Tools/ue_character_materials.py` (wiring the generated textures into the skin and hair materials) **segfaults in the headless commandlet** when it edits the material graph. Not wired; the hair keeps the dark base colour it carries from Blender. Needs either an editor-session run or a material asset built from scratch instead of edited in place.
- [ ] Tag `v0.2.2` and write `CHANGELOG.md` entry; then remove this section.