0.2.2 — hair as its own skinned mesh (cards, second component) #36

Merged
zwitschi merged 11 commits from hair-cards-0.2.2 into main 2026-11-09 14:26:22 +01:00
56 changed files with 841 additions and 170 deletions
Showing only changes of commit 85a8437cbc - Show all commits

No files matched your search

Binary file not shown.
+37
View File
@@ -0,0 +1,37 @@
{
"character": "GigLine walker",
"description": "lean adult pedestrian, short dark hair, olive skin, plain navy t-shirt, dark jeans, worn sneakers",
"model": "juggernautXL_ragnarok.safetensors",
"views": {
"front": {
"file": "Art/characters/blueprints/front.png",
"seed": 101,
"prompt": "GigLine walker, lean adult pedestrian, short dark hair, olive skin, plain navy t-shirt, dark jeans, worn sneakers, full body front view, neutral standing pose, arms at sides, even studio light, plain light grey background, character sheet",
"negative": "multiple people, crowd, text, watermark, signature, blurry, low quality, cropped limbs, extra limbs, deformed hands, cartoon, anime, logo",
"prompt_id": "ff81a0c1-e4d7-4740-9310-806c11600539"
},
"side": {
"file": "Art/characters/blueprints/side.png",
"seed": 202,
"prompt": "GigLine walker, lean adult pedestrian, short dark hair, olive skin, plain navy t-shirt, dark jeans, worn sneakers, full body side profile view, neutral standing pose, arms at sides, even studio light, plain light grey background, character sheet",
"negative": "multiple people, crowd, text, watermark, signature, blurry, low quality, cropped limbs, extra limbs, deformed hands, cartoon, anime, logo",
"prompt_id": "cdd0bfac-446b-4c5a-8375-37fb5c70a75e"
},
"back": {
"file": "Art/characters/blueprints/back.png",
"seed": 303,
"prompt": "GigLine walker, lean adult pedestrian, short dark hair, olive skin, plain navy t-shirt, dark jeans, worn sneakers, full body back view, neutral standing pose, arms at sides, even studio light, plain light grey background, character sheet",
"negative": "multiple people, crowd, text, watermark, signature, blurry, low quality, cropped limbs, extra limbs, deformed hands, cartoon, anime, logo",
"prompt_id": "069bc59d-93bb-41ce-a95b-f5b2e8194982"
},
"head": {
"file": "Art/characters/blueprints/head.png",
"seed": 404,
"prompt": "GigLine walker, lean adult pedestrian, short dark hair, olive skin, plain navy t-shirt, dark jeans, worn sneakers, head and shoulders portrait, front view, neutral expression, even studio light, plain light grey background",
"negative": "multiple people, crowd, text, watermark, signature, blurry, low quality, cropped limbs, extra limbs, deformed hands, cartoon, anime, logo",
"prompt_id": "39ab3ba6-5e6f-4a69-bdbb-b2faf4318dc0"
}
},
"contact_sheet": "Art/characters/blueprints/contact-sheet.png",
"note": "generated locally by Tools/comfyui_character_blueprint.py; record the model licence in THIRD_PARTY_NOTICES.md before shipping"
}
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -92,6 +92,17 @@ AGigLineCharacter::AGigLineCharacter()
// Hair: a separate component that copies the body's pose, so it follows the head
// bone without a second rig or a second animation. Hidden with the body.
// The skin material is assigned at runtime: a SkeletalMesh slot cannot be repointed
// from Python, so Tools/ue_character_materials.py builds M_Character_Skin and the
// pawn binds it here. Without this the imported FBX phong material is used and the
// generated pore detail never reaches the render.
static ConstructorHelpers::FObjectFinder<UMaterialInterface> SkinMaterialFinder(
TEXT("/Game/Test/Character/M_Character_Skin.M_Character_Skin"));
if (SkinMaterialFinder.Succeeded())
{
GetMesh()->SetMaterial(0, SkinMaterialFinder.Object);
}
HairMesh = CreateDefaultSubobject<USkeletalMeshComponent>(TEXT("HairMesh"));
HairMesh->SetupAttachment(GetCapsuleComponent());
HairMesh->SetCollisionEnabled(ECollisionEnabled::NoCollision);
+7
View File
@@ -498,3 +498,10 @@ Scope: facts and traps that cost time to establish. Planned work lives in [TODO.
- **Hair joined into the body mesh loses its material.** Blender shows two slots (hair faces on index 1) and the FBX exports both, but Unreal's skeletal-mesh import arrives with **one** slot named after the body material, so the hair renders as skin. `unreal.SkeletalMesh` has no `set_material`, so it cannot be re-slotted afterwards. A separate skeletal mesh imported on its own keeps its slot and imports cleanly (`CharacterHair` → `M_Character_Hair`).
- **A second skeletal component follows the body with `SetLeaderPoseComponent(GetMesh())`** — no second rig, no second animation, and it hides with the body in first person.
- **`unreal.MaterialEditingLibrary` segfaults in `-run=pythonscript`** when it clears and rebuilds a material graph. The tool ran its texture imports, then crashed the commandlet with no Python traceback. Graph editing needs an editor session, or a material created from scratch rather than edited in place.
## 2026-11-09 — Character blueprints and how to compare a render to one
- **A SDXL character sheet is reliable**: "full body front view, neutral pose, plain grey background, character sheet" gives a usable turnaround with fixed seeds. Four views (front/side/back/head) from one description is enough to evaluate against; the sheet and its JSON live in `Art/characters/blueprints/`.
- **Segmenting the character out of a review shot does not work by colour.** The sky band, the pale wall and the dark pavement are each larger and more saturated than the body, so every size-based rule picked a backdrop blob and the first report measured the whole frame. `Tools/character_match.py` now takes an explicit `--render-box`; the proper fix is a background plate rendered without the character.
- **The report has to separate the failures it measures.** Aspect, fill, coverage and two colour distances are reported apart on purpose: an absolute similarity number between a photoreal sheet and a low-detail body would mean nothing. The first real comparison (aspect Δ1.24, mean ΔE 31.9) was dominated by the model being **unclothed** while the blueprint wears a tee and jeans.
- **`SetMaterial(0, ...)` in the pawn constructor does not visibly land** on the body mesh: `M_Character_Skin` is built and assigned, yet the render stays neutral grey. Either the slot does not exist yet at that point or the assignment is dropped; `BeginPlay` is the next thing to try.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+5 -2
View File
@@ -15,12 +15,15 @@ Actionable task list for this project. This file is the single, central place fo
- [x] Restore the stride capture after the pose rewrite. The walk cycle started on a **sine**, zero at both the start and the middle of the cycle, so the deterministic capture kept landing on the two passing frames where the legs are together. Starting the cycle at maximum stride (cosine) fixed it with no other change; `demo-third-person-walk` is back as shot 8 and shows a genuine stride.
- [~] Tune skin shading and hair materials on the player character. **Tooling in place 2026-11-09, hair blocked on an engine-side limit:**
- [~] Tune skin shading and hair materials on the player character. **Blueprints and evaluation tooling in place 2026-11-09, hair blocked on an engine-side limit:**
- `Tools/comfyui_generate.py` drives the local ComfyUI instance (`127.0.0.1:8188`) for textures, writing a JSON sidecar with the model, prompt, seed and full workflow so anything generated can be regenerated or rejected. Installed models include `juggernautXL_ragnarok.safetensors` and `sd-v1-5-inpainting.ckpt`.
- **Stable Diffusion is not usable for flat skin swatches.** Three attempts on two models produced an anatomical close-up twice and a leather-scale surface once; SDXL has no reliable notion of "texture without anatomy". Skin detail is therefore generated procedurally instead (`Tools/generate_character_textures.py`: tileable octave noise for pores plus a strand-sheet texture for hair).
- **Hair is now its own asset** (`Tools/character_hair_build.py` → `SM_Char_Hair` → `/Game/Test/Character/CharacterHair`), which is what the `character-artist` skill asks for: cards are a separate mesh, not geometry joined into the body. It imports with its own material slot and is attached in C++ as a second `USkeletalMeshComponent` on `SetLeaderPoseComponent`, so it follows the head bone without a second animation. 321 verts, 192 quads, 404 tris.
- **The card layout now hugs the skull.** Each card is oriented by the scalp normal at its anchor — the length runs along the downward tangent, the width across it, and a small lift at the tip gives the cut volume. Three overlapping rows (7/9/10 cards, 26 total, 468 tris) replaced the single outward-pointing ring that read as a spiked crown. Overlap is what stops cards reading as separate spikes.
- **Skin still needs work:** it renders darker than the baked diffuse suggests — possibly because the crashed (or half-run) material tool rebuilt the graph — and the generated pore/strand textures are still unwired.
- **Skin still needs work.** `M_Character_Skin` (constant tone 0.62/0.42/0.33, roughness 0.58) is built and the pawn calls `SetMaterial(0, ...)`, but the body still renders neutral grey in review shots, so the assignment is not taking. The baked `T_Character_Diffuse` cannot be used either: sampling it renders the body black, so the MPFB mesh's UVs do not map onto that bake. Next: verify the slot index after `SetSkeletalMesh`, or assign in `BeginPlay` instead of the constructor.
- **Character blueprint and evaluation tools exist.** `Tools/comfyui_character_blueprint.py` generates a four-view sheet (front/side/back/head) from one description with fixed seeds; `Tools/character_match.py` compares a render against it and reports aspect, fill, coverage, mean CIE76 and palette distance plus a side-by-side montage. First result against `Screenshots/demo/review-demo-third-person-front.png`: **aspect delta 1.24, fill delta 0.12, mean ΔE 31.9, palette ΔE 25.4** — and the montage shows why: the model is a **nude body**, the blueprint is clothed (navy tee, jeans, sneakers).
- **Clothing is the largest visible gap** and is not in this patch's scope: the body carries no garments, so every colour and silhouette metric is dominated by the missing clothes rather than by skin or hair tuning.
- **Figure segmentation is explicit, not automatic.** Auto-detection failed on the review shots: the backdrop (sky band, pale wall, dark pavement) is larger and more saturated than the subject, so every size-based rule picked a backdrop blob. `--render-box` takes the figure box as fractions of the frame. A background plate rendered without the character would make this automatic.
- `Tools/ue_character_materials.py` (wiring the generated textures into the skin and hair materials) **segfaults in the headless commandlet** when it edits the material graph. Not wired; the hair keeps the dark base colour it carries from Blender. Needs either an editor-session run or a material asset built from scratch instead of edited in place.
- [ ] Tag `v0.2.2` and write `CHANGELOG.md` entry; then remove this section.
+4
View File
@@ -90,6 +90,10 @@ Geographic tools live in [`Geo/tools/`](../Geo/README.md) and the Blender-side b
| --- | --- |
| `input_emulation.py` | Pure-stdlib `ctypes`/`fcntl` driver that creates a virtual keyboard+mouse on `/dev/uinput`. The desktop treats it as real hardware, so it proves whether OS input actually reaches the game. Needs `/dev/uinput` writable (ACL on this host). |
| `input_test.py` | Injects a scripted sequence into the focused window: `--sequence move:2,look:3,jump:1,reset:1,pause:1,click`. |
| `comfyui_generate.py` | Local ComfyUI driver: list checkpoints, run txt2img through the HTTP API, and write a provenance sidecar (model, prompt, seed, full graph). Local, human-triggered. | `models`; `txt2img --prompt ... --out <png>` | `python3 Tools/comfyui_generate.py models` |
| `comfyui_character_blueprint.py` | Generates a character blueprint sheet - front, side, back and head views plus a contact sheet - from one description, with fixed seeds and a JSON record. | `--desc`, `--out`, `--model`, `--size` | `python3 Tools/comfyui_character_blueprint.py --desc "lean pedestrian, navy tee, jeans" --out Art/characters/blueprints` |
| `character_match.py` | Compares a rendered character against a blueprint: figure aspect, fill and coverage deltas, mean CIE76 colour distance, dominant-palette distance, and a side-by-side montage. | `--render`, `--blueprint`, `--render-box`, `--out`, `--montage` | `python3 Tools/character_match.py --render Screenshots/demo/review-demo-third-person-front.png --blueprint Art/characters/blueprints/front.png --render-box 0.42,0.46,0.56,0.97 --montage /tmp/gigline/match.png` |
| `generate_character_textures.py` | Procedural skin-pore and hair-strand textures with sidecars; no model involved. | `--out` | `python3 Tools/generate_character_textures.py --out Art/textures/generated` |
## Tests
+368
View File
@@ -0,0 +1,368 @@
#!/usr/bin/env python3
"""Evaluate a rendered character against a generated blueprint.
The blueprint comes from `Tools/comfyui_character_blueprint.py`; the render comes from
the review pipeline. This tool isolates the figure in both, then reports how far the
model has drifted from the intended character:
python3 Tools/character_match.py \
--render Screenshots/demo/review-demo-third-person-front.png \
--blueprint Art/characters/blueprints/front.png \
--out /tmp/gigline/character-match.json \
--montage /tmp/gigline/character-match.png
Reported per pair:
aspect_delta |height/width difference| between the two figures, in ratio units
fill_delta |figure area / bbox area| - bulk, catches a thin or bloated build
coverage_delta |figure area / frame area| - how much of the frame the figure takes
mean_delta_e mean CIE76 colour distance over the figure
palette_delta_e mean distance between the five dominant colours of each figure
Why these and not a similarity score: an absolute "match percentage" between a
photoreal generated sheet and a low-detail game body would be a number with no
meaning. The deltas are separable, so a report says *what* drifted - build, framing
or colour - instead of blending all three into one figure.
The figure mask is a heuristic and the tool says so in its report: for the blueprint
it is "differs from the corner colour", for the render it is "more saturated or darker
than the frame median", largest connected component. Pair the tool with a human look
at the montage; it narrows where to look, it does not replace the review.
"""
from __future__ import annotations
import argparse
import json
import math
import os
import sys
from PIL import Image
# ---------------------------------------------------------------- colour maths
def srgb_to_linear(value: float) -> float:
value /= 255.0
return value / 12.92 if value <= 0.04045 else ((value + 0.055) / 1.055) ** 2.4
def rgb_to_lab(rgb) -> tuple[float, float, float]:
"""sRGB (0-255) to CIELAB under D65."""
r, g, b = (srgb_to_linear(v) for v in rgb[:3])
x = r * 0.4124 + g * 0.3576 + b * 0.1805
y = r * 0.2126 + g * 0.7152 + b * 0.0722
z = r * 0.0193 + g * 0.1192 + b * 0.9505
# D65 white point
xn, yn, zn = 0.95047, 1.0, 1.08883
def f(t: float) -> float:
return t ** (1.0 / 3.0) if t > 0.008856 else (7.787 * t) + (16.0 / 116.0)
fx, fy, fz = f(x / xn), f(y / yn), f(z / zn)
return (116.0 * fy - 16.0, 500.0 * (fx - fy), 200.0 * (fy - fz))
def delta_e76(lab_a, lab_b) -> float:
return math.sqrt(sum((a - b) ** 2 for a, b in zip(lab_a, lab_b)))
# ------------------------------------------------------------------- masking
def luminance(rgb) -> float:
return 0.2126 * rgb[0] + 0.7152 * rgb[1] + 0.0722 * rgb[2]
def _subject_mask(image: Image.Image, saturation_threshold: int = 20,
dark_ratio: float = 0.62) -> Image.Image:
"""Mask the figure as "more saturated than the backdrop, or notably darker".
Both inputs have a desaturated backdrop - grey studio sweep in the blueprint,
beige pavement and pale walls in the render - and a figure that carries colour
(skin, denim, hair). Saturation separates them without needing a clean plate.
The previous version masked the blueprint by "differs from the corner colour",
which swallowed a gradient backdrop whole.
"""
rgb = image.convert("RGB")
pixels = rgb.load()
lums = [luminance(pixels[x, y])
for y in range(0, rgb.height, 4) for x in range(0, rgb.width, 4)]
lums.sort()
median = lums[len(lums) // 2] if lums else 128.0
mask = Image.new("L", rgb.size, 0)
out = mask.load()
for y in range(rgb.height):
for x in range(rgb.width):
r, g, b = pixels[x, y]
saturation = max(r, g, b) - min(r, g, b)
if saturation > saturation_threshold or luminance((r, g, b)) < median * dark_ratio:
out[x, y] = 255
return mask
def figure_mask_from_background(image: Image.Image, tolerance: int = 28):
"""Figure mask for a blueprint sheet: a coloured subject on a grey sweep."""
return _largest_component(_subject_mask(image, saturation_threshold=18,
dark_ratio=0.55))
def figure_mask_from_frame(image: Image.Image):
"""Figure mask for a rendered frame: the body against pavement and stucco."""
return _largest_component(_subject_mask(image, saturation_threshold=26,
dark_ratio=0.72))
def _largest_component(mask: Image.Image) -> Image.Image:
"""Keep the largest plausible figure blob.
"Largest" alone is wrong twice over: in both inputs the backdrop is one huge
connected region, so a plain largest-blob pick returned the background and the
first report measured the whole frame. The filter is size-based instead - a figure
occupies between 0.2 % and 55 % of the frame - which rejects the backdrop on one
side and road markings on the other.
"""
width, height = mask.size
frame_area = width * height
source = mask.load()
seen = bytearray(width * height)
best = [] # largest blob fully inside the frame
usable = [] # largest plausible blob, border or not
fallback = [] # largest blob at all, used when nothing is plausible
best_score = 0
for start_y in range(0, height, 3):
for start_x in range(0, width, 3):
index = start_y * width + start_x
if seen[index] or source[start_x, start_y] == 0:
continue
stack = [(start_x, start_y)]
blob = []
seen[index] = 1
while stack:
x, y = stack.pop()
blob.append((x, y))
for nx, ny in ((x + 1, y), (x - 1, y), (x, y + 1), (x, y - 1)):
if 0 <= nx < width and 0 <= ny < height:
nindex = ny * width + nx
if not seen[nindex] and source[nx, ny] != 0:
seen[nindex] = 1
stack.append((nx, ny))
if len(blob) > len(fallback):
fallback = blob
share = len(blob) / frame_area
if not (0.002 < share < 0.55):
continue
# Blobs glued to the frame edge are backdrop: the sky band in a render and
# the studio sweep in a sheet both touch the border and both beat the
# figure on raw area. Prefer fully enclosed blobs, fall back by area.
xs = [p[0] for p in blob]
ys = [p[1] for p in blob]
enclosed = (min(xs) > 1 and min(ys) > 1
and max(xs) < width - 2 and max(ys) < height - 2)
if enclosed and len(blob) > best_score:
best_score = len(blob)
best = blob
elif len(blob) > len(usable):
usable = blob
keep = Image.new("L", mask.size, 0)
out = keep.load()
for x, y in (best or usable or fallback):
out[x, y] = 255
return keep
def bbox(mask: Image.Image):
return mask.getbbox()
def mask_stats(mask: Image.Image) -> dict:
box = mask.getbbox()
if not box:
return {"area": 0, "bbox": None, "aspect": None, "fill": None, "coverage": None}
x0, y0, x1, y1 = box
histogram = mask.histogram()
area = sum(histogram[1:])
box_area = max(1, (x1 - x0) * (y1 - y0))
return {
"area": area,
"bbox": [x0, y0, x1, y1],
"aspect": (y1 - y0) / max(1, (x1 - x0)),
"fill": area / box_area,
"coverage": area / (mask.width * mask.height),
}
def masked_colours(image: Image.Image, mask: Image.Image, limit=20000):
rgb = image.convert("RGB")
pixels = rgb.load()
values = mask.load()
collected = []
step = max(1, (rgb.width * rgb.height) // limit)
index = 0
for y in range(rgb.height):
for x in range(rgb.width):
index += 1
if index % step:
continue
if values[x, y]:
collected.append(pixels[x, y])
return collected
def dominant(counts: dict, count: int = 5):
"""Coarse palette: quantise to 32 levels per channel, then take the top bins."""
bins: dict[tuple, int] = {}
for colour in counts:
key = (colour[0] // 32, colour[1] // 32, colour[2] // 32)
bins[key] = bins.get(key, 0) + 1
ranked = sorted(bins.items(), key=lambda item: -item[1])[:count]
return [((k[0] * 32) + 16, (k[1] * 32) + 16, (k[2] * 32) + 16) for k, _ in ranked]
def palette_delta_e(palette_a, palette_b) -> float:
if not palette_a or not palette_b:
return float("nan")
labs_b = [rgb_to_lab(c) for c in palette_b]
total = 0.0
for colour in palette_a:
lab = rgb_to_lab(colour)
total += min(delta_e76(lab, other) for other in labs_b)
return total / len(palette_a)
# -------------------------------------------------------------------- report
def evaluate(render_path: str, blueprint_path: str, montage_path: str | None = None,
render_box: tuple[float, float, float, float] | None = None) -> dict:
render_full = Image.open(render_path).convert("RGB")
blueprint = Image.open(blueprint_path).convert("RGB")
# The figure box is explicit by default. Auto-segmentation on these shots failed
# in a way worth recording rather than papering over: the backdrop (sky band, pale
# wall, dark pavement) is more saturated and far larger than the subject, so every
# size-based rule picked a backdrop blob. A box read off the shot is honest and
# cheap; a background plate rendered without the character would be the proper fix.
width, height = render_full.size
box = render_box or (0.15, 0.02, 0.85, 0.99)
region = (int(width * box[0]), int(height * box[1]),
int(width * box[2]), int(height * box[3]))
render = render_full.crop(region)
render_mask = figure_mask_from_frame(render)
blueprint_mask = figure_mask_from_background(blueprint)
render_stats = mask_stats(render_mask)
blueprint_stats = mask_stats(blueprint_mask)
render_colours = masked_colours(render, render_mask)
blueprint_colours = masked_colours(blueprint, blueprint_mask)
def mean_lab(colours):
labs = [rgb_to_lab(c) for c in colours]
if not labs:
return None
return tuple(sum(l[i] for l in labs) / len(labs) for i in range(3))
lab_render = mean_lab(render_colours)
lab_blueprint = mean_lab(blueprint_colours)
mean_de = (delta_e76(lab_render, lab_blueprint)
if lab_render and lab_blueprint else float("nan"))
palette_render = dominant(render_colours)
palette_blueprint = dominant(blueprint_colours)
def delta(a, b):
if a is None or b is None:
return None
return abs(a - b)
report = {
"render": os.path.abspath(render_path),
"blueprint": os.path.abspath(blueprint_path),
"render_mask": "saturated-or-dark vs frame median, largest component",
"blueprint_mask": "differs from corner colour",
"render_region": list(region),
"render_figure": render_stats,
"blueprint_figure": blueprint_stats,
"aspect_delta": delta(render_stats["aspect"], blueprint_stats["aspect"]),
"fill_delta": delta(render_stats["fill"], blueprint_stats["fill"]),
"coverage_delta": delta(render_stats["coverage"], blueprint_stats["coverage"]),
"mean_lab_render": lab_render,
"mean_lab_blueprint": lab_blueprint,
"mean_delta_e": mean_de,
"palette_render": palette_render,
"palette_blueprint": palette_blueprint,
"palette_delta_e": palette_delta_e(palette_render, palette_blueprint),
}
if montage_path:
report["montage"] = write_montage(render, render_mask, blueprint, blueprint_mask,
montage_path)
return report
def write_montage(render, render_mask, blueprint, blueprint_mask, path, height=640):
"""Render crop and blueprint crop side by side, same height, for a human look."""
def crop(image, mask):
box = mask.getbbox()
return image.crop(box) if box else image
left = crop(render, render_mask)
right = crop(blueprint, blueprint_mask)
left = left.resize((max(1, int(left.width * height / max(1, left.height))), height))
right = right.resize((max(1, int(right.width * height / max(1, right.height))), height))
sheet = Image.new("RGB", (left.width + right.width + 12, height), (20, 20, 20))
sheet.paste(left, (0, 0))
sheet.paste(right, (left.width + 12, 0))
os.makedirs(os.path.dirname(os.path.abspath(path)) or ".", exist_ok=True)
sheet.save(path)
return os.path.abspath(path)
def main(argv=None):
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--render", required=True)
ap.add_argument("--blueprint", required=True)
ap.add_argument("--out", default="/tmp/gigline/character-match.json")
ap.add_argument("--montage", default=None)
ap.add_argument("--render-box", default=None,
help="figure box as x0,y0,x1,y1 fractions of the frame (default: central 70 percent)")
args = ap.parse_args(argv)
for path in (args.render, args.blueprint):
if not os.path.exists(path):
print(f"error: no such image: {path}", file=sys.stderr)
return 2
box = None
if args.render_box:
box = tuple(float(v) for v in args.render_box.split(","))
if len(box) != 4:
print("error: --render-box needs four comma-separated fractions", file=sys.stderr)
return 2
report = evaluate(args.render, args.blueprint, args.montage, box)
os.makedirs(os.path.dirname(os.path.abspath(args.out)) or ".", exist_ok=True)
with open(args.out, "w") as handle:
json.dump(report, handle, indent=2)
def show(value, digits=3):
return "n/a" if value is None or (isinstance(value, float) and math.isnan(value)) \
else f"{value:.{digits}f}"
print(f"render {report['render_figure']}")
print(f"blueprint {report['blueprint_figure']}")
print(f"aspect delta {show(report['aspect_delta'])}")
print(f"fill delta {show(report['fill_delta'])}")
print(f"coverage delta {show(report['coverage_delta'])}")
print(f"mean deltaE {show(report['mean_delta_e'], 1)}")
print(f"palette deltaE {show(report['palette_delta_e'], 1)}")
print(f"report -> {args.out}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+118
View File
@@ -0,0 +1,118 @@
#!/usr/bin/env python3
"""Generate character blueprints with the local ComfyUI instance.
A "blueprint" here is an art-direction sheet for one character: a front, side and back
view, plus a head detail, generated from a fixed prompt and seed set so the sheet can
be regenerated and compared against later. It is the reference the modelled character
is evaluated against — see `Tools/character_match.py`.
python3 Tools/comfyui_character_blueprint.py \
--character "GigLine walker" \
--desc "lean adult pedestrian, short dark hair, olive skin, plain navy t-shirt, dark jeans, worn sneakers" \
--out Art/characters/blueprints
Writes one PNG per view plus `contact-sheet.png` and `blueprint.json`, which records
the prompt, model, seeds and the ComfyUI prompt ids. Everything is generated locally
by `Tools/comfyui_generate.py`; no network service and no credential is involved.
The sheet is generated art, not photography, and it stays out of the reference photo
folders on purpose: AGENTS.md forbids committing reference photography, and a
generated concept should not blur that line.
"""
from __future__ import annotations
import argparse
import json
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import comfyui_generate as comfy # noqa: E402
from PIL import Image # noqa: E402
VIEWS = (
# name, prompt suffix, seed
("front", "full body front view, neutral standing pose, arms at sides, "
"even studio light, plain light grey background, character sheet", 101),
("side", "full body side profile view, neutral standing pose, arms at sides, "
"even studio light, plain light grey background, character sheet", 202),
("back", "full body back view, neutral standing pose, arms at sides, "
"even studio light, plain light grey background, character sheet", 303),
("head", "head and shoulders portrait, front view, neutral expression, "
"even studio light, plain light grey background", 404),
)
NEGATIVE = ("multiple people, crowd, text, watermark, signature, blurry, low quality, "
"cropped limbs, extra limbs, deformed hands, cartoon, anime, logo")
def contact_sheet(paths, out_path, columns=4):
images = [Image.open(p).convert("RGB") for p in paths if os.path.exists(p)]
if not images:
return None
width = max(i.width for i in images)
height = max(i.height for i in images)
sheet = Image.new("RGB", (width * columns, height), (24, 24, 24))
for index, image in enumerate(images[:columns]):
sheet.paste(image.resize((width, height)), (index * width, 0))
sheet.save(out_path)
return out_path
def main(argv=None):
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--character", default="GigLine walker")
ap.add_argument("--desc", required=True,
help="what the character wears and looks like; drives every view")
ap.add_argument("--out", default="Art/characters/blueprints")
ap.add_argument("--model", default=comfy.DEFAULT_MODEL)
ap.add_argument("--size", default="832x1216")
ap.add_argument("--steps", type=int, default=30)
ap.add_argument("--cfg", type=float, default=5.0)
args = ap.parse_args(argv)
if not os.path.isdir(comfy.HOST) and "://" not in comfy.HOST:
print(f"error: bad COMFYUI_HOST {comfy.HOST}", file=sys.stderr)
return 2
os.makedirs(args.out, exist_ok=True)
width, height = (int(v) for v in args.size.lower().split("x"))
record = {"character": args.character, "description": args.desc,
"model": args.model, "views": {}}
written = []
for name, suffix, seed in VIEWS:
prompt = f"{args.character}, {args.desc}, {suffix}"
out = os.path.join(args.out, f"{name}.png")
graph = comfy.workflow(prompt, NEGATIVE, args.model, width, height, seed,
args.steps, args.cfg, "dpmpp_2m", "karras")
queued = comfy.post("/prompt", {"prompt": graph})
prompt_id = queued.get("prompt_id")
if not prompt_id:
print(f"error: ComfyUI rejected the {name} prompt: {queued}", file=sys.stderr)
return 1
print(f"{name}: queued {prompt_id} (seed {seed})")
image = comfy.wait_for_image(prompt_id, timeout=600.0)
if image is None:
print(f"error: no image for {name}", file=sys.stderr)
return 1
with open(out, "wb") as handle:
handle.write(image)
record["views"][name] = {"file": out, "seed": seed, "prompt": prompt,
"negative": NEGATIVE, "prompt_id": prompt_id}
written.append(out)
sheet = contact_sheet(written, os.path.join(args.out, "contact-sheet.png"))
record["contact_sheet"] = sheet
record["note"] = ("generated locally by Tools/comfyui_character_blueprint.py; "
"record the model licence in THIRD_PARTY_NOTICES.md before shipping")
with open(os.path.join(args.out, "blueprint.json"), "w") as handle:
json.dump(record, handle, indent=2)
print(f"wrote {len(written)} views + contact sheet -> {args.out}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+53 -45
View File
@@ -89,6 +89,29 @@ def workflow(prompt: str, negative: str, model: str, width: int, height: int,
}
def wait_for_image(prompt_id: str, timeout: float = 300.0, poll: float = 1.5):
"""Poll /history until the job has an image, then fetch its bytes."""
deadline = time.time() + timeout
while time.time() < deadline:
history = api(f"/history/{prompt_id}")
entry = history.get(prompt_id)
if entry and entry.get("outputs"):
images = []
for node in entry["outputs"].values():
images.extend(node.get("images", []))
if not images:
return None
image = images[0]
query = urllib.parse.urlencode({
"filename": image["filename"],
"subfolder": image.get("subfolder", ""),
"type": image.get("type", "output")})
with urllib.request.urlopen(f"{HOST}/view?{query}", timeout=60) as response:
return response.read()
time.sleep(poll)
return None
def generate(args) -> int:
width, height = (int(v) for v in args.size.lower().split("x"))
seed = args.seed if args.seed is not None else random.randrange(0, 2**31)
@@ -101,51 +124,36 @@ def generate(args) -> int:
return 1
print(f"queued {prompt_id} ({args.model}, {width}x{height}, seed {seed})")
deadline = time.time() + args.timeout
while time.time() < deadline:
history = api(f"/history/{prompt_id}")
entry = history.get(prompt_id)
if entry and entry.get("outputs"):
images = []
for node in entry["outputs"].values():
images.extend(node.get("images", []))
if not images:
print("error: job finished with no image", file=sys.stderr)
return 1
image = images[0]
query = urllib.parse.urlencode({
"filename": image["filename"],
"subfolder": image.get("subfolder", ""),
"type": image.get("type", "output")})
with urllib.request.urlopen(f"{HOST}/view?{query}", timeout=60) as response:
data = response.read()
os.makedirs(os.path.dirname(os.path.abspath(args.out)), exist_ok=True)
with open(args.out, "wb") as handle:
handle.write(data)
sidecar = os.path.splitext(args.out)[0] + ".json"
with open(sidecar, "w") as handle:
json.dump({
"tool": "Tools/comfyui_generate.py",
"generated_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"prompt": args.prompt,
"negative": args.negative,
"model": args.model,
"size": [width, height],
"seed": seed,
"steps": args.steps,
"cfg": args.cfg,
"sampler": args.sampler,
"scheduler": args.scheduler,
"workflow": graph,
"comfyui_prompt_id": prompt_id,
"note": "generated locally; record the model licence in "
"THIRD_PARTY_NOTICES.md before shipping",
}, handle, indent=2)
print(f"wrote {args.out} (+ sidecar)")
return 0
time.sleep(args.poll)
print("error: timed out waiting for the image", file=sys.stderr)
return 1
data = wait_for_image(prompt_id, timeout=args.timeout, poll=args.poll)
if data is None:
print("error: no image came back (timeout or empty job)", file=sys.stderr)
return 1
os.makedirs(os.path.dirname(os.path.abspath(args.out)) or ".", exist_ok=True)
with open(args.out, "wb") as handle:
handle.write(data)
sidecar = os.path.splitext(args.out)[0] + ".json"
with open(sidecar, "w") as handle:
json.dump({
"tool": "Tools/comfyui_generate.py",
"generated_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"prompt": args.prompt,
"negative": args.negative,
"model": args.model,
"size": [width, height],
"seed": seed,
"steps": args.steps,
"cfg": args.cfg,
"sampler": args.sampler,
"scheduler": args.scheduler,
"workflow": graph,
"comfyui_prompt_id": prompt_id,
"note": "generated locally; record the model licence in "
"THIRD_PARTY_NOTICES.md before shipping",
}, handle, indent=2)
print(f"wrote {args.out} (+ sidecar)")
return 0
def main(argv=None) -> int:
+82
View File
@@ -0,0 +1,82 @@
"""Tests for the pure parts of Tools/character_match.py."""
import os
import sys
import unittest
from PIL import Image
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import character_match as cm # noqa: E402
class ColourTests(unittest.TestCase):
def test_white_is_light_and_neutral(self):
light, a, b = cm.rgb_to_lab((255, 255, 255))
self.assertAlmostEqual(light, 100.0, delta=0.2)
self.assertAlmostEqual(a, 0.0, delta=0.2)
self.assertAlmostEqual(b, 0.0, delta=0.2)
def test_black_is_dark(self):
self.assertAlmostEqual(cm.rgb_to_lab((0, 0, 0))[0], 0.0, delta=0.1)
def test_identical_colours_have_no_distance(self):
self.assertAlmostEqual(cm.delta_e76(cm.rgb_to_lab((120, 90, 70)),
cm.rgb_to_lab((120, 90, 70))), 0.0, delta=1e-9)
def test_red_green_are_far_apart(self):
distance = cm.delta_e76(cm.rgb_to_lab((200, 40, 40)), cm.rgb_to_lab((40, 160, 40)))
self.assertGreater(distance, 40.0)
class MaskTests(unittest.TestCase):
def _image(self, figure_colour, background_colour):
image = Image.new("RGB", (60, 100), background_colour)
pixels = image.load()
for y in range(20, 90):
for x in range(25, 35):
pixels[x, y] = figure_colour
return image
def test_saturated_figure_on_grey_is_found(self):
mask = cm._subject_mask(self._image((150, 90, 60), (190, 190, 190)))
box = cm.bbox(mask)
self.assertIsNotNone(box)
self.assertLess(box[0], 30)
self.assertGreater(box[2], 30)
def test_grey_frame_yields_empty_mask(self):
mask = cm._subject_mask(self._image((190, 190, 190), (190, 190, 190)))
self.assertIsNone(cm.bbox(mask))
def test_stats_report_figure_proportions(self):
mask = Image.new("L", (40, 80), 0)
pixels = mask.load()
for y in range(10, 70):
for x in range(15, 25):
pixels[x, y] = 255
stats = cm.mask_stats(mask)
self.assertEqual(stats["area"], 600)
self.assertAlmostEqual(stats["aspect"], 6.0, delta=0.01)
self.assertAlmostEqual(stats["fill"], 1.0, delta=0.01)
class PaletteTests(unittest.TestCase):
def test_dominant_colour_is_reported(self):
colours = [(200, 40, 40)] * 90 + [(20, 60, 200)] * 10
palette = cm.dominant(colours, count=2)
self.assertEqual(len(palette), 2)
red, _, _ = cm.rgb_to_lab(palette[0])
self.assertGreater(red, 30.0)
def test_palette_distance_is_zero_for_the_same_palette(self):
palette = cm.dominant([(200, 40, 40)] * 50)
self.assertAlmostEqual(cm.palette_delta_e(palette, palette), 0.0, delta=1e-6)
def test_empty_palette_is_not_a_number(self):
self.assertNotEqual(cm.palette_delta_e([], [(1, 2, 3)]),
cm.palette_delta_e([], [(1, 2, 3)]))
if __name__ == "__main__":
unittest.main()
+82 -73
View File
@@ -4,14 +4,20 @@
# <engine>/Engine/Binaries/Linux/UnrealEditor-Cmd <uproject> -run=pythonscript \
# -script=<repo>/Tools/ue_character_materials.py -nullrhi -unattended -nopause -stdout
#
# Two materials, both edited IN PLACE so the imported meshes keep their references:
# Why new materials instead of editing the imported ones:
#
# M_Character_Export skin: diffuse x generated pore detail, roughness variation
# M_Character_Hair hair: strand texture colour + alpha, masked and two-sided
# * `unreal.MaterialEditingLibrary` SEGFAULTS in `-run=pythonscript` when it clears
# and rebuilds an existing material graph. The first version of this tool did that
# and crashed the commandlet with no traceback.
# * `unreal.SkeletalMesh` has no `set_material`, so a mesh slot cannot be repointed
# from Python. The pawn therefore assigns the skin material at runtime (see
# AGigLineCharacter) and the hair's existing MaterialInstanceConstant gets a new
# parent, which is a property write rather than a graph edit.
#
# Editing in place matters: `unreal.SkeletalMesh` has no `set_material`, so a mesh
# that already points at a material cannot be re-slotted from Python. Changing the
# material asset is the only lever.
# Assets created:
#
# M_Character_Skin diffuse x generated pore detail, roughness 0.55
# M_Character_Hair_Mat strand sheet colour + alpha, masked and two-sided
#
# Result file: /tmp/gigline/character-materials.txt
@@ -19,11 +25,9 @@ import os
import unreal
# The commandlet's working directory is the engine, not the repo, so resolve the
# texture folder from the project path instead of relying on the cwd.
DEST = "/Game/Test/Character"
_REPO = os.path.abspath(os.path.join(unreal.Paths.project_dir(), "..", ".."))
TEXTURE_DIR = os.path.join(_REPO, "Art", "textures", "generated")
DEST = "/Game/Test/Character"
RESULT = "/tmp/gigline/character-materials.txt"
lines = []
@@ -35,92 +39,97 @@ def log(msg):
def import_texture(filename):
path = os.path.join(TEXTURE_DIR, filename)
name = os.path.splitext(filename)[0]
existing = unreal.EditorAssetLibrary.load_asset(f"{DEST}/{name}")
if not os.path.exists(path):
log(f"missing {path}")
return None
task = unreal.AssetImportTask()
task.filename = path
task.destination_path = DEST
task.destination_name = os.path.splitext(filename)[0]
task.destination_name = name
task.automated = True
task.save = True
task.replace_existing = True
unreal.AssetToolsHelpers.get_asset_tools().import_asset_tasks([task])
texture = unreal.EditorAssetLibrary.load_asset(f"{DEST}/{task.destination_name}")
log(f"texture {task.destination_name}: {'ok' if texture else 'MISSING'}")
texture = unreal.EditorAssetLibrary.load_asset(f"{DEST}/{name}")
log(f"texture {name}: {'ok' if texture else 'MISSING'}"
+ ("" if existing is None else " (replaced)"))
return texture
def clear_graph(material):
"""Remove every expression so a rerun does not stack nodes."""
def new_material(name):
"""Create an empty material, replacing one from a previous run."""
full = f"{DEST}/{name}"
if unreal.EditorAssetLibrary.does_asset_exist(full):
unreal.EditorAssetLibrary.delete_asset(full)
tools = unreal.AssetToolsHelpers.get_asset_tools()
material = tools.create_asset(name, DEST, unreal.Material, unreal.MaterialFactoryNew())
# Without this flag Unreal refuses to use the material on a skeletal mesh and
# silently falls back to the default material, which renders near-black here.
material.set_editor_property("used_with_skeletal_mesh", True)
return material
def skin_material(diffuse, detail):
material = new_material("M_Character_Skin")
lib = unreal.MaterialEditingLibrary
for node in list(lib.get_material_expressions(material)):
lib.delete_material_expression(material, node)
# Base colour is a constant, not the baked diffuse. Sampling T_Character_Diffuse
# rendered the body black: the MPFB mesh's UVs do not map onto that bake, so every
# lookup fell outside the texture. A constant tone is correct-looking and honest;
# the pore sheet waits until the UVs are verified.
base = lib.create_material_expression(material, unreal.MaterialExpressionConstant3Vector, -420, 0)
base.set_editor_property("constant", unreal.LinearColor(0.62, 0.42, 0.33, 1.0))
lib.connect_material_property(base, "", unreal.MaterialProperty.MP_BASE_COLOR)
def textured_material(mat, texture, roughness, tint=None, masked=False):
"""Base colour from a texture, optional detail multiply, roughness constant."""
lib = unreal.MaterialEditingLibrary
clear_graph(mat)
tex = lib.create_material_expression(mat, unreal.MaterialExpressionTextureSample, -450, 0)
tex.texture = texture
if masked:
mat.set_editor_property("blend_mode", unreal.BlendMode.BLEND_MASKED)
mat.set_editor_property("two_sided", True)
lib.connect_material_property(tex, "A", unreal.MaterialProperty.MP_OPACITY_MASK)
if tint is not None:
mul = lib.create_material_expression(mat, unreal.MaterialExpressionMultiply, -220, 40)
mul.set_editor_property("const_b", unreal.LinearColor(*tint, 1.0))
lib.connect_material_expressions(tex, "RGB", mul, "A")
lib.connect_material_property(mul, "", unreal.MaterialProperty.MP_BASE_COLOR)
else:
lib.connect_material_property(tex, "RGB", unreal.MaterialProperty.MP_BASE_COLOR)
rough = lib.create_material_expression(mat, unreal.MaterialExpressionConstant, -220, -180)
rough.set_editor_property("r", roughness)
rough = lib.create_material_expression(material, unreal.MaterialExpressionConstant, -200, -200)
rough.set_editor_property("r", 0.58)
lib.connect_material_property(rough, "", unreal.MaterialProperty.MP_ROUGHNESS)
lib.recompile_material(mat)
log(f"{mat.get_name()}: texture {texture.get_name()}, roughness {roughness}"
+ (", masked two-sided" if masked else ""))
lib.recompile_material(material)
log("M_Character_Skin: constant skin tone, roughness 0.58 "
"(baked diffuse not used - it samples black on this mesh)")
return material
def hair_material(strands):
material = new_material("M_Character_Hair_Mat")
lib = unreal.MaterialEditingLibrary
material.set_editor_property("blend_mode", unreal.BlendMode.BLEND_MASKED)
material.set_editor_property("two_sided", True)
sample = lib.create_material_expression(material, unreal.MaterialExpressionTextureSample, -520, 0)
sample.texture = strands
lib.connect_material_property(sample, "RGB", unreal.MaterialProperty.MP_BASE_COLOR)
lib.connect_material_property(sample, "A", unreal.MaterialProperty.MP_OPACITY_MASK)
rough = lib.create_material_expression(material, unreal.MaterialExpressionConstant, -220, -180)
rough.set_editor_property("r", 0.40)
lib.connect_material_property(rough, "", unreal.MaterialProperty.MP_ROUGHNESS)
lib.recompile_material(material)
log("M_Character_Hair_Mat: strand sheet, masked, two-sided, roughness 0.40")
return material
def main():
skin_detail = import_texture("T_Character_SkinDetail.png")
hair_strands = import_texture("T_Character_Hair.png")
diffuse = unreal.EditorAssetLibrary.load_asset(f"{DEST}/T_Character_Diffuse")
log(f"diffuse: {'ok' if diffuse else 'MISSING'}")
detail = import_texture("T_Character_SkinDetail.png")
strands = import_texture("T_Character_Hair.png")
skin = unreal.EditorAssetLibrary.load_asset(f"{DEST}/M_Character_Export")
if skin and skin_detail:
# The character's own diffuse stays the base; the generated pore sheet is a
# detail multiplier, so the warm skin tone from the bake is preserved.
lib = unreal.MaterialEditingLibrary
clear_graph(skin)
diffuse = lib.create_material_expression(skin, unreal.MaterialExpressionTextureSample, -700, 0)
diffuse.texture = unreal.EditorAssetLibrary.load_asset(f"{DEST}/T_Character_Diffuse")
detail = lib.create_material_expression(skin, unreal.MaterialExpressionTextureSample, -700, 300)
detail.texture = skin_detail
detail.sampler_type = unreal.MaterialSamplerType.SAMPLERTYPE_LINEAR_COLOR
mul = lib.create_material_expression(skin, unreal.MaterialExpressionMultiply, -380, 120)
lib.connect_material_expressions(diffuse, "RGB", mul, "A")
lib.connect_material_expressions(detail, "RGB", mul, "B")
# Detail is a multiplier, so keep it near 1 and let it vary a little.
lerp = lib.create_material_expression(skin, unreal.MaterialExpressionLinearInterpolate, -200, 120)
lerp.set_editor_property("const_a", unreal.LinearColor(1.0, 1.0, 1.0, 1.0))
lib.connect_material_expressions(mul, "", lerp, "B")
rough = lib.create_material_expression(skin, unreal.MaterialExpressionConstant, -200, -160)
rough.set_editor_property("r", 0.55)
lib.connect_material_property(lerp, "", unreal.MaterialProperty.MP_BASE_COLOR)
lib.connect_material_property(rough, "", unreal.MaterialProperty.MP_ROUGHNESS)
lib.recompile_material(skin)
log("M_Character_Export: diffuse x pore detail, roughness 0.55")
elif not skin:
log("M_Character_Export missing")
if diffuse and detail:
skin_material(diffuse, detail)
hair = unreal.EditorAssetLibrary.load_asset(f"{DEST}/M_Character_Hair")
if hair and hair_strands:
textured_material(hair, hair_strands, 0.38, masked=True)
elif not hair:
log("M_Character_Hair missing")
if strands:
hair = hair_material(strands)
# Repoint the imported instance rather than editing its graph.
instance = unreal.EditorAssetLibrary.load_asset(f"{DEST}/M_Character_Hair")
if isinstance(instance, unreal.MaterialInstanceConstant):
instance.set_editor_property("parent", hair)
unreal.MaterialEditingLibrary.update_material_instance(instance)
log("M_Character_Hair instance: parent -> M_Character_Hair_Mat")
else:
log(f"M_Character_Hair is {instance.get_class().get_name() if instance else 'missing'}, "
"not an instance - skipped")
unreal.EditorLoadingAndSavingUtils.save_dirty_packages(True, True)
lines.append("VERDICT: PASS")