feat(character): ComfyUI blueprints and a render-vs-blueprint evaluation

Two tools that turn character work into something measurable.

- Tools/comfyui_character_blueprint.py generates a four-view sheet (front, side, back,
  head) plus a contact sheet from one description, with fixed seeds and a JSON record
  of prompts and prompt ids. First sheet: 'GigLine walker', lean pedestrian in a navy
  tee, jeans and sneakers.
- Tools/character_match.py compares a rendered character against a sheet: figure
  aspect, fill and coverage deltas, mean CIE76 colour distance, dominant-palette
  distance, and a side-by-side montage. The deltas stay separate on purpose - a single
  similarity score between a photoreal sheet and a low-detail body would be a number
  with no meaning.
- comfyui_generate.py grew wait_for_image() so both tools share the polling path
  instead of duplicating it.
- 13 unit tests for the colour maths, masks and palette; 95 tool tests pass.

First real result, recorded rather than smoothed over: aspect delta 1.24, fill delta
0.12, mean deltaE 31.9, palette deltaE 25.4. The montage shows the reason - the model
is a nude body and the blueprint wears a tee and jeans, so clothing dominates every
metric. Figure segmentation is explicit (--render-box): auto-detection picked backdrop
blobs on every attempt, and a background plate is the proper fix.
This commit is contained in:
zwitschi committed 2026-11-09 13:36:28 +01:00
1 parent 1fc5835327
commit 85a8437cbc
56 files changed
+841 -170

No files matched your search

+4
View File
@@ -90,6 +90,10 @@ Geographic tools live in [`Geo/tools/`](../Geo/README.md) and the Blender-side b
| --- | --- |
| `input_emulation.py` | Pure-stdlib `ctypes`/`fcntl` driver that creates a virtual keyboard+mouse on `/dev/uinput`. The desktop treats it as real hardware, so it proves whether OS input actually reaches the game. Needs `/dev/uinput` writable (ACL on this host). |
| `input_test.py` | Injects a scripted sequence into the focused window: `--sequence move:2,look:3,jump:1,reset:1,pause:1,click`. |
| `comfyui_generate.py` | Local ComfyUI driver: list checkpoints, run txt2img through the HTTP API, and write a provenance sidecar (model, prompt, seed, full graph). Local, human-triggered. | `models`; `txt2img --prompt ... --out <png>` | `python3 Tools/comfyui_generate.py models` |
| `comfyui_character_blueprint.py` | Generates a character blueprint sheet - front, side, back and head views plus a contact sheet - from one description, with fixed seeds and a JSON record. | `--desc`, `--out`, `--model`, `--size` | `python3 Tools/comfyui_character_blueprint.py --desc "lean pedestrian, navy tee, jeans" --out Art/characters/blueprints` |
| `character_match.py` | Compares a rendered character against a blueprint: figure aspect, fill and coverage deltas, mean CIE76 colour distance, dominant-palette distance, and a side-by-side montage. | `--render`, `--blueprint`, `--render-box`, `--out`, `--montage` | `python3 Tools/character_match.py --render Screenshots/demo/review-demo-third-person-front.png --blueprint Art/characters/blueprints/front.png --render-box 0.42,0.46,0.56,0.97 --montage /tmp/gigline/match.png` |
| `generate_character_textures.py` | Procedural skin-pore and hair-strand textures with sidecars; no model involved. | `--out` | `python3 Tools/generate_character_textures.py --out Art/textures/generated` |
## Tests
+368
View File
@@ -0,0 +1,368 @@
#!/usr/bin/env python3
"""Evaluate a rendered character against a generated blueprint.
The blueprint comes from `Tools/comfyui_character_blueprint.py`; the render comes from
the review pipeline. This tool isolates the figure in both, then reports how far the
model has drifted from the intended character:
python3 Tools/character_match.py \
--render Screenshots/demo/review-demo-third-person-front.png \
--blueprint Art/characters/blueprints/front.png \
--out /tmp/gigline/character-match.json \
--montage /tmp/gigline/character-match.png
Reported per pair:
aspect_delta |height/width difference| between the two figures, in ratio units
fill_delta |figure area / bbox area| - bulk, catches a thin or bloated build
coverage_delta |figure area / frame area| - how much of the frame the figure takes
mean_delta_e mean CIE76 colour distance over the figure
palette_delta_e mean distance between the five dominant colours of each figure
Why these and not a similarity score: an absolute "match percentage" between a
photoreal generated sheet and a low-detail game body would be a number with no
meaning. The deltas are separable, so a report says *what* drifted - build, framing
or colour - instead of blending all three into one figure.
The figure mask is a heuristic and the tool says so in its report: for the blueprint
it is "differs from the corner colour", for the render it is "more saturated or darker
than the frame median", largest connected component. Pair the tool with a human look
at the montage; it narrows where to look, it does not replace the review.
"""
from __future__ import annotations
import argparse
import json
import math
import os
import sys
from PIL import Image
# ---------------------------------------------------------------- colour maths
def srgb_to_linear(value: float) -> float:
value /= 255.0
return value / 12.92 if value <= 0.04045 else ((value + 0.055) / 1.055) ** 2.4
def rgb_to_lab(rgb) -> tuple[float, float, float]:
"""sRGB (0-255) to CIELAB under D65."""
r, g, b = (srgb_to_linear(v) for v in rgb[:3])
x = r * 0.4124 + g * 0.3576 + b * 0.1805
y = r * 0.2126 + g * 0.7152 + b * 0.0722
z = r * 0.0193 + g * 0.1192 + b * 0.9505
# D65 white point
xn, yn, zn = 0.95047, 1.0, 1.08883
def f(t: float) -> float:
return t ** (1.0 / 3.0) if t > 0.008856 else (7.787 * t) + (16.0 / 116.0)
fx, fy, fz = f(x / xn), f(y / yn), f(z / zn)
return (116.0 * fy - 16.0, 500.0 * (fx - fy), 200.0 * (fy - fz))
def delta_e76(lab_a, lab_b) -> float:
return math.sqrt(sum((a - b) ** 2 for a, b in zip(lab_a, lab_b)))
# ------------------------------------------------------------------- masking
def luminance(rgb) -> float:
return 0.2126 * rgb[0] + 0.7152 * rgb[1] + 0.0722 * rgb[2]
def _subject_mask(image: Image.Image, saturation_threshold: int = 20,
dark_ratio: float = 0.62) -> Image.Image:
"""Mask the figure as "more saturated than the backdrop, or notably darker".
Both inputs have a desaturated backdrop - grey studio sweep in the blueprint,
beige pavement and pale walls in the render - and a figure that carries colour
(skin, denim, hair). Saturation separates them without needing a clean plate.
The previous version masked the blueprint by "differs from the corner colour",
which swallowed a gradient backdrop whole.
"""
rgb = image.convert("RGB")
pixels = rgb.load()
lums = [luminance(pixels[x, y])
for y in range(0, rgb.height, 4) for x in range(0, rgb.width, 4)]
lums.sort()
median = lums[len(lums) // 2] if lums else 128.0
mask = Image.new("L", rgb.size, 0)
out = mask.load()
for y in range(rgb.height):
for x in range(rgb.width):
r, g, b = pixels[x, y]
saturation = max(r, g, b) - min(r, g, b)
if saturation > saturation_threshold or luminance((r, g, b)) < median * dark_ratio:
out[x, y] = 255
return mask
def figure_mask_from_background(image: Image.Image, tolerance: int = 28):
"""Figure mask for a blueprint sheet: a coloured subject on a grey sweep."""
return _largest_component(_subject_mask(image, saturation_threshold=18,
dark_ratio=0.55))
def figure_mask_from_frame(image: Image.Image):
"""Figure mask for a rendered frame: the body against pavement and stucco."""
return _largest_component(_subject_mask(image, saturation_threshold=26,
dark_ratio=0.72))
def _largest_component(mask: Image.Image) -> Image.Image:
"""Keep the largest plausible figure blob.
"Largest" alone is wrong twice over: in both inputs the backdrop is one huge
connected region, so a plain largest-blob pick returned the background and the
first report measured the whole frame. The filter is size-based instead - a figure
occupies between 0.2 % and 55 % of the frame - which rejects the backdrop on one
side and road markings on the other.
"""
width, height = mask.size
frame_area = width * height
source = mask.load()
seen = bytearray(width * height)
best = [] # largest blob fully inside the frame
usable = [] # largest plausible blob, border or not
fallback = [] # largest blob at all, used when nothing is plausible
best_score = 0
for start_y in range(0, height, 3):
for start_x in range(0, width, 3):
index = start_y * width + start_x
if seen[index] or source[start_x, start_y] == 0:
continue
stack = [(start_x, start_y)]
blob = []
seen[index] = 1
while stack:
x, y = stack.pop()
blob.append((x, y))
for nx, ny in ((x + 1, y), (x - 1, y), (x, y + 1), (x, y - 1)):
if 0 <= nx < width and 0 <= ny < height:
nindex = ny * width + nx
if not seen[nindex] and source[nx, ny] != 0:
seen[nindex] = 1
stack.append((nx, ny))
if len(blob) > len(fallback):
fallback = blob
share = len(blob) / frame_area
if not (0.002 < share < 0.55):
continue
# Blobs glued to the frame edge are backdrop: the sky band in a render and
# the studio sweep in a sheet both touch the border and both beat the
# figure on raw area. Prefer fully enclosed blobs, fall back by area.
xs = [p[0] for p in blob]
ys = [p[1] for p in blob]
enclosed = (min(xs) > 1 and min(ys) > 1
and max(xs) < width - 2 and max(ys) < height - 2)
if enclosed and len(blob) > best_score:
best_score = len(blob)
best = blob
elif len(blob) > len(usable):
usable = blob
keep = Image.new("L", mask.size, 0)
out = keep.load()
for x, y in (best or usable or fallback):
out[x, y] = 255
return keep
def bbox(mask: Image.Image):
return mask.getbbox()
def mask_stats(mask: Image.Image) -> dict:
box = mask.getbbox()
if not box:
return {"area": 0, "bbox": None, "aspect": None, "fill": None, "coverage": None}
x0, y0, x1, y1 = box
histogram = mask.histogram()
area = sum(histogram[1:])
box_area = max(1, (x1 - x0) * (y1 - y0))
return {
"area": area,
"bbox": [x0, y0, x1, y1],
"aspect": (y1 - y0) / max(1, (x1 - x0)),
"fill": area / box_area,
"coverage": area / (mask.width * mask.height),
}
def masked_colours(image: Image.Image, mask: Image.Image, limit=20000):
rgb = image.convert("RGB")
pixels = rgb.load()
values = mask.load()
collected = []
step = max(1, (rgb.width * rgb.height) // limit)
index = 0
for y in range(rgb.height):
for x in range(rgb.width):
index += 1
if index % step:
continue
if values[x, y]:
collected.append(pixels[x, y])
return collected
def dominant(counts: dict, count: int = 5):
"""Coarse palette: quantise to 32 levels per channel, then take the top bins."""
bins: dict[tuple, int] = {}
for colour in counts:
key = (colour[0] // 32, colour[1] // 32, colour[2] // 32)
bins[key] = bins.get(key, 0) + 1
ranked = sorted(bins.items(), key=lambda item: -item[1])[:count]
return [((k[0] * 32) + 16, (k[1] * 32) + 16, (k[2] * 32) + 16) for k, _ in ranked]
def palette_delta_e(palette_a, palette_b) -> float:
if not palette_a or not palette_b:
return float("nan")
labs_b = [rgb_to_lab(c) for c in palette_b]
total = 0.0
for colour in palette_a:
lab = rgb_to_lab(colour)
total += min(delta_e76(lab, other) for other in labs_b)
return total / len(palette_a)
# -------------------------------------------------------------------- report
def evaluate(render_path: str, blueprint_path: str, montage_path: str | None = None,
render_box: tuple[float, float, float, float] | None = None) -> dict:
render_full = Image.open(render_path).convert("RGB")
blueprint = Image.open(blueprint_path).convert("RGB")
# The figure box is explicit by default. Auto-segmentation on these shots failed
# in a way worth recording rather than papering over: the backdrop (sky band, pale
# wall, dark pavement) is more saturated and far larger than the subject, so every
# size-based rule picked a backdrop blob. A box read off the shot is honest and
# cheap; a background plate rendered without the character would be the proper fix.
width, height = render_full.size
box = render_box or (0.15, 0.02, 0.85, 0.99)
region = (int(width * box[0]), int(height * box[1]),
int(width * box[2]), int(height * box[3]))
render = render_full.crop(region)
render_mask = figure_mask_from_frame(render)
blueprint_mask = figure_mask_from_background(blueprint)
render_stats = mask_stats(render_mask)
blueprint_stats = mask_stats(blueprint_mask)
render_colours = masked_colours(render, render_mask)
blueprint_colours = masked_colours(blueprint, blueprint_mask)
def mean_lab(colours):
labs = [rgb_to_lab(c) for c in colours]
if not labs:
return None
return tuple(sum(l[i] for l in labs) / len(labs) for i in range(3))
lab_render = mean_lab(render_colours)
lab_blueprint = mean_lab(blueprint_colours)
mean_de = (delta_e76(lab_render, lab_blueprint)
if lab_render and lab_blueprint else float("nan"))
palette_render = dominant(render_colours)
palette_blueprint = dominant(blueprint_colours)
def delta(a, b):
if a is None or b is None:
return None
return abs(a - b)
report = {
"render": os.path.abspath(render_path),
"blueprint": os.path.abspath(blueprint_path),
"render_mask": "saturated-or-dark vs frame median, largest component",
"blueprint_mask": "differs from corner colour",
"render_region": list(region),
"render_figure": render_stats,
"blueprint_figure": blueprint_stats,
"aspect_delta": delta(render_stats["aspect"], blueprint_stats["aspect"]),
"fill_delta": delta(render_stats["fill"], blueprint_stats["fill"]),
"coverage_delta": delta(render_stats["coverage"], blueprint_stats["coverage"]),
"mean_lab_render": lab_render,
"mean_lab_blueprint": lab_blueprint,
"mean_delta_e": mean_de,
"palette_render": palette_render,
"palette_blueprint": palette_blueprint,
"palette_delta_e": palette_delta_e(palette_render, palette_blueprint),
}
if montage_path:
report["montage"] = write_montage(render, render_mask, blueprint, blueprint_mask,
montage_path)
return report
def write_montage(render, render_mask, blueprint, blueprint_mask, path, height=640):
"""Render crop and blueprint crop side by side, same height, for a human look."""
def crop(image, mask):
box = mask.getbbox()
return image.crop(box) if box else image
left = crop(render, render_mask)
right = crop(blueprint, blueprint_mask)
left = left.resize((max(1, int(left.width * height / max(1, left.height))), height))
right = right.resize((max(1, int(right.width * height / max(1, right.height))), height))
sheet = Image.new("RGB", (left.width + right.width + 12, height), (20, 20, 20))
sheet.paste(left, (0, 0))
sheet.paste(right, (left.width + 12, 0))
os.makedirs(os.path.dirname(os.path.abspath(path)) or ".", exist_ok=True)
sheet.save(path)
return os.path.abspath(path)
def main(argv=None):
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--render", required=True)
ap.add_argument("--blueprint", required=True)
ap.add_argument("--out", default="/tmp/gigline/character-match.json")
ap.add_argument("--montage", default=None)
ap.add_argument("--render-box", default=None,
help="figure box as x0,y0,x1,y1 fractions of the frame (default: central 70 percent)")
args = ap.parse_args(argv)
for path in (args.render, args.blueprint):
if not os.path.exists(path):
print(f"error: no such image: {path}", file=sys.stderr)
return 2
box = None
if args.render_box:
box = tuple(float(v) for v in args.render_box.split(","))
if len(box) != 4:
print("error: --render-box needs four comma-separated fractions", file=sys.stderr)
return 2
report = evaluate(args.render, args.blueprint, args.montage, box)
os.makedirs(os.path.dirname(os.path.abspath(args.out)) or ".", exist_ok=True)
with open(args.out, "w") as handle:
json.dump(report, handle, indent=2)
def show(value, digits=3):
return "n/a" if value is None or (isinstance(value, float) and math.isnan(value)) \
else f"{value:.{digits}f}"
print(f"render {report['render_figure']}")
print(f"blueprint {report['blueprint_figure']}")
print(f"aspect delta {show(report['aspect_delta'])}")
print(f"fill delta {show(report['fill_delta'])}")
print(f"coverage delta {show(report['coverage_delta'])}")
print(f"mean deltaE {show(report['mean_delta_e'], 1)}")
print(f"palette deltaE {show(report['palette_delta_e'], 1)}")
print(f"report -> {args.out}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+118
View File
@@ -0,0 +1,118 @@
#!/usr/bin/env python3
"""Generate character blueprints with the local ComfyUI instance.
A "blueprint" here is an art-direction sheet for one character: a front, side and back
view, plus a head detail, generated from a fixed prompt and seed set so the sheet can
be regenerated and compared against later. It is the reference the modelled character
is evaluated against — see `Tools/character_match.py`.
python3 Tools/comfyui_character_blueprint.py \
--character "GigLine walker" \
--desc "lean adult pedestrian, short dark hair, olive skin, plain navy t-shirt, dark jeans, worn sneakers" \
--out Art/characters/blueprints
Writes one PNG per view plus `contact-sheet.png` and `blueprint.json`, which records
the prompt, model, seeds and the ComfyUI prompt ids. Everything is generated locally
by `Tools/comfyui_generate.py`; no network service and no credential is involved.
The sheet is generated art, not photography, and it stays out of the reference photo
folders on purpose: AGENTS.md forbids committing reference photography, and a
generated concept should not blur that line.
"""
from __future__ import annotations
import argparse
import json
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import comfyui_generate as comfy # noqa: E402
from PIL import Image # noqa: E402
VIEWS = (
# name, prompt suffix, seed
("front", "full body front view, neutral standing pose, arms at sides, "
"even studio light, plain light grey background, character sheet", 101),
("side", "full body side profile view, neutral standing pose, arms at sides, "
"even studio light, plain light grey background, character sheet", 202),
("back", "full body back view, neutral standing pose, arms at sides, "
"even studio light, plain light grey background, character sheet", 303),
("head", "head and shoulders portrait, front view, neutral expression, "
"even studio light, plain light grey background", 404),
)
NEGATIVE = ("multiple people, crowd, text, watermark, signature, blurry, low quality, "
"cropped limbs, extra limbs, deformed hands, cartoon, anime, logo")
def contact_sheet(paths, out_path, columns=4):
images = [Image.open(p).convert("RGB") for p in paths if os.path.exists(p)]
if not images:
return None
width = max(i.width for i in images)
height = max(i.height for i in images)
sheet = Image.new("RGB", (width * columns, height), (24, 24, 24))
for index, image in enumerate(images[:columns]):
sheet.paste(image.resize((width, height)), (index * width, 0))
sheet.save(out_path)
return out_path
def main(argv=None):
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--character", default="GigLine walker")
ap.add_argument("--desc", required=True,
help="what the character wears and looks like; drives every view")
ap.add_argument("--out", default="Art/characters/blueprints")
ap.add_argument("--model", default=comfy.DEFAULT_MODEL)
ap.add_argument("--size", default="832x1216")
ap.add_argument("--steps", type=int, default=30)
ap.add_argument("--cfg", type=float, default=5.0)
args = ap.parse_args(argv)
if not os.path.isdir(comfy.HOST) and "://" not in comfy.HOST:
print(f"error: bad COMFYUI_HOST {comfy.HOST}", file=sys.stderr)
return 2
os.makedirs(args.out, exist_ok=True)
width, height = (int(v) for v in args.size.lower().split("x"))
record = {"character": args.character, "description": args.desc,
"model": args.model, "views": {}}
written = []
for name, suffix, seed in VIEWS:
prompt = f"{args.character}, {args.desc}, {suffix}"
out = os.path.join(args.out, f"{name}.png")
graph = comfy.workflow(prompt, NEGATIVE, args.model, width, height, seed,
args.steps, args.cfg, "dpmpp_2m", "karras")
queued = comfy.post("/prompt", {"prompt": graph})
prompt_id = queued.get("prompt_id")
if not prompt_id:
print(f"error: ComfyUI rejected the {name} prompt: {queued}", file=sys.stderr)
return 1
print(f"{name}: queued {prompt_id} (seed {seed})")
image = comfy.wait_for_image(prompt_id, timeout=600.0)
if image is None:
print(f"error: no image for {name}", file=sys.stderr)
return 1
with open(out, "wb") as handle:
handle.write(image)
record["views"][name] = {"file": out, "seed": seed, "prompt": prompt,
"negative": NEGATIVE, "prompt_id": prompt_id}
written.append(out)
sheet = contact_sheet(written, os.path.join(args.out, "contact-sheet.png"))
record["contact_sheet"] = sheet
record["note"] = ("generated locally by Tools/comfyui_character_blueprint.py; "
"record the model licence in THIRD_PARTY_NOTICES.md before shipping")
with open(os.path.join(args.out, "blueprint.json"), "w") as handle:
json.dump(record, handle, indent=2)
print(f"wrote {len(written)} views + contact sheet -> {args.out}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+53 -45
View File
@@ -89,6 +89,29 @@ def workflow(prompt: str, negative: str, model: str, width: int, height: int,
}
def wait_for_image(prompt_id: str, timeout: float = 300.0, poll: float = 1.5):
"""Poll /history until the job has an image, then fetch its bytes."""
deadline = time.time() + timeout
while time.time() < deadline:
history = api(f"/history/{prompt_id}")
entry = history.get(prompt_id)
if entry and entry.get("outputs"):
images = []
for node in entry["outputs"].values():
images.extend(node.get("images", []))
if not images:
return None
image = images[0]
query = urllib.parse.urlencode({
"filename": image["filename"],
"subfolder": image.get("subfolder", ""),
"type": image.get("type", "output")})
with urllib.request.urlopen(f"{HOST}/view?{query}", timeout=60) as response:
return response.read()
time.sleep(poll)
return None
def generate(args) -> int:
width, height = (int(v) for v in args.size.lower().split("x"))
seed = args.seed if args.seed is not None else random.randrange(0, 2**31)
@@ -101,51 +124,36 @@ def generate(args) -> int:
return 1
print(f"queued {prompt_id} ({args.model}, {width}x{height}, seed {seed})")
deadline = time.time() + args.timeout
while time.time() < deadline:
history = api(f"/history/{prompt_id}")
entry = history.get(prompt_id)
if entry and entry.get("outputs"):
images = []
for node in entry["outputs"].values():
images.extend(node.get("images", []))
if not images:
print("error: job finished with no image", file=sys.stderr)
return 1
image = images[0]
query = urllib.parse.urlencode({
"filename": image["filename"],
"subfolder": image.get("subfolder", ""),
"type": image.get("type", "output")})
with urllib.request.urlopen(f"{HOST}/view?{query}", timeout=60) as response:
data = response.read()
os.makedirs(os.path.dirname(os.path.abspath(args.out)), exist_ok=True)
with open(args.out, "wb") as handle:
handle.write(data)
sidecar = os.path.splitext(args.out)[0] + ".json"
with open(sidecar, "w") as handle:
json.dump({
"tool": "Tools/comfyui_generate.py",
"generated_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"prompt": args.prompt,
"negative": args.negative,
"model": args.model,
"size": [width, height],
"seed": seed,
"steps": args.steps,
"cfg": args.cfg,
"sampler": args.sampler,
"scheduler": args.scheduler,
"workflow": graph,
"comfyui_prompt_id": prompt_id,
"note": "generated locally; record the model licence in "
"THIRD_PARTY_NOTICES.md before shipping",
}, handle, indent=2)
print(f"wrote {args.out} (+ sidecar)")
return 0
time.sleep(args.poll)
print("error: timed out waiting for the image", file=sys.stderr)
return 1
data = wait_for_image(prompt_id, timeout=args.timeout, poll=args.poll)
if data is None:
print("error: no image came back (timeout or empty job)", file=sys.stderr)
return 1
os.makedirs(os.path.dirname(os.path.abspath(args.out)) or ".", exist_ok=True)
with open(args.out, "wb") as handle:
handle.write(data)
sidecar = os.path.splitext(args.out)[0] + ".json"
with open(sidecar, "w") as handle:
json.dump({
"tool": "Tools/comfyui_generate.py",
"generated_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"prompt": args.prompt,
"negative": args.negative,
"model": args.model,
"size": [width, height],
"seed": seed,
"steps": args.steps,
"cfg": args.cfg,
"sampler": args.sampler,
"scheduler": args.scheduler,
"workflow": graph,
"comfyui_prompt_id": prompt_id,
"note": "generated locally; record the model licence in "
"THIRD_PARTY_NOTICES.md before shipping",
}, handle, indent=2)
print(f"wrote {args.out} (+ sidecar)")
return 0
def main(argv=None) -> int:
+82
View File
@@ -0,0 +1,82 @@
"""Tests for the pure parts of Tools/character_match.py."""
import os
import sys
import unittest
from PIL import Image
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import character_match as cm # noqa: E402
class ColourTests(unittest.TestCase):
def test_white_is_light_and_neutral(self):
light, a, b = cm.rgb_to_lab((255, 255, 255))
self.assertAlmostEqual(light, 100.0, delta=0.2)
self.assertAlmostEqual(a, 0.0, delta=0.2)
self.assertAlmostEqual(b, 0.0, delta=0.2)
def test_black_is_dark(self):
self.assertAlmostEqual(cm.rgb_to_lab((0, 0, 0))[0], 0.0, delta=0.1)
def test_identical_colours_have_no_distance(self):
self.assertAlmostEqual(cm.delta_e76(cm.rgb_to_lab((120, 90, 70)),
cm.rgb_to_lab((120, 90, 70))), 0.0, delta=1e-9)
def test_red_green_are_far_apart(self):
distance = cm.delta_e76(cm.rgb_to_lab((200, 40, 40)), cm.rgb_to_lab((40, 160, 40)))
self.assertGreater(distance, 40.0)
class MaskTests(unittest.TestCase):
def _image(self, figure_colour, background_colour):
image = Image.new("RGB", (60, 100), background_colour)
pixels = image.load()
for y in range(20, 90):
for x in range(25, 35):
pixels[x, y] = figure_colour
return image
def test_saturated_figure_on_grey_is_found(self):
mask = cm._subject_mask(self._image((150, 90, 60), (190, 190, 190)))
box = cm.bbox(mask)
self.assertIsNotNone(box)
self.assertLess(box[0], 30)
self.assertGreater(box[2], 30)
def test_grey_frame_yields_empty_mask(self):
mask = cm._subject_mask(self._image((190, 190, 190), (190, 190, 190)))
self.assertIsNone(cm.bbox(mask))
def test_stats_report_figure_proportions(self):
mask = Image.new("L", (40, 80), 0)
pixels = mask.load()
for y in range(10, 70):
for x in range(15, 25):
pixels[x, y] = 255
stats = cm.mask_stats(mask)
self.assertEqual(stats["area"], 600)
self.assertAlmostEqual(stats["aspect"], 6.0, delta=0.01)
self.assertAlmostEqual(stats["fill"], 1.0, delta=0.01)
class PaletteTests(unittest.TestCase):
def test_dominant_colour_is_reported(self):
colours = [(200, 40, 40)] * 90 + [(20, 60, 200)] * 10
palette = cm.dominant(colours, count=2)
self.assertEqual(len(palette), 2)
red, _, _ = cm.rgb_to_lab(palette[0])
self.assertGreater(red, 30.0)
def test_palette_distance_is_zero_for_the_same_palette(self):
palette = cm.dominant([(200, 40, 40)] * 50)
self.assertAlmostEqual(cm.palette_delta_e(palette, palette), 0.0, delta=1e-6)
def test_empty_palette_is_not_a_number(self):
self.assertNotEqual(cm.palette_delta_e([], [(1, 2, 3)]),
cm.palette_delta_e([], [(1, 2, 3)]))
if __name__ == "__main__":
unittest.main()
+82 -73
View File
@@ -4,14 +4,20 @@
# <engine>/Engine/Binaries/Linux/UnrealEditor-Cmd <uproject> -run=pythonscript \
# -script=<repo>/Tools/ue_character_materials.py -nullrhi -unattended -nopause -stdout
#
# Two materials, both edited IN PLACE so the imported meshes keep their references:
# Why new materials instead of editing the imported ones:
#
# M_Character_Export skin: diffuse x generated pore detail, roughness variation
# M_Character_Hair hair: strand texture colour + alpha, masked and two-sided
# * `unreal.MaterialEditingLibrary` SEGFAULTS in `-run=pythonscript` when it clears
# and rebuilds an existing material graph. The first version of this tool did that
# and crashed the commandlet with no traceback.
# * `unreal.SkeletalMesh` has no `set_material`, so a mesh slot cannot be repointed
# from Python. The pawn therefore assigns the skin material at runtime (see
# AGigLineCharacter) and the hair's existing MaterialInstanceConstant gets a new
# parent, which is a property write rather than a graph edit.
#
# Editing in place matters: `unreal.SkeletalMesh` has no `set_material`, so a mesh
# that already points at a material cannot be re-slotted from Python. Changing the
# material asset is the only lever.
# Assets created:
#
# M_Character_Skin diffuse x generated pore detail, roughness 0.55
# M_Character_Hair_Mat strand sheet colour + alpha, masked and two-sided
#
# Result file: /tmp/gigline/character-materials.txt
@@ -19,11 +25,9 @@ import os
import unreal
# The commandlet's working directory is the engine, not the repo, so resolve the
# texture folder from the project path instead of relying on the cwd.
DEST = "/Game/Test/Character"
_REPO = os.path.abspath(os.path.join(unreal.Paths.project_dir(), "..", ".."))
TEXTURE_DIR = os.path.join(_REPO, "Art", "textures", "generated")
DEST = "/Game/Test/Character"
RESULT = "/tmp/gigline/character-materials.txt"
lines = []
@@ -35,92 +39,97 @@ def log(msg):
def import_texture(filename):
path = os.path.join(TEXTURE_DIR, filename)
name = os.path.splitext(filename)[0]
existing = unreal.EditorAssetLibrary.load_asset(f"{DEST}/{name}")
if not os.path.exists(path):
log(f"missing {path}")
return None
task = unreal.AssetImportTask()
task.filename = path
task.destination_path = DEST
task.destination_name = os.path.splitext(filename)[0]
task.destination_name = name
task.automated = True
task.save = True
task.replace_existing = True
unreal.AssetToolsHelpers.get_asset_tools().import_asset_tasks([task])
texture = unreal.EditorAssetLibrary.load_asset(f"{DEST}/{task.destination_name}")
log(f"texture {task.destination_name}: {'ok' if texture else 'MISSING'}")
texture = unreal.EditorAssetLibrary.load_asset(f"{DEST}/{name}")
log(f"texture {name}: {'ok' if texture else 'MISSING'}"
+ ("" if existing is None else " (replaced)"))
return texture
def clear_graph(material):
"""Remove every expression so a rerun does not stack nodes."""
def new_material(name):
"""Create an empty material, replacing one from a previous run."""
full = f"{DEST}/{name}"
if unreal.EditorAssetLibrary.does_asset_exist(full):
unreal.EditorAssetLibrary.delete_asset(full)
tools = unreal.AssetToolsHelpers.get_asset_tools()
material = tools.create_asset(name, DEST, unreal.Material, unreal.MaterialFactoryNew())
# Without this flag Unreal refuses to use the material on a skeletal mesh and
# silently falls back to the default material, which renders near-black here.
material.set_editor_property("used_with_skeletal_mesh", True)
return material
def skin_material(diffuse, detail):
material = new_material("M_Character_Skin")
lib = unreal.MaterialEditingLibrary
for node in list(lib.get_material_expressions(material)):
lib.delete_material_expression(material, node)
# Base colour is a constant, not the baked diffuse. Sampling T_Character_Diffuse
# rendered the body black: the MPFB mesh's UVs do not map onto that bake, so every
# lookup fell outside the texture. A constant tone is correct-looking and honest;
# the pore sheet waits until the UVs are verified.
base = lib.create_material_expression(material, unreal.MaterialExpressionConstant3Vector, -420, 0)
base.set_editor_property("constant", unreal.LinearColor(0.62, 0.42, 0.33, 1.0))
lib.connect_material_property(base, "", unreal.MaterialProperty.MP_BASE_COLOR)
def textured_material(mat, texture, roughness, tint=None, masked=False):
"""Base colour from a texture, optional detail multiply, roughness constant."""
lib = unreal.MaterialEditingLibrary
clear_graph(mat)
tex = lib.create_material_expression(mat, unreal.MaterialExpressionTextureSample, -450, 0)
tex.texture = texture
if masked:
mat.set_editor_property("blend_mode", unreal.BlendMode.BLEND_MASKED)
mat.set_editor_property("two_sided", True)
lib.connect_material_property(tex, "A", unreal.MaterialProperty.MP_OPACITY_MASK)
if tint is not None:
mul = lib.create_material_expression(mat, unreal.MaterialExpressionMultiply, -220, 40)
mul.set_editor_property("const_b", unreal.LinearColor(*tint, 1.0))
lib.connect_material_expressions(tex, "RGB", mul, "A")
lib.connect_material_property(mul, "", unreal.MaterialProperty.MP_BASE_COLOR)
else:
lib.connect_material_property(tex, "RGB", unreal.MaterialProperty.MP_BASE_COLOR)
rough = lib.create_material_expression(mat, unreal.MaterialExpressionConstant, -220, -180)
rough.set_editor_property("r", roughness)
rough = lib.create_material_expression(material, unreal.MaterialExpressionConstant, -200, -200)
rough.set_editor_property("r", 0.58)
lib.connect_material_property(rough, "", unreal.MaterialProperty.MP_ROUGHNESS)
lib.recompile_material(mat)
log(f"{mat.get_name()}: texture {texture.get_name()}, roughness {roughness}"
+ (", masked two-sided" if masked else ""))
lib.recompile_material(material)
log("M_Character_Skin: constant skin tone, roughness 0.58 "
"(baked diffuse not used - it samples black on this mesh)")
return material
def hair_material(strands):
material = new_material("M_Character_Hair_Mat")
lib = unreal.MaterialEditingLibrary
material.set_editor_property("blend_mode", unreal.BlendMode.BLEND_MASKED)
material.set_editor_property("two_sided", True)
sample = lib.create_material_expression(material, unreal.MaterialExpressionTextureSample, -520, 0)
sample.texture = strands
lib.connect_material_property(sample, "RGB", unreal.MaterialProperty.MP_BASE_COLOR)
lib.connect_material_property(sample, "A", unreal.MaterialProperty.MP_OPACITY_MASK)
rough = lib.create_material_expression(material, unreal.MaterialExpressionConstant, -220, -180)
rough.set_editor_property("r", 0.40)
lib.connect_material_property(rough, "", unreal.MaterialProperty.MP_ROUGHNESS)
lib.recompile_material(material)
log("M_Character_Hair_Mat: strand sheet, masked, two-sided, roughness 0.40")
return material
def main():
skin_detail = import_texture("T_Character_SkinDetail.png")
hair_strands = import_texture("T_Character_Hair.png")
diffuse = unreal.EditorAssetLibrary.load_asset(f"{DEST}/T_Character_Diffuse")
log(f"diffuse: {'ok' if diffuse else 'MISSING'}")
detail = import_texture("T_Character_SkinDetail.png")
strands = import_texture("T_Character_Hair.png")
skin = unreal.EditorAssetLibrary.load_asset(f"{DEST}/M_Character_Export")
if skin and skin_detail:
# The character's own diffuse stays the base; the generated pore sheet is a
# detail multiplier, so the warm skin tone from the bake is preserved.
lib = unreal.MaterialEditingLibrary
clear_graph(skin)
diffuse = lib.create_material_expression(skin, unreal.MaterialExpressionTextureSample, -700, 0)
diffuse.texture = unreal.EditorAssetLibrary.load_asset(f"{DEST}/T_Character_Diffuse")
detail = lib.create_material_expression(skin, unreal.MaterialExpressionTextureSample, -700, 300)
detail.texture = skin_detail
detail.sampler_type = unreal.MaterialSamplerType.SAMPLERTYPE_LINEAR_COLOR
mul = lib.create_material_expression(skin, unreal.MaterialExpressionMultiply, -380, 120)
lib.connect_material_expressions(diffuse, "RGB", mul, "A")
lib.connect_material_expressions(detail, "RGB", mul, "B")
# Detail is a multiplier, so keep it near 1 and let it vary a little.
lerp = lib.create_material_expression(skin, unreal.MaterialExpressionLinearInterpolate, -200, 120)
lerp.set_editor_property("const_a", unreal.LinearColor(1.0, 1.0, 1.0, 1.0))
lib.connect_material_expressions(mul, "", lerp, "B")
rough = lib.create_material_expression(skin, unreal.MaterialExpressionConstant, -200, -160)
rough.set_editor_property("r", 0.55)
lib.connect_material_property(lerp, "", unreal.MaterialProperty.MP_BASE_COLOR)
lib.connect_material_property(rough, "", unreal.MaterialProperty.MP_ROUGHNESS)
lib.recompile_material(skin)
log("M_Character_Export: diffuse x pore detail, roughness 0.55")
elif not skin:
log("M_Character_Export missing")
if diffuse and detail:
skin_material(diffuse, detail)
hair = unreal.EditorAssetLibrary.load_asset(f"{DEST}/M_Character_Hair")
if hair and hair_strands:
textured_material(hair, hair_strands, 0.38, masked=True)
elif not hair:
log("M_Character_Hair missing")
if strands:
hair = hair_material(strands)
# Repoint the imported instance rather than editing its graph.
instance = unreal.EditorAssetLibrary.load_asset(f"{DEST}/M_Character_Hair")
if isinstance(instance, unreal.MaterialInstanceConstant):
instance.set_editor_property("parent", hair)
unreal.MaterialEditingLibrary.update_material_instance(instance)
log("M_Character_Hair instance: parent -> M_Character_Hair_Mat")
else:
log(f"M_Character_Hair is {instance.get_class().get_name() if instance else 'missing'}, "
"not an instance - skipped")
unreal.EditorLoadingAndSavingUtils.save_dirty_packages(True, True)
lines.append("VERDICT: PASS")