gig-line/Tools/README.md

Tools

Repeatable helpers that drive Blender, Unreal, and the reference/asset sources. Nothing here is hand-run twice: every step that produced committed content has a script.

Geographic tools live in Geo/tools/ and the Blender-side builders live next to their output in Art/blender/*/; this folder holds everything that talks to Unreal, the network, or the repository.

Layout

Sixty scripts in one directory stopped being navigable, so they are grouped by what they talk to. The folder is also the sandbox boundary: everything in unreal/ needs the engine, everything in blender/ needs the Flatpak, and the rest is host Python.

Folder	Holds	Needs
unreal/	Unreal commandlet scripts and the two map builders; ue_common.py and ue_placement.py are the shared imports	the engine pin from Docs/toolchain.md
blender/	Blender-side builders and exporters, the FBX preset, the MPFB/MakeHuman character pipeline, blender_screenshots.py	Blender Flatpak
review/	The visual loop: render_screenshots.sh, judge_screenshot.py, hero_compare.py, build_comparison_md.py, e2e_asset_check.py, hero_iterate.sh, visual_loop.py, visual_targets.json	engine + optional OpenRouter key
gen/	Image and texture generation: local ComfyUI driver, deterministic procedural texture writers, reference fetching	local ComfyUI / network
input/	input_emulation.py, input_test.py, run_pie.sh — OS-level input injection	/dev/uinput
_common.py, tests/	Shared helpers and the stdlib test suite	none

Scripts resolve the repository root as ../.. from their own folder and put the tools root and their siblings on sys.path, so import ue_common keeps working from unreal/.

Conventions

• One script, one job. Every input is an argument; no interactive prompts.
• Absolute paths where a tool crosses into a sandbox (Blender Flatpak, the Unreal commandlet). Those processes do not share the repo's working directory.
• Unreal commandlets swallow print(), so each writes a result file under /tmp/gigline/. Read the file; do not trust the exit code alone.
• No credentials, no writes outside the project by default.

Repository and assets

Tool	Purpose
check-lfs.sh	Fails if the lfs filter is missing or an LFS-tracked file was committed as a plain blob
blender/facade_kit_build.py	Builds the reusable façade module kit at the measured 4.96 m bay: pilaster, sill, lintel, window bay (surround + glazing), shopfront bay (piers, glass, fascia), door bay (jambs, door, threshold), cornice and parapet + coping. Exports each piece with the shared preset plus a JSON part report (dims + evidence per piece).
unreal/ue_import_retail_detail.py	Imports the two retail trim meshes into /Game/Hollywood/Frontage with complex-as-simple collision — the trim decorates a solid the player already collides with, so it must not add snag points. Staged by the demo builder as RetailTrim_*.
unreal/ue_import_facade_kit.py	Imports the kit into /Game/Hollywood/Kit/* as static meshes with complex-as-simple collision; writes /tmp/gigline/kit-import.txt.
blender/retail_frontage_build.py	Retail frontage trim for 6800 Lee Drug and 6806–6820 (0.1.6): bronze cornice, mural row, lettering band, stepped Deco parapet and rounded corner tower on Lee Drug; burgundy fascia, pilasters, white-framed windows and banded parapet on 6806–6820. Built in world coordinates from the blockout bounds, one joined mesh per building, JSON part report with the evidence behind every piece. 6822 Budget is not built — no photograph exists.
blender/retail_frontage_build.py	Retail frontage trim for 6800 Lee Drug and 6806–6820 (0.1.6): bronze cornice, mural row, lettering band, stepped Deco parapet and rounded corner tower on Lee Drug; burgundy fascia, pilasters, white-framed windows and banded parapet on 6806–6820. Built in world coordinates from the blockout bounds, one joined mesh per building, JSON part report with the evidence behind every piece. 6822 Budget is not built — no photograph exists.
blender/theatre_trim_build.py	El Capitan Theatre trim: projecting marquee canopy, illuminated fascia, recessed soffit, vertical blade sign with frame and bracket. Built in world coordinates (span and pavement height baked in) with a JSON part report, so staging is a single spawn at the origin.
unreal/ue_stage_theatre_trim.py	Imports the trim to /Game/Hollywood/Theatre and stages one actor per piece in the DemoMap with site materials; clears a previous run first.
unreal/ue_apply_facade_kit.py	Lays the kit over a building's street face from its own bounding box: bay count = round(width / 4.96), storeys from the height, shopfronts on the ground floor, windows above, pilasters on bay boundaries, cornice and parapet at the top. Clears a previous application first; unique labels per building.
gen/generate_facade_textures.py	Deterministic stdlib PNG generator for the facade maps (T_StoneAshlar, T_Stucco, T_Asphalt, T_Concrete, each _BC base colour + _R roughness). Pure-zlib writer, hash-based value noise, identical bytes on every run; --size and --out (/tmp/gigline/textures by default). ue_build_demo_map.py imports them into /Game/Hollywood/Textures and samples them world-aligned.
unreal/ue_audit_scene.py	Gate on the DemoMap: no unmaterialed actor, every verified frontage span within tolerance (ue_placement), no duplicate actor labels. Writes /tmp/gigline/scene-audit.txt; render_screenshots.sh --verify fails on a non-zero failure count. Catches the class of defect that still renders plausibly.
unreal/ue_placement.py	Single source of truth for where staged actors belong in the DemoMap (hero position, verified frontage spans) plus span_error/describe_span_failure validators. Exists because the hero position was written twice and drifted — the importer kept the pre-0.1.0 value and moved the hero 13 m east onto the wrong side of the boulevard. No unreal import, so it is unit-tested.
unreal/ue_common.py	Shared helpers for the Unreal commandlet scripts: make_writable() clears the read-only bit an LFS .umap checks out with (otherwise save_map() returns False silently), and require_saved() makes that failure explicit.
blender/fbx_export_preset.py	The single FBX export preset (ue_metric_v1 / ue_character_v1): metres→centimetres via FBX_SCALE_NONE, axis mapping east +X / north −Y / up +Z, mesh_smooth_type="FACE" (smoothing groups survive the import). Every Blender exporter imports it instead of copying the argument block; preset_kwargs() is bpy-free and unit-tested.
sibling_asset_audit.py	Reads the sibling project's prop library from GLB headers into Art/props/reuse-manifest.json; copies nothing
scale_test_export.py / ue_scale_test_import.py	The 1 m = 100 uu scale test; the named presets ue_metric_v1 live in these two files

Reference gathering

Tool	Purpose
gen/commons_fetch.py	Wikimedia Commons: search, exact File: titles, or a whole category; writes a provenance sidecar.json
gen/page_image_fetch.py	Resolves a web page's og:image and downloads it (Flickr, Commons file pages, photo blogs)
addon_compat_probe.py	Verifies MPFB/Blosm can load and generate headless on the pinned Blender

Blender content

Tool	Purpose
blender/hero_facade_build.py	Parametric hero façade; emits a JSON report naming the evidence behind every part
blender/character_anim_build.py	Authors the basic animation set on the MPFB rig — GigLine_Idle (60 f) and GigLine_Walk (30 f) sampled from parametric cycles at 30 fps — and exports one animated FBX with the ue_character_v1 preset. Rotation axes are measured on the rig, not assumed (probe in the docstring). --out, --report.
blender/mpfb_character_export.py	MPFB human, standard rig, baked skin, FBX export

Unreal import and level building

Tool	Purpose
unreal/ue_import_site.py	Imports the six site FBXs into /Game/Hollywood/Site, setting collision complexity per mesh
unreal/ue_hero_facade_import.py	Imports the façade and places it on the measured building line
unreal/ue_build_demo_map.py	Builds /Game/Hollywood/DemoMap: lighting, façade, site meshes, player start
review/hero_iterate.sh	Full hero-facade loop in one command: chmod the Blender files, build the facade, import it, render the capture set, judge the shot and write a comparison plate. bash Tools/hero_iterate.sh [shot].
review/hero_compare.py	Side-by-side plate of a reference photograph and a rendered capture, scaled to one height and labelled. --reference, --render, --out.
review/e2e_asset_check.py	End-to-end check in one command: Blender renders → both Unreal maps rebuilt → captures for both the game map and the staging map → two-model judge → Screenshots/README.md regenerated. --skip-blender, --skip-judge, --skip-recreate. Report in /tmp/gigline/e2e-report.txt; non-zero exit if any step failed.
unreal/ue_recreate_maps.py	Deletes DemoMap and ReviewMap and creates both as empty levels, so a run never inherits an actor from an experiment. Chmods the LFS-read-only .umap first and parks the editor on a scratch level when it has the map open.
unreal/ue_clear_simple_collision.py	Removes the FBX import's auto-generated convex hulls from the site and street meshes and sets them to complex-as-simple, so physics bodies are not pushed out of a 279 m hull.
unreal/ue_vehicle_probe_map.py	Copies ReviewMap to VehicleProbe, optionally adds one demo mesh and the Chaos pawn, and saves it, to bisect which map content throws a physics body into the air. Set GIGLINE_PROBE_ASSET to stage one mesh; unset for the control.
unreal/ue_build_review_map.py	Rebuilds Test/ReviewMap, stages the shot camera list, and stages the demo-map views
unreal/ue_create_input_assets.py	Creates /Game/Input actions and the mapping context headlessly
unreal/ue_character_anim_import.py	Imports the animated FBX into /Game/Test/Character and verifies an AnimSequence exists — an FBX exported without bake_anim imports a mesh and no clips, with no error. Writes VERDICT: PASS/FAIL to /tmp/gigline/character-anim-import.txt.
unreal/ue_character_import.py	Skeletal import plus the material/texture chain Interchange skips

Visual review

Tool	Purpose
render_screenshots.py / render_screenshots.sh	Rebuilds the review map, runs the game, captures the shot list into Screenshots/
review/judge_screenshot.py	Vision-model pre-check: judges a screenshot against an expected-scene description (--image + --expect, or --shots for every shot in the README). Reads OPENROUTER_API_KEY from .env (ignored). Judged by two models by default (--models a,b), reconciled: agreement carries the verdict, disagreement reports uncertain with both reasons. Optional, advisory only — never replaces a human verdict.
review/visual_loop.py	The reference-vs-render loop. --verify gates each capture against its expectation in visual_targets.json (blank/short/wrong-size/tone-less shots fail); --compare measures luminance, contrast, edge density, palette and 3×3 composition against the reference photograph and turns the deltas into modelling actions naming the skill to consult (blender-modeler, archviz, materials, lookdev, camera-cinematography); --iteration N writes the artefacts under Art/reference/iterations/N/, which is gitignored until the owner confirms a shot.
review/visual_targets.json	Per-shot expectation and reference photograph for visual_loop.py. Every entry must be a demo-map shot.

Gameplay QA

Command	Purpose
UnrealEditor <uproject> /Game/Hollywood/DemoMap -game -qaselftest	Headless behaviour check: drives forward input for 2 s, then resets and pauses. Logs GigLine QA: ... PASS/FAIL lines and a final RESULT. Runs only with -qaselftest, so normal play is unaffected.
-qaselftest phases	Walk, reset, pause/unpause, a 16-row seam transect (0 holes), a ground-referenced wall probe (0/35) and a facade occlusion probe that fires sideways from the road centre and requires a blocking wall wherever a building is staged (0/9), plus a frame-time capture writing Saved/Profiling/perf-walk-frametimes.csv and logging GigLine PERF:.
gigline.SetupLogs cvar	Spawn/input setup logging, off by default (DefaultEngine.ini [ConsoleVariables]); enable at runtime with gigline.SetupLogs 1.

Reference and concept generation

Tool	Purpose
gen/comfyui_generate.py	Drives the local ComfyUI at http://127.0.0.1:8188 (Stability Matrix install). models lists checkpoints; txt2img renders through the HTTP API; texture renders material sheets with the project's documented texture workflows (see below). Every run writes a provenance sidecar (model, prompt, seed, settings, full graph). Two candidates by default — same prompt and seed, both scored by high-frequency detail, the winner takes --out, the loser is kept beside it. Local and human-triggered; no credential.
gen/generate_facade_textures.py	Procedural, deterministic facade texture maps (ashlar base colour + roughness, stucco, asphalt, concrete) written with a pure-zlib PNG encoder — no model, no network, byte-identical on every run.
gen/generate_character_textures.py	Procedural character texture maps, same deterministic approach as the facade set.
gen/generate_banner_texture.py	Draws the hero-facade banner (JIMMY KIMMEL LIVE in gold on crimson) with Pillow + a system font — exact text, deterministic, no model. An image model would mangle a trademark string, and Qwen-Image 2.1 is research-only, so this is the lettering path. --out Art/textures/generated/T_Banner_KIMMEL.png by default.
gen/facade_detail_concepts.py	Generates frontal building plates (whole elevation, entrance in its facade, or building with neighbours) from a local reference photograph with an image model (OpenRouter); close-up detail crops are deliberately not requested because they cannot be measured for proportion. --list is free and shows which studies have their reference present; --all/--only <name> generates. Never overwrites (writes -v2, -v3), writes a .refgen.json sidecar per image (model, prompt, source photo, SHA-256), and puts everything under Art/reference/facade-details/ — gitignored, local-only, never shipped. Costs money per image, so it is never automatic.
review/judge_screenshot.py	(see Visual review)

Texture generation workflows (from the Stability Matrix install)

The install documents four verified, core-node graphs in Workflows/texture_generation.md. Those files are ComfyUI UI format, which /prompt cannot accept, so comfyui_generate.py texture rebuilds the equivalent API graphs with exactly their settings:

Profile	Model	Steps / CFG / sampler	VAE	Best for	Licence
albedobase	albedobaseXL_v31Large.safetensors	30 / 6.0 / dpmpp_2m karras	baked	stone, brick, metal, wood albedo	Civitai — commercial, credit required
krea2	krea2_turbo_nvfp4.safetensors	8 / 1.0 / euler simple	texture-fix (Qwen 1 / Krea-2)	fast stochastic detail: stone, bark, foliage, fabric	Krea-2 community licence
qwen21	qwen_image_2.1_int8_convrot.safetensors	25 / 1.0 / euler simple	texture-fix (Qwen 2.1)	material sheets with readable labels/signage, 2K	Qwen research — non-commercial

python3 Tools/gen/comfyui_generate.py texture \
    --material "weathered grey limestone ashlar, fine grain" \
    --out Art/textures/generated/T_StoneAshlar_AI.png

The default pair is albedobase + krea2 — both commercially usable — generated from one prompt and seed, scored, and compared before the winner takes --out. --profile/--alt-profile pick another pair, and --alt-profile "" runs one.

Limits this tool inherits from the workflows, and does not hide:

• The graphs produce a colour/albedo sheet only. Normal, roughness and metallic maps are derived downstream with a material tool.
• ComfyUI 0.44 core has no seamless-tiling sampler, so "seamless tileable" in the prompt is a hint, not a guarantee. Offset the image 50 % and check for a seam before using it on a tiled surface; the installed SDXL tile ControlNet is the fix-up path.
• A generated texture ships only once its model and licence are in THIRD_PARTY_NOTICES.md.

Local image generation: start Stability Matrix first

comfyui_generate.py talks to ComfyUI on http://127.0.0.1:8188. That server is not started by this repository — it lives in a Stability Matrix install and must already be running.

If the tool reports a connection error, start Stability Matrix and launch ComfyUI, then re-run. The install is documented in /home/zwitschi/StabilityMatrix/Data/README.md: package version, launch arguments, and the shared model library (Models/, ~80 GB, mapped in through extra_model_paths.yaml). Reading that file before writing a prompt is the cheap way to see which checkpoints actually exist — python3 Tools/gen/comfyui_generate.py models asks the running server the same question, and it is the authoritative list — the default pair is juggernautXL_ragnarok.safetensors and epicrealismXL_pureFix.safetensors, both verified installed.

Checkpoints are not committed. A generated texture ships only after its model and licence are recorded in THIRD_PARTY_NOTICES.md; the sidecar carries what that needs.

Input injection

Tool	Purpose
input/input_emulation.py	Pure-stdlib ctypes/fcntl driver that creates a virtual keyboard+mouse on /dev/uinput. The desktop treats it as real hardware, so it proves whether OS input actually reaches the game. Needs /dev/uinput writable (ACL on this host).
input/input_test.py	Injects a scripted sequence into the focused window: --sequence move:2,look:3,jump:1,reset:1,pause:1,click.
gen/comfyui_generate.py	Local ComfyUI driver: list checkpoints, run txt2img through the HTTP API, and write a provenance sidecar (model, prompt, seed, full graph). Two checkpoints by default: both are generated from the same prompt and seed, scored by high-frequency detail, and the better one takes --out while the loser is kept beside it. Local, human-triggered.
blender/comfyui_character_blueprint.py	Generates a character blueprint sheet - front, side, back and head views plus a contact sheet - from one description, with fixed seeds and a JSON record.
blender/character_match.py	Compares a rendered character against a blueprint: figure aspect, fill and coverage deltas, mean CIE76 colour distance, dominant-palette distance, and a side-by-side montage.
gen/generate_character_textures.py	Procedural skin-pore and hair-strand textures with sidecars; no model involved.
blender/vehicle_build.py	Models the drivable sedan: three-tier body, four wheels, lamps and bumpers, plus four UCX_ convex collision hulls; saves the .blend, exports an FBX with the ue_metric_v1 preset, and writes a report with every dimension the vehicle config needs.
unreal/ue_vehicle_import.py	Imports the sedan's six systems into /Game/Hollywood/Vehicle as separate static meshes, plus the two LOD meshes; the body carries the UCX_ hulls as simple collision.
blender/vehicle_from_template.py	Converts the owner's hot hatch.blend into UE-ready assets: normalises the scale to a documented 4.05 m, splits systems (body/trim/glass/lights/wheels), adds a UCX_ hull, decimates two LODs, exports FBX per system and writes a report.
blender/blender_screenshots.py	Renders review screenshots out of a .blend: one per mesh object (--mode objects) and four turntable angles (--mode scene), framed from the scene's own bounds, into Screenshots/blender/.
review/build_comparison_md.py	Regenerates the Blender vs in-game section of Screenshots/README.md between <!-- blender-comparison:begin/end --> markers: in-game captures beside their Blender renders in one table, plus a signoff table with one row per in-game screen. Pairs by name with an alias map for camera-named shots. Run it after every capture or Blender render.
review/shadow_gap.py	Measures the vertical pixel gap between an object's lowest pixels and the top of its shadow, as a fraction of the object's height, and calls it contact or floating. Objective replacement for asking a vision model whether something floats.

Tests

python3 -m pip install -r requirements-tools.txt
python3 -m unittest discover -s Tools/tests -t Tools -p 'test_*.py'

Stdlib unittest, no Blender/Unreal process. See tests/README.md: test the pure parts (encoding, classification, maths, parsing); if logic is buried in main(), extract it into a function so it can be tested.