diff --git a/Docs/build.md b/Docs/build.md index 45845c5..2570f50 100644 --- a/Docs/build.md +++ b/Docs/build.md @@ -68,6 +68,34 @@ Both content-acceptance tests run headless and are the template for all asset im # Note: 5.8 swallows Python print() in the commandlet; scripts write result files to /tmp. ``` +## Content pipeline (current) + +The hero block and the demo level are built by a fixed sequence. Each step writes +a result file under `/tmp/gigline/`; read it, because the commandlet's own stdout +can be swallowed. + +| Step | Tool | Output | +| --- | --- | --- | +| OSM blockout | `Art/blender/blockout/blosm_blockout.py` | `Art/blender/blockout/blockout-*.blend` | +| Terrain | `Art/blender/terrain/terrain_from_dem.py` | `Art/blender/terrain/terrain-*.blend` | +| Street | `Art/blender/street/street_mesh.py` | `Art/blender/street/street-*.blend` | +| Site assembly | `Art/blender/site/site_assembly.py` | `Art/blender/site/site-*.blend` | +| Site export | `Art/blender/site/export_site_fbx.py` | six FBXs, one per mesh | +| Hero façade | `Tools/hero_facade_build.py` | `Art/blender/hero/hero-facade-*.blend` + JSON report | +| Site import | `Tools/ue_import_site.py` | `/Game/Hollywood/Site/*` | +| Hero import | `Tools/ue_hero_facade_import.py` | `/Game/Hollywood/ElCapitan/SM_HeroFacade` | +| Demo map | `Tools/ue_build_demo_map.py` | `/Game/Hollywood/DemoMap` | +| Review shots | `Tools/ue_build_review_map.py` + `Tools/render_screenshots.sh` | `Screenshots/` | + +Two rules that the pipeline enforces rather than documents: + +- **One conversion.** Metres to Unreal centimetres happens only in + `Geo/tools/osm_common.py::to_unreal_local`. Sidecars (`.geo.json`) record + the result and `Geo/tools/asset_metadata.py audit` re-derives it. +- **Collision is decided at import,** not later: terrain, road and sidewalks are + complex-as-simple; kerbs and props stay simple. A simple hull over a kerbed + street ramps straight across the kerb. + ## Known engine quirks (2026-11-07) - One handled `PSOPrecache` ensure fires at packaged-build exit only; not blocking. diff --git a/Docs/release-checks.md b/Docs/release-checks.md index ae04fd0..bc77de0 100644 --- a/Docs/release-checks.md +++ b/Docs/release-checks.md @@ -6,6 +6,8 @@ Record results in the table at the end of this file, with the date, the exact bu ## General checks (every release) +0. **Blender pre-check (before the engine pass).** `Art/blender/site/site_assembly.py` raycasts both sidewalks and the designated crossing on a 1 m grid and reports holes and step heights. It is evidence, not the check: the engine traversal below is still required. + 1. **Fresh checkout builds.** Clone into a clean directory, `git lfs pull`, build and package using only the documented external engine installation. No machine-specific paths, no uncommitted steps. 2. **No packaging errors.** The packaged build launches, reaches gameplay, and quits cleanly. Logs contain no missing-asset, missing-map, or shader-compile errors. 3. **Licensing.** Every new asset has provenance in `THIRD_PARTY_NOTICES.md`, or is recorded as unresolved and kept out of the build. diff --git a/Docs/setup.md b/Docs/setup.md index cffe8aa..6d85ae7 100644 --- a/Docs/setup.md +++ b/Docs/setup.md @@ -18,9 +18,10 @@ Anything marked **human step** needs credentials, licence acceptance, or a decis | 2. Engine | done — UE `5.8.4` at `/run/media/system/DATA/Engines/UE_5.8.4`, editor launches with Vulkan on the 5070 Ti | | 3. Tooling pins | done except the optional geo venv; Blender 5.2, MPFB 2.0.18, Blosm 2.7.28, QGIS 4.2.3 all verified | | 4. Project bootstrap | done — C++ project, packaged blank build, static-mesh scale test, MPFB character import | +| 6. Content pipeline | in progress — blockout, terrain, street, hero façade and the walkable `DemoMap` exist; façades and materials are planned patches (`0.1.4`) | | 5. Repository | procedure documented; a fresh checkout has not yet been exercised end to end | -Next: `0.1.0` content work (façade, block frontage, gameplay). Current board is [ROADMAP.md](../ROADMAP.md). +Next: `0.1.0` gameplay (attribution display, lighting, crossing) and the El Capitan Theatre, then the planned patches. Current board is [ROADMAP.md](../ROADMAP.md). ## 1. System diff --git a/Docs/toolchain.md b/Docs/toolchain.md index 0d07215..204586f 100644 --- a/Docs/toolchain.md +++ b/Docs/toolchain.md @@ -4,7 +4,7 @@ Pinned versions for this project. Record the exact build string a tool reports, Audited 2026-11-06, updated 2026-11-07 on the development workstation. -**State:** initial setup is complete. Every pinned tool below has been exercised on real work — the engine compiled game code and packaged a build, Blender/MPFB/Blosm produced committed assets, and QGIS's PROJ produced the project origin. The only unexercised path is a fresh checkout, tracked as a `v0.1.0` release check. +**State:** initial setup is complete. Every pinned tool below has been exercised on real work — the engine compiled game code and packaged a build, Blender/MPFB/Blosm produced committed assets, and QGIS's PROJ produced the project origin. The demo level (`/Game/Hollywood/DemoMap`) and packaged build exercise the engine path end to end; the only unexercised path is a fresh checkout, tracked as a `v0.1.0` release check. ## Pins diff --git a/MEMORY.md b/MEMORY.md index b115d8e..cdd2945 100644 --- a/MEMORY.md +++ b/MEMORY.md @@ -43,7 +43,7 @@ Scope: facts and traps that cost time to establish. Planned work lives in [ROADM - **`flatpak run` does not work here.** Call the binary inside the app tree: `B=/var/lib/flatpak/app/org.blender.Blender/x86_64/stable/active/files`, then `LD_LIBRARY_PATH="$B/blender/lib" "$B/blender/blender" -b …` with `BLENDER_USER_SCRIPTS="$HOME/.var/app/org.blender.Blender/config/blender/5.2/scripts"`. -- **Blosm needs two headless workarounds**, both already in `Tools/blosm_blockout.py`: +- **Blosm needs two headless workarounds**, both already in `Art/blender/blockout/blosm_blockout.py`: 1. it refuses to import until an asset directory is set, and the check fires before the mode is honoured; 2. in background mode the default level table is empty, and the add-on then takes a path with a broken relative import (`from gui import addDefaultLevels`). Seeding one entry avoids it. - **Blosm projects on a sphere, not an ellipsoid.** Its frame is about 0.4 % oversized and rotated by meridian convergence — roughly 2 m at the theatre. Fixed with a six-parameter affine into EPSG:26911 (0.0 cm residual). @@ -135,3 +135,8 @@ Scope: facts and traps that cost time to establish. Planned work lives in [ROADM - **`StaticMesh` has no trace-flag property in Python.** Collision complexity is set through `mesh.get_editor_property("body_setup").set_editor_property("collision_trace_flag", …)`; `StaticMeshEditorSubsystem.get_collision_complexity` reads it back but there is no setter. - **A `-game` run with a paused world still exits on timeout**, so a smoke test that "hangs" may simply be running. +## 2026-11-07 — Documentation sweep + +- **Docs drift silently after a refactor.** Moving the geo scripts to `Geo/tools/` and the Blender builders into `Art/blender/*/` left stale paths in six documents and two tool READMEs. Grep for old paths (`.md` and `.py`) after any move, not just the ones that error at runtime. +- **Every new tool needs a row in `Tools/README.md` and `Docs/build.md`**, or the next session will not know it exists. Both now list the full content pipeline in order. + diff --git a/PLAN.md b/PLAN.md index 7e91f9b..436825f 100644 --- a/PLAN.md +++ b/PLAN.md @@ -47,6 +47,8 @@ Target workstation: Bazzite Linux, Ryzen 7 7700X, 64 GB RAM, RTX 5070 Ti 16 GB. ### Repository +Progress lives in `ROADMAP.md`; session findings live in `MEMORY.md`; rendered review captures and their approval state live in `Screenshots/`. + - `Game/` — Unreal project, configuration, source, game-ready content. - `Art/` — editable Blender files and original textures. - `Geo/` — source manifests, permitted geographic data, processing scripts. diff --git a/README.md b/README.md index 41432f5..8d72eb6 100644 --- a/README.md +++ b/README.md @@ -4,17 +4,20 @@ A Linux-first Unreal Engine game about walking and driving through a real city. ## Status -Pre-production. The Unreal project, prototype content, and geographic data are tracked; the engine itself is not. Current state is tracked in [ROADMAP.md](ROADMAP.md), and the per-task setup state in [Docs/setup.md](Docs/setup.md). +`0.1.0` is in progress. The workstation, engine, geo pipeline and project bootstrap are done and verified; a walkable `DemoMap` exists with terrain, street, sidewalks, kerbs and the hero façade placed from measured coordinates. Façades are still blockouts, and materials are flat colours — both are planned patches. + +Current state is in [ROADMAP.md](ROADMAP.md); what was learned the hard way is in [MEMORY.md](MEMORY.md). ## Layout | Path | Contents | | --- | --- | | `Game/` | Unreal project, configuration, source, game-ready content | -| `Art/` | Editable Blender files and original textures | -| `Geo/` | Source manifests, permitted geographic data, processing scripts | -| `Tools/` | Repeatable import/export helpers | -| `Docs/` | Setup, toolchain pins, licensing, release checks | +| `Art/` | Editable Blender sources, blockouts, terrain, street, hero façade, and a local (uncommitted) reference library | +| `Geo/` | Source manifests, permitted geographic data, and every geographic tool | +| `Tools/` | Unreal-side importers, level builders, and reference gathering | +| `Docs/` | Setup, toolchain pins, build and content pipeline, licensing, release checks | +| `Screenshots/` | Rendered review captures and their approval state | ## Getting started @@ -24,19 +27,20 @@ git lfs pull Tools/check-lfs.sh # verifies the lfs filter is active and files are stored as pointers ``` -The engine is installed outside the repository and pinned in [Docs/toolchain.md](Docs/toolchain.md). The full procedure is in [Docs/setup.md](Docs/setup.md). +The engine is installed outside the repository and pinned in [Docs/toolchain.md](Docs/toolchain.md). Setup is in [Docs/setup.md](Docs/setup.md); building and the content pipeline are in [Docs/build.md](Docs/build.md). ## Documents - [ROADMAP.md](ROADMAP.md) — living task list, source of truth for progress -- [PLAN.md](PLAN.md) — scope, stack, release policy, timeline +- [MEMORY.md](MEMORY.md) — shared session memory: discoveries, traps, decisions - [AGENTS.md](AGENTS.md) — contributor and agent rules -- [MEMORY.md](MEMORY.md) — shared session memory: discoveries, gotchas, bugs -- [Docs/toolchain.md](Docs/toolchain.md) — pinned versions -- [Docs/release-checks.md](Docs/release-checks.md) — release gate definitions -- [Docs/licensing.md](Docs/licensing.md) and [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) — licensing +- [PLAN.md](PLAN.md) — scope, stack, release policy, timeline +- [Docs/toolchain.md](Docs/toolchain.md) · [Docs/environment.md](Docs/environment.md) — pins and workstation audit +- [Docs/build.md](Docs/build.md) — fresh-checkout build and content pipeline +- [Docs/release-checks.md](Docs/release-checks.md) — release gates +- [Docs/licensing.md](Docs/licensing.md) · [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) — licensing - [CHANGELOG.md](CHANGELOG.md) — release history ## Licensing -Project-owned code, art, and documentation are dedicated under [CC0 1.0](LICENSE). The engine, third-party assets, and geographic data have separate terms; see [Docs/licensing.md](Docs/licensing.md). OSM-derived output carries "© OpenStreetMap contributors". +Code licence is an open decision (MIT proposed). Engine, art, and geographic data are licensed separately; see [Docs/licensing.md](Docs/licensing.md). OSM-derived output carries "© OpenStreetMap contributors". diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.md index e31c78e..bc532cb 100644 --- a/THIRD_PARTY_NOTICES.md +++ b/THIRD_PARTY_NOTICES.md @@ -50,3 +50,7 @@ Rules that still apply, because they are about shipped output rather than local | Character mesh, standard rig, and baked skin | MPFB 2.0.18 bundled base mesh, rig, and textures; `Tools/mpfb_character_export.py` | CC0 1.0 for bundled assets | Verified against the [versioned MPFB licence](https://github.com/makehumancommunity/mpfb2/blob/v2.0.18/LICENSE.md); imported character assets and Blender source are tracked. | | OSM blockout | OSM extract and Blosm-generated geometry | ODbL 1.0 for source data | Source attribution above; review ODbL obligations for distributed derived databases. | | Terrain and street prototype | USGS 3DEP DEM, OSM data, project scripts | Public domain USGS data; ODbL 1.0 for OSM data | Source manifests in `Geo/manifests/`; review ODbL obligations for distributed derived databases. | + +| Hero façade blockout | Project-owned geometry from `Tools/hero_facade_build.py`, dimensions from OSM and the reference photography | Project-owned; ODbL 1.0 applies to the OSM measurements it derives from | Committed with a per-part evidence report (`Art/blender/hero/hero-facade-*.json`) and a position sidecar (`hero-facade.geo.json`). | +| Street props (candidate reuse) | Sibling project `…/06-games/security-game`, `assets/models/blender_src/*.blend` | Project-owned by the same author | **Not yet imported.** Sources copied to `Art/props/blender_src/` with `Art/props/reuse-manifest.json`; a licence statement is required before any of them ship. | + diff --git a/Tools/README.md b/Tools/README.md index d74488b..29279ac 100644 --- a/Tools/README.md +++ b/Tools/README.md @@ -1,26 +1,52 @@ # Tools -Repeatable import/export and processing helpers. Anything that was done twice by hand belongs here. +Repeatable helpers that drive Blender, Unreal, and the reference/asset sources. Nothing here is hand-run twice: every step that produced committed content has a script. + +Geographic tools live in [`Geo/tools/`](../Geo/README.md) and the Blender-side builders live next to their output in `Art/blender/*/`; this folder holds everything that talks to Unreal, the network, or the repository. ## Conventions -- One script, one job. No interactive prompts; every input is an argument. -- Each script documents its input format, output format, and the exact tool version it was tested with. -- Blender scripts run headless (`blender --background --python`), so they can be re-run for a regression check. -- No credentials, no absolute machine paths, no writes outside the project by default. -- A script that produces game content logs what it produced, so `Game/Content` can be audited against it. +- One script, one job. Every input is an argument; no interactive prompts. +- Absolute paths where a tool crosses into a sandbox (Blender Flatpak, the Unreal commandlet). Those processes do not share the repo's working directory. +- Unreal commandlets swallow `print()`, so each writes a result file under `/tmp/gigline/`. Read the file; do not trust the exit code alone. +- No credentials, no writes outside the project by default. -## Helpers +## Repository and assets -- `render_screenshots.sh` — visual-review pipeline: rebuilds `/Game/Test/ReviewMap` from current content (`ue_build_review_map.py`), runs the game on it in `-game` mode, lets `AGigLineShotActor` (`Game/GigLine/Source/GigLine/`) capture the fixed camera list, and copies the PNGs into the gitignored `Screenshots/`. See `Screenshots/README.md`. -- `check-lfs.sh` — repository guard: fails if `git-lfs` or the `lfs` filter is missing, or if an LFS-tracked file was committed as a plain blob. -- Geographic tools and their shared helpers live in `Geo/tools/` (see `Geo/README.md`). -- Blender-side builders that turn geo data into meshes live beside their output: `Art/blender/terrain/terrain_from_dem.py`, `Art/blender/street/street_mesh.py`, `Art/blender/site/site_assembly.py`, `Art/blender/blockout/blosm_blockout.py`. They import their helpers from `Geo/tools/osm_common.py`, which is also the only place that converts metres to Unreal centimetres. -- `sibling_asset_audit.py` — reads the sibling project's prop library straight from GLB headers and writes `Art/props/reuse-manifest.json`; copies nothing. -- `page_image_fetch.py` — resolves a web page's `og:image` and downloads it, with page URL and hash in a local sidecar. Works for Flickr, Commons file pages, and most photo blogs. -- `commons_fetch.py` — Wikimedia Commons reference fetcher. Search, exact `File:` titles, or a whole Commons category (`--category`), thumbnails, provenance `sidecar.json` per folder. Wikimedia throttles `upload.wikimedia.org` (HTTP 429), so keep batches to six or fewer files and pass `--delay 3`; a throttled file is skipped and recorded rather than aborting the run. Writes only into the gitignored `Art/reference/`; reference images are never committed. +| Tool | Purpose | +| --- | --- | +| `check-lfs.sh` | Fails if the `lfs` filter is missing or an LFS-tracked file was committed as a plain blob | +| `sibling_asset_audit.py` | Reads the sibling project's prop library from GLB headers into `Art/props/reuse-manifest.json`; copies nothing | +| `scale_test_export.py` / `ue_scale_test_import.py` | The 1 m = 100 uu scale test; the named presets `ue_metric_v1` live in these two files | -## Planned helpers -- Blender export preset runner (named presets, metric scale assertion) -- OSM extract fetch and manifest writer (records date, bounds, checksum) -- QGIS processing steps as batch scripts, so the geo cleanup is reproducible rather than GUI-only +## Reference gathering + +| Tool | Purpose | +| --- | --- | +| `commons_fetch.py` | Wikimedia Commons: search, exact `File:` titles, or a whole category; writes a provenance `sidecar.json` | +| `page_image_fetch.py` | Resolves a web page's `og:image` and downloads it (Flickr, Commons file pages, photo blogs) | +| `addon_compat_probe.py` | Verifies MPFB/Blosm can load and generate headless on the pinned Blender | + +## Blender content + +| Tool | Purpose | +| --- | --- | +| `hero_facade_build.py` | Parametric hero façade; emits a JSON report naming the evidence behind every part | +| `mpfb_character_export.py` | MPFB human, standard rig, baked skin, FBX export | + +## Unreal import and level building + +| Tool | Purpose | +| --- | --- | +| `ue_import_site.py` | Imports the six site FBXs into `/Game/Hollywood/Site`, setting collision complexity per mesh | +| `ue_hero_facade_import.py` | Imports the façade and places it on the measured building line | +| `ue_build_demo_map.py` | Builds `/Game/Hollywood/DemoMap`: lighting, façade, site meshes, player start | +| `ue_build_review_map.py` | Rebuilds `Test/ReviewMap`, stages the shot camera list, and stages the demo-map views | +| `ue_create_input_assets.py` | Creates `/Game/Input` actions and the mapping context headlessly | +| `ue_character_import.py` | Skeletal import plus the material/texture chain Interchange skips | + +## Visual review + +| Tool | Purpose | +| --- | --- | +| `render_screenshots.py` / `render_screenshots.sh` | Rebuilds the review map, runs the game, captures the shot list into `Screenshots/` |