diff --git a/.gitignore b/.gitignore index 003d0b3..c8803aa 100644 --- a/.gitignore +++ b/.gitignore @@ -43,3 +43,6 @@ Thumbs.db # Local, private reference material — never committed (see Docs/reference-notes-hollywood.md) /Art/reference/ + +# Blender/Blosm working caches +/Art/blender/.cache/ diff --git a/Art/blender/README.md b/Art/blender/README.md new file mode 100644 index 0000000..e54bfaa --- /dev/null +++ b/Art/blender/README.md @@ -0,0 +1,55 @@ +# Blender sources + +Editable `.blend` files. Everything here is regenerable or hand-editable, and every generated file records the exact command that produced it. + +## Blockout + +`blockout/blockout-2026-11-06.blend` — OSM blockout for the demo bounds, imported with Blosm from the committed extract. 276 objects, 87 meshes, in collections split by feature (`…_buildings`, `…_roads_secondary`, `…_paths_footway`, `…_roads_primary`, …). + +### Rerun command + +The Flatpak Blender cannot be launched with `flatpak run` from an agent shell, so the binary inside the app tree is called directly with the Flatpak user scripts path. This is exact, not approximate: + +```bash +B=/var/lib/flatpak/app/org.blender.Blender/x86_64/stable/active/files +C="$HOME/.var/app/org.blender.Blender/config/blender/5.2" + +LD_LIBRARY_PATH="$B/blender/lib" \ +BLENDER_USER_SCRIPTS="$C/scripts" \ +"$B/blender/blender" -b --factory-startup \ + --python Tools/blosm_blockout.py -- \ + --osm Geo/raw/osm-api-map-2026-11-06.osm \ + --bounds 34.1006,-118.3422,34.1020,-118.3370 \ + --assets-dir /run/media/system/DATA/Downloads/blosm-assets/assets \ + --out Art/blender/blockout/blockout-2026-11-06.blend +``` + +Run time is about 36 s for this extent. Two headless-only workarounds are baked into the script and must stay there: + +1. Blosm refuses to import until an asset directory is set; the paid asset pack path is passed explicitly and is **not** committed. +2. In background mode the GUI never fills Blosm's default level table, and the add-on then takes a code path with a broken relative import. The script seeds one entry. + +## Scale check (blockout, 2026-11-06) + +Blender works in metres, and the imported meshes match the OSM geometry, so the blockout is 1:1 before any Unreal export: + +| Building | Measured in Blender | OSM geometry | Height source | +| --- | --- | --- | --- | +| El Capitan Entertainment Centre | 25.68 × 54.88 × 14.50 m | 25.75 × 54.88 m | `height` tag 14.5 | +| El Capitan Theatre | 38.59 × 54.87 × 20.62 m | 38.59 × 54.87 m | `building:levels=6` × 3.2 m, plus roof | +| Madame Tussaud's | 42.41 × 57.20 × 15.90 m | 42.0 m frontage | `height` tag 15.9 | +| TCL Chinese Theatre | 43.75 × 76.19 × 18.10 m | 44.0 m frontage | `height` tag 18.1 | + +The 0.07 m width difference on the hero is Blosm's own projection of the outline, not a units problem. Anything larger than a few centimetres would be a flag. + +Buildings **without** a `height` tag get levels × 3.2 m, which is a placeholder, not evidence. Every such height is a modelling decision to be replaced from the reference photography. + +## Relation completeness + +The extract is complete for the block but not for every relation inside its bbox: Blosm's optional member fetch pulled six ways from Overpass, all belonging to one 213 × 152 m multipart complex about 100 m north of the boulevard — outside the `0.1.0` bounds and out of scope. That fetch is therefore **off by default** (`--load-missing` enables it), so the committed blockout is reproducible offline from the committed extract alone. If background massing later needs that complex, refetch a wider extract rather than reintroducing a network dependency. + +## What this blockout is not + +- Not cleaned, not subdivided into façade modules, not UV-mapped, not detailed. +- Materials are placeholders from Blosm's simple mode. Realistic mode needs the paid material pack and was not run. +- Terrain, kerbs, and crossing ramps are not in it: those come from the DEM cross-sections in `Geo/verification/2026-11-06-cross-sections.md`. diff --git a/Art/blender/blockout/blockout-2026-11-06.blend b/Art/blender/blockout/blockout-2026-11-06.blend new file mode 100644 index 0000000..a87d37d --- /dev/null +++ b/Art/blender/blockout/blockout-2026-11-06.blend @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6e50ca1a5c9e5c8513719d1dfed6ba92987221db115924dbd38e68ed19201d59 +size 250112 diff --git a/Docs/setup.md b/Docs/setup.md index 365bf25..eeb05ef 100644 --- a/Docs/setup.md +++ b/Docs/setup.md @@ -47,7 +47,7 @@ The archive on disk is **Unreal Engine 5.2.0**, released May 2023, three and a h - `[x]` Blender 5.2 LTS (Flatpak) and MPFB 2.0.18 recorded. - `[x]` Install QGIS and record its version. QGIS **4.2.3**, Flatpak `stable`, runtime `org.kde.Platform 6.11`, bundled PROJ `9.9.0`. The stale `lts` 3.28.9 install is still beside it and should be uninstalled. -- `[ ]` Install Blosm in Blender, record its version, and confirm it loads on Blender 5.2. Fall back to a side-by-side Blender 4.5 LTS only for the blockout step if it does not. +- `[x]` Install Blosm in Blender, record its version, and confirm it loads on Blender 5.2. Blosm **2.7.28** imports the block extract headless in `3Dsimple` mode; the exact command and the two required headless workarounds are in `Art/blender/README.md`. No second Blender version was needed. - `[ ]` Confirm the geo tool Python environment (venv with `pyproj`, `GDAL`, `shapely` from the QGIS or distrobox environment) and record it. ## 4. Project bootstrap diff --git a/Docs/toolchain.md b/Docs/toolchain.md index 1fcb6a1..a859982 100644 --- a/Docs/toolchain.md +++ b/Docs/toolchain.md @@ -12,9 +12,9 @@ Audited 2026-11-06 on the development workstation. | NVIDIA driver | `615.78.08`, open kernel module | Bazzite image | Verified: desktop session renders through the RTX 5070 Ti; Vulkan GPU0 is the RTX 5070 Ti on driver `615.78.08`, conformance `1.4.3.3`. | | Unreal Engine | `5.8.4` — download in progress 2026-11-06; `5.2.0` archive verified on disk as fallback | Epic download | Not extracted yet. Pin becomes 5.8.4 once the archive verifies and the editor launches on Bazzite; see [setup.md](setup.md#engine-version-decision). | | Unreal Linux toolchain | engine-bundled clang | Engine source tree | Not installed. Do not substitute the system compiler. | -| Blender | `5.2` LTS (upstream patch `5.2.3`) | Flatpak `org.blender.Blender` | Installed. | +| Blender | `5.2.0` LTS (build 2026-07-14) | Flatpak `org.blender.Blender` | Installed. Run headless from the app tree with `LD_LIBRARY_PATH` + `BLENDER_USER_SCRIPTS`; `flatpak run` is unavailable to the agent shell. | | MPFB | `2.0.18` (extension `mpfb`, declares Blender >= 4.2) | Blender extensions platform | Installed. Load-and-export check on 5.2 still pending. | -| Blosm | latest release at install time | blosm.com / Gumroad | Not installed. Version must be recorded before first use. | +| Blosm | `2.7.28` (legacy add-on, `bl_info.blender = (4, 5, 0)`, manifest min 4.2) | blosm.com | Installed and importing on Blender 5.2 in `3Dsimple` mode. Two headless workarounds are required; see `Art/blender/README.md`. Paid material asset pack lives in `~/Downloads/blosm-assets/` and is not committed. | | QGIS | `4.2.3`, Flatpak `stable`, runtime `org.kde.Platform 6.11`; bundled PROJ `9.9.0` | flathub | Installed 2026-11-06. The stale `lts` branch (`3.28.9`, EOL Qt5 runtime) is still installed beside it and should be removed. | | Git | `2.56.0` | Bazzite image | Installed. | | Git LFS | `3.7.1` | `rpm-ostree` layer | Installed; `git lfs install` run, lfs filter active. | @@ -34,7 +34,7 @@ Unreal Engine 5.9 is in preview and Unreal Engine 6 has been announced as of Nov ## Open compatibility risks -1. **Blosm on Blender 5.2.** Blosm versions historically lag Blender releases. Verify Blosm against the installed Blender 5.2 before the blockout step. If it fails, add a side-by-side Blender 4.5 LTS Flatpak and convert that step's output only; do not mix Blender versions within one asset. +1. **Blosm on Blender 5.2 — worked around, not resolved.** `3Dsimple` import works headless (see the blockout). Two headless-only defects are patched around in the script rather than in the add-on: an asset-directory check that fires before `mode` is honoured, and a background-mode import path that uses a broken relative import when the default level table is empty. `3Drealistic` mode has not been exercised; it loads the paid material pack and previously failed inside the realistic renderer setup. If realistic materials are wanted, test that mode separately before depending on it. 2. **MPFB 2.0.18 on Blender 5.2.** The extension declares Blender >= 4.2 but predates 5.x. The acceptance test is the `0.1.0` requirement to export one MPFB character with baked materials and a deform skeleton and import it into Unreal. 3. **Two QGIS installs.** `stable` (4.2.3, current runtime) and the old `lts` (3.28.9, EOL runtime) are both present. Nothing breaks, but launching the wrong one silently gives older PROJ data. Uninstall `lts` and keep one pin. QGIS 4.x also uses Qt6, so any processing recipe copied from a 3.x tutorial needs re-checking before it is trusted. 4. **Driver and Vulkan validation on Bazzite.** Not an officially supported Unreal distribution. Validate before asset production, as `ROADMAP.md` requires. diff --git a/ROADMAP.md b/ROADMAP.md index 90c2335..adf656b 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -31,7 +31,7 @@ Status detail in [Docs/setup.md](Docs/setup.md); workstation audit in [Docs/envi - [x] Download the 3DEP 1 m DEM for the demo bounds, reproject it to EPSG:26911 with the recorded origin and vertical datum, and record the source, clip bounds, and checksum in `Geo/`. - [x] In QGIS, set a suitable local projected CRS (e.g. EPSG:26911) and record a local origin. EPSG:26911, origin and projected easting/northing are recorded in `Geo/manifests/local-origin.md` with PROJ 9.9.0 control points; QGIS 4.2.3 is installed for the DEM reprojection step. - [x] Verify the address, building footprints, and demo boundaries for 6840 Hollywood Blvd against source data. -- [ ] Import OSM blockouts through Blosm; clean and detail them in Blender. +- [-] Import OSM blockouts through Blosm; clean and detail them in Blender. Imported 2026-11-06 (`Art/blender/blockout/blockout-2026-11-06.blend`, 87 meshes, scale verified 1:1 against OSM); cleaning and detailing not started. - [ ] Preserve geographic coordinates in metadata; define explicit meter→centimeter conversion to local Unreal coordinates. ### Asset production diff --git a/Tools/README.md b/Tools/README.md index b5e4424..8c2af35 100644 --- a/Tools/README.md +++ b/Tools/README.md @@ -13,6 +13,7 @@ Repeatable import/export and processing helpers. Anything that was done twice by ## Helpers - `check-lfs.sh` — repository guard: fails if `git-lfs` or the `lfs` filter is missing, or if an LFS-tracked file was committed as a plain blob. +- `blosm_blockout.py` — headless Blosm import of the committed `.osm` extract into a `.blend`, with a per-building scale report. Exact invocation in `Art/blender/README.md`. - `commons_fetch.py` — Wikimedia Commons reference fetcher. Search or exact `File:` titles, 1600 px thumbnails, provenance `sidecar.json` per folder. Writes only into the gitignored `Art/reference/`; reference images are never committed. ## Planned helpers diff --git a/Tools/blosm_blockout.py b/Tools/blosm_blockout.py new file mode 100644 index 0000000..df8a251 --- /dev/null +++ b/Tools/blosm_blockout.py @@ -0,0 +1,179 @@ +"""Headless Blosm blockout: import an OSM extract into a .blend, no GUI. + +Run with Blender in background mode. The add-on path has to point at the folder +that contains `blosm/` (for the Flatpak install that is +`~/.var/app/org.blender.Blender/config/blender/5.2/scripts`): + + B=/var/lib/flatpak/app/org.blender.Blender/x86_64/stable/active/files + LD_LIBRARY_PATH="$B/blender/lib" \ + BLENDER_USER_SCRIPTS="$HOME/.var/app/org.blender.Blender/config/blender/5.2/scripts" \ + "$B/blender/blender" -b --factory-startup \ + --python Tools/blosm_blockout.py -- \ + --osm Geo/raw/osm-api-map-2026-11-06.osm \ + --bounds 34.1006,-118.3422,34.1020,-118.3370 \ + --out Art/blender/blockout/blockout-2026-11-06.blend + +What it does: + +1. enables Blosm and checks that `scene.blosm` exists; +2. imports OSM data from a local `.osm` file (no network, no Overpass); +3. reports what arrived, per import category and in metres, so a scale or + missing-category failure is visible in the log rather than at first render; +4. saves the .blend. + +No source assets are modified in place: the output path is the only write. +""" + +import argparse +import math +import os +import sys + +import addon_utils +import bpy + + +def parse_args(): + argv = sys.argv + argv = argv[argv.index("--") + 1 :] if "--" in argv else [] + ap = argparse.ArgumentParser() + ap.add_argument("--osm", required=True, help="local .osm file to import") + ap.add_argument("--bounds", required=True, help="south,west,north,east in WGS84 degrees") + ap.add_argument("--out", required=True, help="output .blend path") + ap.add_argument("--level-height", type=float, default=3.2, help="metres per building level") + ap.add_argument("--no-subdivide", action="store_true", help="skip OSM area subdivision") + ap.add_argument("--load-missing", action="store_true", + help="let Blosm fetch relation members missing from the extract (needs network, breaks offline reproducibility)") + ap.add_argument("--mode", default="3Dsimple", choices=("3Dsimple", "3Drealistic", "2D"), + help="Blosm import mode; 3Drealistic needs the paid material asset pack") + ap.add_argument("--assets-dir", default=None, + help="directory with building_materials.blend/vegetation.blend, only needed for 3Drealistic") + ap.add_argument("--data-dir", default="Art/blender/.cache/blosm", + help="where Blosm may cache downloads") + return ap.parse_args(argv) + + +def enable_blosm(): + present = [m.__name__ for m in addon_utils.modules() if m.__name__ == "blosm"] + if not present: + raise SystemExit( + "Blosm is not on Blender's script path. Set BLENDER_USER_SCRIPTS to the " + "scripts folder that contains blosm/, e.g. " + "$HOME/.var/app/org.blender.Blender/config/blender/5.2/scripts" + ) + addon_utils.enable("blosm", default_set=True, persistent=False) + if not hasattr(bpy.context.scene, "blosm"): + raise SystemExit("Blosm enabled but scene.blosm is missing; add-on registration failed") + + +def configure_preferences(args): + """Blosm refuses to import until an asset directory is set. + + The check only requires the directory to exist; the paid asset pack + (building_materials.blend, vegetation.blend) supplies richer materials, and + without it the blockout still gets geometry and placeholder materials. + """ + import blosm + + prefs = bpy.context.preferences.addons["blosm"].preferences + assets = args.assets_dir or os.path.join(os.path.dirname(blosm.__file__), "assets") + if os.path.isdir(assets): + prefs.assetsDir = assets + elif args.mode == "3Drealistic": + raise SystemExit(f"3Drealistic needs an asset directory; {assets} does not exist") + data_dir = os.path.abspath(args.data_dir) + os.makedirs(data_dir, exist_ok=True) + prefs.dataDir = data_dir + # In background mode the GUI never populates the default level table, and the + # add-on then takes a code path that imports a module relative to the wrong + # package ("from gui import addDefaultLevels") and dies. One entry avoids it. + scene = bpy.context.scene + levels = scene.blosm.defaultLevels + if len(levels) == 0: + item = levels.add() + item.levels = 4 + item.weight = 1 + print("seeded one default level entry (headless guard)") + print(f"blosm assets: {assets}") + print(f"blosm data: {data_dir}") + + +def import_osm(args): + south, west, north, east = (float(v) for v in args.bounds.split(",")) + scene = bpy.context.scene + osmp = scene.blosm + + osmp.commandLineMode = True + osmp.mode = args.mode + assert osmp.mode == args.mode + osmp.dataType = "osm" + osmp.osmSource = "file" + osmp.osmFilepath = os.path.abspath(args.osm) + osmp.coordinatesAsFilter = True + osmp.minLat = south + osmp.maxLat = north + osmp.minLon = west + osmp.maxLon = east + + # what to bring in: buildings and the street network are what 0.1.0 needs + osmp.buildings = True + osmp.highways = True + osmp.water = False + osmp.forests = False + osmp.vegetation = False + osmp.railways = False + osmp.singleObject = False + osmp.levelHeight = args.level_height + # default: stay offline and reproducible from the committed extract alone + osmp.loadMissingMembers = bool(args.load_missing) + osmp.subdivide = not args.no_subdivide + osmp.subdivisionSize = 1.0 + + print(f"importing {args.osm} bounds {args.bounds}") + bpy.ops.blosm.import_data() + print("import finished") + + +def report(): + """Per-object measurements in metres, so a scale bug is obvious in the log.""" + collections = {c.name: c for c in bpy.data.collections} + meshes = [o for o in bpy.data.objects if o.type == "MESH"] + print(f"objects: {len(bpy.data.objects)} total, {len(meshes)} mesh, " + f"collections: {sorted(collections)}") + if not meshes: + print("WARNING: no mesh objects were created") + return + + def dimension(obj): + return max(obj.dimensions) + + meshes.sort(key=dimension, reverse=True) + print("largest meshes (name, dimensions in metres):") + for obj in meshes[:12]: + dx, dy, dz = obj.dimensions + print(f" {obj.name[:40]:40} {dx:7.2f} x {dy:7.2f} x {dz:7.2f}") + + widths = [dimension(o) for o in meshes] + print(f"mesh extent: max {max(widths):.2f} m, min {min(widths):.3f} m, " + f"median-ish {sorted(widths)[len(widths)//2]:.2f} m") + if max(widths) > 1000: + print("WARNING: something is over 1 km across; check units and bounds") + if max(widths) < 5: + print("WARNING: nothing larger than 5 m; the filter probably matched nothing") + + +def main(): + args = parse_args() + enable_blosm() + configure_preferences(args) + import_osm(args) + report() + + out = os.path.abspath(args.out) + os.makedirs(os.path.dirname(out), exist_ok=True) + bpy.ops.wm.save_as_mainfile(filepath=out) + print(f"saved {out}") + + +if __name__ == "__main__": + main()