feat: headless Blosm blockout of the demo block

Tools/blosm_blockout.py imports the committed extract in 3Dsimple mode and
reports per-building dimensions so a units failure is visible in the log. The
result (87 meshes) is committed as the blockout baseline: hero 25.68 x 54.88 x
14.50 m against 25.75 x 54.88 m of OSM geometry, so the import is 1:1 metres.

Two headless defects needed working around: the asset-directory check that
fires before mode is honoured, and an empty default-level table sending the
add-on down a broken relative import. Member fetching stays off so the import
does not depend on Overpass.
This commit is contained in:
zwitschi committed 2026-11-06 23:33:41 +01:00
1 parent 72a50d8074
commit bf23a27939
8 files changed
+246 -5

No files matched your search

+3
View File
@@ -43,3 +43,6 @@ Thumbs.db
# Local, private reference material — never committed (see Docs/reference-notes-hollywood.md)
/Art/reference/
# Blender/Blosm working caches
/Art/blender/.cache/
+55
View File
@@ -0,0 +1,55 @@
# Blender sources
Editable `.blend` files. Everything here is regenerable or hand-editable, and every generated file records the exact command that produced it.
## Blockout
`blockout/blockout-2026-11-06.blend` — OSM blockout for the demo bounds, imported with Blosm from the committed extract. 276 objects, 87 meshes, in collections split by feature (`…_buildings`, `…_roads_secondary`, `…_paths_footway`, `…_roads_primary`, …).
### Rerun command
The Flatpak Blender cannot be launched with `flatpak run` from an agent shell, so the binary inside the app tree is called directly with the Flatpak user scripts path. This is exact, not approximate:
```bash
B=/var/lib/flatpak/app/org.blender.Blender/x86_64/stable/active/files
C="$HOME/.var/app/org.blender.Blender/config/blender/5.2"
LD_LIBRARY_PATH="$B/blender/lib" \
BLENDER_USER_SCRIPTS="$C/scripts" \
"$B/blender/blender" -b --factory-startup \
--python Tools/blosm_blockout.py -- \
--osm Geo/raw/osm-api-map-2026-11-06.osm \
--bounds 34.1006,-118.3422,34.1020,-118.3370 \
--assets-dir /run/media/system/DATA/Downloads/blosm-assets/assets \
--out Art/blender/blockout/blockout-2026-11-06.blend
```
Run time is about 36 s for this extent. Two headless-only workarounds are baked into the script and must stay there:
1. Blosm refuses to import until an asset directory is set; the paid asset pack path is passed explicitly and is **not** committed.
2. In background mode the GUI never fills Blosm's default level table, and the add-on then takes a code path with a broken relative import. The script seeds one entry.
## Scale check (blockout, 2026-11-06)
Blender works in metres, and the imported meshes match the OSM geometry, so the blockout is 1:1 before any Unreal export:
| Building | Measured in Blender | OSM geometry | Height source |
| --- | --- | --- | --- |
| El Capitan Entertainment Centre | 25.68 × 54.88 × 14.50 m | 25.75 × 54.88 m | `height` tag 14.5 |
| El Capitan Theatre | 38.59 × 54.87 × 20.62 m | 38.59 × 54.87 m | `building:levels=6` × 3.2 m, plus roof |
| Madame Tussaud's | 42.41 × 57.20 × 15.90 m | 42.0 m frontage | `height` tag 15.9 |
| TCL Chinese Theatre | 43.75 × 76.19 × 18.10 m | 44.0 m frontage | `height` tag 18.1 |
The 0.07 m width difference on the hero is Blosm's own projection of the outline, not a units problem. Anything larger than a few centimetres would be a flag.
Buildings **without** a `height` tag get levels × 3.2 m, which is a placeholder, not evidence. Every such height is a modelling decision to be replaced from the reference photography.
## Relation completeness
The extract is complete for the block but not for every relation inside its bbox: Blosm's optional member fetch pulled six ways from Overpass, all belonging to one 213 × 152 m multipart complex about 100 m north of the boulevard — outside the `0.1.0` bounds and out of scope. That fetch is therefore **off by default** (`--load-missing` enables it), so the committed blockout is reproducible offline from the committed extract alone. If background massing later needs that complex, refetch a wider extract rather than reintroducing a network dependency.
## What this blockout is not
- Not cleaned, not subdivided into façade modules, not UV-mapped, not detailed.
- Materials are placeholders from Blosm's simple mode. Realistic mode needs the paid material pack and was not run.
- Terrain, kerbs, and crossing ramps are not in it: those come from the DEM cross-sections in `Geo/verification/2026-11-06-cross-sections.md`.
Binary file not shown.
+1 -1
View File
@@ -47,7 +47,7 @@ The archive on disk is **Unreal Engine 5.2.0**, released May 2023, three and a h
- `[x]` Blender 5.2 LTS (Flatpak) and MPFB 2.0.18 recorded.
- `[x]` Install QGIS and record its version. QGIS **4.2.3**, Flatpak `stable`, runtime `org.kde.Platform 6.11`, bundled PROJ `9.9.0`. The stale `lts` 3.28.9 install is still beside it and should be uninstalled.
- `[ ]` Install Blosm in Blender, record its version, and confirm it loads on Blender 5.2. Fall back to a side-by-side Blender 4.5 LTS only for the blockout step if it does not.
- `[x]` Install Blosm in Blender, record its version, and confirm it loads on Blender 5.2. Blosm **2.7.28** imports the block extract headless in `3Dsimple` mode; the exact command and the two required headless workarounds are in `Art/blender/README.md`. No second Blender version was needed.
- `[ ]` Confirm the geo tool Python environment (venv with `pyproj`, `GDAL`, `shapely` from the QGIS or distrobox environment) and record it.
## 4. Project bootstrap
+3 -3
View File
@@ -12,9 +12,9 @@ Audited 2026-11-06 on the development workstation.
| NVIDIA driver | `615.78.08`, open kernel module | Bazzite image | Verified: desktop session renders through the RTX 5070 Ti; Vulkan GPU0 is the RTX 5070 Ti on driver `615.78.08`, conformance `1.4.3.3`. |
| Unreal Engine | `5.8.4` — download in progress 2026-11-06; `5.2.0` archive verified on disk as fallback | Epic download | Not extracted yet. Pin becomes 5.8.4 once the archive verifies and the editor launches on Bazzite; see [setup.md](setup.md#engine-version-decision). |
| Unreal Linux toolchain | engine-bundled clang | Engine source tree | Not installed. Do not substitute the system compiler. |
| Blender | `5.2` LTS (upstream patch `5.2.3`) | Flatpak `org.blender.Blender` | Installed. |
| Blender | `5.2.0` LTS (build 2026-07-14) | Flatpak `org.blender.Blender` | Installed. Run headless from the app tree with `LD_LIBRARY_PATH` + `BLENDER_USER_SCRIPTS`; `flatpak run` is unavailable to the agent shell. |
| MPFB | `2.0.18` (extension `mpfb`, declares Blender >= 4.2) | Blender extensions platform | Installed. Load-and-export check on 5.2 still pending. |
| Blosm | latest release at install time | blosm.com / Gumroad | Not installed. Version must be recorded before first use. |
| Blosm | `2.7.28` (legacy add-on, `bl_info.blender = (4, 5, 0)`, manifest min 4.2) | blosm.com | Installed and importing on Blender 5.2 in `3Dsimple` mode. Two headless workarounds are required; see `Art/blender/README.md`. Paid material asset pack lives in `~/Downloads/blosm-assets/` and is not committed. |
| QGIS | `4.2.3`, Flatpak `stable`, runtime `org.kde.Platform 6.11`; bundled PROJ `9.9.0` | flathub | Installed 2026-11-06. The stale `lts` branch (`3.28.9`, EOL Qt5 runtime) is still installed beside it and should be removed. |
| Git | `2.56.0` | Bazzite image | Installed. |
| Git LFS | `3.7.1` | `rpm-ostree` layer | Installed; `git lfs install` run, lfs filter active. |
@@ -34,7 +34,7 @@ Unreal Engine 5.9 is in preview and Unreal Engine 6 has been announced as of Nov
## Open compatibility risks
1. **Blosm on Blender 5.2.** Blosm versions historically lag Blender releases. Verify Blosm against the installed Blender 5.2 before the blockout step. If it fails, add a side-by-side Blender 4.5 LTS Flatpak and convert that step's output only; do not mix Blender versions within one asset.
1. **Blosm on Blender 5.2 — worked around, not resolved.** `3Dsimple` import works headless (see the blockout). Two headless-only defects are patched around in the script rather than in the add-on: an asset-directory check that fires before `mode` is honoured, and a background-mode import path that uses a broken relative import when the default level table is empty. `3Drealistic` mode has not been exercised; it loads the paid material pack and previously failed inside the realistic renderer setup. If realistic materials are wanted, test that mode separately before depending on it.
2. **MPFB 2.0.18 on Blender 5.2.** The extension declares Blender >= 4.2 but predates 5.x. The acceptance test is the `0.1.0` requirement to export one MPFB character with baked materials and a deform skeleton and import it into Unreal.
3. **Two QGIS installs.** `stable` (4.2.3, current runtime) and the old `lts` (3.28.9, EOL runtime) are both present. Nothing breaks, but launching the wrong one silently gives older PROJ data. Uninstall `lts` and keep one pin. QGIS 4.x also uses Qt6, so any processing recipe copied from a 3.x tutorial needs re-checking before it is trusted.
4. **Driver and Vulkan validation on Bazzite.** Not an officially supported Unreal distribution. Validate before asset production, as `ROADMAP.md` requires.
+1 -1
View File
@@ -31,7 +31,7 @@ Status detail in [Docs/setup.md](Docs/setup.md); workstation audit in [Docs/envi
- [x] Download the 3DEP 1 m DEM for the demo bounds, reproject it to EPSG:26911 with the recorded origin and vertical datum, and record the source, clip bounds, and checksum in `Geo/`.
- [x] In QGIS, set a suitable local projected CRS (e.g. EPSG:26911) and record a local origin. EPSG:26911, origin and projected easting/northing are recorded in `Geo/manifests/local-origin.md` with PROJ 9.9.0 control points; QGIS 4.2.3 is installed for the DEM reprojection step.
- [x] Verify the address, building footprints, and demo boundaries for 6840 Hollywood Blvd against source data.
- [ ] Import OSM blockouts through Blosm; clean and detail them in Blender.
- [-] Import OSM blockouts through Blosm; clean and detail them in Blender. Imported 2026-11-06 (`Art/blender/blockout/blockout-2026-11-06.blend`, 87 meshes, scale verified 1:1 against OSM); cleaning and detailing not started.
- [ ] Preserve geographic coordinates in metadata; define explicit meter→centimeter conversion to local Unreal coordinates.
### Asset production
+1
View File
@@ -13,6 +13,7 @@ Repeatable import/export and processing helpers. Anything that was done twice by
## Helpers
- `check-lfs.sh` — repository guard: fails if `git-lfs` or the `lfs` filter is missing, or if an LFS-tracked file was committed as a plain blob.
- `blosm_blockout.py` — headless Blosm import of the committed `.osm` extract into a `.blend`, with a per-building scale report. Exact invocation in `Art/blender/README.md`.
- `commons_fetch.py` — Wikimedia Commons reference fetcher. Search or exact `File:` titles, 1600 px thumbnails, provenance `sidecar.json` per folder. Writes only into the gitignored `Art/reference/`; reference images are never committed.
## Planned helpers
+179
View File
@@ -0,0 +1,179 @@
"""Headless Blosm blockout: import an OSM extract into a .blend, no GUI.
Run with Blender in background mode. The add-on path has to point at the folder
that contains `blosm/` (for the Flatpak install that is
`~/.var/app/org.blender.Blender/config/blender/5.2/scripts`):
B=/var/lib/flatpak/app/org.blender.Blender/x86_64/stable/active/files
LD_LIBRARY_PATH="$B/blender/lib" \
BLENDER_USER_SCRIPTS="$HOME/.var/app/org.blender.Blender/config/blender/5.2/scripts" \
"$B/blender/blender" -b --factory-startup \
--python Tools/blosm_blockout.py -- \
--osm Geo/raw/osm-api-map-2026-11-06.osm \
--bounds 34.1006,-118.3422,34.1020,-118.3370 \
--out Art/blender/blockout/blockout-2026-11-06.blend
What it does:
1. enables Blosm and checks that `scene.blosm` exists;
2. imports OSM data from a local `.osm` file (no network, no Overpass);
3. reports what arrived, per import category and in metres, so a scale or
missing-category failure is visible in the log rather than at first render;
4. saves the .blend.
No source assets are modified in place: the output path is the only write.
"""
import argparse
import math
import os
import sys
import addon_utils
import bpy
def parse_args():
argv = sys.argv
argv = argv[argv.index("--") + 1 :] if "--" in argv else []
ap = argparse.ArgumentParser()
ap.add_argument("--osm", required=True, help="local .osm file to import")
ap.add_argument("--bounds", required=True, help="south,west,north,east in WGS84 degrees")
ap.add_argument("--out", required=True, help="output .blend path")
ap.add_argument("--level-height", type=float, default=3.2, help="metres per building level")
ap.add_argument("--no-subdivide", action="store_true", help="skip OSM area subdivision")
ap.add_argument("--load-missing", action="store_true",
help="let Blosm fetch relation members missing from the extract (needs network, breaks offline reproducibility)")
ap.add_argument("--mode", default="3Dsimple", choices=("3Dsimple", "3Drealistic", "2D"),
help="Blosm import mode; 3Drealistic needs the paid material asset pack")
ap.add_argument("--assets-dir", default=None,
help="directory with building_materials.blend/vegetation.blend, only needed for 3Drealistic")
ap.add_argument("--data-dir", default="Art/blender/.cache/blosm",
help="where Blosm may cache downloads")
return ap.parse_args(argv)
def enable_blosm():
present = [m.__name__ for m in addon_utils.modules() if m.__name__ == "blosm"]
if not present:
raise SystemExit(
"Blosm is not on Blender's script path. Set BLENDER_USER_SCRIPTS to the "
"scripts folder that contains blosm/, e.g. "
"$HOME/.var/app/org.blender.Blender/config/blender/5.2/scripts"
)
addon_utils.enable("blosm", default_set=True, persistent=False)
if not hasattr(bpy.context.scene, "blosm"):
raise SystemExit("Blosm enabled but scene.blosm is missing; add-on registration failed")
def configure_preferences(args):
"""Blosm refuses to import until an asset directory is set.
The check only requires the directory to exist; the paid asset pack
(building_materials.blend, vegetation.blend) supplies richer materials, and
without it the blockout still gets geometry and placeholder materials.
"""
import blosm
prefs = bpy.context.preferences.addons["blosm"].preferences
assets = args.assets_dir or os.path.join(os.path.dirname(blosm.__file__), "assets")
if os.path.isdir(assets):
prefs.assetsDir = assets
elif args.mode == "3Drealistic":
raise SystemExit(f"3Drealistic needs an asset directory; {assets} does not exist")
data_dir = os.path.abspath(args.data_dir)
os.makedirs(data_dir, exist_ok=True)
prefs.dataDir = data_dir
# In background mode the GUI never populates the default level table, and the
# add-on then takes a code path that imports a module relative to the wrong
# package ("from gui import addDefaultLevels") and dies. One entry avoids it.
scene = bpy.context.scene
levels = scene.blosm.defaultLevels
if len(levels) == 0:
item = levels.add()
item.levels = 4
item.weight = 1
print("seeded one default level entry (headless guard)")
print(f"blosm assets: {assets}")
print(f"blosm data: {data_dir}")
def import_osm(args):
south, west, north, east = (float(v) for v in args.bounds.split(","))
scene = bpy.context.scene
osmp = scene.blosm
osmp.commandLineMode = True
osmp.mode = args.mode
assert osmp.mode == args.mode
osmp.dataType = "osm"
osmp.osmSource = "file"
osmp.osmFilepath = os.path.abspath(args.osm)
osmp.coordinatesAsFilter = True
osmp.minLat = south
osmp.maxLat = north
osmp.minLon = west
osmp.maxLon = east
# what to bring in: buildings and the street network are what 0.1.0 needs
osmp.buildings = True
osmp.highways = True
osmp.water = False
osmp.forests = False
osmp.vegetation = False
osmp.railways = False
osmp.singleObject = False
osmp.levelHeight = args.level_height
# default: stay offline and reproducible from the committed extract alone
osmp.loadMissingMembers = bool(args.load_missing)
osmp.subdivide = not args.no_subdivide
osmp.subdivisionSize = 1.0
print(f"importing {args.osm} bounds {args.bounds}")
bpy.ops.blosm.import_data()
print("import finished")
def report():
"""Per-object measurements in metres, so a scale bug is obvious in the log."""
collections = {c.name: c for c in bpy.data.collections}
meshes = [o for o in bpy.data.objects if o.type == "MESH"]
print(f"objects: {len(bpy.data.objects)} total, {len(meshes)} mesh, "
f"collections: {sorted(collections)}")
if not meshes:
print("WARNING: no mesh objects were created")
return
def dimension(obj):
return max(obj.dimensions)
meshes.sort(key=dimension, reverse=True)
print("largest meshes (name, dimensions in metres):")
for obj in meshes[:12]:
dx, dy, dz = obj.dimensions
print(f" {obj.name[:40]:40} {dx:7.2f} x {dy:7.2f} x {dz:7.2f}")
widths = [dimension(o) for o in meshes]
print(f"mesh extent: max {max(widths):.2f} m, min {min(widths):.3f} m, "
f"median-ish {sorted(widths)[len(widths)//2]:.2f} m")
if max(widths) > 1000:
print("WARNING: something is over 1 km across; check units and bounds")
if max(widths) < 5:
print("WARNING: nothing larger than 5 m; the filter probably matched nothing")
def main():
args = parse_args()
enable_blosm()
configure_preferences(args)
import_osm(args)
report()
out = os.path.abspath(args.out)
os.makedirs(os.path.dirname(out), exist_ok=True)
bpy.ops.wm.save_as_mainfile(filepath=out)
print(f"saved {out}")
if __name__ == "__main__":
main()