geo: one tool directory, one unit boundary, per-asset provenance

Geo/tools/ is now every geographic script, with Geo/README.md documenting each
one; the Blender-side builders moved beside their outputs in Art/blender/*/ and
import their helpers from Geo/tools/osm_common.py.

That module gains to_unreal_local() and to_blender_local(): the only place in the
repository that multiplies metres by 100, so a second silent scale cannot be
introduced. block_height_report.py adds height provenance for the block — 3 of
15 buildings have no height data and render at a placeholder 6.4 m.
asset_metadata.py writes and audits <asset>.geo.json sidecars; four now exist
(hero-facade, terrain, street, blockout) and the audit recomputes each centre from
the origin, failing anything that disagrees.
This commit is contained in:
zwitschi committed 2026-11-07 12:28:26 +01:00
1 parent 5e5e5ae9b4
commit 386d4bee78
26 files changed
+471 -57

No files matched your search

+3 -4
View File
@@ -14,10 +14,9 @@ Repeatable import/export and processing helpers. Anything that was done twice by
- `render_screenshots.sh` — visual-review pipeline: rebuilds `/Game/Test/ReviewMap` from current content (`ue_build_review_map.py`), runs the game on it in `-game` mode, lets `AGigLineShotActor` (`Game/GigLine/Source/GigLine/`) capture the fixed camera list, and copies the PNGs into the gitignored `Screenshots/`. See `Screenshots/README.md`.
- `check-lfs.sh` — repository guard: fails if `git-lfs` or the `lfs` filter is missing, or if an LFS-tracked file was committed as a plain blob.
- `site_assembly.py` — assembles terrain + street + blockout into one file and raycasts the walking routes for holes and steps.
- `street_mesh.py` — road, kerbs, and sidewalks on the DEM, in the project frame; kerbs flush at the three crossings.
- `terrain_from_dem.py` — terrain grid from the committed DEM, with building footprints cut out.
- `blosm_blockout.py` — headless Blosm import of the committed `.osm` extract into a `.blend`, with a per-building scale report. Exact invocation in `Art/blender/README.md`.
- Geographic tools and their shared helpers live in `Geo/tools/` (see `Geo/README.md`).
- Blender-side builders that turn geo data into meshes live beside their output: `Art/blender/terrain/terrain_from_dem.py`, `Art/blender/street/street_mesh.py`, `Art/blender/site/site_assembly.py`, `Art/blender/blockout/blosm_blockout.py`. They import their helpers from `Geo/tools/osm_common.py`, which is also the only place that converts metres to Unreal centimetres.
- `sibling_asset_audit.py` — reads the sibling project's prop library straight from GLB headers and writes `Art/props/reuse-manifest.json`; copies nothing.
- `page_image_fetch.py` — resolves a web page's `og:image` and downloads it, with page URL and hash in a local sidecar. Works for Flickr, Commons file pages, and most photo blogs.
- `commons_fetch.py` — Wikimedia Commons reference fetcher. Search, exact `File:` titles, or a whole Commons category (`--category`), thumbnails, provenance `sidecar.json` per folder. Wikimedia throttles `upload.wikimedia.org` (HTTP 429), so keep batches to six or fewer files and pass `--delay 3`; a throttled file is skipped and recorded rather than aborting the run. Writes only into the gitignored `Art/reference/`; reference images are never committed.
-302
View File
@@ -1,302 +0,0 @@
"""Headless Blosm blockout: import an OSM extract into a .blend, no GUI.
Run with Blender in background mode. The add-on path has to point at the folder
that contains `blosm/` (for the Flatpak install that is
`~/.var/app/org.blender.Blender/config/blender/5.2/scripts`):
B=/var/lib/flatpak/app/org.blender.Blender/x86_64/stable/active/files
LD_LIBRARY_PATH="$B/blender/lib" \
BLENDER_USER_SCRIPTS="$HOME/.var/app/org.blender.Blender/config/blender/5.2/scripts" \
"$B/blender/blender" -b --factory-startup \
--python Tools/blosm_blockout.py -- \
--osm Geo/raw/osm-api-map-2026-11-06.osm \
--bounds 34.1006,-118.3422,34.1020,-118.3370 \
--out Art/blender/blockout/blockout-2026-11-06.blend
What it does:
1. enables Blosm and checks that `scene.blosm` exists;
2. imports OSM data from a local `.osm` file (no network, no Overpass);
3. reports what arrived, per import category and in metres, so a scale or
missing-category failure is visible in the log rather than at first render;
4. saves the .blend.
No source assets are modified in place: the output path is the only write.
"""
import argparse
import math
import os
import sys
import addon_utils
import bpy
def parse_args():
argv = sys.argv
argv = argv[argv.index("--") + 1 :] if "--" in argv else []
ap = argparse.ArgumentParser()
ap.add_argument("--osm", required=True, help="local .osm file to import")
ap.add_argument("--bounds", required=True, help="south,west,north,east in WGS84 degrees")
ap.add_argument("--out", required=True, help="output .blend path")
ap.add_argument("--level-height", type=float, default=3.2, help="metres per building level")
ap.add_argument("--origin", default=None,
help="lat,lon to use as the Blender origin (project origin). Default: Blosm centres on the extract")
ap.add_argument("--no-subdivide", action="store_true", help="skip OSM area subdivision")
ap.add_argument("--load-missing", action="store_true",
help="let Blosm fetch relation members missing from the extract (needs network, breaks offline reproducibility)")
ap.add_argument("--mode", default="3Dsimple", choices=("3Dsimple", "3Drealistic", "2D"),
help="Blosm import mode; 3Drealistic needs the paid material asset pack")
ap.add_argument("--assets-dir", default=None,
help="directory with building_materials.blend/vegetation.blend, only needed for 3Drealistic")
ap.add_argument("--data-dir", default="Art/blender/.cache/blosm",
help="where Blosm may cache downloads")
return ap.parse_args(argv)
def enable_blosm():
present = [m.__name__ for m in addon_utils.modules() if m.__name__ == "blosm"]
if not present:
raise SystemExit(
"Blosm is not on Blender's script path. Set BLENDER_USER_SCRIPTS to the "
"scripts folder that contains blosm/, e.g. "
"$HOME/.var/app/org.blender.Blender/config/blender/5.2/scripts"
)
addon_utils.enable("blosm", default_set=True, persistent=False)
if not hasattr(bpy.context.scene, "blosm"):
raise SystemExit("Blosm enabled but scene.blosm is missing; add-on registration failed")
def configure_preferences(args):
"""Blosm refuses to import until an asset directory is set.
The check only requires the directory to exist; the paid asset pack
(building_materials.blend, vegetation.blend) supplies richer materials, and
without it the blockout still gets geometry and placeholder materials.
"""
import blosm
prefs = bpy.context.preferences.addons["blosm"].preferences
assets = args.assets_dir or os.path.join(os.path.dirname(blosm.__file__), "assets")
if os.path.isdir(assets):
prefs.assetsDir = assets
elif args.mode == "3Drealistic":
raise SystemExit(f"3Drealistic needs an asset directory; {assets} does not exist")
data_dir = os.path.abspath(args.data_dir)
os.makedirs(data_dir, exist_ok=True)
prefs.dataDir = data_dir
# In background mode the GUI never populates the default level table, and the
# add-on then takes a code path that imports a module relative to the wrong
# package ("from gui import addDefaultLevels") and dies. One entry avoids it.
scene = bpy.context.scene
levels = scene.blosm.defaultLevels
if len(levels) == 0:
item = levels.add()
item.levels = 4
item.weight = 1
print("seeded one default level entry (headless guard)")
print(f"blosm assets: {assets}")
print(f"blosm data: {data_dir}")
def import_osm(args):
south, west, north, east = (float(v) for v in args.bounds.split(","))
scene = bpy.context.scene
osmp = scene.blosm
if args.origin:
lat, lon = (float(v) for v in args.origin.split(","))
scene["lat"] = lat
scene["lon"] = lon
osmp.relativeToInitialImport = True
print(f"origin set to {lat}, {lon} (project origin)")
osmp.commandLineMode = True
osmp.mode = args.mode
assert osmp.mode == args.mode
osmp.dataType = "osm"
osmp.osmSource = "file"
osmp.osmFilepath = os.path.abspath(args.osm)
osmp.coordinatesAsFilter = True
osmp.minLat = south
osmp.maxLat = north
osmp.minLon = west
osmp.maxLon = east
# what to bring in: buildings and the street network are what 0.1.0 needs
osmp.buildings = True
osmp.highways = True
osmp.water = False
osmp.forests = False
osmp.vegetation = False
osmp.railways = False
osmp.singleObject = False
osmp.levelHeight = args.level_height
# default: stay offline and reproducible from the committed extract alone
osmp.loadMissingMembers = bool(args.load_missing)
osmp.subdivide = not args.no_subdivide
osmp.subdivisionSize = 1.0
print(f"importing {args.osm} bounds {args.bounds}")
bpy.ops.blosm.import_data()
print("import finished")
def proj_batch(points, proj_bin):
"""WGS84 (lat, lon) -> EPSG:26911 (E, N) in one PROJ call."""
import subprocess
app_files = os.path.dirname(os.path.dirname(proj_bin))
env = dict(os.environ)
env["LD_LIBRARY_PATH"] = f"{app_files}/lib:{env.get('LD_LIBRARY_PATH', '')}"
payload = "\n".join(f"{lon:.9f} {lat:.9f}" for lat, lon in points) + "\n"
out = subprocess.run(
[proj_bin, "+proj=utm", "+zone=11", "+datum=NAD83", "-f", "%.5f"],
input=payload, capture_output=True, text=True, env=env, check=True,
).stdout.strip().splitlines()
return [tuple(float(v) for v in line.split()) for line in out]
def find_proj():
import glob
import shutil
env = os.environ.get("PROJ_BIN")
if env and os.path.exists(env):
return env
which = shutil.which("proj")
if which:
return which
pattern = os.path.expanduser(
"~/.local/share/flatpak/app/org.qgis.qgis/x86_64/stable/*/files/bin/proj"
)
matches = sorted(glob.glob(pattern))
if not matches:
raise SystemExit("no PROJ binary; set PROJ_BIN")
return matches[0]
def correct_frame(args):
"""Map Blosm's spherical Transverse Mercator onto true EPSG:26911.
Blosm projects with a sphere of radius 6378137 about the origin, which is a
conformal but ~0.3 % oversized frame relative to ellipsoidal UTM. The
difference between the two over a 500 m block is an affine map to well under
a centimetre, so fit a similarity transform on four spread control points and
apply it to the imported objects. After this, the blockout uses the same
metric frame as the DEM and the origin documented in Geo/manifests.
"""
import math
import bpy
from mathutils import Matrix
from util.transverse_mercator import TransverseMercator
lat, lon = (float(v) for v in args.origin.split(","))
tm = TransverseMercator(lat=lat, lon=lon)
south, west, north, east = (float(v) for v in args.bounds.split(","))
corners = [(south, west), (south, east), (north, west), (north, east),
((south + north) / 2, (west + east) / 2)]
blosm_xy = [tm.fromGeographic(la, lo)[:2] for la, lo in corners]
utm_xy = proj_batch(corners, find_proj())
# full affine fit (6 parameters): absorbs the small shear the spherical
# projection introduces across the block, which a similarity leaves as ~30 cm
n = len(blosm_xy)
bxy = [p for p in blosm_xy]
uxy = [q for q in utm_xy]
# solve [x, y, 1] * [a, b, c]^T = u for both components, least squares
def solve(target):
ata = [[0.0] * 3 for _ in range(3)]
atb = [0.0] * 3
for (x, y), t in zip(bxy, target):
row = (x, y, 1.0)
for i in range(3):
for j in range(3):
ata[i][j] += row[i] * row[j]
atb[i] += row[i] * t
# Gaussian elimination
m = [ata[i][:] + [atb[i]] for i in range(3)]
for i in range(3):
piv = max(range(i, 3), key=lambda r: abs(m[r][i]))
m[i], m[piv] = m[piv], m[i]
for r in range(3):
if r != i:
f = m[r][i] / m[i][i]
for c in range(i, 4):
m[r][c] -= f * m[i][c]
return [m[i][3] / m[i][i] for i in range(3)]
a, b, tx = solve([q[0] for q in uxy])
d, e, ty = solve([q[1] for q in uxy])
residuals = [
math.hypot(a * p[0] + b * p[1] + tx - q[0], d * p[0] + e * p[1] + ty - q[1])
for p, q in zip(blosm_xy, utm_xy)
]
print(f"frame correction affine: X = {a:.6f}*x {b:+.6f}*y {tx:+.2f}, "
f"Y = {d:+.6f}*x {e:.6f}*y {ty:+.2f}")
print(f"frame correction residual: max {max(residuals)*100:.1f} cm")
# local origin for the UTM frame: the project origin
origin_lat_lon = (lat, lon)
origin_xy = proj_batch([origin_lat_lon], find_proj())[0]
m = Matrix(((a, b, 0.0, tx - origin_xy[0]), (d, e, 0.0, ty - origin_xy[1]),
(0.0, 0.0, 1.0, 0.0), (0.0, 0.0, 0.0, 1.0)))
# only transform roots: children inherit their parent's transform, and
# transforming both would apply the correction twice
roots = [o for o in bpy.data.objects if o.parent is None]
for obj in roots:
obj.matrix_world = m @ obj.matrix_world
print(f"frame correction applied to {len(roots)} root objects "
f"({len(bpy.data.objects) - len(roots)} children inherit)")
def report():
"""Per-object measurements in metres, so a scale bug is obvious in the log."""
collections = {c.name: c for c in bpy.data.collections}
meshes = [o for o in bpy.data.objects if o.type == "MESH"]
print(f"objects: {len(bpy.data.objects)} total, {len(meshes)} mesh, "
f"collections: {sorted(collections)}")
if not meshes:
print("WARNING: no mesh objects were created")
return
def dimension(obj):
return max(obj.dimensions)
meshes.sort(key=dimension, reverse=True)
print("largest meshes (name, dimensions in metres):")
for obj in meshes[:12]:
dx, dy, dz = obj.dimensions
print(f" {obj.name[:40]:40} {dx:7.2f} x {dy:7.2f} x {dz:7.2f}")
widths = [dimension(o) for o in meshes]
print(f"mesh extent: max {max(widths):.2f} m, min {min(widths):.3f} m, "
f"median-ish {sorted(widths)[len(widths)//2]:.2f} m")
if max(widths) > 1000:
print("WARNING: something is over 1 km across; check units and bounds")
if max(widths) < 5:
print("WARNING: nothing larger than 5 m; the filter probably matched nothing")
def main():
args = parse_args()
enable_blosm()
configure_preferences(args)
import_osm(args)
if args.origin:
correct_frame(args)
report()
out = os.path.abspath(args.out)
os.makedirs(os.path.dirname(out), exist_ok=True)
bpy.ops.wm.save_as_mainfile(filepath=out)
print(f"saved {out}")
if __name__ == "__main__":
main()
-201
View File
@@ -1,201 +0,0 @@
"""Assemble the site: blockout + terrain + street in one file, and pre-check walkability.
B=/var/lib/flatpak/app/org.blender.Blender/x86_64/stable/active/files
LD_LIBRARY_PATH="$B/blender/lib" "$B/blender/blender" -b --factory-startup \
--python Tools/site_assembly.py -- --out Art/blender/site/site-2026-11-06.blend
The three source files are opened/appended, not regenerated. Objects are grouped
into `Site_Blockout`, `Site_Terrain`, and `Site_Street` so the next step — collision
authoring in Unreal — can address them by category.
The walkability check is a pre-check, not the `0.1.0` release check: it raycasts
straight down on a 1 m grid over the walking route (both sidewalks and the
designated crossing) and reports holes (no hit within 5 m) and steps (height jumps
between adjacent samples). Anything that would drop the player is caught here
rather than in the engine.
"""
import argparse
import glob
import math
import os
import shutil
import subprocess
import sys
import bpy
HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, os.path.join(HERE, "..", "Geo", "scripts"))
import osm_common as oc # noqa: E402
ORIGIN_E = 376515.389
ORIGIN_N = 3774224.083
def find_proj():
env = os.environ.get("PROJ_BIN")
if env and os.path.exists(env):
return env
which = shutil.which("proj")
if which:
return which
matches = sorted(glob.glob(os.path.expanduser(
"~/.local/share/flatpak/app/org.qgis.qgis/x86_64/stable/*/files/bin/proj")))
if not matches:
raise SystemExit("no PROJ binary; set PROJ_BIN")
return matches[0]
def project_to_local(points, proj_bin):
"""WGS84 (lat, lon) -> project-local X/Y in one PROJ call."""
app_files = os.path.dirname(os.path.dirname(proj_bin))
env = dict(os.environ)
env["LD_LIBRARY_PATH"] = f"{app_files}/lib:{env.get('LD_LIBRARY_PATH', '')}"
payload = "\n".join(f"{lon:.9f} {lat:.9f}" for lat, lon in points) + "\n"
out = subprocess.run([proj_bin, "+proj=utm", "+zone=11", "+datum=NAD83", "-f", "%.4f"],
input=payload, capture_output=True, text=True, env=env, check=True)
return [(float(a) - ORIGIN_E, float(b) - ORIGIN_N)
for a, b in (line.split() for line in out.stdout.strip().splitlines())]
APPEND = {
"Art/blender/terrain/terrain-2026-11-06.blend": ["Terrain_DEM"],
"Art/blender/street/street-2026-11-06.blend": [
"Road_Hollywood", "Sidewalk_South", "Sidewalk_North", "Kerb_South", "Kerb_North",
],
}
BLOCKOUT = "Art/blender/blockout/blockout-2026-11-06.blend"
def parse_args():
argv = sys.argv
argv = argv[argv.index("--") + 1:] if "--" in argv else []
ap = argparse.ArgumentParser()
ap.add_argument("--out", required=True)
ap.add_argument("--osm", default="Geo/raw/osm-api-map-2026-11-06.osm")
ap.add_argument("--sample", type=float, default=1.0, help="walkability sample spacing in metres")
return ap.parse_args(argv)
def collection(name):
coll = bpy.data.collections.get(name)
if coll is None:
coll = bpy.data.collections.new(name)
bpy.context.scene.collection.children.link(coll)
return coll
def move_to(obj, coll):
for parent in list(obj.users_collection):
parent.objects.unlink(obj)
coll.objects.link(obj)
def main():
args = parse_args()
bpy.ops.wm.open_mainfile(filepath=os.path.abspath(BLOCKOUT))
for path, names in APPEND.items():
blend = os.path.abspath(path)
if not os.path.exists(blend):
raise SystemExit(f"missing source file: {blend}")
for name in names:
bpy.ops.wm.append(
filepath=f"{blend}/Object/{name}",
directory=f"{blend}/Object",
filename=name,
)
terrain_coll = collection("Site_Terrain")
street_coll = collection("Site_Street")
blockout_coll = collection("Site_Blockout")
for name in APPEND["Art/blender/street/street-2026-11-06.blend"]:
obj = bpy.data.objects.get(name)
if obj and name != "Terrain_DEM":
move_to(obj, street_coll)
for name in ("Terrain_DEM",):
obj = bpy.data.objects.get(name)
if obj:
move_to(obj, terrain_coll)
for obj in list(bpy.data.objects):
if obj.users_collection and obj.users_collection[0].name.startswith("osm-api-map"):
move_to(obj, blockout_coll)
meshes = [o for o in bpy.data.objects if o.type == "MESH"]
for obj in meshes:
obj.data.calc_loop_triangles()
tris = sum(len(o.data.loop_triangles) for o in meshes)
print(f"site objects: {len(bpy.data.objects)} ({len(meshes)} mesh), "
f"triangles {tris}, collections: {sorted(c.name for c in bpy.data.collections)}")
# walkability pre-check
depsgraph = bpy.context.evaluated_depsgraph_get()
scene = bpy.context.scene
block_coll = bpy.data.collections.get("Site_Blockout")
blockout_names = {o.name for o in block_coll.all_objects} if block_coll else set()
print(f"walk check ignores {len(blockout_names)} blockout objects (marquees, canopies, roofs)")
def ground(x, y):
"""Lowest-ish walkable hit: skip building meshes such as marquees overhead."""
origin_z = 60.0
for _ in range(8):
hit, loc, _n, _i, obj, _m = scene.ray_cast(
depsgraph, (x, y, origin_z), (0.0, 0.0, -1.0), distance=200.0)
if not hit:
return None, None
if obj.name not in blockout_names:
return loc.z, obj.name
origin_z = loc.z - 0.02
return None, None
nodes, ways = oc.load(os.path.abspath(args.osm))
line, _lat0, _lon0, _lengths, block_len = oc.block_extent(
nodes, ways, "Hollywood Boulevard", "North Orange Drive", "North Highland Avenue")
# route in local metres: both sidewalks and the designated crossing. Points are
# computed in the mesh frame via PROJ — the OSM spherical approximation is
# rotated by ~0.75 deg against UTM here, which is metres over a 280 m block.
lat0 = sum(p[0] for p in line) / len(line)
lon0 = sum(p[1] for p in line) / len(line)
def lat_lon(s, offset):
return oc.point_at(line, _lengths, s, offset, lat0, lon0)
route_specs = {
"south sidewalk": [(s, -12.5) for s in [i * args.sample for i in range(int(block_len))]],
"north sidewalk": [(s, 13.0) for s in [i * args.sample for i in range(int(block_len))]],
"crossing at 130 m": [(130.0, o) for o in [i * args.sample - 12.5 for i in range(26)]],
}
flat = [(spec, lat_lon(s, off)) for spec, samples in route_specs.items() for s, off in samples]
local = project_to_local([ll for _spec, ll in flat], find_proj())
routes = {}
for (spec, _ll), xy in zip(flat, local):
routes.setdefault(spec, []).append(xy)
for label, samples in routes.items():
heights, holes = [], 0
for x, y in samples:
z, _hit = ground(x, y)
if z is None:
holes += 1
heights.append(None)
else:
heights.append(z)
deltas = [abs(b - a) for a, b in zip(heights, heights[1:])
if a is not None and b is not None]
max_step = max(deltas) if deltas else float("nan")
known = [h for h in heights if h is not None]
print(f" {label:18} samples {len(heights):3} holes {holes} "
f"Z {min(known):+.2f}..{max(known):+.2f} max step {max_step*100:.1f} cm")
out = os.path.abspath(args.out)
os.makedirs(os.path.dirname(out), exist_ok=True)
bpy.ops.wm.save_as_mainfile(filepath=out)
print(f"saved {out}")
if __name__ == "__main__":
main()
-269
View File
@@ -1,269 +0,0 @@
"""Build the street corridor — road, kerbs, sidewalks — on the DEM.
B=/var/lib/flatpak/app/org.blender.Blender/x86_64/stable/active/files
LD_LIBRARY_PATH="$B/blender/lib" "$B/blender/blender" -b --factory-startup \
--python Tools/street_mesh.py -- \
--osm Geo/raw/osm-api-map-2026-11-06.osm \
--dem Geo/raw/dem-3dep-1m-26911-2026-11-06.tif \
--out Art/blender/street/street-2026-11-06.blend
Everything lands in the project frame from `Geo/manifests/local-origin.md`:
X east, Y north, Z up, metres, origin at Hollywood × Highland.
Geometry follows the measured profile rather than a guess:
* carriageway between gutters at ±10.5 m from the centreline (cross-sections
put them at −10 and +10…+12; the crossing length says 20.9 m);
* kerb upstand **0.15 m**, which is an assumption — the DEM does not resolve a
kerb (see `Geo/verification/2026-11-06-cross-sections.md`);
* kerbs are flushed to 0.02 m within ±2.5 m of each signalised crossing, with a
1.5 m blend, so the crossings are actually usable on foot;
* sidewalk widths 4.0 m south and 5.3 m north, from the DEM-derived setback
arithmetic;
* the DEM supplies every elevation, so the 3.4 m fall and the ~2 % crown come
through unchanged.
The road surface is lifted 2 cm above the DEM so it does not z-fight the terrain
mesh. When the terrain is cut to the corridor later, that lift can go.
"""
import argparse
import glob
import math
import os
import shutil
import subprocess
import sys
import bpy
HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, os.path.join(HERE, "..", "Geo", "scripts"))
import dem_stats # noqa: E402
import osm_common as oc # noqa: E402
ORIGIN_E = 376515.389
ORIGIN_N = 3774224.083
ORIGIN_Z = 115.911
SOUTH_GUTTER = -10.5
NORTH_GUTTER = 10.5
SOUTH_SIDEWALK = -14.5
NORTH_SIDEWALK = 15.8
CROSSINGS = (36.6, 130.1, 266.1)
def parse_args():
argv = sys.argv
argv = argv[argv.index("--") + 1:] if "--" in argv else []
ap = argparse.ArgumentParser()
ap.add_argument("--osm", required=True)
ap.add_argument("--dem", required=True)
ap.add_argument("--out", required=True)
ap.add_argument("--along", type=float, default=1.0)
ap.add_argument("--cross", type=float, default=0.75)
ap.add_argument("--kerb", type=float, default=0.15)
ap.add_argument("--ramp-half-width", type=float, default=2.5)
ap.add_argument("--ramp-blend", type=float, default=1.5)
return ap.parse_args(argv)
def find_proj():
env = os.environ.get("PROJ_BIN")
if env and os.path.exists(env):
return env
which = shutil.which("proj")
if which:
return which
matches = sorted(glob.glob(os.path.expanduser(
"~/.local/share/flatpak/app/org.qgis.qgis/x86_64/stable/*/files/bin/proj")))
if not matches:
raise SystemExit("no PROJ binary; set PROJ_BIN")
return matches[0]
def project_to_utm(points, proj_bin):
app_files = os.path.dirname(os.path.dirname(proj_bin))
env = dict(os.environ)
env["LD_LIBRARY_PATH"] = f"{app_files}/lib:{env.get('LD_LIBRARY_PATH', '')}"
payload = "\n".join(f"{lon:.9f} {lat:.9f}" for lat, lon in points) + "\n"
out = subprocess.run([proj_bin, "+proj=utm", "+zone=11", "+datum=NAD83", "-f", "%.4f"],
input=payload, capture_output=True, text=True, env=env, check=True)
return [tuple(float(v) for v in line.split()) for line in out.stdout.strip().splitlines()]
class Dem:
def __init__(self, path):
entries, w, h, values = dem_stats.read_tiff(path)
self.w, self.h, self.v = w, h, values
scale, tie = entries[33550], entries[33922]
self.x0 = tie[3] - tie[0] * scale[0]
self.y0 = tie[4] + tie[1] * scale[1]
self.sx, self.sy = scale[0], scale[1]
nodata = entries.get(42113)
self.nodata = float(nodata) if nodata not in (None, "") else None
def at(self, e, n):
col = (e - self.x0) / self.sx - 0.5
row = (self.y0 - n) / self.sy - 0.5
c0, r0 = int(col // 1), int(row // 1)
if c0 < 0 or r0 < 0 or c0 + 1 >= self.w or r0 + 1 >= self.h:
return None
fx, fy = col - c0, row - r0
p = []
for r, c in ((r0, c0), (r0, c0 + 1), (r0 + 1, c0), (r0 + 1, c0 + 1)):
val = self.v[r * self.w + c]
if self.nodata is not None and val == self.nodata:
return None
p.append(val)
return (p[0] * (1 - fx) * (1 - fy) + p[1] * fx * (1 - fy)
+ p[2] * (1 - fx) * fy + p[3] * fx * fy)
def kerb_height(s, base, half_width, blend):
"""Full kerb, except flushed near a crossing so people can step off."""
for crossing in CROSSINGS:
d = abs(s - crossing)
if d <= half_width:
return 0.02
if d <= half_width + blend:
t = (d - half_width) / blend
return 0.02 + (base - 0.02) * t
return base
def strip(rows, offsets, z_at, name, material):
"""Build a quad grid from per-row sampled vertices."""
verts, faces = [], []
cols = len(offsets)
for r, (_s, base, normal) in enumerate(rows):
for off in offsets:
x = base[0] + off * normal[0]
y = base[1] + off * normal[1]
verts.append((x, y, z_at(r, off, x, y)))
for r in range(len(rows) - 1):
for c in range(cols - 1):
i = r * cols + c
faces.append((i, i + 1, i + cols + 1, i + cols))
mesh = bpy.data.meshes.new(name)
mesh.from_pydata(verts, [], faces)
mesh.update()
if material:
mesh.materials.append(material)
obj = bpy.data.objects.new(name, mesh)
bpy.context.scene.collection.objects.link(obj)
return obj
def main():
args = parse_args()
nodes, ways = oc.load(args.osm)
line, _, _, _, block_len = oc.block_extent(
nodes, ways, "Hollywood Boulevard", "North Orange Drive", "North Highland Avenue")
utm = project_to_utm(line, find_proj())
local = [(e - ORIGIN_E, n - ORIGIN_N) for e, n in utm]
# arc length + unit normals in the local frame
acc, lengths = 0.0, [0.0]
for a, b in zip(local, local[1:]):
acc += math.hypot(b[0] - a[0], b[1] - a[1])
lengths.append(acc)
total = lengths[-1]
print(f"centreline {total:.1f} m")
def at(s):
for i in range(len(local) - 1):
if lengths[i] <= s <= lengths[i + 1]:
seg = lengths[i + 1] - lengths[i]
t = 0.0 if seg == 0 else (s - lengths[i]) / seg
ax, ay = local[i]
bx, by = local[i + 1]
dx, dy = bx - ax, by - ay
norm = math.hypot(dx, dy) or 1.0
return ((ax + t * dx, ay + t * dy), (-dy / norm, dx / norm), (dx / norm, dy / norm))
ax, ay = local[-1]
return ((ax, ay), (0.0, 1.0), (1.0, 0.0))
dem = Dem(args.dem)
rows = []
s = 0.0
while s <= total + 1e-9:
base, normal, tangent = at(s)
rows.append((s, base, normal))
s += args.along
# materials
asphalt = bpy.data.materials.new("Asphalt")
asphalt.diffuse_color = (0.09, 0.09, 0.095, 1.0)
concrete = bpy.data.materials.new("Concrete")
concrete.diffuse_color = (0.55, 0.53, 0.50, 1.0)
def dem_z(x, y):
z = dem.at(x + ORIGIN_E, y + ORIGIN_N)
return None if z is None else z - ORIGIN_Z
road_offsets = [SOUTH_GUTTER + i * (NORTH_GUTTER - SOUTH_GUTTER) / 28 for i in range(29)]
south_offsets = [SOUTH_SIDEWALK + i * (SOUTH_GUTTER - SOUTH_SIDEWALK) / 8 for i in range(9)]
north_offsets = [NORTH_GUTTER + i * (NORTH_SIDEWALK - NORTH_GUTTER) / 10 for i in range(11)]
def road_z(r, off, x, y):
z = dem_z(x, y)
return (z if z is not None else 0.0) + 0.02
def south_z(r, off, x, y):
z = dem_z(x, y)
base = z if z is not None else 0.0
return base + kerb_height(rows[r][0], args.kerb, args.ramp_half_width, args.ramp_blend)
def north_z(r, off, x, y):
return south_z(r, off, x, y)
road = strip(rows, road_offsets, road_z, "Road_Hollywood", asphalt)
sw_s = strip(rows, south_offsets, south_z, "Sidewalk_South", concrete)
sw_n = strip(rows, north_offsets, north_z, "Sidewalk_North", concrete)
# kerb faces: vertical quads joining road edge to sidewalk edge
def kerb_faces(offset, name):
verts, faces = [], []
for idx, (s_val, base, normal) in enumerate(rows):
x = base[0] + offset * normal[0]
y = base[1] + offset * normal[1]
z_road = road_z(idx, offset, x, y)
h = kerb_height(s_val, args.kerb, args.ramp_half_width, args.ramp_blend)
z_walk = (dem_z(x, y) or 0.0) + h
verts.extend([(x, y, z_road), (x, y, z_walk)])
for idx in range(len(rows) - 1):
i = idx * 2
faces.append((i, i + 2, i + 3, i + 1))
mesh = bpy.data.meshes.new(name)
mesh.from_pydata(verts, [], faces)
mesh.update()
mesh.materials.append(concrete)
obj = bpy.data.objects.new(name, mesh)
bpy.context.scene.collection.objects.link(obj)
return obj
kerb_faces(SOUTH_GUTTER, "Kerb_South")
kerb_faces(NORTH_GUTTER, "Kerb_North")
for obj in (road, sw_s, sw_n):
print(f"{obj.name:16} verts {len(obj.data.vertices):5} faces {len(obj.data.polygons):5}")
# crown check on the built mesh
mid = total / 2
for off in (SOUTH_GUTTER, -5.0, 0.0, 5.0, NORTH_GUTTER):
base, normal, _ = at(mid)
x = base[0] + off * normal[0]
y = base[1] + off * normal[1]
zr = road_z(0, off, x, y)
print(f" crown at {off:+5.1f} m: road Z {zr:+.2f} m")
out = os.path.abspath(args.out)
os.makedirs(os.path.dirname(out), exist_ok=True)
bpy.ops.wm.save_as_mainfile(filepath=out)
print(f"saved {out}")
if __name__ == "__main__":
main()
-220
View File
@@ -1,220 +0,0 @@
"""Build a terrain mesh from the committed 3DEP DEM, in project-local metres.
Run inside Blender (background or GUI):
B=/var/lib/flatpak/app/org.blender.Blender/x86_64/stable/active/files
LD_LIBRARY_PATH="$B/blender/lib" "$B/blender/blender" -b --factory-startup \
--python Tools/terrain_from_dem.py -- \
--dem Geo/raw/dem-3dep-1m-26911-2026-11-06.tif \
--out Art/blender/terrain/terrain-2026-11-06.blend
Coordinates follow `Geo/manifests/local-origin.md`:
* X = UTM easting − 376 515.389 (east positive)
* Y = UTM northing − 3 774 224.083 (north positive)
* Z = NAVD88 elevation − 115.911 (the DEM value at the origin)
The DEM is already EPSG:26911, so no reprojection happens here: pixel centres are
UTM, and only the origin subtraction is applied. Negative X is west, negative Y
is south — the whole block sits at negative X.
The mesh is a regular grid at `--step` metres (default 2, which still resolves
the ~2 % road crown over the ~21 m carriageway). Nodata cells are skipped by
dropping faces that touch them, so no fake geometry is invented at the edges.
"""
import argparse
import os
import sys
import subprocess
import bpy
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "Geo", "scripts"))
import dem_stats # noqa: E402
import osm_common as oc # noqa: E402
ORIGIN_E = 376515.389
ORIGIN_N = 3774224.083
ORIGIN_Z = 115.911
def parse_args():
argv = sys.argv
argv = argv[argv.index("--") + 1:] if "--" in argv else []
ap = argparse.ArgumentParser()
ap.add_argument("--dem", required=True)
ap.add_argument("--out", required=True)
ap.add_argument("--step", type=float, default=2.0, help="grid spacing in metres")
ap.add_argument("--xmin", type=float, default=-330.0)
ap.add_argument("--xmax", type=float, default=50.0)
ap.add_argument("--ymin", type=float, default=-100.0)
ap.add_argument("--ymax", type=float, default=110.0)
ap.add_argument("--osm", default=None,
help="OSM extract; building outlines are cut out so rooftops never become terrain")
return ap.parse_args(argv)
def find_proj():
import glob
import shutil
env = os.environ.get("PROJ_BIN")
if env and os.path.exists(env):
return env
which = shutil.which("proj")
if which:
return which
matches = sorted(glob.glob(os.path.expanduser(
"~/.local/share/flatpak/app/org.qgis.qgis/x86_64/stable/*/files/bin/proj")))
if not matches:
raise SystemExit("no PROJ binary; set PROJ_BIN")
return matches[0]
def project_to_utm(points, proj_bin):
app_files = os.path.dirname(os.path.dirname(proj_bin))
env = dict(os.environ)
env["LD_LIBRARY_PATH"] = f"{app_files}/lib:{env.get('LD_LIBRARY_PATH', '')}"
payload = "\n".join(f"{lon:.9f} {lat:.9f}" for lat, lon in points) + "\n"
out = subprocess.run([proj_bin, "+proj=utm", "+zone=11", "+datum=NAD83", "-f", "%.4f"],
input=payload, capture_output=True, text=True, env=env, check=True)
return [tuple(float(v) for v in line.split()) for line in out.stdout.strip().splitlines()]
def building_polygons(osm_path):
"""Building footprints as local X/Y rings, via PROJ into EPSG:26911."""
nodes, ways = oc.load(osm_path)
rings = []
for _, tags, refs in ways:
if "building" not in tags:
continue
pts = [nodes[r] for r in refs if r in nodes]
if len(pts) < 3:
continue
rings.append(pts)
if not rings:
return []
flat = [p for ring in rings for p in pts_flat(ring)]
utm = project_to_utm(flat, find_proj())
out, i = [], 0
for ring in rings:
n = len(ring)
out.append([(utm[j][0] - ORIGIN_E, utm[j][1] - ORIGIN_N) for j in range(i, i + n)])
i += n
return out
def pts_flat(ring):
return ring
def inside(x, y, ring):
"""Ray casting point-in-polygon."""
hit = False
n = len(ring)
for i in range(n):
x1, y1 = ring[i]
x2, y2 = ring[(i + 1) % n]
if (y1 > y) != (y2 > y):
xin = (x2 - x1) * (y - y1) / (y2 - y1 + 1e-12) + x1
if x < xin:
hit = not hit
return hit
def main():
args = parse_args()
entries, width, height, values = dem_stats.read_tiff(args.dem)
scale = entries[33550]
tie = entries[33922]
e0 = tie[3] - tie[0] * scale[0]
n0 = tie[4] + tie[1] * scale[1]
sx, sy = scale[0], scale[1]
nodata = None
if 42113 in entries:
try:
nodata = float(entries[42113])
except (TypeError, ValueError):
nodata = None
step = args.step
cols = int((args.xmax - args.xmin) / step) + 1
rows = int((args.ymax - args.ymin) / step) + 1
print(f"grid {cols} x {rows} at {step} m over "
f"X {args.xmin}..{args.xmax}, Y {args.ymin}..{args.ymax}")
polys = []
if args.osm:
polys = building_polygons(args.osm)
print(f"masking {len(polys)} building footprints")
verts = []
ok = []
for r in range(rows):
y = args.ymin + r * step
for c in range(cols):
x = args.xmin + c * step
e, n = x + ORIGIN_E, y + ORIGIN_N
col = (e - e0) / sx
row = (n0 - n) / sy
ci, ri = int(col), int(row)
if ci < 0 or ri < 0 or ci >= width or ri >= height:
verts.append((x, y, 0.0))
ok.append(False)
continue
if polys and any(inside(x, y, ring) for ring in polys):
verts.append((x, y, 0.0))
ok.append(False)
continue
z = values[ri * width + ci]
if nodata is not None and z == nodata:
verts.append((x, y, 0.0))
ok.append(False)
continue
verts.append((x, y, z - ORIGIN_Z))
ok.append(True)
faces = []
skipped = 0
for r in range(rows - 1):
for c in range(cols - 1):
i = r * cols + c
quad = (i, i + 1, i + cols + 1, i + cols)
if not all(ok[k] for k in quad):
skipped += 1
continue
faces.append(quad)
mesh = bpy.data.meshes.new("Terrain_DEM")
mesh.from_pydata(verts, [], faces)
mesh.update()
obj = bpy.data.objects.new("Terrain_DEM", mesh)
bpy.context.scene.collection.objects.link(obj)
zs = [v[2] for v, good in zip(verts, ok) if good]
print(f"vertices {len(verts)}, faces {len(faces)}, skipped cells {skipped}")
print(f"Z range {min(zs):+.2f} .. {max(zs):+.2f} m (relative to origin ground {ORIGIN_Z} m)")
# spot checks against the cross-section findings: gutters near +/-10.5 m
for name, x, y in (("origin (Highland)", 0.0, 0.0),
("Orange Dr end", -279.6, 0.0),
("hero frontage", -133.0, -14.5),
("north sidewalk", -133.0, 13.6)):
if not (args.xmin <= x <= args.xmax and args.ymin <= y <= args.ymax):
continue
c = int((x - args.xmin) / step)
r = int((y - args.ymin) / step)
i = r * cols + c
if ok[i]:
print(f" {name:22} ({x:7.1f}, {y:6.1f}) -> Z {verts[i][2]:+.2f} m")
out = os.path.abspath(args.out)
os.makedirs(os.path.dirname(out), exist_ok=True)
bpy.ops.wm.save_as_mainfile(filepath=out)
print(f"saved {out}")
if __name__ == "__main__":
main()