diff --git a/.gitignore b/.gitignore index d500a38..dac8763 100644 --- a/.gitignore +++ b/.gitignore @@ -49,3 +49,6 @@ Thumbs.db # Scratch files .tmp-* + +# Unreal per-project build artefacts (FileOpenOrder logs and the like) +Build/ diff --git a/AGENTS.md b/AGENTS.md index c9a1741..fdb7c51 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -81,7 +81,20 @@ Always use a consistent scale test, named export presets, and repeatable import The living roadmap with per-version task checklists is in **[ROADMAP.md](ROADMAP.md)**. +- Planned work is listed as checkbox items in `ROADMAP.md`, each marked **open** (`- [ ]`), **in progress** (`- [-]`), or **done** (`- [x]`). Update the marks as work changes state; a checkbox that no longer matches reality is worse than none. + - Each version is a `##` heading; task groups are `###` headings with `- [ ]` steps (`[-]` in progress, `[x]` done). - When **all** steps in a version section are done: write a short descriptive entry to `CHANGELOG.md`, then remove the version section from `ROADMAP.md` to keep it concise. - Finish each milestone's release checks **before** expanding scope. Cut map size and simulation complexity before compromising the core walking/driving experience. - Define the release checks in `Docs/release-checks.md`. If a check fails, do not tag the release; open a patch task in `ROADMAP.md` and report the failing check in `CHANGELOG.md`. + +## Memory + +**[MEMORY.md](MEMORY.md)** is the shared working memory between sessions and collaborators. Write to it whenever you: + +- discover a fact about the environment, a tool, or a data source that cost time to establish; +- hit a gotcha, trap, or surprising failure mode, and what actually caused it; +- find or fix a bug, and how it was found so it can be found again; +- make a decision that does not fit a `ROADMAP.md` checkbox, such as a workaround, an assumption, or a rejected approach and why. + +Keep entries short and dated, with the evidence: the command, the log line, or the file path. Anything that belongs in a formal document (setup steps, tool versions, licensing) goes in `Docs/` as well — memory is for coordination, not a replacement for the record. Do not let `MEMORY.md` become a second roadmap: planned work stays in `ROADMAP.md`. diff --git a/Docs/toolchain.md b/Docs/toolchain.md index e34af7b..125f616 100644 --- a/Docs/toolchain.md +++ b/Docs/toolchain.md @@ -10,8 +10,8 @@ Audited 2026-11-06, updated 2026-11-07 on the development workstation. | --- | --- | --- | --- | | Bazzite | `45.20261105` (Kinoite, `bazzite-dx-nvidia`), kernel `7.2.15-ogc1.1.fc45` | `ujust update` / `rpm-ostree upgrade` | Booted and default. `44.20261026` retained as rollback. | | NVIDIA driver | `615.78.08`, open kernel module | Bazzite image | Verified: desktop session renders through the RTX 5070 Ti; Vulkan GPU0 is the RTX 5070 Ti on driver `615.78.08`, conformance `1.4.3.3`. | -| Unreal Engine | `5.8.4` — CL 58902052, `++UE5+Release-5.8`, promoted build | Epic download | Fully extracted to `/run/media/system/DATA/Engines/UE_5.8.4` (73 GiB on disk); headless smoke test passed 2026-11-07 (`UnrealEditor-Cmd -nullrhi -run=BlankProgram`, clean startup and shutdown). Pin is 5.8.4; formal validation completes when the editor GUI launches on Bazzite. The verified 5.2.0 archive stays as fallback until then. | -| Unreal Linux toolchain | system `clang 23.1.4` (Fedora 45) | `rpm-ostree` layer | This 5.8.4 build ships **no** bundled Linux clang (no `clang++` anywhere in the engine tree, verified 2026-11-07); UBT compiles game code with the system toolchain. Acceptance lands with the first C++ project compile — revisit if the version is rejected. | +| Unreal Engine | `5.8.4` — CL 58902052, `++UE5+Release-5.8`, promoted build | Epic download | Fully extracted to `/run/media/system/DATA/Engines/UE_5.8.4` (73 GiB on disk). **Validated 2026-11-07:** headless `-NullRHI` smoke test and full editor GUI launch with Vulkan RHI on the RTX 5070 Ti (loader 1.4.350, `VULKAN_SM5`, map check 0 errors). Pin is final. The verified 5.2.0 archive stays as fallback. | +| Unreal Linux toolchain | system `clang 23.1.4` (Fedora 45) | `rpm-ostree` layer | This 5.8.4 build ships **no** bundled Linux clang (no `clang++` anywhere in the engine tree, verified 2026-11-07); UBT compiles game code with the system toolchain. **Accepted 2026-11-07:** first game-module compile and link succeeded (`GigLineEditor` target, `libUnrealEditor-GigLine.so`). Note: game targets must use `BuildSettingsVersion.V7` to share the engine's build environment. | | Blender | `5.2.0` LTS (build 2026-07-14) | Flatpak `org.blender.Blender` | Installed. Run headless from the app tree with `LD_LIBRARY_PATH` + `BLENDER_USER_SCRIPTS`; `flatpak run` is unavailable to the agent shell. | | MPFB | `2.0.18` (extension `mpfb`, declares Blender >= 4.2) | Blender extensions platform | Installed. Load-and-export check on 5.2 still pending. | | Blosm | `2.7.28` (legacy add-on, `bl_info.blender = (4, 5, 0)`, manifest min 4.2) | blosm.com | Installed and importing on Blender 5.2 in `3Dsimple` mode. Two headless workarounds are required; see `Art/blender/README.md`. Paid material asset pack lives in `~/Downloads/blosm-assets/` and is not committed. | diff --git a/Game/GigLine/Config/DefaultEngine.ini b/Game/GigLine/Config/DefaultEngine.ini new file mode 100644 index 0000000..c74b3c9 --- /dev/null +++ b/Game/GigLine/Config/DefaultEngine.ini @@ -0,0 +1,22 @@ +[/Script/EngineSettings.GameMapsSettings] +GlobalDefaultGameMode="/Script/GigLine.GigLineGameModeBase" + +[/Script/Engine.RendererSettings] +; Leave defaults for 0.1.0; quality preset is documented at release time. + +[URL] +GameName=GigLine + +[/Script/AndroidFileServerEditor.AndroidFileServerRuntimeSettings] +bEnablePlugin=False +bAllowNetworkConnection=False +bIncludeInShipping=False +bAllowExternalStartInShipping=False +bCompileAFSProject=False +bUseCompression=False +bLogFiles=False +bReportStats=False +ConnectionType=USBOnly +bUseManualIPAddress=False +ManualIPAddress= + diff --git a/Game/GigLine/Config/DefaultGame.ini b/Game/GigLine/Config/DefaultGame.ini new file mode 100644 index 0000000..4dda0d3 --- /dev/null +++ b/Game/GigLine/Config/DefaultGame.ini @@ -0,0 +1,9 @@ +[/Script/EngineSettings.GeneralProjectSettings] +ProjectName=Gig Line +CompanyName= +CopyrightNotice= + +[/Script/UnrealEd.ProjectPackagingSettings] +BuildConfiguration=PPBC_Development +FullRebuild=False +UsePakFile=True diff --git a/Game/GigLine/Config/DefaultInput.ini b/Game/GigLine/Config/DefaultInput.ini new file mode 100644 index 0000000..494e827 --- /dev/null +++ b/Game/GigLine/Config/DefaultInput.ini @@ -0,0 +1,5 @@ +[/Script/Engine.InputSettings] +DefaultPlayerInputClass=/Script/EnhancedInput.EnhancedPlayerInput +DefaultInputComponentClass=/Script/EnhancedInput.EnhancedInputComponent +bEnableLegacyInputScales=False +bEnableDefaultMapping=1 diff --git a/Game/GigLine/GigLine.uproject b/Game/GigLine/GigLine.uproject new file mode 100644 index 0000000..fe0b5d4 --- /dev/null +++ b/Game/GigLine/GigLine.uproject @@ -0,0 +1,19 @@ +{ + "FileVersion": 3, + "EngineAssociation": "5.8", + "Category": "", + "Description": "Linux-first walking/driving demo around 6840 Hollywood Blvd. See repository AGENTS.md and ROADMAP.md.", + "Modules": [ + { + "Name": "GigLine", + "Type": "Runtime", + "LoadingPhase": "Default" + } + ], + "Plugins": [ + { + "Name": "EnhancedInput", + "Enabled": true + } + ] +} diff --git a/Game/GigLine/Source/GigLine.Target.cs b/Game/GigLine/Source/GigLine.Target.cs new file mode 100644 index 0000000..5c4fa88 --- /dev/null +++ b/Game/GigLine/Source/GigLine.Target.cs @@ -0,0 +1,13 @@ +using UnrealBuildTool; +using System.Collections.Generic; + +public class GigLineTarget : TargetRules +{ + public GigLineTarget(TargetInfo Target) : base(Target) + { + Type = TargetType.Game; + DefaultBuildSettings = BuildSettingsVersion.V7; + IncludeOrderVersion = EngineIncludeOrderVersion.Latest; + ExtraModuleNames.Add("GigLine"); + } +} diff --git a/Game/GigLine/Source/GigLine/GigLine.Build.cs b/Game/GigLine/Source/GigLine/GigLine.Build.cs new file mode 100644 index 0000000..5f9b939 --- /dev/null +++ b/Game/GigLine/Source/GigLine/GigLine.Build.cs @@ -0,0 +1,18 @@ +using UnrealBuildTool; + +public class GigLine : ModuleRules +{ + public GigLine(ReadOnlyTargetRules Target) : base(Target) + { + PCHUsage = PCHUsageMode.UseExplicitOrSharedPCHs; + + PublicDependencyModuleNames.AddRange(new string[] + { + "Core", + "CoreUObject", + "Engine", + "InputCore", + "EnhancedInput" + }); + } +} diff --git a/Game/GigLine/Source/GigLine/GigLine.cpp b/Game/GigLine/Source/GigLine/GigLine.cpp new file mode 100644 index 0000000..b425852 --- /dev/null +++ b/Game/GigLine/Source/GigLine/GigLine.cpp @@ -0,0 +1,4 @@ +#include "GigLine.h" +#include "Modules/ModuleManager.h" + +IMPLEMENT_PRIMARY_GAME_MODULE(FDefaultGameModuleImpl, GigLine, "GigLine"); diff --git a/Game/GigLine/Source/GigLine/GigLine.h b/Game/GigLine/Source/GigLine/GigLine.h new file mode 100644 index 0000000..56c5ef1 --- /dev/null +++ b/Game/GigLine/Source/GigLine/GigLine.h @@ -0,0 +1,3 @@ +#pragma once + +#include "CoreMinimal.h" diff --git a/Game/GigLine/Source/GigLine/GigLineGameModeBase.cpp b/Game/GigLine/Source/GigLine/GigLineGameModeBase.cpp new file mode 100644 index 0000000..cba6c19 --- /dev/null +++ b/Game/GigLine/Source/GigLine/GigLineGameModeBase.cpp @@ -0,0 +1 @@ +#include "GigLineGameModeBase.h" diff --git a/Game/GigLine/Source/GigLine/GigLineGameModeBase.h b/Game/GigLine/Source/GigLine/GigLineGameModeBase.h new file mode 100644 index 0000000..b047761 --- /dev/null +++ b/Game/GigLine/Source/GigLine/GigLineGameModeBase.h @@ -0,0 +1,16 @@ +#pragma once + +#include "CoreMinimal.h" +#include "GameFramework/GameModeBase.h" +#include "GigLineGameModeBase.generated.h" + +/** + * Default game mode for the 0.1.0 walkable Hollywood block demo. + * First-person movement is Blueprint-driven on top of Enhanced Input; + * this class exists so the project has a valid primary game module. + */ +UCLASS() +class GIGLINE_API AGigLineGameModeBase : public AGameModeBase +{ + GENERATED_BODY() +}; diff --git a/Game/GigLine/Source/GigLineEditor.Target.cs b/Game/GigLine/Source/GigLineEditor.Target.cs new file mode 100644 index 0000000..a3d73ce --- /dev/null +++ b/Game/GigLine/Source/GigLineEditor.Target.cs @@ -0,0 +1,13 @@ +using UnrealBuildTool; +using System.Collections.Generic; + +public class GigLineEditorTarget : TargetRules +{ + public GigLineEditorTarget(TargetInfo Target) : base(Target) + { + Type = TargetType.Editor; + DefaultBuildSettings = BuildSettingsVersion.V7; + IncludeOrderVersion = EngineIncludeOrderVersion.Latest; + ExtraModuleNames.Add("GigLine"); + } +} diff --git a/MEMORY.md b/MEMORY.md new file mode 100644 index 0000000..c6b7b35 --- /dev/null +++ b/MEMORY.md @@ -0,0 +1,56 @@ +# MEMORY.md + +Shared working memory between sessions and collaborators. Short, dated entries with evidence: the command, the log line, or the file path. + +Scope: facts and traps that cost time to establish. Planned work lives in [ROADMAP.md](ROADMAP.md); formal setup, pins, and licensing live in [Docs/](Docs/). + +--- + +## 2026-11-07 — Unreal Engine 5.8.4 + +- **Extracted** to `/run/media/system/DATA/Engines/UE_5.8.4` (73 GiB, `unzip` exit 0). CL `58902052`, `++UE5+Release-5.8`. +- **Gotcha: the editor needs a writable `~/.config/Epic`.** In the agent shell it crashed with signal 11 twice and looked like a broken install. The log named it: + `create dir('/home/zwitschi/.config/Epic/') failed: errno=30 (Read-only file system)` → + `Fatal error: Failed to create Intermediate directory '…/.config/Epic/UnrealEngine/5.8/Intermediate/'`. + Rerunning with `HOME=/tmp/uehome XDG_CONFIG_HOME=/tmp/uehome/.config` reached `LogInit: Display: Engine is initialized.` +- **Vulkan RHI is still unproven.** The successful run used `-NullRHI`. Untested: `UnrealEditor -vulkan` from a desktop terminal. +- A packaged game does not need `~/.config/Epic`, so this failure does not follow the build. + +## 2026-11-07 — Agent shell limits + +- The agent shell has a **read-only home** and read-only `/run`: `sudo` is blocked (`no new privileges`), `flatpak run` and `podman` cannot create instances, `/dev/nvidia*` is absent so `nvidia-smi` always fails there. None of this is a broken machine; verify GPU things from a normal terminal. +- Anything outside the session workspace (`/run/media/system/DATA/Engines`, `/tmp`) needs the wider sandbox with an approval prompt. + +## 2026-11-07 — Reference photography + +- **Licensing is not a gate for local development.** Reference images live in the gitignored `Art/reference/` and are never committed. What each set establishes is written into `Docs/reference-notes-hollywood.md`. +- **Commons text search runs dry; categories do not.** `Tools/commons_fetch.py --category "Category:Hollywood Masonic Temple"` surfaced 15 files a search never showed, including 4–5k px façades. +- **`upload.wikimedia.org` throttles hard.** Batches of six or fewer with `--delay 3`; a throttled file is skipped and recorded rather than killing the run (HTTP 429). +- **Unqualified queries return junk.** `Hollywood Boulevard shops` returned a Münster street, `Hollywood Masonic Temple entrance` a Seattle card index. Check the hit list before accepting a set. +- **OpenRouter web search is expensive.** One `gpt-5.2` + `openrouter:web_search` call ran 17 searches and cost **$0.38**; two of its eight URLs were Commons files already held. Use it for a known gap, not a sweep. `Tools/page_image_fetch.py` turns results into files by reading `og:image`. +- **The inherited "eight stone steps, 12 m wide" is wrong.** Photographs show three portals, each with its own short flight. Do not copy that number into geometry. + +## 2026-11-06 — Blender and Blosm, headless + +- **`flatpak run` does not work here.** Call the binary inside the app tree: + `B=/var/lib/flatpak/app/org.blender.Blender/x86_64/stable/active/files`, then `LD_LIBRARY_PATH="$B/blender/lib" "$B/blender/blender" -b …` with `BLENDER_USER_SCRIPTS="$HOME/.var/app/org.blender.Blender/config/blender/5.2/scripts"`. +- **Blosm needs two headless workarounds**, both already in `Tools/blosm_blockout.py`: + 1. it refuses to import until an asset directory is set, and the check fires before the mode is honoured; + 2. in background mode the default level table is empty, and the add-on then takes a path with a broken relative import (`from gui import addDefaultLevels`). Seeding one entry avoids it. +- **Blosm projects on a sphere, not an ellipsoid.** Its frame is about 0.4 % oversized and rotated by meridian convergence — roughly 2 m at the theatre. Fixed with a six-parameter affine into EPSG:26911 (0.0 cm residual). +- **Transform roots only.** 256 of 275 objects are children; transforming both parent and child applies the correction twice and reads as a 2 m placement error. +- **Blosm quietly fetches from Overpass** for relation members missing from the extract. `loadMissingMembers` is off by default in the tool so imports stay offline and reproducible. + +## 2026-11-06 — Geo data traps + +- **The 3DEP export is a tiled TIFF, not strips.** A strip reader silently scrambles rows and produces fake 0 m / 127 m bands. `Geo/scripts/dem_stats.py` handles tiles. +- **Sample walk routes through PROJ.** Placing samples with the OSM spherical approximation put them metres off the UTM meshes and reported 91 phantom holes. `/tmp` is not shared between tool calls here either — regenerate instead of saving intermediate files there. +- **3DEP cannot resolve kerbs.** Sharpest break near the kerb line is 0.09–0.12 m/m, a gutter transition, not a 0.15 m upstand. Kerb height stays a stated assumption. +- **Building interiors are not ground.** Points inside the hero footprint returned 114.5–117.1 m — structure, not terrain. Excluded from the relief figure. +- Two measurements independently agree on the carriageway: crossing length 20.9 m and gutter spacing −10 / +10…12 m. + +## 2026-11-06 — Repository traps + +- **Without `git lfs install` the lfs filter is inactive and `git add` does not warn** — it stores the file as a plain blob. `Tools/check-lfs.sh` is the guard. +- **Never commit reference images or `.env`.** `.gitignore` covers `/Art/reference/`, `.env`, `*.env`. When images slipped in early, history was rewritten to remove them. +- The project origin is Hollywood × Highland: `E 376 515.389 / N 3 774 224.083` in EPSG:26911. Blender X east, Y north, Z up, metres, Z measured from the origin ground (NAVD88 115.911 m). The whole block sits at negative X. diff --git a/README.md b/README.md index 6807947..f843ba9 100644 --- a/README.md +++ b/README.md @@ -31,6 +31,7 @@ The engine is installed outside the repository and pinned in [Docs/toolchain.md] - [ROADMAP.md](ROADMAP.md) — living task list, source of truth for progress - [PLAN.md](PLAN.md) — scope, stack, release policy, timeline - [AGENTS.md](AGENTS.md) — contributor and agent rules +- [MEMORY.md](MEMORY.md) — shared session memory: discoveries, gotchas, bugs - [Docs/toolchain.md](Docs/toolchain.md) — pinned versions - [Docs/environment.md](Docs/environment.md) — workstation audit - [Docs/release-checks.md](Docs/release-checks.md) — release gate definitions diff --git a/ROADMAP.md b/ROADMAP.md index 46791f3..0798252 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -11,10 +11,10 @@ Workstation and geo pipeline are done and verified. The art pipeline has a first | Workstation | Bazzite `45.20261105`, driver `615.78.08`, Vulkan good, `git-lfs`/`clang`/`cmake`/`ninja` installed | | Tooling | Blender 5.2 + MPFB 2.0.18 + Blosm 2.7.28 working headless; QGIS 4.2.3; PROJ 9.9.0; Unreal 5.8.4 installed | | Geo | extract, address + footprints + bounds, CRS + origin, 3DEP DEM, street profile, cross-sections — all verified and committed | -| Engine | 5.8.4 extracted, headless `-NullRHI` init passes, Vulkan RHI unproven | +| Engine | 5.8.4 installed and validated: headless + Vulkan GUI editor launch pass on the RTX 5070 Ti | | Blender | blockout (87 meshes), terrain, street (road/kerbs/sidewalks), site assembly — all in one frame, walk-checked | | Reference | 72 images local, uncommitted; hero entrance located; inherited eight-step claim overturned | -| Unreal project | not created — C++ project, packaging, and imports all open | +| Unreal project | `Game/GigLine` C++ project created; editor target compiles; blank Linux build packaged and launch-tested | **Open decisions:** code licence (MIT proposed); whether shipped builds carry a derived OSM database (ODbL posture). @@ -26,16 +26,16 @@ Status detail in [Docs/setup.md](Docs/setup.md); workstation audit in [Docs/envi - [x] Update Bazzite using its supported mechanism; verify NVIDIA driver and Vulkan operation. - [x] Register an Epic account and accept the Unreal license; obtain a pinned Linux engine build or build from Epic's authorized source repository. -- [-] Validate Unreal runs on Bazzite before asset production (not an officially supported distribution). 5.8.4 fully extracted (73 GiB, CL 58902052, promoted build); headless smoke test passed 2026-11-07 (`UnrealEditor-Cmd -nullrhi -run=BlankProgram`, clean startup and shutdown). Editor GUI launch on Vulkan still pending. -- [-] Install the engine-required compiler/SDK, Git, Git LFS, and an editor; use Distrobox for isolated CLI dependencies where useful. clang 23.1.4 / cmake 4.3.0 / ninja 1.13.2 installed; verified 2026-11-07 that this engine build bundles no Linux clang, so the system compiler is the toolchain — final proof lands with the first C++ project compile. -- [-] Pin Blender, MPFB, Blosm, and QGIS versions; confirm add-on compatibility; record in `docs/toolchain.md`. Done: Blender 5.2.0 LTS, MPFB 2.0.18, Blosm 2.7.28 (imports headless on 5.2 in `3Dsimple`; two headless workarounds documented), QGIS 4.2.3 with PROJ 9.9.0, stale QGIS `lts` 3.28.9 uninstalled 2026-11-07, Unreal 5.8.4 recorded as pin in `Docs/toolchain.md`. Left: finalize the Unreal pin once the editor GUI launches. +- [x] Validate Unreal runs on Bazzite before asset production (not an officially supported distribution). 2026-11-07: headless `-NullRHI` smoke test **and** full editor GUI launch with the Vulkan RHI passed — device `NVIDIA GeForce RTX 5070 Ti`, Vulkan loader 1.4.350, `VULKAN_SM5`, map check 0 errors. Only benign log noise (Android Turnkey status, telemetry DNS failures). +- [x] Install the engine-required compiler/SDK, Git, Git LFS, and an editor; use Distrobox for isolated CLI dependencies where useful. clang 23.1.4 / cmake 4.3.0 / ninja 1.13.2 installed; this engine build bundles no Linux clang, so the system compiler is the toolchain. **Accepted 2026-11-07:** the first C++ game-module compile and link (`libUnrealEditor-GigLine.so`) succeeded with system clang. +- [x] Pin Blender, MPFB, Blosm, and QGIS versions; confirm add-on compatibility; record in `docs/toolchain.md`. Done: Blender 5.2.0 LTS, MPFB 2.0.18, Blosm 2.7.28, QGIS 4.2.3 with PROJ 9.9.0 (stale `lts` removed 2026-11-07), Unreal 5.8.4 finalized as pin after the editor GUI launch. - [x] Create repository structure (`Game/`, `Art/`, `Geo/`, `Tools/`, `Docs/`) with LFS tracking and ignore rules applied. - [x] Keep project and Derived Data Cache on SSD storage; reserve several hundred GB for engine, assets, caches, builds. ### Project bootstrap -- [ ] Create a C++ Unreal project with Blueprint gameplay iteration and Enhanced Input. -- [ ] Package and launch a blank Linux build. +- [x] Create a C++ Unreal project with Blueprint gameplay iteration and Enhanced Input. Created 2026-11-07 at `Game/GigLine/GigLine.uproject` (primary game module + default game mode, Enhanced Input plugin + input classes in `Config/DefaultInput.ini`); `GigLineEditor` target compiles clean with system clang (UBT: Result Succeeded). +- [x] Package and launch a blank Linux build. 2026-11-07: UAT `BuildCookRun` (cook + build + stage + pak, Development config) succeeded in 56 s; 1.1 GiB archive at `/run/media/system/DATA/Builds/GigLine` (outside the repo, uncommitted). `GigLine.sh` launched on the desktop session, created a Vulkan swapchain, and rendered 759 frames across a 60 s test. Known minor: one handled `PSOPrecache` ensure fires at exit only; not blocking, watch it in `0.1.x`. - [ ] Export one static mesh from Blender and import into Unreal; verify scale and materials. - [ ] Export one MPFB character with baked materials and deform skeleton; verify import. - [ ] Document initial setup in `Docs/`: engine install, project checkout, fresh-checkout build procedure.