docs(agents): tooling + architecture-upkeep rules, with tool tests #24
No files matched your search
@@ -110,6 +110,59 @@ Keep entries short and dated, with the evidence: the command, the log line, or t
|
||||
|
||||
Memory edits on an agent's branch are not shared live. Communicate important findings to the coordinator during work and reconcile them into the shared `main` copy when integrating.
|
||||
|
||||
## Python and shell tooling
|
||||
|
||||
Every committed asset and every measured claim in this repository is produced by a script. Tools live in `Tools/` (Unreal, network, repository), `Geo/tools/` (projection, OSM/DEM parsing, metadata) and `Art/blender/*/` (Blender-side builders, next to their output).
|
||||
|
||||
**Fix the tool, not the artefact.** If an asset is corrected by hand, script the correction (or regenerate it) in the same change. A second manual fix of the same kind means the tool is missing a step.
|
||||
|
||||
**Improve over fork.** Extend the existing tool rather than adding a near-duplicate. When two or three scripts start sharing logic, factor the shared part into `Tools/_common.py`, `Geo/tools/osm_common.py` or `Art/blender/common.py`; keep the single metres→centimetres conversion where it already lives.
|
||||
|
||||
Rules that keep tools reusable and reviewable:
|
||||
|
||||
- **One script, one job.** Every input is an argument; no interactive prompts; deterministic output paths.
|
||||
- **Fail loudly.** A tool that cannot do its job exits non-zero with a message. Never leave a partially written asset and a success code.
|
||||
- **Result files for Unreal.** Unreal commandlets swallow `print()`; write the outcome to `/tmp/gigline/<tool>.txt` and read that instead of trusting the exit code alone.
|
||||
- **Absolute paths into sandboxes.** Blender Flatpak (`LD_LIBRARY_PATH="$B/blender/lib"`) and Unreal do not share the repo's working directory; pass absolute paths.
|
||||
- **Clean worktree for generators.** Tools that rewrite committed assets (site/street/hero builders, importers, map builders) run from a clean worktree; inspect the LFS diff before committing. Note that `.blend` and `.umap` check out read-only — `chmod u+w` first or the save silently fails.
|
||||
- **Network and paid-API tools are human-triggered.** `Tools/commons_fetch.py`, `Tools/page_image_fetch.py`, `Tools/judge_screenshot.py` (OpenRouter) and anything similar never sit in a gate or CI. Handle HTTP 429 with back-off rather than retrying immediately.
|
||||
- **Measure, don't assert.** Verification tools print the sampled numbers — counts, holes, Z ranges, worst step and its location, frame-time percentiles — so a claim in `TODO.md`, `CHANGELOG.md` or `Docs/` can be re-checked by re-running the command.
|
||||
|
||||
**Document every tool.** A new or changed tool gets a row in the inventory table of the README beside it: [`Tools/README.md`](Tools/README.md), [`Geo/README.md`](Geo/README.md), or [`Art/blender/README.md`](Art/blender/README.md). What it does, its arguments, and a typical command. A tool without a README row is unfinished.
|
||||
|
||||
**Test the pure parts.** `Tools/tests/` holds stdlib `unittest` tests (no Blender or Unreal process; they must not start a game or editor). Extract the testable core — argument parsing, geometry maths, licence gating, report formatting — into functions that a test can call directly.
|
||||
|
||||
```bash
|
||||
python3 -m unittest discover -s Tools/tests -t Tools -p 'test_*.py'
|
||||
```
|
||||
|
||||
Keep the tools standard-library-only where practical; if a third-party import is genuinely needed, add it to `requirements-tools.txt` with a version floor and mention it in the tool's README row.
|
||||
|
||||
## Architecture documentation
|
||||
|
||||
[`Docs/Architecture/`](Docs/Architecture/) is the arc42 documentation: twelve numbered section files, `about-arc42.md`, and the combined [`architecture.md`](Docs/Architecture/architecture.md). **Markdown is canonical** — the `.adoc` sources were removed on 2026-11-08.
|
||||
|
||||
Keep it current **in the same change as the implementation**, not in a later documentation pass. When work alters any of the following, edit the matching section:
|
||||
|
||||
| Change | Section |
|
||||
| --- | --- |
|
||||
| A new or restructured module, tool, data source or external interface | `05_building_block_view.md` |
|
||||
| A runtime flow, tick/order dependency, or lifecycle (possession, map load, QA self-test) | `06_runtime_view.md` |
|
||||
| Packaging, engine pin, checkout/build/launch procedure | `07_deployment_view.md` |
|
||||
| A pattern, convention or cross-cutting rule (units, collision policy, provenance) | `08_concepts.md` |
|
||||
| A decision that constrains future work | new ADR in `09_architecture_decisions.md` |
|
||||
| A quality goal or a measured budget (frame time, traversal coverage) | `10_quality_requirements.md` |
|
||||
| A known risk or a risk that retired | `11_technical_risks.md` |
|
||||
| A new term, tool or abbreviation | `12_glossary.md` |
|
||||
|
||||
Rules:
|
||||
|
||||
- **Describe what exists.** No aspirational architecture: anything planned belongs in `ROADMAP.md` and `TODO.md`. Every structural claim names the file, script or flag that implements it.
|
||||
- **ADRs are append-only.** Never rewrite an accepted decision; supersede it with a new ADR that says what it replaces and why.
|
||||
- **Keep the combined document in sync.** `architecture.md` mirrors the section files; when a section changes, update the combined document too.
|
||||
- **Images** live in `Docs/Architecture/images/` and are referenced, not duplicated.
|
||||
- **Verify:** all Markdown must pass `prettier --check` (config in `.prettierrc`), and a command you quote must still run.
|
||||
|
||||
## Geographic & Art Pipeline
|
||||
|
||||
1. Save a versioned OSM extract; document date, bounds, and source.
|
||||
|
||||
@@ -60,8 +60,10 @@ Python scripts in `Tools/` drive the content pipeline:
|
||||
| `ue_create_input_assets` | Enhanced Input actions + mapping context |
|
||||
| `render_screenshots.sh` | Screenshot capture pipeline |
|
||||
| `judge_screenshot.py` | Vision-model pre-check (optional) |
|
||||
| `input_emulation.py`, `input_test.py` | OS-level virtual keyboard/mouse injection for input verification |
|
||||
| `Tools/tests/` | Stdlib `unittest` tests for the pure parts of the tools (`requirements-tools.txt`) |
|
||||
|
||||
Source data: `Art/blender/` (`.blend` files), `Geo/` (OSM extracts, DEM, verification docs).
|
||||
Source data: `Art/blender/` (`.blend` files), `Geo/` (OSM extracts, DEM, verification docs). Tool rules (one script one job, result files for commandlets, clean worktree for generators, documented inventory, tests for the pure parts) are in [`AGENTS.md`](../../AGENTS.md#python-and-shell-tooling).
|
||||
|
||||
### QA & Review
|
||||
|
||||
|
||||
@@ -199,6 +199,8 @@ Python scripts in `Tools/` drive the content pipeline:
|
||||
| `ue_create_input_assets` | Enhanced Input actions + mapping context |
|
||||
| `render_screenshots.sh` | Screenshot capture pipeline |
|
||||
| `judge_screenshot.py` | Vision-model pre-check (optional) |
|
||||
| `input_emulation.py`, `input_test.py` | OS-level virtual keyboard/mouse injection for input verification |
|
||||
| `Tools/tests/` | Stdlib `unittest` tests for the pure parts of the tools (`requirements-tools.txt`) |
|
||||
|
||||
Source data: `Art/blender/` (`.blend` files), `Geo/` (OSM extracts, DEM, verification docs).
|
||||
|
||||
|
||||
@@ -22,6 +22,24 @@ import osm_common as oc
|
||||
LEVEL_HEIGHT = 3.2
|
||||
|
||||
|
||||
def classify_height(tags, level_height=LEVEL_HEIGHT):
|
||||
"""Return (height_m, source) for one OSM way's tags.
|
||||
|
||||
Extracted from `main()` so the classification is testable: the roof/shelter
|
||||
rule and the "height tag beats levels beats guess" order are the parts that
|
||||
actually decide what the blockout should render.
|
||||
"""
|
||||
if tags.get("building") == "roof" or tags.get("amenity") == "shelter":
|
||||
kind = "roof" if tags.get("building") == "roof" else "shelter"
|
||||
return 0.0, f"not a building volume ({kind})"
|
||||
if tags.get("height"):
|
||||
return float(tags["height"]), "height tag"
|
||||
if tags.get("building:levels"):
|
||||
return (float(tags["building:levels"]) * level_height,
|
||||
f"levels x {level_height} m")
|
||||
return 2 * level_height, f"GUESS: default 2 levels x {level_height} m"
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("osm")
|
||||
@@ -43,21 +61,7 @@ def main():
|
||||
pts = oc.way_points(nodes, refs)
|
||||
if not pts or not in_block(pts):
|
||||
continue
|
||||
tag_height = tags.get("height")
|
||||
levels = tags.get("building:levels")
|
||||
# Non-building volumes must not be extruded as two-storey blocks: a
|
||||
# `building=roof` is an elevated slab and a public-transport shelter is a
|
||||
# small street structure. Treating them as "2 levels x 3.2 m" produced a
|
||||
# 6.4 m block for a bus shelter and a canopy.
|
||||
if tags.get("building") == "roof" or tags.get("amenity") == "shelter":
|
||||
kind = "roof" if tags.get("building") == "roof" else "shelter"
|
||||
height, source = 0.0, f"not a building volume ({kind})"
|
||||
elif tag_height:
|
||||
height, source = float(tag_height), "height tag"
|
||||
elif levels:
|
||||
height, source = float(levels) * LEVEL_HEIGHT, f"levels x {LEVEL_HEIGHT} m"
|
||||
else:
|
||||
height, source = 2 * LEVEL_HEIGHT, f"GUESS: default 2 levels x {LEVEL_HEIGHT} m"
|
||||
height, source = classify_height(tags)
|
||||
nearest = min(oc.project(p, line, lat0, lon0, lengths)[0] for p in pts)
|
||||
side = "S" if oc.project(pts[0], line, lat0, lon0, lengths)[2] < 0 else "N"
|
||||
rows.append((side, nearest, wid, tags.get("name", "") or tags.get("addr:housenumber", ""),
|
||||
|
||||
+18
-1
@@ -57,4 +57,21 @@ Geographic tools live in [`Geo/tools/`](../Geo/README.md) and the Blender-side b
|
||||
| Command | Purpose |
|
||||
| --- | --- |
|
||||
| `UnrealEditor <uproject> /Game/Hollywood/DemoMap -game -qaselftest` | Headless behaviour check: drives forward input for 2 s, then resets and pauses. Logs `GigLine QA: ... PASS/FAIL` lines and a final `RESULT`. Runs only with `-qaselftest`, so normal play is unaffected. |
|
||||
| `GigLine SIM` / `GigLine GROUND` log lines | Emitted every second by `AGigLineCharacter` (TEMP QA): position, fall state, movement mode, and what the ground probe hits underneath. Remove before release. |
|
||||
| `-qaselftest` phases | Walk, reset, pause/unpause, a 16-row seam transect (0 holes) and a ground-referenced wall probe (0/35), plus a frame-time capture writing `Saved/Profiling/perf-walk-frametimes.csv` and logging `GigLine PERF:`. |
|
||||
| `gigline.SetupLogs` cvar | Spawn/input setup logging, **off by default** (`DefaultEngine.ini [ConsoleVariables]`); enable at runtime with `gigline.SetupLogs 1`. |
|
||||
|
||||
## Input injection
|
||||
|
||||
| Tool | Purpose |
|
||||
| --- | --- |
|
||||
| `input_emulation.py` | Pure-stdlib `ctypes`/`fcntl` driver that creates a virtual keyboard+mouse on `/dev/uinput`. The desktop treats it as real hardware, so it proves whether OS input actually reaches the game. Needs `/dev/uinput` writable (ACL on this host). |
|
||||
| `input_test.py` | Injects a scripted sequence into the focused window: `--sequence move:2,look:3,jump:1,reset:1,pause:1,click`. |
|
||||
|
||||
## Tests
|
||||
|
||||
```bash
|
||||
python3 -m pip install -r requirements-tools.txt
|
||||
python3 -m unittest discover -s Tools/tests -t Tools -p 'test_*.py'
|
||||
```
|
||||
|
||||
Stdlib `unittest`, no Blender/Unreal process. See [`tests/README.md`](tests/README.md): test the pure parts (encoding, classification, maths, parsing); if logic is buried in `main()`, extract it into a function so it can be tested.
|
||||
@@ -0,0 +1,26 @@
|
||||
# Tooling tests
|
||||
|
||||
Standard-library `unittest` tests for the host-side tools in `Tools/`. They never start Blender, Unreal, or the game; anything that needs those processes is verified by the tool itself and recorded in `Docs/release-checks.md` instead.
|
||||
|
||||
Run from the repository root:
|
||||
|
||||
```bash
|
||||
python3 -m pip install -r requirements-tools.txt
|
||||
python3 -m unittest discover -s Tools/tests -t Tools -p 'test_*.py'
|
||||
```
|
||||
|
||||
The `-t Tools` top-level directory is required: the tests import the tool modules by their plain names (`import input_emulation`), which is why `Tools/tests/` carries an `__init__.py`.
|
||||
|
||||
Individual test:
|
||||
|
||||
```bash
|
||||
python3 -m unittest Tools.tests.test_input_emulation -v
|
||||
```
|
||||
|
||||
## What belongs here
|
||||
|
||||
Test the **pure** parts — argument parsing, key/coordinate encoding, geometry maths, licence gating, report formatting, decision classification. Network, Blender and Unreal calls stay out: a test that needs a paid API, a display, or an engine process is a manual check, not a unit test.
|
||||
|
||||
When a tool's logic is buried in `main()`, extract it into a function so it can be tested directly. That is a tooling improvement, not test scaffolding.
|
||||
|
||||
Geo (`Geo/tools/`) and Blender (`Art/blender/*/`) builders are tested the same way once their pure parts are extracted; note the module path in the test.
|
||||
@@ -0,0 +1 @@
|
||||
"""Package marker so `python3 -m unittest Tools.tests.<module>` resolves."""
|
||||
@@ -0,0 +1,53 @@
|
||||
"""Tests for Geo/tools/block_height_report.py::classify_height.
|
||||
|
||||
The Geo tools live outside `Tools/`, so the test adds `Geo/tools` to `sys.path`
|
||||
before importing the module by name.
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", ".."))
|
||||
sys.path.insert(0, os.path.join(ROOT, "Geo", "tools"))
|
||||
|
||||
import block_height_report as bhr # noqa: E402
|
||||
|
||||
|
||||
class ClassifyHeightTest(unittest.TestCase):
|
||||
def test_height_tag_wins(self):
|
||||
height, source = bhr.classify_height({"building": "yes", "height": "18.1"})
|
||||
self.assertEqual(height, 18.1)
|
||||
self.assertEqual(source, "height tag")
|
||||
|
||||
def test_height_tag_beats_levels(self):
|
||||
height, _ = bhr.classify_height(
|
||||
{"building": "yes", "height": "12.5", "building:levels": "4"})
|
||||
self.assertEqual(height, 12.5)
|
||||
|
||||
def test_levels_derive_height(self):
|
||||
height, source = bhr.classify_height(
|
||||
{"building": "yes", "building:levels": "4"}, level_height=3.2)
|
||||
self.assertEqual(height, 12.8)
|
||||
self.assertIn("levels x", source)
|
||||
|
||||
def test_roof_is_not_a_building_volume(self):
|
||||
# building=roof at layer=1 is an elevated canopy, not a two-storey block.
|
||||
height, source = bhr.classify_height({"building": "roof", "layer": "1"})
|
||||
self.assertEqual(height, 0.0)
|
||||
self.assertIn("not a building volume (roof)", source)
|
||||
|
||||
def test_transit_shelter_is_not_a_building_volume(self):
|
||||
height, source = bhr.classify_height(
|
||||
{"building": "yes", "amenity": "shelter", "shelter_type": "public_transport"})
|
||||
self.assertEqual(height, 0.0)
|
||||
self.assertIn("not a building volume (shelter)", source)
|
||||
|
||||
def test_missing_data_falls_back_to_a_flagged_guess(self):
|
||||
height, source = bhr.classify_height({"building": "yes"})
|
||||
self.assertEqual(height, 2 * bhr.LEVEL_HEIGHT)
|
||||
self.assertTrue(source.startswith("GUESS"))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,57 @@
|
||||
"""Tests for Tools/input_emulation.py — the pure ioctl/key-code parts.
|
||||
|
||||
The device path itself needs /dev/uinput, so only the encoding helpers are
|
||||
covered here; the end-to-end injection is verified by running the tool (see
|
||||
Docs/release-checks.md and MEMORY.md).
|
||||
"""
|
||||
|
||||
import unittest
|
||||
|
||||
import input_emulation as ie
|
||||
|
||||
|
||||
class IocEncodingTest(unittest.TestCase):
|
||||
"""_IOC must match the Linux ioctl bit layout."""
|
||||
|
||||
def test_layout_constants(self):
|
||||
self.assertEqual(ie._IOC_NRSHIFT, 0)
|
||||
self.assertEqual(ie._IOC_TYPESHIFT, 8)
|
||||
self.assertEqual(ie._IOC_SIZESHIFT, 16)
|
||||
self.assertEqual(ie._IOC_DIRSHIFT, 30)
|
||||
|
||||
def test_known_values(self):
|
||||
# UI_SET_EVBIT = _IOW('U', 100, int) -> 0x40045564 on Linux.
|
||||
self.assertEqual(ie.UI_SET_EVBIT, 0x40045564)
|
||||
# UI_DEV_CREATE = _IO('U', 1) -> 0x5501.
|
||||
self.assertEqual(ie.UI_DEV_CREATE, 0x5501)
|
||||
|
||||
def test_type_accepts_char_or_int(self):
|
||||
self.assertEqual(ie._ioc(1, 4, "U", 100), ie._ioc(1, 4, ord("U"), 100))
|
||||
|
||||
|
||||
class KeyCodeTest(unittest.TestCase):
|
||||
"""KEYCODES must hold the Linux input-event codes the demo uses."""
|
||||
|
||||
EXPECTED = {
|
||||
"w": 17, "s": 31, "a": 30, "d": 32,
|
||||
"space": 57, "r": 19, "esc": 1,
|
||||
}
|
||||
|
||||
def test_demo_controls(self):
|
||||
for name, code in self.EXPECTED.items():
|
||||
self.assertEqual(ie.KEYCODES[name], code, f"{name} code")
|
||||
|
||||
def test_lookup_is_case_insensitive(self):
|
||||
dev = ie.InputDevice.__new__(ie.InputDevice) # no device opened
|
||||
self.assertEqual(dev._key_code("W"), 17)
|
||||
self.assertEqual(dev._key_code("SPACE"), 57)
|
||||
self.assertEqual(dev._key_code(19), 19) # a raw code passes through
|
||||
|
||||
def test_unknown_key_raises(self):
|
||||
dev = ie.InputDevice.__new__(ie.InputDevice)
|
||||
with self.assertRaises(ie.UInputError):
|
||||
dev._key_code("not-a-key")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1 @@
|
||||
requests>=2.28
|
||||
Reference in new issue
Block a user