172 lines
4.8 KiB
Markdown
172 lines
4.8 KiB
Markdown
# QA Gate Matrix
|
|
|
|
Run from the repository root. Capture the exit code and the useful report line,
|
|
not only the final “OK”.
|
|
|
|
## 1. Source and docs
|
|
|
|
```bash
|
|
git status --short
|
|
git log --oneline -5
|
|
python3 tools/check_docs.py
|
|
git diff --check
|
|
```
|
|
|
|
Expected: intended files only; `check_docs: OK`; no whitespace errors.
|
|
|
|
## 2. Automated suites
|
|
|
|
```bash
|
|
godot --headless --path . --script res://src/test/test_runner.gd
|
|
python3 -m unittest discover -s tools/tests -t tools -p 'test_*.py'
|
|
```
|
|
|
|
Expected: the current Godot suite passes (historically 272+ tests) and the
|
|
Python tooling suite passes (currently 160 tests). Do not edit expected counts
|
|
without inspecting the suite.
|
|
|
|
## 3. Asset gates
|
|
|
|
```bash
|
|
python3 tools/blender_assets.py audit
|
|
python3 tools/blender_assets.py fit
|
|
python3 tools/blender_assets.py texels
|
|
python3 tools/npc_pipeline.py audit
|
|
python3 tools/npc_proportions.py report
|
|
```
|
|
|
|
Expected: exit 0, no `BAD` assets, all eight characters inside the stylized
|
|
proportion guide, no budget/contract regression.
|
|
|
|
Optional texture/model audits when those assets changed:
|
|
|
|
```bash
|
|
python3 tools/asset_pipeline.py audit assets/textures/ai/game_ready
|
|
python3 tools/model_pipeline.py audit
|
|
```
|
|
|
|
## 4. Smoke matrix
|
|
|
|
Headless:
|
|
|
|
```bash
|
|
for s in smoke_audio smoke_crowd smoke_emergency smoke_npc_budget \
|
|
smoke_plan_verbs smoke_scenario_content smoke_session_shell \
|
|
smoke_timeline smoke_ui_flows; do
|
|
echo "== $s"
|
|
godot --headless --path . --script res://src/test/$s.gd || exit 1
|
|
done
|
|
```
|
|
|
|
Isolated data:
|
|
|
|
```bash
|
|
rm -rf /tmp/gig-line-release-campaign /tmp/gig-line-release-tutorial
|
|
XDG_DATA_HOME=/tmp/gig-line-release-campaign godot --path . --fixed-fps 60 \
|
|
--script res://src/test/smoke_campaign.gd
|
|
XDG_DATA_HOME=/tmp/gig-line-release-tutorial godot --path . --fixed-fps 60 \
|
|
--script res://src/test/smoke_tutorial.gd -- --tutorial
|
|
```
|
|
|
|
Windowed input/UI:
|
|
|
|
```bash
|
|
godot --path . --script res://src/test/smoke_movement.gd
|
|
godot --path . --script res://src/test/smoke_props.gd
|
|
godot --path . --script res://src/test/smoke_focus_radial.gd
|
|
```
|
|
|
|
Expected: 14 smokes green. `smoke_npc_budget` should print the current mesh,
|
|
triangle and frame-time evidence; the 50 ms budget is a safety ceiling.
|
|
|
|
## 5. Screenshot and design gates
|
|
|
|
```bash
|
|
./tools/check_screenshots.sh
|
|
python3 tools/screen_designs.py compare
|
|
```
|
|
|
|
Expected: 21 good / 0 bad; no unexplained compare output. The judge is optional:
|
|
|
|
```bash
|
|
python3 tools/screen_designs.py judge --model auto
|
|
```
|
|
|
|
Record model, confidence/verdict summary, `diff` count and any manual review.
|
|
`docs/screen-design-report.md` is the durable record.
|
|
|
|
## 6. Realism judge (when visual assets/lighting changed)
|
|
|
|
```bash
|
|
godot --path . --script res://src/test/capture_realism.gd \
|
|
-- --out=res://docs/screens/realism/<run>/after
|
|
python3 tools/scene_concepts.py realism --run docs/screens/realism/<run> \
|
|
--judge --model <pinned-model>
|
|
```
|
|
|
|
Expected: a complete set of required stage captures, no stale pauses/modals, and
|
|
a report under the gitignored run directory. Record:
|
|
|
|
- pinned model/fallbacks
|
|
- shots attempted/completed
|
|
- spend if the tool exposes it
|
|
- mean score and Pillow gap
|
|
- every `review` verdict and disposition
|
|
|
|
Do not commit the run. Commit only the decision/report summary and any durable
|
|
comparison sheet explicitly intended for review.
|
|
|
|
## 7. Package checks
|
|
|
|
Use the clean worktree from `release-runbook.md`.
|
|
|
|
```bash
|
|
PUBLISH_DRY_RUN=1 ./tools/publish_release.sh <version>
|
|
sha256sum -c build/SHA256SUMS-<version>.txt
|
|
unzip -l build/gig-line-web-<version>.zip
|
|
unzip -l build/gig-line-desktop-hq-<version>.zip
|
|
unzip -p build/gig-line-desktop-hq-<version>.zip override.cfg
|
|
./build/gig-line-linux-<version>.x86_64 --headless --quit-after 60
|
|
```
|
|
|
|
Expected:
|
|
|
|
- names match the version;
|
|
- Linux/Windows/macOS/Web/HQ present locally;
|
|
- Web contains HTML/wasm/pck and docs;
|
|
- HQ contains `override.cfg`, executables and docs;
|
|
- Linux/HQ start without a script error.
|
|
|
|
Windows installer/AppImage and macOS launch require CI or the target matrix. If
|
|
not available, mark them as unverified rather than passing them by analogy.
|
|
|
|
## 8. Release verification
|
|
|
|
After the tag push:
|
|
|
|
- CI checks job succeeded;
|
|
- CI build job succeeded;
|
|
- release page has seven assets;
|
|
- each artifact name includes the released version;
|
|
- the tag points at the release commit;
|
|
- README/changelog/status and release notes agree.
|
|
|
|
## Evidence summary template
|
|
|
|
```text
|
|
Release: vX.Y.Z (commit)
|
|
Godot: pass (N/N)
|
|
Python tooling: pass (N/N)
|
|
Smokes: 14/14
|
|
Assets: audit/fit/proportions/npc audit pass
|
|
Budget: 24 NPCs, N meshes, N triangles, N ms
|
|
Gallery: 21 good / 0 bad
|
|
Screen judge: N/N match, N review, N diff
|
|
Realism: model, attempts, mean, gap, review dispositions
|
|
Build: clean worktree, artifact sizes/checksums
|
|
Launch: Linux/HQ/Web checks; unverified platforms
|
|
CI/release: run id, seven assets
|
|
Skipped checks + reason:
|
|
Known limitations:
|
|
```
|