Add tracked AI skills for Blender assets and release QA
This commit is contained in:
1 parent
e4e1566e55
commit
724e8b183f
13 files changed
+1124
-25
No files matched your search
@@ -0,0 +1,171 @@
|
||||
# QA Gate Matrix
|
||||
|
||||
Run from the repository root. Capture the exit code and the useful report line,
|
||||
not only the final “OK”.
|
||||
|
||||
## 1. Source and docs
|
||||
|
||||
```bash
|
||||
git status --short
|
||||
git log --oneline -5
|
||||
python3 tools/check_docs.py
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Expected: intended files only; `check_docs: OK`; no whitespace errors.
|
||||
|
||||
## 2. Automated suites
|
||||
|
||||
```bash
|
||||
godot --headless --path . --script res://src/test/test_runner.gd
|
||||
python3 -m unittest discover -s tools/tests -t tools -p 'test_*.py'
|
||||
```
|
||||
|
||||
Expected: the current Godot suite passes (historically 272+ tests) and the
|
||||
Python tooling suite passes (currently 160 tests). Do not edit expected counts
|
||||
without inspecting the suite.
|
||||
|
||||
## 3. Asset gates
|
||||
|
||||
```bash
|
||||
python3 tools/blender_assets.py audit
|
||||
python3 tools/blender_assets.py fit
|
||||
python3 tools/blender_assets.py texels
|
||||
python3 tools/npc_pipeline.py audit
|
||||
python3 tools/npc_proportions.py report
|
||||
```
|
||||
|
||||
Expected: exit 0, no `BAD` assets, all eight characters inside the stylized
|
||||
proportion guide, no budget/contract regression.
|
||||
|
||||
Optional texture/model audits when those assets changed:
|
||||
|
||||
```bash
|
||||
python3 tools/asset_pipeline.py audit assets/textures/ai/game_ready
|
||||
python3 tools/model_pipeline.py audit
|
||||
```
|
||||
|
||||
## 4. Smoke matrix
|
||||
|
||||
Headless:
|
||||
|
||||
```bash
|
||||
for s in smoke_audio smoke_crowd smoke_emergency smoke_npc_budget \
|
||||
smoke_plan_verbs smoke_scenario_content smoke_session_shell \
|
||||
smoke_timeline smoke_ui_flows; do
|
||||
echo "== $s"
|
||||
godot --headless --path . --script res://src/test/$s.gd || exit 1
|
||||
done
|
||||
```
|
||||
|
||||
Isolated data:
|
||||
|
||||
```bash
|
||||
rm -rf /tmp/gig-line-release-campaign /tmp/gig-line-release-tutorial
|
||||
XDG_DATA_HOME=/tmp/gig-line-release-campaign godot --path . --fixed-fps 60 \
|
||||
--script res://src/test/smoke_campaign.gd
|
||||
XDG_DATA_HOME=/tmp/gig-line-release-tutorial godot --path . --fixed-fps 60 \
|
||||
--script res://src/test/smoke_tutorial.gd -- --tutorial
|
||||
```
|
||||
|
||||
Windowed input/UI:
|
||||
|
||||
```bash
|
||||
godot --path . --script res://src/test/smoke_movement.gd
|
||||
godot --path . --script res://src/test/smoke_props.gd
|
||||
godot --path . --script res://src/test/smoke_focus_radial.gd
|
||||
```
|
||||
|
||||
Expected: 14 smokes green. `smoke_npc_budget` should print the current mesh,
|
||||
triangle and frame-time evidence; the 50 ms budget is a safety ceiling.
|
||||
|
||||
## 5. Screenshot and design gates
|
||||
|
||||
```bash
|
||||
./tools/check_screenshots.sh
|
||||
python3 tools/screen_designs.py compare
|
||||
```
|
||||
|
||||
Expected: 21 good / 0 bad; no unexplained compare output. The judge is optional:
|
||||
|
||||
```bash
|
||||
python3 tools/screen_designs.py judge --model auto
|
||||
```
|
||||
|
||||
Record model, confidence/verdict summary, `diff` count and any manual review.
|
||||
`docs/screen-design-report.md` is the durable record.
|
||||
|
||||
## 6. Realism judge (when visual assets/lighting changed)
|
||||
|
||||
```bash
|
||||
godot --path . --script res://src/test/capture_realism.gd \
|
||||
-- --out=res://docs/screens/realism/<run>/after
|
||||
python3 tools/scene_concepts.py realism --run docs/screens/realism/<run> \
|
||||
--judge --model <pinned-model>
|
||||
```
|
||||
|
||||
Expected: a complete set of required stage captures, no stale pauses/modals, and
|
||||
a report under the gitignored run directory. Record:
|
||||
|
||||
- pinned model/fallbacks
|
||||
- shots attempted/completed
|
||||
- spend if the tool exposes it
|
||||
- mean score and Pillow gap
|
||||
- every `review` verdict and disposition
|
||||
|
||||
Do not commit the run. Commit only the decision/report summary and any durable
|
||||
comparison sheet explicitly intended for review.
|
||||
|
||||
## 7. Package checks
|
||||
|
||||
Use the clean worktree from `release-runbook.md`.
|
||||
|
||||
```bash
|
||||
PUBLISH_DRY_RUN=1 ./tools/publish_release.sh <version>
|
||||
sha256sum -c build/SHA256SUMS-<version>.txt
|
||||
unzip -l build/gig-line-web-<version>.zip
|
||||
unzip -l build/gig-line-desktop-hq-<version>.zip
|
||||
unzip -p build/gig-line-desktop-hq-<version>.zip override.cfg
|
||||
./build/gig-line-linux-<version>.x86_64 --headless --quit-after 60
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- names match the version;
|
||||
- Linux/Windows/macOS/Web/HQ present locally;
|
||||
- Web contains HTML/wasm/pck and docs;
|
||||
- HQ contains `override.cfg`, executables and docs;
|
||||
- Linux/HQ start without a script error.
|
||||
|
||||
Windows installer/AppImage and macOS launch require CI or the target matrix. If
|
||||
not available, mark them as unverified rather than passing them by analogy.
|
||||
|
||||
## 8. Release verification
|
||||
|
||||
After the tag push:
|
||||
|
||||
- CI checks job succeeded;
|
||||
- CI build job succeeded;
|
||||
- release page has seven assets;
|
||||
- each artifact name includes the released version;
|
||||
- the tag points at the release commit;
|
||||
- README/changelog/status and release notes agree.
|
||||
|
||||
## Evidence summary template
|
||||
|
||||
```text
|
||||
Release: vX.Y.Z (commit)
|
||||
Godot: pass (N/N)
|
||||
Python tooling: pass (N/N)
|
||||
Smokes: 14/14
|
||||
Assets: audit/fit/proportions/npc audit pass
|
||||
Budget: 24 NPCs, N meshes, N triangles, N ms
|
||||
Gallery: 21 good / 0 bad
|
||||
Screen judge: N/N match, N review, N diff
|
||||
Realism: model, attempts, mean, gap, review dispositions
|
||||
Build: clean worktree, artifact sizes/checksums
|
||||
Launch: Linux/HQ/Web checks; unverified platforms
|
||||
CI/release: run id, seven assets
|
||||
Skipped checks + reason:
|
||||
Known limitations:
|
||||
```
|
||||
Reference in new issue
Block a user