Add tracked AI skills for Blender assets and release QA
This commit is contained in:
zwitschi committed
1 parent
e4e1566e55
commit
724e8b183f
13 files changed
+1124
-25
No files matched your search
@@ -0,0 +1,171 @@
|
||||
# QA Gate Matrix
|
||||
|
||||
Run from the repository root. Capture the exit code and the useful report line,
|
||||
not only the final “OK”.
|
||||
|
||||
## 1. Source and docs
|
||||
|
||||
```bash
|
||||
git status --short
|
||||
git log --oneline -5
|
||||
python3 tools/check_docs.py
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Expected: intended files only; `check_docs: OK`; no whitespace errors.
|
||||
|
||||
## 2. Automated suites
|
||||
|
||||
```bash
|
||||
godot --headless --path . --script res://src/test/test_runner.gd
|
||||
python3 -m unittest discover -s tools/tests -t tools -p 'test_*.py'
|
||||
```
|
||||
|
||||
Expected: the current Godot suite passes (historically 272+ tests) and the
|
||||
Python tooling suite passes (currently 160 tests). Do not edit expected counts
|
||||
without inspecting the suite.
|
||||
|
||||
## 3. Asset gates
|
||||
|
||||
```bash
|
||||
python3 tools/blender_assets.py audit
|
||||
python3 tools/blender_assets.py fit
|
||||
python3 tools/blender_assets.py texels
|
||||
python3 tools/npc_pipeline.py audit
|
||||
python3 tools/npc_proportions.py report
|
||||
```
|
||||
|
||||
Expected: exit 0, no `BAD` assets, all eight characters inside the stylized
|
||||
proportion guide, no budget/contract regression.
|
||||
|
||||
Optional texture/model audits when those assets changed:
|
||||
|
||||
```bash
|
||||
python3 tools/asset_pipeline.py audit assets/textures/ai/game_ready
|
||||
python3 tools/model_pipeline.py audit
|
||||
```
|
||||
|
||||
## 4. Smoke matrix
|
||||
|
||||
Headless:
|
||||
|
||||
```bash
|
||||
for s in smoke_audio smoke_crowd smoke_emergency smoke_npc_budget \
|
||||
smoke_plan_verbs smoke_scenario_content smoke_session_shell \
|
||||
smoke_timeline smoke_ui_flows; do
|
||||
echo "== $s"
|
||||
godot --headless --path . --script res://src/test/$s.gd || exit 1
|
||||
done
|
||||
```
|
||||
|
||||
Isolated data:
|
||||
|
||||
```bash
|
||||
rm -rf /tmp/gig-line-release-campaign /tmp/gig-line-release-tutorial
|
||||
XDG_DATA_HOME=/tmp/gig-line-release-campaign godot --path . --fixed-fps 60 \
|
||||
--script res://src/test/smoke_campaign.gd
|
||||
XDG_DATA_HOME=/tmp/gig-line-release-tutorial godot --path . --fixed-fps 60 \
|
||||
--script res://src/test/smoke_tutorial.gd -- --tutorial
|
||||
```
|
||||
|
||||
Windowed input/UI:
|
||||
|
||||
```bash
|
||||
godot --path . --script res://src/test/smoke_movement.gd
|
||||
godot --path . --script res://src/test/smoke_props.gd
|
||||
godot --path . --script res://src/test/smoke_focus_radial.gd
|
||||
```
|
||||
|
||||
Expected: 14 smokes green. `smoke_npc_budget` should print the current mesh,
|
||||
triangle and frame-time evidence; the 50 ms budget is a safety ceiling.
|
||||
|
||||
## 5. Screenshot and design gates
|
||||
|
||||
```bash
|
||||
./tools/check_screenshots.sh
|
||||
python3 tools/screen_designs.py compare
|
||||
```
|
||||
|
||||
Expected: 21 good / 0 bad; no unexplained compare output. The judge is optional:
|
||||
|
||||
```bash
|
||||
python3 tools/screen_designs.py judge --model auto
|
||||
```
|
||||
|
||||
Record model, confidence/verdict summary, `diff` count and any manual review.
|
||||
`docs/screen-design-report.md` is the durable record.
|
||||
|
||||
## 6. Realism judge (when visual assets/lighting changed)
|
||||
|
||||
```bash
|
||||
godot --path . --script res://src/test/capture_realism.gd \
|
||||
-- --out=res://docs/screens/realism/<run>/after
|
||||
python3 tools/scene_concepts.py realism --run docs/screens/realism/<run> \
|
||||
--judge --model <pinned-model>
|
||||
```
|
||||
|
||||
Expected: a complete set of required stage captures, no stale pauses/modals, and
|
||||
a report under the gitignored run directory. Record:
|
||||
|
||||
- pinned model/fallbacks
|
||||
- shots attempted/completed
|
||||
- spend if the tool exposes it
|
||||
- mean score and Pillow gap
|
||||
- every `review` verdict and disposition
|
||||
|
||||
Do not commit the run. Commit only the decision/report summary and any durable
|
||||
comparison sheet explicitly intended for review.
|
||||
|
||||
## 7. Package checks
|
||||
|
||||
Use the clean worktree from `release-runbook.md`.
|
||||
|
||||
```bash
|
||||
PUBLISH_DRY_RUN=1 ./tools/publish_release.sh <version>
|
||||
sha256sum -c build/SHA256SUMS-<version>.txt
|
||||
unzip -l build/gig-line-web-<version>.zip
|
||||
unzip -l build/gig-line-desktop-hq-<version>.zip
|
||||
unzip -p build/gig-line-desktop-hq-<version>.zip override.cfg
|
||||
./build/gig-line-linux-<version>.x86_64 --headless --quit-after 60
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- names match the version;
|
||||
- Linux/Windows/macOS/Web/HQ present locally;
|
||||
- Web contains HTML/wasm/pck and docs;
|
||||
- HQ contains `override.cfg`, executables and docs;
|
||||
- Linux/HQ start without a script error.
|
||||
|
||||
Windows installer/AppImage and macOS launch require CI or the target matrix. If
|
||||
not available, mark them as unverified rather than passing them by analogy.
|
||||
|
||||
## 8. Release verification
|
||||
|
||||
After the tag push:
|
||||
|
||||
- CI checks job succeeded;
|
||||
- CI build job succeeded;
|
||||
- release page has seven assets;
|
||||
- each artifact name includes the released version;
|
||||
- the tag points at the release commit;
|
||||
- README/changelog/status and release notes agree.
|
||||
|
||||
## Evidence summary template
|
||||
|
||||
```text
|
||||
Release: vX.Y.Z (commit)
|
||||
Godot: pass (N/N)
|
||||
Python tooling: pass (N/N)
|
||||
Smokes: 14/14
|
||||
Assets: audit/fit/proportions/npc audit pass
|
||||
Budget: 24 NPCs, N meshes, N triangles, N ms
|
||||
Gallery: 21 good / 0 bad
|
||||
Screen judge: N/N match, N review, N diff
|
||||
Realism: model, attempts, mean, gap, review dispositions
|
||||
Build: clean worktree, artifact sizes/checksums
|
||||
Launch: Linux/HQ/Web checks; unverified platforms
|
||||
CI/release: run id, seven assets
|
||||
Skipped checks + reason:
|
||||
Known limitations:
|
||||
```
|
||||
@@ -0,0 +1,171 @@
|
||||
# Release Runbook
|
||||
|
||||
Authoritative source: `AGENTS.md` (Release Procedure), `docs/RELEASING.md`.
|
||||
This runbook is the operational checklist; if it conflicts with those, they win.
|
||||
|
||||
## 0. Decide the release
|
||||
|
||||
Determine whether this is a major, minor, patch, or release-candidate checkpoint.
|
||||
Do not assign a version to an unversioned development track until its scope is
|
||||
accepted into a release.
|
||||
|
||||
Check the current release constraints in `TODO.md`:
|
||||
|
||||
- external playtest evidence
|
||||
- legal/AI-provider/storefront review
|
||||
- macOS signing/notarization policy
|
||||
- target hardware matrix
|
||||
- any manual QA gaps
|
||||
|
||||
If a final release would ship with unresolved blockers, prepare an RC or stop;
|
||||
do not create a final tag.
|
||||
|
||||
## 1. Prepare the version
|
||||
|
||||
Files to update:
|
||||
|
||||
| File | Change |
|
||||
| --- | --- |
|
||||
| `project.godot` | `config/version="X.Y.Z"` |
|
||||
| `CHANGELOG.md` | dated `## [X.Y.Z] — YYYY-MM-DD` entry |
|
||||
| `README.md` | latest/current version and summary |
|
||||
| `PLAN.md` / `docs/architecture.md` | current version reference |
|
||||
| `docs/development-status.md` | shipped capability/release result |
|
||||
| `TODO.md` / `DONE.md` | close/move milestone when complete |
|
||||
| `MEMORY.md` | release state and gotchas |
|
||||
|
||||
Then commit the preparation before building.
|
||||
|
||||
## 2. Run gates
|
||||
|
||||
Use `qa-gates.md` as the command matrix. The minimum release gate is:
|
||||
|
||||
- Godot suite
|
||||
- Python tooling tests
|
||||
- documentation gate
|
||||
- asset audits/fit/proportions
|
||||
- 24-NPC budget
|
||||
- 14 smoke scripts
|
||||
- screenshot compare
|
||||
- optional screen/realism judge with recorded model/attempts
|
||||
- clean build and artifact checks
|
||||
|
||||
Any skipped check must be written into `MEMORY.md` or `TODO.md` with a reason.
|
||||
|
||||
## 3. Build in a clean worktree
|
||||
|
||||
Why: Godot can pack ignored `tmp/` captures and `build/` outputs into the export.
|
||||
A dirty build can produce 1.6–3.1 GB artifacts instead of ~160–240 MB.
|
||||
|
||||
```bash
|
||||
git worktree add --detach /tmp/gig-line-release HEAD
|
||||
cd /tmp/gig-line-release
|
||||
./tools/build_releases.sh
|
||||
```
|
||||
|
||||
Expected local artifacts:
|
||||
|
||||
```text
|
||||
build/gig-line-linux-<version>.x86_64
|
||||
build/gig-line-windows-<version>.exe
|
||||
build/gig-line-macos-<version>.zip
|
||||
build/gig-line-web-<version>.zip
|
||||
build/gig-line-desktop-hq-<version>.zip
|
||||
```
|
||||
|
||||
CI additionally produces the NSIS setup and AppImage, for seven release assets.
|
||||
|
||||
## 4. Preflight and checksum
|
||||
|
||||
```bash
|
||||
PUBLISH_DRY_RUN=1 ./tools/publish_release.sh <version>
|
||||
sha256sum -c build/SHA256SUMS-<version>.txt
|
||||
```
|
||||
|
||||
`publish_release.sh` checks non-empty artifacts and local free space before any
|
||||
API call. It writes checksums under `build/` and does not upload them by default.
|
||||
|
||||
Inspect packages:
|
||||
|
||||
```bash
|
||||
unzip -l build/gig-line-web-<version>.zip | head -20
|
||||
unzip -l build/gig-line-desktop-hq-<version>.zip | head -20
|
||||
unzip -p build/gig-line-desktop-hq-<version>.zip override.cfg
|
||||
```
|
||||
|
||||
Web must contain `index.html`, `index.wasm`, `index.pck`, and player docs.
|
||||
Desktop HQ must contain the Linux/Windows executables, `override.cfg`,
|
||||
`README.txt`, `CREDITS.md`, `PRIVACY.md`, `accessibility.md` and
|
||||
`known-limitations.md`.
|
||||
|
||||
## 5. Runtime launch smoke
|
||||
|
||||
```bash
|
||||
./build/gig-line-linux-<version>.x86_64 --headless --quit-after 60
|
||||
unzip -q build/gig-line-desktop-hq-<version>.zip -d /tmp/gig-line-hq
|
||||
/tmp/gig-line-hq/gig-line-linux-<version>.x86_64 --headless --quit-after 60
|
||||
```
|
||||
|
||||
For Web, serve `build/web` and check HTTP 200 for `index.html`, `index.wasm`,
|
||||
`index.pck` and docs. Do not claim browser interaction from a curl check.
|
||||
|
||||
Remove the temporary worktree after checks:
|
||||
|
||||
```bash
|
||||
git worktree remove --force /tmp/gig-line-release
|
||||
```
|
||||
|
||||
## 6. Tag and CI
|
||||
|
||||
From the real repository, on the release commit:
|
||||
|
||||
```bash
|
||||
git tag -a "v<version>" -m "Gig Line <version>"
|
||||
git push origin main "v<version>"
|
||||
```
|
||||
|
||||
The tag triggers `.gitea/workflows/build.yml`: checks, export, installer,
|
||||
AppImage, release publish. Verify the run and assets:
|
||||
|
||||
- check job success
|
||||
- build job success
|
||||
- seven assets present
|
||||
- tag/version/artifact names equal
|
||||
- download links respond
|
||||
|
||||
Do not reuse a published tag. If a fix is needed, bump patch and repeat.
|
||||
|
||||
## 7. Manual publish fallback
|
||||
|
||||
Only if CI cannot publish:
|
||||
|
||||
```bash
|
||||
RETAIN_RELEASES=7 ./tools/publish_release.sh <version>
|
||||
```
|
||||
|
||||
The script reuses or creates the release, uploads present artifacts, verifies the
|
||||
asset count, and rolls back newly uploaded assets/new release on failure. The
|
||||
default retention is `0` (no deletion). Never delete older releases implicitly.
|
||||
|
||||
## 8. Post-release record
|
||||
|
||||
Commit a release record that includes:
|
||||
|
||||
- tag and commit
|
||||
- CI run/result
|
||||
- seven assets
|
||||
- checksums or size evidence
|
||||
- skipped checks
|
||||
- known limitations
|
||||
- release URL
|
||||
|
||||
Update `DONE.md`/`TODO.md` and `MEMORY.md`; keep the changelog and README
|
||||
consistent. Push the record commit.
|
||||
|
||||
## Versioning pitfalls
|
||||
|
||||
- A `vX.Y.Z` tag is immutable in practice; do not move/replace it.
|
||||
- A release candidate is not a final release. Do not label it as such in the
|
||||
README or release page if blockers remain.
|
||||
- `config/version` must match the tag and artifact filenames.
|
||||
- The clean-worktree build is a release requirement, not an optimization.
|
||||
Reference in new issue
Block a user