From afd086ba11bfa8bfd85837956e254583be1143e4 Mon Sep 17 00:00:00 2001 From: Zwitschi Date: Sat, 31 Oct 2026 21:38:12 +0100 Subject: [PATCH] Fail loudly when release uploads do not land Follow-up to the publishing fix: CI created the v0.2.11 release and logged each upload, but the assets never arrived, so the job looked successful while the release was empty. The host root filesystem is full (7.8G, 100%), and Gitea answered HTTP 500 "Attachments.Save: no space left on device". - publish_release.sh now checks the HTTP status of every asset upload and prints the response body on failure, then verifies the release reports as many assets as were sent. Empty releases can no longer pass silently. - Deleted the empty v0.2.11 release so the "latest" release (v0.2.10) still has all five artifacts; the tag itself remains. - TODO/MEMORY record the host disk blocker with concrete cleanup commands and note that v0.2.11 artifacts are built locally in build/ for upload once space exists. --- MEMORY.md | 11 ++++++++++- TODO.md | 11 ++++++++++- tools/publish_release.sh | 25 ++++++++++++++++++++++++- 3 files changed, 44 insertions(+), 3 deletions(-) diff --git a/MEMORY.md b/MEMORY.md index 47e4a8b..6320e84 100644 --- a/MEMORY.md +++ b/MEMORY.md @@ -148,7 +148,16 @@ Session memory for durable decisions, state, and gotchas. Keep concise, remove o `actions/checkout` writes (`git config --local --get-regexp '^http\..*\.extraheader$'`), which carries the runner's basic-auth credentials for the same server. Verified locally by adding that extraheader and publishing v0.2.10 (all five artifacts). **v0.2.9 and v0.2.10 - are published**; v0.2.4–v0.2.8 tags have no artifacts. Verify the CI path on the next tag. PLAN.md + ADRs 0007–0010 still describe the intended UX. + are published**; v0.2.4–v0.2.8 tags have no artifacts. Verify the CI path on the next tag. +- **Host disk full (2026-10-31):** the v0.2.11 tag build created release id 27 and logged the + uploads, but they failed with `HTTP 500 Attachments.Save: write … no space left on device` + (`/dev/mapper/pve-vm--500--disk--0`, 7.8G, 100%, 0 free). The empty v0.2.11 release was + deleted so "latest" is v0.2.10 (5 assets). `publish_release.sh` now checks the HTTP code and + verifies the asset count, so this cannot look successful again. Host cleanup needs root on + 192.168.88.30: `sudo journalctl --vacuum-size=100M` (journal is 837 MB), `sudo apt-get clean` + (91 MB), clear `/var/lib/gitea/data/attachments/tmp`. SSH works as `zwitschi` but `sudo -n` + requires a password. v0.2.11 artifacts are built locally in `build/` and can be uploaded by + running `./tools/publish_release.sh 0.2.11` once space is available. PLAN.md + ADRs 0007–0010 still describe the intended UX. - **Release v0.2.1 is LIVE** (2026-10-31): CI run 1110 built the NSIS installer + AppImage and published — installers/AppImage are on the releases page. Releasing = bump version → tag → push. - NSIS gotchas (validated in a local ubuntu:24.04 docker repro): makensis uses `-D` defines diff --git a/TODO.md b/TODO.md index 997330a..b22f5c0 100644 --- a/TODO.md +++ b/TODO.md @@ -33,7 +33,16 @@ Verified via API+host credentials (2026-10-31): `GITEA_TOKEN` secret). - [x] Fall back to the checkout's git extraheader when no token/credentials exist (the Gitea runner does not expose `github.token`). -- [ ] Verify the fallback on the next tag push produces a release automatically. +- [x] Uploads now fail loudly and the release is verified to carry every artifact + (a silent HTTP 4xx used to leave an empty release with a green job). +- [ ] **Host disk is full** (`/` 7.8G, 100%, 0 bytes free) so release uploads fail with + HTTP 500 `Attachments.Save: no space left on device`. Root-owned data is using the + space; `sudo journalctl --vacuum-size=100M` (~700 MB in `/var/log/journal`), + `sudo apt-get clean` (~90 MB), and clearing `attachments/tmp` would unblock it. + Alternative: prune superseded releases (v0.2.0 + v0.2.9 assets ≈ 565 MB). +- [ ] Verify the fallback on the next tag push produces a release automatically (the + v0.2.11 tag build reached the publish step and created the release; only the + upload failed on disk space). - [ ] Optionally backfill releases for v0.2.4–v0.2.8 (their CI artifacts were not retained; would need local rebuilds) or leave them superseded by the latest. diff --git a/tools/publish_release.sh b/tools/publish_release.sh index 047a09d..035fba3 100755 --- a/tools/publish_release.sh +++ b/tools/publish_release.sh @@ -99,8 +99,31 @@ for f in "${all_artifacts[@]}"; do exit 1 fi echo "==> Uploading $(basename "$f")" - curl -s -X POST "$API/$RELEASE_ID/assets" "${AUTH[@]}" -F "attachment=@$f" >/dev/null + # Fail loudly: a silent 4xx here used to leave a release with no artifacts while + # the job still reported success. + code="$(curl -sS -o /tmp/release-upload.json -w '%{http_code}' --retry 2 \ + -X POST "$API/$RELEASE_ID/assets" "${AUTH[@]}" -F "attachment=@$f" || true)" + if [[ "$code" != "201" && "$code" != "200" ]]; then + echo "error: upload of $(basename "$f") failed with HTTP $code" >&2 + cat /tmp/release-upload.json >&2 || true + echo "" >&2 + exit 1 + fi done +# Verify the release actually carries the artifacts we just sent. +have="$(curl -sS "${AUTH[@]}" "$API/$RELEASE_ID" | python3 -c " +import sys, json +try: + d = json.load(sys.stdin) +except Exception: + print(0); raise SystemExit +print(len(d.get('assets', [])))" 2>/dev/null || echo 0)" +if [[ "${have:-0}" -lt "${#all_artifacts[@]}" ]]; then + echo "error: release has $have assets, expected ${#all_artifacts[@]}" >&2 + exit 1 +fi +echo "Verified $have artifacts on the release" + echo "" echo "Release live: $HOST/$REPO/releases/tag/$TAG"