Fail loudly when release uploads do not land
Follow-up to the publishing fix: CI created the v0.2.11 release and logged each upload, but the assets never arrived, so the job looked successful while the release was empty. The host root filesystem is full (7.8G, 100%), and Gitea answered HTTP 500 "Attachments.Save: no space left on device". - publish_release.sh now checks the HTTP status of every asset upload and prints the response body on failure, then verifies the release reports as many assets as were sent. Empty releases can no longer pass silently. - Deleted the empty v0.2.11 release so the "latest" release (v0.2.10) still has all five artifacts; the tag itself remains. - TODO/MEMORY record the host disk blocker with concrete cleanup commands and note that v0.2.11 artifacts are built locally in build/ for upload once space exists.
This commit is contained in:
1 parent
b4d189c325
commit
afd086ba11
3 files changed
+44
-3
No files matched your search
@@ -148,7 +148,16 @@ Session memory for durable decisions, state, and gotchas. Keep concise, remove o
|
|||||||
`actions/checkout` writes (`git config --local --get-regexp '^http\..*\.extraheader$'`),
|
`actions/checkout` writes (`git config --local --get-regexp '^http\..*\.extraheader$'`),
|
||||||
which carries the runner's basic-auth credentials for the same server. Verified locally by
|
which carries the runner's basic-auth credentials for the same server. Verified locally by
|
||||||
adding that extraheader and publishing v0.2.10 (all five artifacts). **v0.2.9 and v0.2.10
|
adding that extraheader and publishing v0.2.10 (all five artifacts). **v0.2.9 and v0.2.10
|
||||||
are published**; v0.2.4–v0.2.8 tags have no artifacts. Verify the CI path on the next tag. PLAN.md + ADRs 0007–0010 still describe the intended UX.
|
are published**; v0.2.4–v0.2.8 tags have no artifacts. Verify the CI path on the next tag.
|
||||||
|
- **Host disk full (2026-10-31):** the v0.2.11 tag build created release id 27 and logged the
|
||||||
|
uploads, but they failed with `HTTP 500 Attachments.Save: write … no space left on device`
|
||||||
|
(`/dev/mapper/pve-vm--500--disk--0`, 7.8G, 100%, 0 free). The empty v0.2.11 release was
|
||||||
|
deleted so "latest" is v0.2.10 (5 assets). `publish_release.sh` now checks the HTTP code and
|
||||||
|
verifies the asset count, so this cannot look successful again. Host cleanup needs root on
|
||||||
|
192.168.88.30: `sudo journalctl --vacuum-size=100M` (journal is 837 MB), `sudo apt-get clean`
|
||||||
|
(91 MB), clear `/var/lib/gitea/data/attachments/tmp`. SSH works as `zwitschi` but `sudo -n`
|
||||||
|
requires a password. v0.2.11 artifacts are built locally in `build/` and can be uploaded by
|
||||||
|
running `./tools/publish_release.sh 0.2.11` once space is available. PLAN.md + ADRs 0007–0010 still describe the intended UX.
|
||||||
- **Release v0.2.1 is LIVE** (2026-10-31): CI run 1110 built the NSIS installer + AppImage
|
- **Release v0.2.1 is LIVE** (2026-10-31): CI run 1110 built the NSIS installer + AppImage
|
||||||
and published — installers/AppImage are on the releases page. Releasing = bump version → tag → push.
|
and published — installers/AppImage are on the releases page. Releasing = bump version → tag → push.
|
||||||
- NSIS gotchas (validated in a local ubuntu:24.04 docker repro): makensis uses `-D` defines
|
- NSIS gotchas (validated in a local ubuntu:24.04 docker repro): makensis uses `-D` defines
|
||||||
|
|||||||
@@ -33,7 +33,16 @@ Verified via API+host credentials (2026-10-31):
|
|||||||
`GITEA_TOKEN` secret).
|
`GITEA_TOKEN` secret).
|
||||||
- [x] Fall back to the checkout's git extraheader when no token/credentials exist (the
|
- [x] Fall back to the checkout's git extraheader when no token/credentials exist (the
|
||||||
Gitea runner does not expose `github.token`).
|
Gitea runner does not expose `github.token`).
|
||||||
- [ ] Verify the fallback on the next tag push produces a release automatically.
|
- [x] Uploads now fail loudly and the release is verified to carry every artifact
|
||||||
|
(a silent HTTP 4xx used to leave an empty release with a green job).
|
||||||
|
- [ ] **Host disk is full** (`/` 7.8G, 100%, 0 bytes free) so release uploads fail with
|
||||||
|
HTTP 500 `Attachments.Save: no space left on device`. Root-owned data is using the
|
||||||
|
space; `sudo journalctl --vacuum-size=100M` (~700 MB in `/var/log/journal`),
|
||||||
|
`sudo apt-get clean` (~90 MB), and clearing `attachments/tmp` would unblock it.
|
||||||
|
Alternative: prune superseded releases (v0.2.0 + v0.2.9 assets ≈ 565 MB).
|
||||||
|
- [ ] Verify the fallback on the next tag push produces a release automatically (the
|
||||||
|
v0.2.11 tag build reached the publish step and created the release; only the
|
||||||
|
upload failed on disk space).
|
||||||
- [ ] Optionally backfill releases for v0.2.4–v0.2.8 (their CI artifacts were not
|
- [ ] Optionally backfill releases for v0.2.4–v0.2.8 (their CI artifacts were not
|
||||||
retained; would need local rebuilds) or leave them superseded by the latest.
|
retained; would need local rebuilds) or leave them superseded by the latest.
|
||||||
|
|
||||||
|
|||||||
@@ -99,8 +99,31 @@ for f in "${all_artifacts[@]}"; do
|
|||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
echo "==> Uploading $(basename "$f")"
|
echo "==> Uploading $(basename "$f")"
|
||||||
curl -s -X POST "$API/$RELEASE_ID/assets" "${AUTH[@]}" -F "attachment=@$f" >/dev/null
|
# Fail loudly: a silent 4xx here used to leave a release with no artifacts while
|
||||||
|
# the job still reported success.
|
||||||
|
code="$(curl -sS -o /tmp/release-upload.json -w '%{http_code}' --retry 2 \
|
||||||
|
-X POST "$API/$RELEASE_ID/assets" "${AUTH[@]}" -F "attachment=@$f" || true)"
|
||||||
|
if [[ "$code" != "201" && "$code" != "200" ]]; then
|
||||||
|
echo "error: upload of $(basename "$f") failed with HTTP $code" >&2
|
||||||
|
cat /tmp/release-upload.json >&2 || true
|
||||||
|
echo "" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
done
|
done
|
||||||
|
|
||||||
|
# Verify the release actually carries the artifacts we just sent.
|
||||||
|
have="$(curl -sS "${AUTH[@]}" "$API/$RELEASE_ID" | python3 -c "
|
||||||
|
import sys, json
|
||||||
|
try:
|
||||||
|
d = json.load(sys.stdin)
|
||||||
|
except Exception:
|
||||||
|
print(0); raise SystemExit
|
||||||
|
print(len(d.get('assets', [])))" 2>/dev/null || echo 0)"
|
||||||
|
if [[ "${have:-0}" -lt "${#all_artifacts[@]}" ]]; then
|
||||||
|
echo "error: release has $have assets, expected ${#all_artifacts[@]}" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "Verified $have artifacts on the release"
|
||||||
|
|
||||||
echo ""
|
echo ""
|
||||||
echo "Release live: $HOST/$REPO/releases/tag/$TAG"
|
echo "Release live: $HOST/$REPO/releases/tag/$TAG"
|
||||||
Reference in new issue
Block a user