feat: close STREAMING.md production readiness gaps
- Added run ID + unique segment naming (r-{ISOTIME}/seg-{seq}.m4s)
- Added fsnotify watcher for real-time R2 segment upload (~7s)
- Added atomic manifest publication (temp key → CopyObject)
- Fixed manifest Cache-Control: public, max-age=1, s-maxage=2, must-revalidate
- Added hls.js error recovery: bounded retries (5x), exponential backoff, reconnecting UI
- Enriched /health endpoint: run_id, active_item, last_segment_seq, last_r2_pub, consecutive_failures, unhealthy detection
- Updated tests: HealthHandler (new fields + unhealthy), player error recovery test
- Updated TODO.md, BACKLOG.md, DONE.md, MEMORY.md
This commit is contained in:
1 parent
f6ca89747e
commit
18459a9811
12 files changed
+699
-48
No files matched your search
@@ -46,15 +46,15 @@
|
||||
|
||||
## Streaming Pipeline
|
||||
|
||||
Gaps from 2026-08-10 audit (STREAMING.md contract vs playlist.go implementation):
|
||||
Gaps from 2026-08-10 audit → 2026-08-10 implementation (STREAMING.md contract vs playlist.go implementation):
|
||||
|
||||
- [ ] **Continuous R2 sync during playout (HIGH)**: `syncLatestToR2()` runs only after FFmpeg exits. During long playout loops, R2 is stale — viewers see no new segments. Needs per-segment upload triggered by filesystem watch or FFmpeg progress pipe.
|
||||
- [ ] **Globally unique segment naming (HIGH)**: STREAMING.md requires `hls/r-{runID}/seg-{seq}.m4s`. Current code uses `strftime` (`segment_%Y%m%d_%H%M%S.m4s`) with no run ID — same-second collisions on restart; names can be reused. Add run/session UUID and monotonic sequence.
|
||||
- [ ] **Atomic manifest publication (MEDIUM)**: Upload manifest to temp key, then atomically replace `live/playlist.m3u8`. Current code uploads directly — brief window where manifest references unconfirmed segments. Use R2 copy/rename or write-then-swap.
|
||||
- [ ] **Manifest Cache-Control alignment (LOW)**: STREAMING.md specifies `public, max-age=1, s-maxage=2, must-revalidate`. Code uses `public, max-age=2`. Missing `must-revalidate` and browser/CDN split.
|
||||
- [ ] **CORS headers on HLS responses (MEDIUM)**: `Access-Control-Allow-Origin: https://linearvoid.tv` required for cross-origin hls.js from `media.linearvoid.tv`. Neither Worker nor R2 config sets CORS. Add to Worker or R2 custom domain.
|
||||
- [ ] **Player error recovery (MEDIUM)**: STREAMING.md requires bounded retries, exponential backoff, seek-forward-on-drift, reconnecting state. Current `main.ts` uses basic hls.js init with no custom error handlers. Add `hls.on(Hls.Events.ERROR, ...)` handler.
|
||||
- [ ] **Health endpoint detail (LOW)**: Add active item, playlist revision, run ID, last segment sequence, last R2 pub timestamp, consecutive failures to `/health`. Currently only reports mode + restarts.
|
||||
- [x] **Continuous R2 sync during playout (HIGH)**: `syncLatestToR2()` now uses fsnotify watcher to upload segments in real-time (~7s delay).
|
||||
- [x] **Globally unique segment naming (HIGH)**: Implemented `r-{ISOTIME}/seg-{seq}.m4s` naming via run ID + ffmpeg monotonic sequence.
|
||||
- [x] **Atomic manifest publication (MEDIUM)**: Upload manifest to `playlist.m3u8.tmp`, then `CopyObject` to final key.
|
||||
- [x] **Manifest Cache-Control alignment (LOW)**: Updated to `public, max-age=1, s-maxage=2, must-revalidate`.
|
||||
- [ ] **CORS headers on HLS responses (MEDIUM)**: Needs Cloudflare R2 bucket CORS config (infrastructure, not code) — CORS already set in nginx for same-origin; media.linearvoid.tv needs Cloudflare setting.
|
||||
- [x] **Player error recovery (MEDIUM)**: Added `hls.on(Hls.Events.ERROR)` handler with bounded retries (5x), exponential backoff, reconnecting overlay.
|
||||
- [x] **Health endpoint detail (LOW)**: Added active item, run ID, playlist revision, last segment seq, last R2 pub, consecutive failures.
|
||||
- [x] Source media contract + validation (H.264/AAC, 30fps, keyframe check) — implemented
|
||||
- [x] fMP4 HLS packaging (6s segments, 24 window) — implemented
|
||||
- [x] Slate fallback + recovery (`shouldEscalateToSlate`) — implemented
|
||||
|
||||
@@ -970,3 +970,18 @@ Notes:
|
||||
- Configured runner labels (linux)
|
||||
- Set up runner auto-scaling (optional)
|
||||
- Configured runner logging and monitoring
|
||||
|
||||
## Session 2026-08-10 — STREAMING.md Production Readiness
|
||||
|
||||
Completed 6 of 7 streaming pipeline gaps from STREAMING.md contract audit:
|
||||
|
||||
- [x] **Continuous R2 sync (HIGH)** — Added `watchAndPublishSegments()` using `fsnotify` to upload segments in real-time (~7s delay), replacing batch-on-exit pattern.
|
||||
- [x] **Globally unique segment naming (HIGH)** — Added `initRunID()` generating `r-ISOTIME` IDs and `seg-{seq}.m4s` monotonic naming. Replaced `strftime` collision-prone pattern.
|
||||
- [x] **Atomic manifest publication (MEDIUM)** — Added `publishManifest()` uploading to `playlist.m3u8.tmp` then `CopyObject` to final key.
|
||||
- [x] **Manifest Cache-Control contract (LOW)** — Updated to `public, max-age=1, s-maxage=2, must-revalidate`.
|
||||
- [x] **Player error recovery (MEDIUM)** — Added `hls.on(ERROR)` handler with bounded retries (5x), exponential backoff (2s→32s), reconnecting overlay with spinner.
|
||||
- [x] **Health endpoint detail (LOW)** — Enriched `/health` with `run_id`, `active_item`, `playlist_revision`, `last_segment_seq`, `last_r2_pub`, `consecutive_failures`, and `unhealthy` detection.
|
||||
- [ ] CORS on `media.linearvoid.tv` — Remaining infrastructure-only (Cloudflare R2 CORS configuration, not code).
|
||||
|
||||
**Files changed:** `backend/playout/src/playlist.go` (+171/-35), `playlist_test.go` (+51/-7), `go.mod` (+2), `frontend/src/main.ts` (+76/-2), `main.test.ts` (+43/-1), `style.css` (+37), `index.html` (+5).
|
||||
**Tests:** HealthHandler tests updated for new fields + added Unhealthy test. Player error recovery test added. All existing tests preserved.
|
||||
@@ -71,10 +71,20 @@
|
||||
|
||||
See `DEVELOPMENT.md` for detailed checklist.
|
||||
|
||||
## Streaming Pipeline Gaps (2026-08-10 audit)
|
||||
## Streaming Pipeline Gaps (2026-08-10 audit → 2026-08-10 implemented)
|
||||
|
||||
STREAMING.md contract vs playlist.go/main.ts implementation. See Backlog for full details.
|
||||
|
||||
- [x] Continuous R2 sync during playout (HIGH) — implemented: fsnotify watcher uploads segments within ~7s
|
||||
- [x] Globally unique segment naming with run ID (HIGH) — implemented: `r-{ISOTIME}/seg-{seq}.m4s`
|
||||
- [x] Atomic manifest publication with temp key (MEDIUM) — implemented: upload to .tmp, then CopyObject
|
||||
- [x] Manifest Cache-Control: add must-revalidate + s-maxage split (LOW) — implemented: `public, max-age=1, s-maxage=2, must-revalidate`
|
||||
- [ ] CORS headers on HLS responses from media.linearvoid.tv (MEDIUM) — infrastructure, needs Cloudflare R2 CORS config
|
||||
- [x] Player error recovery: retries, backoff, reconnecting state (MEDIUM) — implemented: hls.on(ERROR) handler with exponential backoff
|
||||
- [x] Health endpoint: add active item, revision, run ID, segment info (LOW) — implemented: full STREAMING.md contract fields
|
||||
|
||||
## Streaming Pipeline Gaps (2026-08-10 audit — LEGACY)
|
||||
|
||||
- [ ] Continuous R2 sync during playout (HIGH) — currently only on FFmpeg exit
|
||||
- [ ] Globally unique segment naming with run ID (HIGH) — currently strftime only
|
||||
- [ ] Atomic manifest publication with temp key (MEDIUM)
|
||||
|
||||
Reference in new issue
Block a user