diff --git a/STREAMING.md b/STREAMING.md new file mode 100644 index 0000000..e44f88c --- /dev/null +++ b/STREAMING.md @@ -0,0 +1,186 @@ +# Streaming Pipeline + +This document is the operational contract for Linear Void video ingest, playout, publication, caching, and browser playback. General deployment, chat, and admin details remain in `DEPLOYMENT.md`, `SETUP.md`, and `API.md`. + +## Production flow + +```mermaid +flowchart LR + A["Normalized MP4 assets"] --> B["Playlist and schedule"] + B --> C["Playout service and FFmpeg"] + C --> D["Local HLS staging"] + D --> E["R2 object storage"] + E --> F["Cloudflare edge"] + F --> G["hls.js or native HLS"] +``` + +The playout service is the timeline authority. It reads the ordered playlist, runs FFmpeg continuously, stages a live HLS window under `data/hls/`, and publishes complete objects to R2. Viewers read only the public R2 custom domain through Cloudflare: + +```text +https://media.linearvoid.tv/hls/playlist.m3u8 +``` + +Nginx `/hls/` is a local/test fallback, not the production media origin. The SPA, admin API, and chat use `linearvoid.tv`; media uses `media.linearvoid.tv`. + +## Source-media contract + +Only validated files may enter the production playlist. + +| Property | Required profile | +| ---------- | ------------------------------------------------------------------------ | +| Container | MP4 | +| Video | H.264/AVC, `yuv420p` | +| Resolution | 1280x720 or 1920x1080; avoid changing resolution within a playlist block | +| Frame rate | Constant 30 fps | +| GOP | Closed, fixed 2 seconds: `-g 60 -keyint_min 60 -sc_threshold 0` | +| Audio | AAC-LC, stereo, 48 kHz, 128–192 kbit/s | + +The ingest gate uses `ffprobe` to reject incompatible codecs, variable frame rate, missing audio, invalid dimensions, or incorrect keyframe cadence. Assets are immutable after acceptance; corrections create a new asset ID and object key. + +Reference normalization: + +```bash +ffmpeg -i input.mov \ + -c:v libx264 -preset medium -crf 21 \ + -r 30 -g 60 -keyint_min 60 -sc_threshold 0 \ + -pix_fmt yuv420p \ + -c:a aac -b:a 160k -ar 48000 -ac 2 \ + -movflags +faststart \ + output-normalized.mp4 +``` + +## Playlist and timeline + +- `data/playlist/playlist.txt` contains one local media path per line; blank lines and `#` comments are ignored. +- The service validates the complete playlist before activating a revision. +- Playlist edits are written atomically and receive a revision ID. The active FFmpeg run keeps its current item and adopts the new revision at the next item boundary. +- All viewers follow the same live manifest; clients do not calculate the schedule independently. +- Schedule metadata must be derived from the same active playlist revision and timeline so “now/next” agrees with the video. +- When the last item ends, playout returns to the first item unless an active schedule override says otherwise. + +## Playout and HLS packaging + +The playout service wraps FFmpeg, tracks the active item and playlist revision, restarts failed processes, and exposes `/health` plus Prometheus metrics. + +Baseline packaging: + +| Setting | Value | +| ---------------- | ------------------------------------------------------------------------------------- | +| Protocol | Standard live HLS, not LL-HLS | +| Segment format | fMP4 (`.m4s`) with an initialization segment; `.ts` may be supported during migration | +| Segment target | 6 seconds (`HLS_SEGMENT_DURATION=6`) | +| Live window | 24 segments (`HLS_WINDOW_SIZE=24`), about 144 seconds | +| Codec path | Stream copy for conforming assets; no per-viewer transcoding | +| Expected latency | 15–35 seconds | + +Segment boundaries must align with keyframes. Timestamps must remain monotonic across item boundaries. If FFmpeg cannot preserve a seamless timeline, insert `#EXT-X-DISCONTINUITY` and publish the corresponding new initialization segment. + +Every segment name is globally unique and never reused, including after a restart or playlist loop. Use a run/session ID plus a monotonically increasing media sequence, for example: + +```text +hls/r-20260807T170412Z/init-000001.mp4 +hls/r-20260807T170412Z/seg-000001234.m4s +``` + +This naming rule is essential: a URL advertised as immutable must never later contain different bytes. + +## Safe publication to R2 + +Publication order prevents a manifest from referencing an unavailable segment: + +1. FFmpeg writes a segment to a temporary local name and closes it. +2. The publisher uploads the initialization segment, if new, and media segment to their final immutable R2 keys. +3. It verifies successful upload and records object size/checksum. +4. It renders the next media manifest referencing only confirmed objects. +5. It uploads the manifest to a temporary key, then replaces `hls/playlist.m3u8` as the final commit step. + +Never upload an actively written segment or publish the manifest first. Use a single publisher instance, or fencing/leader election, so two playout workers cannot update the live manifest concurrently. + +The local `data/hls/` directory is staging data and may be regenerated. Source media, playlist revisions, and schedule state are authoritative; generated HLS is not backed up as application state. + +## Cache contract + +| Objects | Edge policy | Browser policy | Required headers/behavior | +| -------------------------- | ---------------------------------- | --------------------------------------- | ---------------------------------------------------------------------------- | +| `.m4s`, `.ts`, init `.mp4` | Cache eligible; TTL 1 year | TTL 1 year | `Cache-Control: public, max-age=31536000, immutable`; full path in cache key | +| Live `.m3u8` | Cache eligible; edge TTL 2 seconds | Do not retain beyond freshness interval | `Cache-Control: public, max-age=1, s-maxage=2, must-revalidate` | +| Versioned JS/CSS | Cache eligible; TTL 1 year | TTL 1 year | Content-hashed filenames and `immutable` | + +Further rules: + +- Do not add query-string cache busters to segment URLs; uniqueness belongs in the path. +- Cache keys must preserve hostname and full path. Ignore only explicitly approved tracking parameters. +- HLS responses require `Access-Control-Allow-Origin: https://linearvoid.tv` (or the reviewed origin allowlist), `GET`, `HEAD`, and `OPTIONS` support, and correct MIME types. +- Enable Smart Tiered Cache for `media.linearvoid.tv` to reduce R2 reads on regional misses. +- Purge manifests when necessary, but never purge immutable segments during normal operation. +- Retain expired-window segments in R2 for at least the maximum manifest TTL plus client retry/grace time. A practical minimum is one hour; lifecycle deletion may run later according to storage policy. + +Expected validation: + +```bash +curl -I https://media.linearvoid.tv/hls/playlist.m3u8 +curl -I https://media.linearvoid.tv/hls/.m4s +``` + +The manifest must refresh within a few seconds. Repeated segment requests should progress from `CF-Cache-Status: MISS` to `HIT`, retain the same ETag/content length, and carry the immutable cache header. + +## Player behavior + +The SPA uses native HLS where supported and hls.js elsewhere. + +- Start at the live edge with a conservative buffer; do not seek to playlist start. +- Treat the manifest as the synchronization source and target roughly three segments behind the newest segment unless testing supports a smaller margin. +- Recover transient network/media errors with bounded retries and exponential backoff. +- On a fatal hls.js network error, reload the manifest; on a fatal media error, attempt media recovery once before recreating the player. +- When tab suspension or drift leaves playback materially behind the target live position, seek forward to the target rather than playing the backlog. +- Surface a clear reconnecting state. Never silently loop stale media. +- Schedule/chat timestamps should account for the measured player latency rather than assuming wall-clock playout is already visible. + +## Failure and recovery + +The service retries an item failure with a bounded backoff. After the configured failure threshold it switches to a pre-normalized looping slate and reports `mode=slate`. It periodically retries normal playout and returns only after a segment and committed manifest have been published successfully. + +On process or host restart: + +- create a new run/session ID and continue with globally unique segment paths; +- resume from the schedule-derived current position when reliable state exists, otherwise start the current playlist item; +- never overwrite old segment objects; +- keep the last valid manifest available until the replacement stream is playable; +- emit a discontinuity when timeline continuity cannot be guaranteed. + +Health is not merely “process running.” `/health` should report at least mode (`live`, `slate`, `starting`, `degraded`), active item, playlist revision, run ID, last segment sequence/time, last successful R2 publication, consecutive failures, and uptime. Mark the service unhealthy if the manifest or media sequence has not advanced for more than two segment durations. + +## Monitoring and acceptance checks + +Monitor and alert on: + +- manifest age and media-sequence advancement; +- time since last successfully published segment; +- FFmpeg restarts and consecutive item failures; +- live/slate mode; +- R2 upload latency and errors; +- segment 404/5xx rate and manifest/segment request latency; +- Cloudflare cache-hit ratio, origin requests, and R2 Class B operations; +- player startup time, fatal errors, rebuffer ratio, and live-edge drift; +- local staging disk usage. + +A release is streaming-ready only when one full playlist boundary has played without timestamp errors, the manifest never references a missing object, a second viewer joins the same timeline, cache headers match the table above, a forced FFmpeg failure activates the slate, and normal playout recovers without reusing an object URL. + +## Required configuration + +| Variable | Purpose | +| ------------------------------------------ | -------------------------------------------------- | +| `PLAYLIST_PATH` | Active playlist file | +| `HLS_SEGMENT_DURATION` | Target segment duration; baseline `6` | +| `HLS_WINDOW_SIZE` | Manifest window; baseline `24` | +| `R2_ENDPOINT`, `R2_REGION` | S3-compatible publication endpoint | +| `R2_BUCKET_NAME` | Media bucket | +| `R2_ACCESS_KEY_ID`, `R2_SECRET_ACCESS_KEY` | Write credentials, available only to the publisher | +| `MEDIA_DOMAIN` | Public R2 custom domain | +| `VITE_PUBLIC_HLS_URL` | Public live manifest URL | + +Production values are `MEDIA_DOMAIN=media.linearvoid.tv` and `VITE_PUBLIC_HLS_URL=https://media.linearvoid.tv/hls/playlist.m3u8`. Local fallback may use `http://localhost/hls/playlist.m3u8`. + +## Explicitly deferred + +Adaptive-bitrate ladders, LL-HLS, DVR/catch-up, multi-origin playout, and regional chat affinity are outside the v1 contract. Adding any of them requires an ADR and an update to this document.