zwitschi/linearvoid
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 9 Wiki Activity
Files
06ab5b4655c3bd8347ee1ab4b2c60279231a4584
linearvoid/docs/architecture/adr/ADR-001.md
T

9.6 KiB
Raw Blame History

ADR-001: Cost-Efficient Self-Hosted 24/7 Synchronized Linear Video Streaming with Custom Chat (US-Centric, Cloudflare + R2 Optimized)

Status: Accepted
Date: 2026-08-02
Authors: Synthesized from Grok, GPT-Terra, DeepSeek, Gemini (research-assisted)
Decision Owners: Product & Engineering
Scope: One 24/7 linear video channel, custom viewer website, public/private chat, playlist-driven playout. Related ADRs: authentication, moderation, payments, multi-channel.

Context

We require a completely independent 24/7 linear video channel that broadcasts a continuous playlist of self-owned pre-recorded videos. Viewers consume it via a lightweight custom HTML/CSS/JavaScript single-page application (no reliance on YouTube, Twitch, Vimeo, or paid Cloudflare Stream tiers). The experience must include:

  • True (or near-true) synchronized playback — all viewers see the same program timeline.
  • Embedded custom chat with public rooms, private 1:1/direct messages, presence, moderation (mute/ban/report), rate limiting.
  • “Now playing” schedule/metadata.
  • Full control over source media, frontend, schedule, and chat data.
  • Primary audience and operational origin in the United States (East/Central preferred for uploads and low latency).

Key distinctions:

  • Playout: Continuous scheduled program from playlist.
  • Packaging: FFmpeg → standard HLS (.m3u8 + .m4s segments, 4–6s duration).
  • Delivery: Immutable segments served via Cloudflare edge caching + R2 (zero egress cost).
  • Interaction: Separate lightweight WebSocket service for chat.

Bandwidth (not storage or compute) is the dominant cost driver. A single viewer at ~4 Mbps consumes ~1.3–1.8 TB/month; 100 concurrent viewers = ~130–180 TB/month. Direct VPS delivery is only viable for PoC/small audiences. HLS segments are typically .m4s files, with .m4s being the default for fMP4 segments.

Decision Drivers

  1. Minimize monthly cost (target <$25 for moderate scale).
  2. Full independence and ownership (no video platform lock-in).
  3. Strong US performance and low latency.
  4. True/near-true synchronization via live-style HLS (not pure VOD).
  5. Aggressive immutable caching of .m4s segments to achieve near-zero egress.
  6. Browser-native playback (hls.js + native HLS).
  7. Lightweight, maintainable stack with modest operational complexity.
  8. Support for custom multi-room + private chat with persistence and moderation.
  9. Scalability from tens to thousands of concurrent viewers without linear cost growth.
  10. Compliance with Cloudflare ToS (use R2 as origin for large video files).

Decision

Adopt a self-managed US-East playout + Cloudflare R2 + edge-cached HLS architecture:

1. Video Pipeline & Playout (US-East VPS)

  • Location: Hetzner Cloud CX22 (or equivalent) in Ashburn, VA (~$4–8/mo, 2 vCPU/4 GB, 40 GB NVMe, 20 TB included traffic). Excellent peering.
  • Tools:
    • FFmpeg (concat demuxer + while true loop for auto-restart) for playlist playout and normalization (-c copy where possible, CRF encoding).
    • MediaMTX (preferred), SRS, or Nginx-RTMP as lightweight media server/packager for continuous HLS output.
    • Sliding window HLS: 4–6s segments, playlist length ~24–60s.
  • Source Normalization (mandatory before scheduling):
    • Container: MP4
    • Video: H.264/AVC, yuv420p, constant 30 fps, keyframe interval 2s
    • Audio: AAC-LC, 48 kHz, stereo, 128–192 kbps
    • Bitrate targets: 720p30 (2.5–5 Mbps), 1080p30 (5–8 Mbps)
  • Dashboard for playlist management, current file, health.

2. Storage & Delivery (Cloudflare R2 + CDN)

  • HLS segments (.m4s) and manifests uploaded to Cloudflare R2 (Eastern North America hint enam).
  • Custom domain (media.linearvoid.tv) fronted by Cloudflare with Cache Rules:
    • *.m4s (and init .mp4 if using fMP4 later): Cache Everything, immutable, Edge TTL 24h–1yr (Cache-Control: public, max-age=31536000, immutable).
    • *.m3u8: Short TTL (1–10s) or must-revalidate (Cache-Control: public, max-age=1..10, must-revalidate).
    • Master playlist: TTL ~30s.
    • Tiered Cache enabled where possible.
  • R2 as origin satisfies ToS for large video; $0 egress, free 10 GB, then ~$0.015/GB-mo (Standard).
  • Viewer player hits Cloudflare edge → vast majority of traffic served from cache (origin/R2 hit only once per region).

3. Frontend

  • Static SPA (HTML + custom CSS + vanilla JS) hosted on Cloudflare Pages or same VPS/Nginx.
  • Video player: hls.js (Chromium/Firefox) or native HLS (Safari). Single live master.m3u8.
  • Embedded chat sidebar with room switcher, private DMs, now-playing overlay.

4. Custom Chat

  • Lightweight backend (Go, Node.js + Socket.io, or Elixir) on same VPS.
  • WebSockets (wss://), Redis (presence, pub/sub, rate limits), PostgreSQL/SQLite (message history, moderation log, users).
  • Features: public rooms, private 1:1, presence, rate limiting, mute/timeout/ban/report, basic search.
  • Persistence with retention policy. Moderation audit log. No E2EE by default (separate ADR if required).

5. Infrastructure Summary & Cost Model

  • Primary: Hetzner Ashburn VPS (playout + chat + Nginx) + Cloudflare R2 + Pages.
  • Monitoring: MediaMTX/SRS dashboard, UptimeRobot, systemd logs, Cloudflare analytics.
  • Backup: Weekly R2/S3-compatible snapshots of playlist, DB, config.
  • Fallback: Emergency slate/loop playlist, automated restart.
  • Monthly Cost (early production, moderate US audience): $5–25 (VPS $4–8 + R2 $1–3 + negligible requests). Scales to hundreds/thousands of viewers with near-zero additional egress.

Initial Targets:

  • Protocol: Standard HLS (.m4s segments). Defer LL-HLS/CMAF/WebRTC.
  • Max rendition: 720p30 initially (upgrade to 1080p30 or ABR later).
  • Latency: 15–35s (acceptable for linear programming/chat).

Storage for 48 Hours of Source Footage

Formula: GB ≈ (bitrate_Mbps × duration_seconds) / 8 / 1024 (+5–10% overhead).

Resolution Video Bitrate Total w/ Audio 48h Raw Recommended Allocation R2 Cost/mo (Standard)
720p30 2.5–4 Mbps ~2.6–4.2 Mbps 57–90 GB ~80–110 GB ~$1.20–$1.65
1080p30 5–8 Mbps ~5.1–8.2 Mbps 110–175 GB ~130–180 GB ~$1.95–$2.70

Plan 2–3× headroom for variants, backups, artwork, processing. Use R2 Infrequent Access if latency acceptable.

Rationale

  • Hetzner Ashburn + Cloudflare R2 delivers unmatched price/performance for US traffic. R2’s $0 egress + edge caching eliminates the dominant bandwidth cost.
  • HLS is the most compatible, cache-friendly, operationally simple protocol for one-to-many linear broadcast.
  • FFmpeg + MediaMTX/SRS provides reliable playlist-driven continuous playout with low CPU.
  • Immutable segment naming + targeted Cache Rules ensure high hit ratios while keeping live manifests fresh.
  • Lightweight static frontend + separate lightweight chat service keeps complexity low.
  • Starts simple (single VPS) and evolves (ABR, secondary origins, managed DBs, LL-HLS) without breaking changes.

This hybrid synthesizes the strongest elements from all evaluated ADRs while resolving minor variances (Hetzner vs generic VPS, direct origin vs R2, SQLite vs PostgreSQL+Redis, exact bitrates).

Considered Alternatives

  • Pure VPS origin (no R2/CDN): Rejected for production (20 TB cap limits ~12–20 concurrent HD viewers).
  • Managed platforms (YouTube/Twitch/Cloudflare Stream): Rejected (loss of control, higher cost at scale, ToS restrictions).
  • WebRTC / P2P: High complexity, poor caching, not suited for passive linear broadcast.
  • Owncast: Good for turnkey but less flexible for custom playlist/schedule/chat.
  • LL-HLS or CMAF initially: Deferred (added complexity; standard HLS sufficient for v1).

Consequences

Positive:

  • Near-zero egress cost → unlimited viewer scaling.
  • Full ownership, US performance, synchronized linear experience.
  • Extremely low TCO (~$5–25/mo).
  • Cacheable, maintainable, evolvable architecture.

Negative / Risks & Technical Debt:

  • Cloudflare ToS scrutiny for heavy video (mitigated by R2 path; monitor & have BunnyCDN fallback).
  • HLS latency (15–35s) affects real-time chat feel.
  • Single-origin SPOF (mitigate with DNS failover/secondary region later).
  • Video normalization and cache-rule tuning required upfront.
  • Chat moderation and abuse prevention operational burden.
  • R2 request costs on cache misses (measure in production; Tiered Cache helps).
  • No built-in E2EE for private chats (future ADR).

Operational Requirements:

  • Monitor playout health, segment generation, cache hit ratio, R2 ops, WebSocket connections.
  • Maintain fallback slate, emergency playlist, backup/restore procedures.
  • Strict CORS, HTTPS, WAF, rate limiting, authenticated admin paths.

Implementation Plan

Phase 1 (Prototype): Hetzner VPS, normalized playlist, FFmpeg+MediaMTX → HLS to R2, basic hls.js page + Socket.io chat, cache rules, load test.

Phase 2 (MVP): Scheduler/metadata, PostgreSQL+Redis, full moderation, dashboards/alerts, emergency fallback.

Phase 3 (Scale): ABR, multi-region, LL-HLS evaluation, paid access controls.

References

  • Cloudflare R2 pricing, custom domains, cache rules, Tiered Cache.
  • MediaMTX / SRS / FFmpeg HLS documentation.
  • Hetzner Cloud Ashburn pricing (2026).
  • hls.js, Video.js, standard HLS best practices.
  • Video bitrate/storage calculators.

This consolidated ADR merges all prior drafts. Future ADRs will detail Docker Compose, exact cache rules, playlist normalization scripts, chat schema, monitoring, and access control.

Reference in new issue View Git Blame Copy Permalink