linearvoid/.github/instructions/BACKLOG.md

BACKLOG

Consolidated from BACKLOG.md, DEVELOPMENT.md, and TODO.md on 2026-08-11.

Cloudflare Configuration

DNS & Pages

• Configure DNS for linearvoid.tv apex and www redirect (manual)
• Configure media.linearvoid.tv for split-domain HLS delivery (manual)
• Configure Cloudflare Pages deploy workflow (.gitea/workflows/pages-deploy.yml) and setup docs (docs/operations/cloudflare-pages-setup.md, docs/operations/gitea-cloudflare-pages.md)
• Create Cloudflare Pages project in dashboard (unavailable, created linearvoid via API)
• Cloudflare Pages: link custom domain (manual)
• Create Cloudflare Pages project and cut over frontend hosting (manual)

Storage & Cache

• Create R2 bucket (manual)
• Configure Cache Rules: .ts/.m4s + .m3u8 (manual)
• Implement Worker for immutable cache headers on segment paths
• CORS headers on HLS responses from media.linearvoid.tv (MEDIUM) — needs Cloudflare R2 CORS config

Security & TLS

• Enable Tiered Cache, WAF, rate limiting (manual)
• Tiered Cache: enable Smart Tiered Cache, validate Cf-Cache-Status: HIT from upper tier
• WAF: enable OWASP Managed Ruleset, set to Simulate → review 48h → Block, enable Bot Fight Mode
• Rate limiting: /api/admin/login + /api/admin/* → 10 req/10s per IP, Challenge action
• HTTPS/WSS: Always Use HTTPS, HSTS max-age=31536000 includeSubDomains preload, min TLS 1.2
• HTTPS/WSS: enforce at edge (Cloudflare) and origin (nginx), redirect HTTP → HTTPS

Monitoring & Observability

Prometheus & Grafana

• Set up Prometheus with chat/playout/nginx/redis/postgres/node scrape targets
• Add nginx-exporter, redis-exporter, postgres-exporter, node-exporter
• Import Grafana app + operational dashboards
• Define Prometheus alerting rules (8 rules: ChatDown, PlayoutDown, RedisDown, etc.)

External Monitoring

• Set up UptimeRobot external monitoring
• UptimeRobot: add HTTP monitor for https://linearvoid.tv/health (5min interval)
• UptimeRobot: add keyword monitor for https://linearvoid.tv/hls/playlist.m3u8 (keyword: #EXTM3U, 5min)
• Alert on 2 consecutive failures, notify via email + Discord/Slack webhook

Security

• Implement admin cookie-session auth (HttpOnly, SameSite=Strict, Secure when TLS)
• Implement server-side session expiration
• Implement rate limiting for chat messages (Redis windowing)
• Implement WebSocket connection rate limiting (5/IP/60s)
• Implement admin login rate limiting (5/IP/5min)
• Enforce HTTPS/WSS at edge and origin

Test Fixtures

• Create test media fixtures in backend/playout/testdata/: valid H.264/AAC, invalid codec, invalid framerate
• Create test playlist fixtures: valid_playlist.txt, empty_playlist.txt, comments_only.txt, mixed_valid_invalid.txt
• Document ffmpeg fixture-generation commands in test file headers

Frontend Tests

User Frontend

• Video player behavior (play, pause, seek, buffering, error handling) — HLS player startup, native/fallback, URL resolution, error recovery with retries/backoff/reconnecting overlay
• Chat functionality (send/receive messages, presence, rate limiting, moderation) — send, receive, presence join/leave, DM flow, reconnect covered
• State management and data flow
• Responsive design and layout across different screen sizes — chat-collapsed toggle, CSS grid, now-playing aria
• Accessibility testing (keyboard navigation, screen reader support) — video controls, aria-labels, aria-expanded, input labels, reconnecting overlay aria-live/role
• Integration with backend APIs (mocking API responses for testing)
• Test coverage for all critical frontend components and interactions
• Add code coverage reporting and enforce minimum coverage threshold in CI/CD pipeline
• Document test cases and expected outcomes

Admin Frontend

• Playlist management UI (add/remove videos, reorder, normalize) — save/error handling, sortable rendering
• User moderation tools (mute/ban/report users, view moderation logs)
• Metrics and monitoring dashboards (Prometheus/Grafana integration) — overview + analytics rendering
• State management and data flow
• Responsive design and layout across different screen sizes — tab content visibility, essential tabs
• Accessibility testing (keyboard navigation, screen reader support) — nav aria-label, active tab aria-current, nav button focusability
• Integration with backend APIs (mocking API responses for testing)
• Test coverage for all critical admin UI components and interactions
• Add code coverage reporting and enforce minimum coverage threshold in CI/CD pipeline
• Document test cases and expected outcomes

Backend Integration Tests

Cloudflare R2

• Upload test HLS segments and manifests to R2 bucket (SyncToR2 with mock, verified bucket/key/body)
• Verify correct cache headers are applied (immutable segments via segmentCacheControl, short-lived manifests via manifestCacheControl)
• Test retrieval of segments and manifests from R2 via Cloudflare edge
• Validate cache hit/miss behavior and performance metrics
• Validate edge cache hit/miss behavior with repeat requests — script scripts/validate-edge-cache.sh created
• Ensure integration test can run in CI/CD pipeline with proper mocking/stubbing of R2 API (putObject mock, no network calls)
• Document test cases and expected outcomes
• Document required stubs or mocks (putObject variable injection pattern)

Streaming Pipeline

• Continuous R2 sync during playout (HIGH): fsnotify watcher uploads segments in real-time (~7s delay)
• Globally unique segment naming (HIGH): r-{ISOTIME}/seg-{seq}.m4s via run ID + ffmpeg monotonic sequence
• Atomic manifest publication (MEDIUM): Upload to playlist.m3u8.tmp, then CopyObject to final key
• Manifest Cache-Control alignment (LOW): public, max-age=1, s-maxage=2, must-revalidate
• CORS headers on HLS responses (MEDIUM): Needs Cloudflare R2 bucket CORS config (infrastructure, not code)
• Player error recovery (MEDIUM): hls.on(Hls.Events.ERROR) handler with bounded retries (5x), exponential backoff, reconnecting overlay
• Health endpoint detail (LOW): Added active item, run ID, playlist revision, last segment seq, last R2 pub, consecutive failures
• Source media contract + validation (H.264/AAC, 30fps, keyframe check)
• fMP4 HLS packaging (6s segments, 24 window)
• Slate fallback + recovery (shouldEscalateToSlate)
• R2 cache headers (immutable segments, short-lived manifest)
• Cloudflare Worker immutable .m4s headers

External Services

Evidence tracker: docs/operations/cloudflare-uptimerobot-checklist.md

• UptimeRobot: add HTTP monitor for https://linearvoid.tv/health (5min interval)
• UptimeRobot: add keyword monitor for https://linearvoid.tv/hls/playlist.m3u8 (keyword: #EXTM3U, 5min)
• Alert on 2 consecutive failures, notify via email + Discord/Slack webhook

Production Deployment

Host Provisioning (Hetzner — not provisioned)

• Provision Hetzner CX22 (2 vCPU, 4 GB, 40 GB) in Ashburn VA, Ubuntu 24.04
• Install Docker Engine + Compose plugin, open firewall: 80/443 (public), 2375 (runner IP only)
• Configure Docker daemon for remote TLS access (-H tcp://0.0.0.0:2376 with certs)
• Create /srv/linearvoid/data/{playlist,media,hls,chat,slate} directories

CI/CD & Secrets

• Add CI_PROD_* secrets to Gitea: DOCKER_HOST, R2 keys, admin password, DB password, Grafana password
• Trigger CD workflow with DOCKER_HOST pointing to Hetzner IP

Smoke Test & Cutover

• Verify all 11 containers start healthy, check /health, /hls/playlist.m3u8, WebSocket
• Update Cloudflare A record linearvoid.tv → Hetzner IP, SSL Full (strict), test HTTPS + WSS
• Smoke test: viewer SPA loads, HLS plays, chat connects, admin login → upload → playlist → playout

Production Operations

• Set up Prometheus alerting notifications + UptimeRobot on production URL
• Run scripts/backup-db.sh + scripts/verify-backups.sh on production host
• Document production IP, SSH access, rollback procedure, scaling thresholds

Scaling & Performance

Load Testing

• Load-test scaffolding: ws-chat.yml, admin-api.js, run-loadtests.ps1, summarizers, README
• Cache-hit ratio exporter (scripts/cache-hit-ratio.js + .ps1)
• Saturation alerts: chat p95 latency, Redis memory usage, cache-hit ratio
• Execute WebSocket load tests with real infrastructure — ramp 10→100→500→1000 connections, measure p95 latency + errors
• Execute admin API benchmarks with real infrastructure — wrk/k6 against /api/admin/*, target p95 <1s

Cache & TTFB Optimization

• Optimize segment immutability and manifest TTL for cache efficiency
• Tune manifest TTL: measure Cf-Cache-Status distribution, adjust to 0.5× segment duration
• Measure playback TTFB from multiple US regions — WebPageTest from 3+ regions, target <4s warm cache
• Track cache hit ratio and R2 ops in production — Cloudflare GraphQL → Grafana panel, alert <90%

Capacity & Future Planning

• Capacity triggers: Prometheus rules for cache-miss >10%/15min, CPU >80%/10min
• Plan Phase 2 service separation — document boundaries (playout CPU, chat latency, DB state, monitoring), estimate per-service resources
• Plan Phase 3 multi-origin resilience — research second Hetzner location, design warm standby + DNS failover + DB/Redis replication

Feature List (Future Enhancements)

• Admin backend
  • Analytics: implement a settings entry to set matomo url and page id
• User frontend
  • Chat: add user registration and login flows
  • Chat: private DM history and search
  • Analytics: add javascript snippet to send pageview events to matomo