docs: fix filename typo and add Cloudflare Pages deploy details across docs

.github/AGENTS.md

@@ -2,10 +2,6 @@
 
 Agents instructions for the linearvoid project.
 
-## Output
-
-- use /caveman skill, latest version from <https://github.com/JuliusBrussee/caveman> is <https://github.com/JuliusBrussee/caveman/tree/main/skills/caveman>
-
 ## General Guidelines
 
 - Use the architecture documentation (arc42 template and ADRs in `docs`) as the source of truth for design decisions.

.github/instructions/BACKLOG.md

@@ -1,62 +1,173 @@
 # BACKLOG
 
+> Consolidated from `BACKLOG.md`, `DEVELOPMENT.md`, and `TODO.md` on 2026-08-11.
+
 ## Cloudflare Configuration
 
+### DNS & Pages
+
 - [x] Configure DNS for linearvoid.tv apex and www redirect (manual)
 - [x] Configure media.linearvoid.tv for split-domain HLS delivery (manual)
+- [x] Configure Cloudflare Pages deploy workflow (`.gitea/workflows/pages-deploy.yml`) and setup docs (`docs/operations/cloudflare-pages-setup.md`, `docs/operations/gitea-cloudflare-pages.md`)
+- [x] Create Cloudflare Pages project in dashboard (unavailable, created `linearvoid` via API)
+- [ ] Cloudflare Pages: link custom domain (manual)
+- [ ] Create Cloudflare Pages project and cut over frontend hosting (manual)
+
+### Storage & Cache
+
 - [x] Create R2 bucket (manual)
 - [x] Configure Cache Rules: .ts/.m4s + .m3u8 (manual)
-- [x] Configure Cloudflare Pages deploy workflow (`.gitea/workflows/pages-deploy.yml`) and setup docs (`docs/operations/cloudflare-pages-setup.md`, `docs/operations/gitea-cloudflare.pages.md`)
-- [ ] Create Cloudflare Pages project in dashboard and link custom domain (manual)
-- [ ] Enable Tiered Cache, WAF, rate limiting (manual)
 - [x] Implement Worker for immutable cache headers on segment paths
-- [ ] Enforce HTTPS/WSS, redirect HTTP → HTTPS (manual)
+- [ ] CORS headers on HLS responses from media.linearvoid.tv (MEDIUM) — needs Cloudflare R2 CORS config
+
+### Security & TLS
+
+- [ ] Enable Tiered Cache, WAF, rate limiting (manual)
+- [ ] Tiered Cache: enable Smart Tiered Cache, validate `Cf-Cache-Status: HIT` from upper tier
+- [ ] WAF: enable OWASP Managed Ruleset, set to Simulate → review 48h → Block, enable Bot Fight Mode
+- [ ] Rate limiting: `/api/admin/login` + `/api/admin/*` → 10 req/10s per IP, Challenge action
+- [ ] HTTPS/WSS: Always Use HTTPS, HSTS max-age=31536000 includeSubDomains preload, min TLS 1.2
+- [ ] HTTPS/WSS: enforce at edge (Cloudflare) and origin (nginx), redirect HTTP → HTTPS
 
 ## Monitoring & Observability
 
+### Prometheus & Grafana
+
 - [x] Set up Prometheus with chat/playout/nginx/redis/postgres/node scrape targets
 - [x] Add nginx-exporter, redis-exporter, postgres-exporter, node-exporter
 - [x] Import Grafana app + operational dashboards
 - [x] Define Prometheus alerting rules (8 rules: ChatDown, PlayoutDown, RedisDown, etc.)
+
+### External Monitoring
+
 - [ ] Set up UptimeRobot external monitoring
+- [ ] UptimeRobot: add HTTP monitor for `https://linearvoid.tv/health` (5min interval)
+- [ ] UptimeRobot: add keyword monitor for `https://linearvoid.tv/hls/playlist.m3u8` (keyword: `#EXTM3U`, 5min)
+- [ ] Alert on 2 consecutive failures, notify via email + Discord/Slack webhook
 
 ## Security
 
 - [x] Implement admin cookie-session auth (HttpOnly, SameSite=Strict, Secure when TLS)
 - [x] Implement server-side session expiration
-- [ ] Enforce HTTPS/WSS at edge and origin
 - [x] Implement rate limiting for chat messages (Redis windowing)
 - [x] Implement WebSocket connection rate limiting (5/IP/60s)
 - [x] Implement admin login rate limiting (5/IP/5min)
+- [ ] Enforce HTTPS/WSS at edge and origin
 
-## Scaling & Performance
+## Test Fixtures
 
-- [x] Create scaling/performance playbook (methods, thresholds, execution order)
-- [x] Load-test scaffolding: ws-chat.yml, admin-api.js, run-loadtests.ps1, summarizers
-- [x] Cache-hit ratio exporter (scripts/cache-hit-ratio.js + .ps1)
-- [x] Saturation alerts: chat p95, Redis memory, cache-hit ratio
-- [ ] Create Cloudflare Pages project and cut over frontend hosting (manual)
-- [ ] Optimize segment immutability and manifest TTL for cache efficiency
-- [ ] Measure playback TTFB from multiple US regions
-- [ ] Execute WebSocket load tests with real infrastructure
-- [ ] Execute admin API benchmarks with real infrastructure
-- [ ] Track cache hit ratio and R2 ops in production
-- [ ] Plan Phase 2 service separation
-- [ ] Plan Phase 3 multi-origin resilience
+- [x] Create test media fixtures in `backend/playout/testdata/`: valid H.264/AAC, invalid codec, invalid framerate
+- [x] Create test playlist fixtures: `valid_playlist.txt`, `empty_playlist.txt`, `comments_only.txt`, `mixed_valid_invalid.txt`
+- [x] Document ffmpeg fixture-generation commands in test file headers
+
+## Frontend Tests
+
+### User Frontend
+
+- [x] Video player behavior (play, pause, seek, buffering, error handling) — HLS player startup, native/fallback, URL resolution, error recovery with retries/backoff/reconnecting overlay
+- [x] Chat functionality (send/receive messages, presence, rate limiting, moderation) — send, receive, presence join/leave, DM flow, reconnect covered
+- [x] State management and data flow
+- [x] Responsive design and layout across different screen sizes — chat-collapsed toggle, CSS grid, now-playing aria
+- [x] Accessibility testing (keyboard navigation, screen reader support) — video controls, aria-labels, aria-expanded, input labels, reconnecting overlay aria-live/role
+- [x] Integration with backend APIs (mocking API responses for testing)
+- [ ] Test coverage for all critical frontend components and interactions
+- [ ] Add code coverage reporting and enforce minimum coverage threshold in CI/CD pipeline
+- [ ] Document test cases and expected outcomes
+
+### Admin Frontend
+
+- [x] Playlist management UI (add/remove videos, reorder, normalize) — save/error handling, sortable rendering
+- [ ] User moderation tools (mute/ban/report users, view moderation logs)
+- [x] Metrics and monitoring dashboards (Prometheus/Grafana integration) — overview + analytics rendering
+- [x] State management and data flow
+- [x] Responsive design and layout across different screen sizes — tab content visibility, essential tabs
+- [x] Accessibility testing (keyboard navigation, screen reader support) — nav aria-label, active tab aria-current, nav button focusability
+- [x] Integration with backend APIs (mocking API responses for testing)
+- [ ] Test coverage for all critical admin UI components and interactions
+- [ ] Add code coverage reporting and enforce minimum coverage threshold in CI/CD pipeline
+- [ ] Document test cases and expected outcomes
+
+## Backend Integration Tests
+
+### Cloudflare R2
+
+- [x] Upload test HLS segments and manifests to R2 bucket (SyncToR2 with mock, verified bucket/key/body)
+- [x] Verify correct cache headers are applied (immutable segments via segmentCacheControl, short-lived manifests via manifestCacheControl)
+- [ ] Test retrieval of segments and manifests from R2 via Cloudflare edge
+- [ ] Validate cache hit/miss behavior and performance metrics
+- [ ] Validate edge cache hit/miss behavior with repeat requests — script `scripts/validate-edge-cache.sh` created
+- [x] Ensure integration test can run in CI/CD pipeline with proper mocking/stubbing of R2 API (putObject mock, no network calls)
+- [x] Document test cases and expected outcomes
+- [x] Document required stubs or mocks (putObject variable injection pattern)
 
 ## Streaming Pipeline
 
-Gaps from 2026-08-10 audit → 2026-08-10 implementation (STREAMING.md contract vs playlist.go implementation):
+- [x] **Continuous R2 sync during playout (HIGH)**: fsnotify watcher uploads segments in real-time (~7s delay)
+- [x] **Globally unique segment naming (HIGH)**: `r-{ISOTIME}/seg-{seq}.m4s` via run ID + ffmpeg monotonic sequence
+- [x] **Atomic manifest publication (MEDIUM)**: Upload to `playlist.m3u8.tmp`, then `CopyObject` to final key
+- [x] **Manifest Cache-Control alignment (LOW)**: `public, max-age=1, s-maxage=2, must-revalidate`
+- [ ] **CORS headers on HLS responses (MEDIUM)**: Needs Cloudflare R2 bucket CORS config (infrastructure, not code)
+- [x] **Player error recovery (MEDIUM)**: `hls.on(Hls.Events.ERROR)` handler with bounded retries (5x), exponential backoff, reconnecting overlay
+- [x] **Health endpoint detail (LOW)**: Added active item, run ID, playlist revision, last segment seq, last R2 pub, consecutive failures
+- [x] Source media contract + validation (H.264/AAC, 30fps, keyframe check)
+- [x] fMP4 HLS packaging (6s segments, 24 window)
+- [x] Slate fallback + recovery (`shouldEscalateToSlate`)
+- [x] R2 cache headers (immutable segments, short-lived manifest)
+- [x] Cloudflare Worker immutable `.m4s` headers
 
-- [x] **Continuous R2 sync during playout (HIGH)**: `syncLatestToR2()` now uses fsnotify watcher to upload segments in real-time (~7s delay).
-- [x] **Globally unique segment naming (HIGH)**: Implemented `r-{ISOTIME}/seg-{seq}.m4s` naming via run ID + ffmpeg monotonic sequence.
-- [x] **Atomic manifest publication (MEDIUM)**: Upload manifest to `playlist.m3u8.tmp`, then `CopyObject` to final key.
-- [x] **Manifest Cache-Control alignment (LOW)**: Updated to `public, max-age=1, s-maxage=2, must-revalidate`.
-- [ ] **CORS headers on HLS responses (MEDIUM)**: Needs Cloudflare R2 bucket CORS config (infrastructure, not code) — CORS already set in nginx for same-origin; media.linearvoid.tv needs Cloudflare setting.
-- [x] **Player error recovery (MEDIUM)**: Added `hls.on(Hls.Events.ERROR)` handler with bounded retries (5x), exponential backoff, reconnecting overlay.
-- [x] **Health endpoint detail (LOW)**: Added active item, run ID, playlist revision, last segment seq, last R2 pub, consecutive failures.
-- [x] Source media contract + validation (H.264/AAC, 30fps, keyframe check) — implemented
-- [x] fMP4 HLS packaging (6s segments, 24 window) — implemented
-- [x] Slate fallback + recovery (`shouldEscalateToSlate`) — implemented
-- [x] R2 cache headers (immutable segments, short-lived manifest) — implemented
-- [x] Cloudflare Worker immutable `.m4s` headers — implemented
+## External Services
+
+> Evidence tracker: `docs/operations/cloudflare-uptimerobot-checklist.md`
+
+- [ ] UptimeRobot: add HTTP monitor for `https://linearvoid.tv/health` (5min interval)
+- [ ] UptimeRobot: add keyword monitor for `https://linearvoid.tv/hls/playlist.m3u8` (keyword: `#EXTM3U`, 5min)
+- [ ] Alert on 2 consecutive failures, notify via email + Discord/Slack webhook
+
+## Production Deployment
+
+### Host Provisioning (Hetzner — not provisioned)
+
+- [ ] Provision Hetzner CX22 (2 vCPU, 4 GB, 40 GB) in Ashburn VA, Ubuntu 24.04
+- [ ] Install Docker Engine + Compose plugin, open firewall: 80/443 (public), 2375 (runner IP only)
+- [ ] Configure Docker daemon for remote TLS access (`-H tcp://0.0.0.0:2376` with certs)
+- [ ] Create `/srv/linearvoid/data/{playlist,media,hls,chat,slate}` directories
+
+### CI/CD & Secrets
+
+- [ ] Add `CI_PROD_*` secrets to Gitea: DOCKER_HOST, R2 keys, admin password, DB password, Grafana password
+- [ ] Trigger CD workflow with `DOCKER_HOST` pointing to Hetzner IP
+
+### Smoke Test & Cutover
+
+- [ ] Verify all 11 containers start healthy, check `/health`, `/hls/playlist.m3u8`, WebSocket
+- [ ] Update Cloudflare A record `linearvoid.tv` → Hetzner IP, SSL Full (strict), test HTTPS + WSS
+- [ ] Smoke test: viewer SPA loads, HLS plays, chat connects, admin login → upload → playlist → playout
+
+### Production Operations
+
+- [ ] Set up Prometheus alerting notifications + UptimeRobot on production URL
+- [ ] Run `scripts/backup-db.sh` + `scripts/verify-backups.sh` on production host
+- [ ] Document production IP, SSH access, rollback procedure, scaling thresholds
+
+## Scaling & Performance
+
+### Load Testing
+
+- [x] Load-test scaffolding: `ws-chat.yml`, `admin-api.js`, `run-loadtests.ps1`, summarizers, README
+- [x] Cache-hit ratio exporter (`scripts/cache-hit-ratio.js` + `.ps1`)
+- [x] Saturation alerts: chat p95 latency, Redis memory usage, cache-hit ratio
+- [ ] Execute WebSocket load tests with real infrastructure — ramp 10→100→500→1000 connections, measure p95 latency + errors
+- [ ] Execute admin API benchmarks with real infrastructure — `wrk`/`k6` against `/api/admin/*`, target p95 <1s
+
+### Cache & TTFB Optimization
+
+- [ ] Optimize segment immutability and manifest TTL for cache efficiency
+- [ ] Tune manifest TTL: measure `Cf-Cache-Status` distribution, adjust to 0.5× segment duration
+- [ ] Measure playback TTFB from multiple US regions — WebPageTest from 3+ regions, target <4s warm cache
+- [ ] Track cache hit ratio and R2 ops in production — Cloudflare GraphQL → Grafana panel, alert <90%
+
+### Capacity & Future Planning
+
+- [ ] Capacity triggers: Prometheus rules for cache-miss >10%/15min, CPU >80%/10min
+- [ ] Plan Phase 2 service separation — document boundaries (playout CPU, chat latency, DB state, monitoring), estimate per-service resources
+- [ ] Plan Phase 3 multi-origin resilience — research second Hetzner location, design warm standby + DNS failover + DB/Redis replication