linearvoid/CHANGELOG.md

Change Log for linearvoid

[Unreleased]

Added

• Automatic media normalization on upload: Chat backend auto-transcodes non-conforming uploads. New media.NormalizeVideo wraps ffmpeg with the Linear Void profile (H.264/AAC, 30fps, 2s keyframes, yuv420p, 48kHz stereo), re-verifies output with ffprobe, and writes *.normalized.mp4. Media records gain normalizing / normalization_failed statuses; POST /api/admin/media/{id}/normalize re-triggers manually. Admin UI shows a "Normalize" action with status polling.
• R2 segment retention: Playout cleanupOldSegments deletes hls/r-*/* objects older than R2_SEGMENT_RETENTION_HOURS (default 24h), run hourly + at startup, never touching the live manifest. Exposes linearvoid_playout_r2_cleanup_objects_total metric. .env.example adds R2_SEGMENT_RETENTION_HOURS.
• Playout loop metrics: linearvoid_playout_loops_total (clean completions) and linearvoid_playout_run_duration_seconds (last run duration).

Changed

• Playout seamless looping: main() now only resets HLS output when the playlist changed (SHA-256 playlistRevision), on crash/slate recovery, or first run; otherwise it continues segment numbering via -start_number and prunes stale segments. Eliminates the viewer gap on clean playlist loops.
• Single playout entrypoint: start-playout.sh now runs /app/playlist-manager -validate then execs the Go binary, removing the duplicate shell ffmpeg loop. Added a -validate mode to main().

Tests

• Media: TestNormalizationArgs, TestNormalizeVideo_FFmpegError.
• Handlers: TestNormalizeMediaRecord_Success, TestNormalizeMediaRecord_Failure, TestHandleAdminMediaByID_Normalize.
• Playout: TestCleanupOldSegments, TestR2SegmentRetentionHours, TestPlaylistFingerprint, TestHLSNextStartNumber.

[1.0.12] - 2026-08-11

Changed

• Task tracking consolidated: Removed DEVELOPMENT.md and moved all development tasks to .github/instructions/BACKLOG.md as the single source of truth. TODO.md updated to reference BACKLOG.md and DONE.md.

Fixed

• Playout R2 endpoint validation: initR2Client() now validates all required R2 environment variables (R2_ENDPOINT, R2_ACCESS_KEY_ID, R2_SECRET_ACCESS_KEY, R2_BUCKET_NAME) before creating the S3 client. Previously, an empty R2_ENDPOINT caused cryptic Custom endpoint '' was not a valid URI errors on every segment upload attempt. Playout now gracefully degrades when R2 is not configured — local HLS via nginx continues to work.
• CD workflow missing R2 secrets: .gitea/workflows/cd.yml "Prepare environment" step now injects all R2_ variables (R2_ACCESS_KEY_ID, R2_SECRET_ACCESS_KEY, R2_BUCKET_NAME, R2_ENDPOINT, plus admin variants) from Gitea secrets. Previously the generated .env was missing R2 entirely, causing the playout container to start with an empty endpoint.
• Deployment docs: docs/DEPLOYMENT.md now includes explicit R2 and Cloudflare variable lists plus a "Gitea Secrets" table documenting all 15 required secrets.

[1.0.11] - 2026-08-10

Added

• Phase 1 test fixtures: Media and playlist test fixtures in backend/playout/testdata/ with ffmpeg generation commands documented in test headers.
• Phase 4 R2 integration: SyncToR2 now sets CacheControl + ContentType based on file extension (immutable segments, short-lived manifests). Three integration tests added.
• Phase 2 viewer tests: Presence updates (join/leave/sorted), private DM flow, message rendering (system/self/other) in main.test.ts (27 total tests).
• Phase 3 admin tests: Playlist management (save via PUT, error handling, sortable rendering) in admin.test.ts (31 total tests passing).

Changed

• Exported savePlaylist and renderPlaylistSortable from admin.ts for testing.

[1.0.10] - 2026-08-09

Added

• WebSocket summary helper: Added scripts/loadtest/summarize-artillery.ps1 to extract counters, rates, and available percentile stats from Artillery JSON output.
• Playbook worksheet mapping: Added artifact-to-worksheet mapping section in docs/operations/scaling-performance-playbook.md for WebSocket and Admin API benchmark artifacts.
• Test fixture generation docs: Added ffmpeg commands for generating valid and invalid media fixtures as comments in backend/playout/src/playlist_test.go. Phase 1 test fixtures now complete.
• R2 cache header support: SyncToR2 now sets CacheControl and ContentType based on file extension:
  • .m4s/.ts/.mp4/.m4a → immutable (max-age=31536000)
  • .m3u8 → short-lived (max-age=2s)
  • unknown → no cache headers
• R2 integration tests: Added TestSyncToR2_ManifestCacheHeaders and TestSyncToR2_NoCacheHeadersForUnknownExt verifying cache header behavior for segment, manifest, and unknown file types.

Changed

• Loadtest runner: scripts/loadtest/run-loadtests.ps1 now generates ws-chat-summary.txt automatically for ws and all scenarios.
• Loadtest docs: scripts/loadtest/README.md now documents summarize-artillery.ps1 usage and the new ws-chat-summary.txt artifact.
• Planning status wording: Updated docs/DEVELOPMENT.md and .github/instructions/TODO.md to mark ws/admin benchmark scaffolding complete while benchmark execution remains pending.
• Phase 1 complete: Test fixture creation and ffmpeg documentation marked done.
• Phase 4 partially complete: R2 upload + cache header validation done; edge cache hit/miss validation remains manual.
• Phase 2 frontend tests: Added presence updates (join/leave rendering), private DM flow, and message rendering tests in main.test.ts (27 total tests passing).
• Phase 3 admin frontend tests: Added playlist management tests (save via PUT, error handling, sortable rendering) in admin.test.ts (31 total tests passing).

[1.0.9] - 2026-08-06

Added

• Prometheus alerts mount: alerts.yml mounted into prometheus container in both dev and prod compose files.
• Grafana provisioning: Prometheus datasource + dashboard provider YAML configs in monitoring/grafana/.

Changed

• Prod prometheus: Switched from empty named volume to bind mounts for config files (consistent with dev).
• CD pipeline: Trigger on push (parallel with CI), removed environment: production (was causing silent abort).
• CI pipeline: Removed redundant release job (handled by release.yml).
• Tag format: Standardized on *.*.* (e.g. 1.0.9), removed v prefix from existing tags.
• DEPLOYMENT.md: Documented parallel CI/CD pipeline flow with workflow table.
• Runtime checks: Executed normalization edge-case tests (MOV, MKV, VFR, 5.1 input) and concat continuity verification in running playout environment.
• Incident runbook: Added quarterly disaster recovery drill procedure with prerequisites, outage simulation, restore validation, and evidence template.
• Manual ops checklist: Added docs/operations/cloudflare-uptimerobot-checklist.md to execute and record Cloudflare + UptimeRobot dashboard tasks with evidence capture.
• Scaling playbook: Added docs/operations/scaling-performance-playbook.md with benchmark methods, trigger thresholds, and Phase 2/3 scaling decision flow.

Fixed

• Prometheus config mount: Switched prod from bind mounts to named volume prometheus-config, populated via docker run on remote host. Fixes not a directory error when bind source path doesn't exist on deployment host.
• Viewer mobile overflow: Replaced 100vw with 100% on full-screen viewer containers to remove horizontal overflow on mobile viewports.

Removed

• .github/MEMORY.md, .github/agents/, .github/instructions/ — tracking files migrated out of repo.

[1.0.8] - 2026-08-06

Changed

• API.md: Added full HTTP status code matrix (table with 12 codes) and rate limit headers documentation.
• DEPLOYMENT.md: Added troubleshooting table (8 common symptoms with check/fix).
• MONITORING.md: Full rewrite with architecture diagram, 7 scrape targets, 8 alert rules, key metrics, troubleshooting.
• SECURITY.md: Full rewrite with threat model diagram, TLS/HSTS config, network isolation, secrets management, incident response.
• Architecture docs: Filled Level 2 building blocks (Mermaid diagram, 5 subsystems with code locations) and Level 3 module breakdowns (chat: 9 modules, playout: 6 modules, frontend: 10 modules).

[1.0.7] - 2026-08-06

Added

• Emergency slate fallback: scripts/generate-slate.sh creates "We'll Be Right Back" slate video + pre-rendered HLS. Playout detects FFmpeg failures (>5 restarts/60s) and auto-switches to slate manifest. Health endpoint reports degraded mode. Auto-recovers when FFmpeg restarts successfully.
• Prometheus metrics: linearvoid_playout_restarts_total, linearvoid_playout_mode (0=live, 1=slate), linearvoid_chat_dms_total exposed on playout + chat /metrics.
• Video normalization pipeline: scripts/normalize.sh with env-configurable CRF/preset/audio-bitrate wrapping reference FFmpeg command.
• Strict ffprobe verification gate: ValidateVideo() in playout parses ffprobe JSON, checks codec (h264/aac), framerate (30/1), resolution (1280x720 or 1920x1080), pix_fmt (yuv420p), sample_rate (48000), keyframe interval (≤2s). ValidateVideoStrict() returns {valid, errors[]}.
• Upload normalization enforcement: validateMediaNormalization() runs on every admin upload (single + chunked). Non-conforming files get needs_normalization status. Visible in media library + playlist validation warnings.
• Chat history on room join: pushChatHistory() sends last 50 messages via WebSocket on connect.
• DM offline delivery: Private messages persisted to DB, appear in history when recipient reconnects.
• Online presence indicators: Green dot per user in chat sidebar, updated via join/leave system messages.
• Schedule metadata overlay: "Now Playing" + "Up Next" below viewer player, polling GET /api/schedule every 30s. Public /api/schedule endpoint added.
• Playlist drag-to-reorder: Sortable list with drag-and-drop, per-entry remove buttons, validation warnings (missing files, needs_normalization).
• Schedule overrides: ScheduleOverride model + GET/POST/DELETE /api/admin/schedule/overrides with conflict detection (409 on overlap). Admin UI with form + list.
• User management actions: POST /api/admin/users with mute/unmute/ban/unban actions. Per-user action buttons in admin users table.
• Inline moderation: Mute/Ban buttons on each chat log entry row in admin.
• CSV export: GET /api/admin/chat-log/export returns CSV download.
• Analytics time range selector: 1h, 6h, 24h, 7d dropdown on admin analytics page.
• Runtime config editor: GET/PUT /api/admin/config for rate limit max + window. Admin system page with config form + restart buttons.
• Production target documentation: DEPLOYMENT.md updated with two-environment model (test lab 192.168.88.0/24 vs Hetzner production TBD), CX22 specs, firewall requirements, DNS cutover plan.
• Network diagram: Mermaid diagram in SETUP.md distinguishing test lab from planned production.
• E2E test script: scripts/test-playout-e2e.sh for runtime validation in test lab.
• Redis pub/sub stub: Architecture ready for multi-instance chat (pending implementation).

Changed

• docker-compose.yml + .prod.yml: added slate volume mount (./data/slate:/app/slate:ro), Redis --maxmemory 256mb --maxmemory-policy allkeys-lru.
• backend/playout/Dockerfile: added /app/slate directory.
• handleConnection: added 30s presence heartbeat + WebSocket ping, banned/muted user checks, env-configurable rate limits (CHAT_RATE_LIMIT_MAX, CHAT_RATE_LIMIT_WINDOW_S).
• handleModerationCommand: added moderator role check, mute duration parsing, Redis key enforcement.
• handlePrivateDM: targeted delivery via clientMeta map instead of broadcast.
• initRedis: added redisReconnectLoop with exponential backoff (2s→60s).
• handleAdminMedia + chunked upload: normalization validation runs on every upload, sets needs_normalization status.
• nginx: OPTIONS preflight → 204 on all 4 HLS locations. /health returns JSON. /nginx-status with stub_status (internal-network restricted).
• CD workflow: health checks now IP-agnostic (docker exec instead of hardcoded IP curls).
• docker-compose.yml + .prod.yml, .gitea/workflows/cd.yml, DEPLOYMENT.md, SETUP.md, docs/architecture/README.md, README.md: all annotated with test-lab vs production distinction, 192.168.88.0/24 flagged as internal test network.
• docs/architecture/README.md: CI/CD deployment view updated to note test lab target, production pending.

Fixed

• CD health check no longer hardcodes 192.168.88.40 in curl targets — uses docker exec for portability.
• Playlist page no longer single-textarea — now shows sortable list with validation warnings.
• Schedule page shows overrides alongside playlist timeline.

[1.0.6] - 2026-08-05

Added

• GET /api/admin/chat-log endpoint with ?room=, ?limit=, ?before=, ?username=, ?text= filter/pagination support.
• GET /api/admin/chat-log/:id endpoint returning a single chat log entry as JSON.
• DELETE /api/admin/chat-log/:id endpoint returning 204 No Content on success.
• DELETE /api/admin/chat-log/bulk endpoint accepting {"ids": [...]} for bulk deletion (max 500).
• Chat Log admin tab with table, individual delete with confirmation dialog, pagination (prev/next), search/filter by room/username/text, select-all checkbox, and bulk delete button.
• Added "lint": "tsc --noEmit" script to frontend/package.json.
• /api/admin/media/{id} GET/PUT/DELETE handlers for media metadata retrieval, updates, and file+record deletion.
• Resumable chunked admin media upload flow with upload-session endpoints and client progress indicators.
• Prometheus-format /metrics endpoint for chat runtime and admin-session monitoring.
• Dedicated operational documentation: API.md, DEPLOYMENT.md, MONITORING.md, SECURITY.md.

Changed

• Split frontend/admin/index.html into a template shell plus 9 per-tab page HTML files loaded dynamically via loadPageContent / switchTab / bindPageEvents.
• Rebuilt viewer, admin, and admin-login pages around custom semantic class structures with accessibility-focused landmarks, tab semantics, and keyboard-focus behavior.
• Replaced legacy and Tailwind-based stylesheet rules with a custom design-token CSS system covering responsive layouts, forms, tables, and reduced-motion behavior.
• Switched frontend theme to dark-mode-first token defaults with light-scheme override.
• Applied new color palette (#32292f dark, #b4c4ae light, #587792 brand, #2ebfa5 focus, #04650d ok) across all 22 CSS custom properties in dark and light themes.
• Replaced hardcoded admin sidebar and tab-trigger colors with CSS custom properties.
• Optimized CI pipeline: removed duplicate docker compose build --no-cache from test job, removed broken playlist validation step, fixed Go module cache path, added Docker Buildx registry cache to docker/release jobs.
• Bumped chat healthcheck start_period from 10s to 30s in docker-compose.prod.yml.
• CD workflow now verifies nginx (port 8080) + chat (3000) + playout (8081); dumps container logs for all failing services.
• Updated Prometheus scrape targets and Grafana dashboard queries to use live chat metrics.

Fixed

• Go module cache in CI now correctly references only backend/chat/go.sum (playout go.sum did not exist); added go mod download step to seed cache.
• Removed broken playlist validation step (go run src/playlist.go without go.mod) from CI test job.
• Converted nginx CI build from custom shell docker buildx to docker/build-push-action@v5 for cache consistency.
• Enforced 2GB upload ceiling plus server-side media file type/size validation for admin uploads.
• Increased nginx client_max_body_size to allow resumable admin upload chunks without 413 errors.
• Removed unused data-admin-* Vite env var attributes from admin HTML template.
• Removed Tailwind and Alpine runtime/build dependencies; eliminated Tailwind directives/warnings during production build.

[1.0.3] - 2026-08-03

Added

• Added Prometheus-format /metrics endpoint for chat runtime and admin-session monitoring.

Changed

• Updated Prometheus scrape targets and Grafana dashboard queries to use live chat metrics.

Fixed

• Fixed nginx routing so /metrics is reachable through localhost and linearvoid.tv.

[1.0.2] - 2026-08-03

Changed

• Updated local environment documentation and examples to prefer real-domain local routing through linearvoid.tv and compose-safe internal service hostnames.

Fixed

• Fixed admin production build output and nginx routing so /admin/ serves its own entrypoint.
• Fixed static asset serving so hashed JS/CSS files return correct MIME types and stale asset URLs return 404 instead of SPA HTML.
• Fixed chat database configuration so DATABASE_URL is honored for compose-safe Postgres or SQLite local development.

[1.0.1] - 2026-08-03

Changed

• Expanded the setup guide with current .env.example variables, development prerequisites, Cloudflare R2 setup, Gitea registry setup, and a production deployment guide.
• Aligned frontend, nginx, and deployment docs with the linearvoid.tv app domain and media.linearvoid.tv HLS domain model.

[1.0.0] - 2026-08-02

Added

Full backend and frontend implementation complete.

• Playout Service (backend/playout/): Go-based playlist management, ffprobe normalization validation, continuous HLS with immutable segment naming, automatic R2 sync, health check endpoint, and Docker integration (ADR-001, ADR-003).
• Chat Backend (backend/chat/): Go + Gorilla WebSocket, Redis for presence/rate limiting/pub/sub, SQLite persistence for message history and ModerationLog (ADR-006/ADR-007), public rooms, private DM routing, moderation commands with immutable audit logging, health check endpoint, and docker-compose integration.
• Frontend (frontend/): Vite + TypeScript + Tailwind + Alpine.js, HLS player with hls.js, chat sidebar with room switching and private messaging, "Now Playing" overlay, and modern UI (ADR-002, ADR-003).
• Complete docker-compose.yml for local development with all services, volumes, and healthchecks.
• All ADRs (001-007) created and linked in documentation.
• CHANGELOG.md updated with full traceability.
• Git repository initialized with comprehensive initial commit.

The project is a fully functional, self-hosted 24/7 linear video channel with synchronized playback, custom chat, R2 delivery, moderation, and admin-ready structure. It matches the architecture defined in the arc42 template and all ADRs.

Next Steps (Future ADRs):

• ADR-008: Admin Interface Implementation (ADR-004/ADR-005).
• ADR-009: CI/CD, monitoring (Prometheus/Grafana), and production deployment.
• Tests, admin UI, and final polish.

See docs/architecture/README.md and the ADR files for complete documentation. The system is ready for use and further extension.