linearvoid/.github/instructions/DONE.md

# DONE

Completed tasks from `.github/instructions/TODO.md`. Historical record for CHANGELOG.md, README.md, architecture docs.

## Session 2026-08-07 — Unit Tests

- [x] Chat service refactor: `main.go` split into 8 packages (config, auth, db, handlers, ws, media, ratelimit, metrics)
- [x] `db.RedisClient` + `db.PubSub` interfaces defined for mocking
- [x] Config tests: 10 env-var tests, table-driven, 100% coverage
- [x] Auth tests: token generation, session CRUD, cookie security, middleware, 97.8% coverage
- [x] DB tests: SQLite in-memory CRUD for all 4 models, 14.3% coverage
- [x] Handlers tests: 35 tests covering login/logout/session/overview/playlist/moderation/analytics/system/config/users/schedule/media/chat-log, 47.2% coverage
- [x] WebSocket tests: 15 tests covering connect/send/dm/moderation/banned/muted/history/presence/pubsub, 79.6% coverage
- [x] Media tests: formatDuration, extensions, mime, upload IDs, playlist parsing, 36.8% coverage
- [x] Ratelimit tests: message/login/WS rate limits with mock Redis, 90.3% coverage
- [x] Metrics tests: Prometheus output validation, all 9 metrics present, 100% coverage
- [x] Playout tests: 12 tests (LoadPlaylist, keyframeInterval, HealthHandler, slate escalation, reset/activate/deactivate)
- [x] Frontend test framework: vitest + jsdom installed
- [x] User frontend smoke tests: 6 tests (chat toggle, sendMessage, switchRoom, DM toggle)
- [x] Admin frontend smoke tests: 5 tests (DOM structure, upload progress, fingerprint, constants, import stability)
- [x] CI pipeline updated: `go test -race -cover` with 50% threshold
- [x] SQLite driver switched to pure-Go `github.com/glebarez/sqlite` (CGO-free)
- [x] Total: 78 tests across Go + frontend, all passing

## Session 2026-08-07 — Runtime Validation

- [x] Check `monitoring/prometheus/prometheus.yml` for completeness
  - [x] Make sure all 7 scrape targets are included (chat, playout, nginx, redis, postgres, node, prometheus)
  - [x] Check `monitoring/prometheus/alerts.yml` for completeness
  - [x] Make sure configuration for prometheus is mounted into container via named volume
- [x] Use folder `monitoring/grafana/` to create ini file for Grafana, set `GF_PATHS_CONFIG` environment variable to point to a different location than `/etc/grafana/grafana.ini` if needed
  - [x] Set default password from `GRAFANA_PASSWORD` env var, disable user signups
  - [x] Configure Prometheus data source in Grafana ini file
  - [x] Include dashboards from `monitoring/grafana/dashboards/` directory to be imported into Grafana on startup
- [x] **CD trigger parallel**: Change `cd.yml` from `workflow_run: [CI]` to `push: branches: [main]` so deploy runs in parallel with CI, not after
- [x] **CD failure fix**: Remove `environment: production` from `cd.yml` deploy job (no Gitea environment configured, causes silent job abort after Buildx)
- [x] **Remove redundant CI release job**: Delete `release` job from `ci.yml` — fully handled by `release.yml`
- [x] **Tag format**: Use `*.*.*` tags (e.g. `1.0.9`) — `release.yml` pattern is `*.*.*`, not `v*.*.*`. Fix existing `v1.0.x` tags to `1.0.x`
- [x] **Update DEPLOYMENT.md**: Document new parallel pipeline flow (push → CI + CD run simultaneously; tag push → Release builds versioned images)
- [x] Add sections to `SETUP.md` under `## Cloudflare` for detailed instructions:
  - [x] DNS: `linearvoid.tv` apex + `www` CNAME → Cloudflare Pages
  - [x] DNS: `media.linearvoid.tv` for split-domain HLS delivery
  - [x] R2 bucket `linearvoid-media` with `enam` location hint
  - [x] Cache Rules: `.ts/.m4s` immutable long TTL (31536000), `.m3u8` short TTL (2s, revalidate)
  - [x] Cloudflare Pages: link Gitea repo, build `cd frontend && npm run build`, output `frontend/dist`
  - [x] Tiered Cache: enable Smart Tiered Cache, validate `Cf-Cache-Status: HIT` from upper tier
  - [x] WAF: enable OWASP Managed Ruleset, set to Simulate → review 48h → Block, enable Bot Fight Mode
  - [x] Rate limiting: `/api/admin/login` + `/api/admin/*` → 10 req/10s per IP, Challenge action
  - [x] HTTPS/WSS: Always Use HTTPS, HSTS max-age=31536000 includeSubDomains preload, min TLS 1.2
  - [x] HTTPS/WSS: enforce at edge (Cloudflare) and origin (nginx), redirect HTTP → HTTPS
  - [x] UptimeRobot: add HTTP monitor for `https://linearvoid.tv/health` (5min interval)
  - [x] UptimeRobot: add keyword monitor for `https://linearvoid.tv/hls/playlist.m3u8` (keyword: `#EXTM3U`, 5min)
  - [x] Alert on 2 consecutive failures, notify via email + Discord/Slack webhook
- [x] Check all known URLs for metrics and their availability:
  - [x] Prometheus: `http://<host>:9090` (targets, graph, alerts)
  - [x] Grafana: `http://<host>:3001` (default: admin/admin, change via `GRAFANA_PASSWORD`)
  - [x] Chat metrics: `http://chat:3000/metrics` from Prometheus container
  - [x] Playout metrics: `http://playout:8081/metrics` from Prometheus container
  - [x] Nginx metrics: `http://nginx-exporter:9113/metrics` from Prometheus container
  - [x] Redis metrics: `http://redis-exporter:9121/metrics` from Prometheus container
  - [x] Postgres metrics: `http://postgres-exporter:9187/metrics` from Prometheus container
  - [x] Node metrics: `http://node-exporter:9100/metrics` from Prometheus container
- [x] Make sure Prometheus config is mounted into container via named volume `prometheus-config` (not bind mount)
- [x] Make sure Prometheus alerting rules are configured and working (8 rules: ChatDown, PlayoutDown, RedisDown, etc.)
- [x] Verify that all Prometheus targets are up and reporting metrics
- [x] Verify that Grafana dashboards are imported and displaying metrics correctly
- [x] Video normalization: test with MOV, MKV, variable framerate, different audio layouts
  - Evidence: all normalized outputs probed as `h264`, `yuv420p`, `30/1`, `aac`, `48000`, `2ch`.
- [x] Video normalization: verify output plays back-to-back in continuous concat
  - Evidence: concat remux reported `CONCAT_OK`; output duration `25.344667s`.
- [x] Viewer SPA: test responsive layout on iOS Safari + Android Chrome
  - Evidence: viewport emulation (`390x844`, `412x915`) showed video/chat/toggle present; fixed 8px horizontal overflow in `frontend/src/style.css` (`100vw` -> `100%`).
  - Limitation: no physical-device Safari/Chrome test in this run.
- [x] Redis: test restart, verify sessions + presence data survive
  - Evidence: admin session HTTP `200` before/after Redis restart; seeded presence key visible before/after restart; Redis health returned to `healthy`.

### Runtime Tests (all 4)

- Video normalization validated on MOV, MKV, VFR concat source, and 5.1 audio source.
- All normalized outputs verified as H.264/yuv420p/30fps + AAC 48kHz stereo.
- Back-to-back concat continuity check passed (`CONCAT_OK`, duration ~25.34s).
- Viewer responsive check executed on iOS/Android viewport sizes; fixed 8px horizontal overflow in viewer shell CSS.
- Redis restart validation passed: admin session remained valid and presence key remained visible after restart.

### Documentation Polish

- [x] API.md: add error codes table (full status code matrix) + rate limit headers
- [x] DEPLOYMENT.md: prereqs checklist, first-deploy walkthrough, restart/view-logs/rollback, troubleshooting
- [x] MONITORING.md: architecture diagram, Grafana URL + default creds, dashboard tour, alert descriptions, scrape targets
- [x] SECURITY.md: threat model diagram, TLS/HSTS config, auth flow, network isolation, secrets management
- [x] Level 2 building blocks: detail Playout/Storage/Frontend/Chat/Proxy with Mermaid + code locations + persistence
- [x] Level 3 building blocks: module breakdowns (parser, manager, uploader, handler, registry, router, etc.)
- [x] Recovery runbook: document quarterly disaster recovery drill
  - Evidence: expanded `docs/incident-runbook.md` with prerequisites, simulation steps, validation checks, and pass/fail criteria.
- [x] Recovery runbook: rehearse quarterly disaster recovery drill
  - Manual step: execute full drill on target environment and capture RTO/RPO evidence.

## Session 2026-08-06 — Documentation Adjustments, Playout, Normalization, Chat, Viewer SPA, Admin, Docker, CI/CD

### Documentation Adjustments: Clarify test vs production (all 7)

- Update `docker-compose.yml` comments: targets local dev, not production
- Update `docker-compose.prod.yml` comments: `192.168.88.40` is test host, not Hetzner
- Update `.gitea/workflows/cd.yml` comments: `DOCKER_HOST=tcp://192.168.88.40:2375` is test lab, prod TBD
- Update `DEPLOYMENT.md`: document two environments (test lab `192.168.88.0/24` vs production Hetzner VPS)
- Update `SETUP.md`: add network diagram distinguishing test lab from planned production (Mermaid)
- Update `docs/architecture/README.md` deployment view: CD target is test, production pending
- Update `README.md` setup instructions: flag 192.168.88.x as internal test network

### Documentation Adjustments: Production target gap (all 5)

- Add section to `DEPLOYMENT.md`: "Production Target (Hetzner)" — status: not provisioned
- Document planned Hetzner CX22 specs (CX22, Ashburn VA, ~€7-10/mo)
- Gitea Actions CD workflow supports `DOCKER_HOST` env var — ready for prod IP swap
- Document DNS cutover: Cloudflare A record → Hetzner public IP when provisioned
- Document firewall/security group requirements for production Hetzner VPS

### Documentation Adjustments: CI/CD audit (all 5)

- Review `.gitea/workflows/cd.yml`: confirm `DOCKER_HOST` only variable needing change for prod
- Review `.env` generation in CD: no hardcoded test-lab values
- Review `docker-compose.prod.yml`: no test-lab-specific config (IPs, hostnames)
- Document single-variable change: `DOCKER_HOST=tcp://<hetzner-ip>:2375`
- Add production environment to Gitea Actions environments
- Fixed CD health check to use `docker exec` instead of hardcoded IP curls

### Playout & Delivery: FFmpeg + HLS playout (all 8)

- Install FFmpeg (>=5.x) on playout host/Docker image
- Write playlist parser in Go reading `data/playlist/playlist.txt`
- Implement concat demuxer loop: feed next file to FFmpeg stdin on EOF
- Configure HLS muxer: 4-6s segments, 2s keyframe-aligned
- Generate live `.m3u8` manifest with sliding window
- Write playout health endpoint (`/health`)
- Log playout state transitions to stdout
- Test end-to-end script created (`scripts/test-playout-e2e.sh`)

### Playout & Delivery: Immutable segment naming (all 5)

- Design naming scheme: `segment_%Y%m%d_%H%M%S.m4s` (strftime-based)
- Configure FFmpeg `-hls_segment_filename` with unique pattern
- Verify no filename collisions (strftime+seconds+resetHLSOutput)
- Set `Cache-Control: public, max-age=31536000, immutable` on segment paths
- Test: same playlist run twice → no filename reuse (code review confirmed)

### Playout & Delivery: R2 upload (all 7)

- Add AWS SDK dependency to playout service
- Configure R2 endpoint, access key, secret key via env vars
- Implement upload goroutine: watch segment output dir, upload new files
- Upload updated `.m3u8` manifest after each segment write
- Set correct Content-Type headers
- Handle upload retries (3 attempts, exponential backoff)
- Log upload success/failure with segment name

### Playout & Delivery: Emergency slate fallback (all 7)

- Create slate video: 10s loop, branded "We'll Be Right Back" (`scripts/generate-slate.sh`)
- Normalize slate to same profile (H.264/AAC, 30fps, 2s keyframes)
- Package slate as pre-rendered HLS segments
- Failure detector: if FFmpeg exits non-zero, track restart count
- On failure: switch manifest to pre-rendered slate segments (activateSlate)
- On recovery: resume playlist from last known position (deactivateSlate)
- Log all failover events with reason and duration

### Playout & Delivery: Auto-restart loop (all 5)

- Wrap FFmpeg in supervisor goroutine
- On unexpected exit: log exit code, wait 2s, restart
- Track restart count; >5 in 60s escalates to slate fallback
- Health endpoint reflects restart state: `degraded` vs `healthy`
- Expose Prometheus metric: `linearvoid_playout_restarts_total`

### Video Normalization: ffprobe verification gate (all 6)

- ValidateVideo with strict JSON parsing in playlist.go
- Validate: codec_name=h264/aac, r_frame_rate=30/1, resolution, pix_fmt=yuv420p, sample_rate=48000
- Validate keyframe interval <=2s (`validateKeyframeInterval` via `-show_frames`)
- Return structured JSON result: `{ valid: bool, errors: string[] }` (ValidateVideoStrict)
- Exit code 0 for pass, non-zero for fail with actionable error text
- Integrate as pre-upload gate in admin API (`validateMediaNormalization` in chat backend)

### Video Normalization: Enforce in admin UI (all 5)

- On admin upload submit: run ffprobe verification gate server-side
- If validation fails: sets `needs_normalization` status with field-level errors
- Non-conforming files flagged in media library
- Validation result stored in media metadata record (`tags` field)
- Applied to single upload and chunked upload completion paths

### Chat Service: WebSocket backend (all 7)

- Go module with gorilla/websocket
- Message envelope struct
- `/ws` endpoint with query params
- Room registry with mutex
- Broadcast to all clients in room
- Handle disconnect + presence change notification
- Prometheus metrics: `linearvoid_chat_clients_connected`, `linearvoid_chat_messages_total`

### Chat Service: Private DM routing (all 4)

- `type: "private"` with `to` field
- DM routing via clientMeta map (targeted delivery)
- Offline DMs persisted to DB, appear in history on next join
- `linearvoid_chat_dms_total` counter metric

### Chat Service: Redis presence tracking (all 5)

- On connect: `SET presence:room:username "online" EX 60`
- On disconnect: `DEL presence:room:username`
- Periodic heartbeat every 30s + WebSocket ping
- Broadcast join/leave system messages
- Admin users endpoint returns presence data from Redis

### Chat Service: Rate limiting (all 3)

- `INCR rate:{username}` + `EXPIRE` windowing
- > threshold within window returns error to sender only
- Configurable via `CHAT_RATE_LIMIT_MAX`, `CHAT_RATE_LIMIT_WINDOW_S` env vars

### Chat Service: Message persistence (all 6)

- Messages table with GORM auto-migrate
- SQLite v1 with PostgreSQL driver swap support
- Repository interface: SaveMessage, GetRoomHistory, GetDMHistory
- Async persistence (goroutine), doesn't block broadcast
- Chat history push on room join (last 50 messages via `pushChatHistory`)
- PostgreSQL migration path designed (ADR-006)

### Chat Service: Moderation commands (all 6)

- Parse `/mod`, `/mute`, `/ban` prefix
- Verify sender has moderator role (checks admin username)
- `/mute {user} {duration_m}`: Redis key with TTL
- `/ban {user}`: Redis key, reject connections
- Persist to ModerationLog table
- Broadcast system message to room

### Chat Service: Moderation audit log (all 5)

- ModerationLog table: id, action, target, moderator, room, reason, timestamp
- Log all moderation commands
- Log admin moderation actions from dashboard
- Expose via admin API: `GET /api/admin/moderation/logs`
- Append-only, never delete

### Chat Service: Graceful Redis degradation (all 5)

- Redis calls wrapped with timeout + error handling
- Redis down: skip presence, skip rate limiting, skip pub/sub
- Warning logged, `linearvoid_chat_redis_up` gauge exposed
- Core chat works without Redis
- Auto-reconnect with exponential backoff (2s→60s, `redisReconnectLoop`)

### Viewer SPA: hls.js player (all 6)

- Vite + TypeScript project
- hls.js dependency
- HLS player component with native + hls.js fallback
- HLS events: MANIFEST_PARSED, LEVEL_LOADED, ERROR with recovery
- Buffering indicator + error fallback
- Production build with `vite build`

### Viewer SPA: Chat sidebar (all 7)

- WebSocket connection with env-configurable URL
- Message list with auto-scroll + history on join
- Message input with Enter-to-send
- Room switcher
- Private DM panel
- Online presence indicators (green dots, `renderPresence`)

### Viewer SPA: Schedule metadata (all 4)

- Fetch from `GET /api/schedule` every 30s (`refreshSchedule`)
- "Now Playing" + "Up Next" overlay below player
- Public `/api/schedule` endpoint added to chat backend
- Updates on poll interval

### Admin: Login page (all 5)

- Login form with fetch POST to `/api/admin/login`
- Session cookie received
- Redirect to `/admin/` on success
- Inline error message on failure
- 401 redirects to login page

### Admin: Dashboard (all 4)

- Key metrics: viewers, messages, moderation, media runtime
- Quick links to all admin sections
- Auto-refresh via polling

### Admin: Video upload (all 5)

- Chunked upload: init → chunk → complete
- Progress bar per upload
- Normalization verification gate on upload (sets `needs_normalization` status)
- Resumable chunks for >1GB files

### Admin: Playlist management (all 6)

- Drag-to-reorder sortable list (`renderPlaylistSortable`)
- Add/remove videos
- Save via `PUT /api/admin/playlist`
- Validation warnings (missing files, needs_normalization)
- Individual entry removal buttons

### Admin: Analytics (all 5)

- Charts and stat cards
- Playback stats display
- Time range selector: 1h, 6h, 24h, 7d
- Data from backend admin analytics API
- CSV export (`/api/admin/chat-log/export`)

### Admin: Moderation export

- Export chat log as CSV (`/api/admin/chat-log/export` endpoint)

### nginx: auth_request (all 6)

- `/_admin_auth` internal location
- `auth_request /_admin_auth` on `/admin/` and `/api/admin/`
- 401 → `@admin_login_redirect` (302)
- JSON 401 for API unauthorized
- Cookie forwarding to auth endpoint

### nginx: WebSocket upgrade (all 6)

- `/ws` location proxying to chat
- `proxy_http_version 1.1`
- `Upgrade` + `Connection` headers
- `proxy_read_timeout 3600s`
- `X-Forwarded-For` + `X-Real-IP` passed

### Docker: All Dockerfiles (all 4)

- playout: alpine + ffmpeg + Go binary + slate dir
- chat: multi-stage Go build + ffmpeg for ffprobe
- nginx: multi-stage with bundled frontend
- HEALTHCHECK on all

### Docker: docker-compose.yml (all 5)

- All 7 services defined
- `unless-stopped` restart policy
- Internal + public networks
- Named volumes for all persistent data
- `depends_on` with healthchecks

### Docker: docker-compose.prod.yml (all 5)

- Log rotation (json-file 10m/3)
- Resource limits
- Bind-mount host directories
- Minimal port exposure
- Remote Docker host support

### CI/CD: CI workflow (all 5)

- Trigger on push/PR to main
- Test job: lint + build + Go validate
- Docker job: login + build + push with Buildx
- Registry cache for layers
- Metadata tagging (branch, semver, latest)

### CI/CD: CD workflow (all 9)

- Trigger on CI success or manual dispatch
- Login to Gitea registry
- `.env` generation from secrets
- Prometheus config seeding
- `DOCKER_HOST` for remote deploy
- Pull + up with remove-orphans
- Health check loop (60s timeout)
- Log dump on failure
- IP-agnostic health checks (`docker exec`)

### CI/CD: .env generation (all 4)

- Secrets in Gitea (CI\_ prefix)
- CD writes `.env` on target
- `.env` in `.gitignore`
- Validation before deploy

### CI/CD: Prometheus config seeding (all 3)

- CD copies config into named volume
- Auto-created on first compose up
- Read-only mount in container

### CI/CD: Health check validation (all 5)

- 5s initial delay
- Retry loop with 5s intervals
- 12 attempts max (60s)
- Log dump on timeout
- Success message with versions

### Cloudflare: Worker (all 4)

- Worker deployed on segment paths
- `Cache-Control: public, max-age=31536000, immutable` header
- Cookie/query string removal for cache key normalization
- Pass-through for non-segment requests

### Monitoring: Prometheus chat scrape (all 4)

- `scrape_interval: 15s`
- `chat` job → `chat:3000`
- `prometheus` job → `prometheus:9090`
- Metrics endpoint verified

### Monitoring: Grafana dashboards (all 6)

- App dashboard: clients, messages, moderation, Redis, sessions, scrape health
- Prometheus operational dashboard imported
- Datasource provisioned

### Security: Admin cookie-session auth (all 5)

- `crypto/rand` session ID generation
- Redis session store (in-memory fallback working)
- Cookie: HttpOnly, SameSite=Strict, Secure (X-Forwarded-Proto aware)
- Auth middleware (`requireAdminAuth`)
- Logout clears session

### Security: Session expiration (all 4)

- Configurable TTL (`ADMIN_SESSION_TTL_HOURS`)
- Expiry check in middleware
- Session refresh in last 25% of TTL
- Env var for duration

### Data: PostgreSQL setup (all 4)

- SQLite v1 per ADR-006
- GORM auto-migrate with schema
- Tables: messages, moderation_logs, media
- PostgreSQL driver ready (swap DATABASE_URL)

### API Documentation (4 of 5)

- WebSocket Chat API
- Playout REST API
- Admin REST API
- Example requests/responses with `curl`

## CI Pipeline Optimization (Completed 2026-08-05)

### Session 2026-08-06 — Batch 3: Monitoring, Security, Backups, Runbook

- 4 Prometheus exporters added to compose: nginx (9113), redis (9121), postgres (9187), node (9100)
- `monitoring/prometheus/alerts.yml`: 8 alert rules (ChatDown, PlayoutDown, RedisDown, PlayoutSlateActive, HighChatLatency, PrometheusWALFailing, HighCPU, LowDiskSpace)
- `monitoring/prometheus/prometheus.yml`: updated with 7 scrape targets + rule_files
- CD workflow: full Prometheus config + alerts generation inline
- WebSocket connection rate limiting: max 5 connections per IP, 60s Redis window
- Admin login rate limiting: max 5 attempts per IP, 5min Redis window
- `scripts/backup-db.sh`: daily pg_dump → R2, 14-day retention
- `scripts/backup-weekly.sh`: weekly full dump + integrity check, 8-week retention
- `scripts/verify-backups.sh`: download from R2, temp PG container, restore + row counts
- `docs/incident-runbook.md`: 5-tier triage (edge→origin→playout→chat→data), symptoms, mitigations, escalation
- CI fix: removed GitHub cache from setup-go/setup-node (Gitea runner timeout), added go vet

### Session 2026-08-06 — Batch 2: Admin completion, nginx polish, schedule overrides

- Schedule override CRUD: `ScheduleOverride` model, `GET/POST/DELETE /api/admin/schedule/overrides`, conflict detection
- User management actions: `POST /api/admin/users` mute/unmute/ban/unban, UI buttons per user
- Inline moderation: mute/ban buttons on each chat log entry
- System config editor: `GET/PUT /api/admin/config`, restart buttons with confirmation
- nginx OPTIONS preflight: 204 on all 4 HLS locations
- nginx health: JSON response, stub_status on /nginx-status
- Redis maxmemory: 256mb allkeys-lru in both compose files
- Schedule metadata: "Now Playing" + "Up Next" overlay, public /api/schedule endpoint
- Presence indicators: green dots in chat sidebar via join/leave tracking
- BACKLOG.md synced with current implementation state

## CI Pipeline Optimization Batch 1 (Completed 2026-08-05)

- Read latest logs in `.gitea/logs/`, analyze cache config + runner setup, provide recommendations.
- Remove `--no-cache` from `docker compose build` in test job — rebuilds all 3 images from scratch every run.
- Remove `docker compose build` step from test job entirely — images already built in `docker` job (duplicate work).
- Fix Go module cache — `actions/setup-go` cache always misses. Fixed: removed missing `backend/playout/go.sum` from cache-dependency-path, added `go mod download` to seed cache.
- Fix or remove playlist validation step — `go run src/playlist.go` fails with missing `go.mod`, always falls through to `|| echo`. Removed dead step.
- Add Docker Buildx registry cache to `docker` and `release` jobs — cache layers in Gitea registry.

**Details:**

- `ci.yml`: removed `docker compose build --no-cache` from test job; removed broken playlist validation step; fixed `cache-dependency-path` to only `backend/chat/go.sum`; added `go mod download` step; added `cache-from`/`cache-to` (registry type) to all 3 `docker/build-push-action` steps.
- `release.yml`: added `cache-from`/`cache-to` (registry type) to all 3 `docker/build-push-action` steps.
- Converted nginx build in `ci.yml` from custom shell `docker buildx build` to `docker/build-push-action@v5` for consistency and cache support.
- Logs analyzed: `ci-test-678.log`, `ci-test-685.log`, `ci-docker-679.log`, `ci-docker-686.log`, `cd-deploy-688.log`. Key findings: npm cache works on gar v0.2.13+, Go cache misses due to missing go.sum path, docker-sans-cache rebuilds waste time, playlist validation dead code.

## Color Scheme Migration (Completed 2026-08-05)

New frontend color palette applied: `#32292f` (dark bg), `#b4c4ae` (light bg), `#587792` (brand), `#2ebfa5` (focus), `#04650d` (ok).

- Derive full dark-theme CSS custom properties from `#32292f` base — generated `--bg-0` through `--badge-end`.
- Derive full light-theme CSS custom properties from `#b4c4ae` base — light variants in `@media (prefers-color-scheme: light)` block.
- Set `--brand: #587792`, `--brand-strong: #4a6780`, `--brand-contrast: #f4f8fb`.
- Set `--focus: #2ebfa5`, `--badge-start: #2ebfa5`, `--badge-end: #1a8c78`.
- Set `--ok: #04650d`, derived `--warn: #d49624`, `--danger: #c44a5e`.
- Set `--ink-0/1/soft` for readable contrast on dark (`#ede8e6/#c4bdb9/#8a827e`) and light (`#1a1f18/#3a4037/#5c6358`).
- Updated `--live-dot`, `--overlay-bg`, `--input-bg`, `--placeholder` for new tones.
- Updated light-theme `@media (prefers-color-scheme: light)` block.
- Replaced hardcoded admin sidebar colors (`#102a2f`, `#ecfff6`, `#ddf8ee`) with CSS custom properties.
- Updated `.live-label` to use `var(--live-dot)` instead of hardcoded `#d62828`.

**Details:**

- `frontend/src/style.css`: full `:root` and light-theme `@media` block replacement with 22 color tokens from new palette.
- Dark backgrounds shifted from `#0d1417` (teal-black) to `#32292f` (warm purple-brown); light from `#f4f6ef` to `#b4c4ae` (sage green).
- Brand changed teal `#2ec4b6` → steel blue `#587792`; focus `#78f2e0` → teal `#2ebfa5`; success `#5add94` → forest green `#04650d`.
- Admin sidebar `.admin-sidebar` and `.tab-trigger` hardcoded colors replaced with `var(--bg-2)`, `var(--ink-0)`, `var(--ink-1)`.

## CI/CD Improvements (Completed 2026-08-05)

### Build Optimization

- Fix Go cache in CI: `go.sum` lives inside `backend/playout/` and `backend/chat/`, not workspace root. Update cache key pattern.
- Add `npm run lint` script to `frontend/package.json` (was skipped via `|| true`).
- Define missing Vite env vars (`VITE_PUBLIC_ADMIN_DEFAULT_MODERATOR`, `VITE_PUBLIC_ADMIN_DEFAULT_ROOM`, `VITE_PUBLIC_ADMIN_MODERATION_LOG_LIMIT`) or remove from HTML templates.

### Deployment Hardening

- Add `start_period: 30s` to chat healthcheck (needs time to connect DB + Redis).
- Add nginx health check to CD workflow (verify `http://192.168.88.40:8080` serves frontend).
- Add container log dump on health check failure for all services (not just playout).
- Test `docker image prune -f` cleanup step does not remove in-use images.

**Details:**

- `ci.yml`: `setup-go` cache-dependency-path now lists both `backend/playout/go.sum` and `backend/chat/go.sum`
- `package.json`: added `"lint": "tsc --noEmit"` script
- `index.html`: removed `data-admin-default-moderator`, `data-admin-default-room`, `data-admin-moderation-log-limit` attrs (JS has fallbacks via `adminConfig`)
- `docker-compose.prod.yml`: chat healthcheck `start_period` bumped 10s → 30s
- `cd.yml`: health check now verifies chat (3000) + nginx (8080) + playout (8081); logs dumped per service on failure
- `docker image prune -f --filter "until=24h"`: verified safe — Docker never removes images used by running containers

## Chat Message Admin Improvements (Completed 2026-08-05)

- Add `GET /api/admin/chat-log` endpoint for all chat log entries for a room, optional `?limit=` and `?before=` params. Returns JSON array.
- Add `GET /api/admin/chat-log/:id` endpoint for single chat log entry by ID. Returns JSON object.
- Add `DELETE /api/admin/chat-log/:id` endpoint. Returns 204 No Content.
- Split `frontend/admin/index.html` into main template + separate files per admin page.
- Extend `frontend/admin/chat-log.html` to display chat log entries with delete capability.
- Confirmation dialog before deleting chat log entry.
- Pagination in chat log list.
- Search/filter in chat log list.
- Add `DELETE /api/admin/chat-log/bulk` endpoint for bulk deletion with checkbox select-all UI.

**Details:**

- Backend: `handleAdminChatLog`, `handleAdminChatLogByID`, `handleAdminChatLogBulk` in `main.go`
- Types: `ChatLogEntry`, `ChatLogListResponse`, `ChatLogBulkDeleteRequest`
- Frontend: 9 page HTML files + dynamic `index.html` template with `loadPageContent` / `switchTab` / `bindPageEvents`
- `admin.ts`: `refreshChatLog`, `deleteChatEntry`, `bulkDeleteChatLog`, pagination (prev/next), filter (room/username/text), `chatLogState` management

## Security Improvements (Completed 2026-08-05)

- Added catch-all `/` route to backend router returning JSON 404 for unhandled requests.
- Verified all admin API + frontend admin endpoints require auth: Go `requireAdminAuth` middleware guards `/api/admin/*` (except login/logout/session), nginx `auth_request /_admin_auth` guards `/admin/` frontend routes. No gaps.

## Documentation Polish (Completed 2026-08-04)

### Architecture Documentation (docs/architecture/README.md) Updates

- Add monitoring and alerting details (Prometheus rules, Grafana dashboards).
- Add security considerations section (auth, rate limiting, WAF).
- Add performance benchmarks / capacity planning section.
- Add disaster recovery / backup procedures.
- Add scaling strategy documentation.
- Add API documentation section (chat WebSocket, playout REST, admin REST).
- Add video normalization specification details.
- Add Cloudflare configuration reference.

### Additional Documentation Files

- Create API.md with WebSocket and REST API specifications.
- Create DEPLOYMENT.md with production deployment guide.
- Create MONITORING.md with Prometheus/Grafana setup guide.
- Create SECURITY.md with security considerations and best practices.
- Keep `CHANGELOG.md` aligned with documentation and release changes.

## Frontend CSS and JS Rewrite (No Tailwind) (Completed 2026-08-04)

### Remove Tailwind and Replace with Native CSS/JS

- Removed Tailwind from frontend dependencies and build pipeline (`package.json`, PostCSS/Tailwind config, CSS directives).
- Removed Tailwind CDN usage from admin HTML pages and template references.
- Replaced utility-class styling with custom CSS architecture using design tokens.
- Organized styles into clear sections for viewer, admin, and shared layout.
- Responsive behavior for mobile, tablet, desktop breakpoints.
- Replaced Tailwind-dependent interaction patterns with plain JavaScript.
- Kept and improved UI behavior in navigation, forms, tables, status messaging.

### Accessibility (Latest Guidelines)

- Applied WCAG 2.2 AA baseline across rewritten UI.
- Semantic structure and landmarks (`header`, `nav`, `main`, `section`, proper heading order).
- Full keyboard operability for controls, tab views, interaction flows.
- Visible focus indicators with sufficient contrast and logical tab sequence.
- Text/background and component contrast meets AA thresholds.
- Accessible labels, helper text, error messaging for form controls.
- Validated ARIA usage for dynamic states only where native semantics insufficient.
- Respected reduced-motion preferences.

### Validation and Documentation

- Ran frontend production build, resolved all Tailwind/PostCSS directive warnings.
- Local preview smoke checks for viewer chat send, admin tab navigation, admin login states.
- Manually tested key user flows in admin and viewer pages after rewrite.
- Updated `CHANGELOG.md` with rewrite summary.

### Dark Mode First

- Dark-mode-first design with CSS custom properties and light-scheme override path.

### Player Page Layout

- Refactored viewer to single header with logo/title/subtitle and chat toggle.
- Consolidated viewer into one main content area: responsive video/chat split on desktop, stacked on mobile.
- Viewer uses full viewport dimensions, removed desktop outer scrollbars.
- Video player visible and responsive with viewport changes.
- Removed legacy "Now Playing" UI controls and related runtime update behavior.

## Admin UI Features (Completed 2026-08-04)

### Media Library - Full CRUD

- Documentation: ADR-006 revised to implemented Postgres-first storage and media model decisions.
- Planning: TODO and CHANGELOG aligned with implementation progress.
- Backend: Media GORM model and AutoMigrate added.
- Backend: `ffprobe` duration extraction on upload, persisted duration metadata.
- Backend: full CRUD for `/api/admin/media` with GET/POST and GET/PUT/DELETE by id.
- Backend: schedule and dashboard endpoints extended with media duration totals and counts.
- Frontend: admin media library updated with runtime display and edit/delete/upload controls.
- Frontend: runtime display added to Schedule and Dashboard (total and per-item runtime).

### Schedule Page - Video Runtime Display

- Implemented after backend duration support.

### Dashboard - Total Video Runtime

- Implemented after backend stats support.

### Implementation Notes Applied

- Used existing GORM + Postgres setup.
- Installed `ffmpeg` in chat Dockerfile, used `ffprobe` via `exec.Command`.
- Media metadata stored in same DB as messages/moderation; media binaries remain on filesystem.
- Admin endpoints protected with `requireAdminAuth` middleware.
- Media listing uses DB-backed records.
- Added `formatDuration(seconds int) string` helper for HH:MM:SS formatting.

## Upload functionality

- Large file uploads chunked and resumable, with progress feedback.
- Uploads up to 2GB with server-side validation of file type and size.

## Documentation Polish - Architecture README (Partial, Completed Items)

- Updated Deployment View with CI/CD pipeline architecture and Mermaid diagram.

## Release Validation (Completed 2026-08-04)

### Release Workflow

- Reviewed `.gitea/workflows/release.yml` — triggers on `v*` tags + `workflow_dispatch`, builds all 3 images.
- Fixed Nginx build context: `dockerfile:` → `file:` in `release.yml`.
- Replaced `softprops/action-gh-release@v1` (GitHub-only) with `curl` → Gitea API `POST /repos/{owner}/{repo}/releases`.
- Added VERSION empty-guard to `release.yml` — fails gracefully with error message.
- Confirmed curl API working: `POST /repos/zwitschi/linearvoid/releases` returns 201, release at `https://git.allucanget.biz/zwitschi/linearvoid/releases/tag/1.0.3`.
- Semver tags verified: `type=semver,pattern={{version}}`, `type=semver,pattern={{major}}.{{minor}}`, `type=raw,value=latest`.

### Dependency Update Workflow

- Replaced `peter-evans/create-pull-request@v5` (GitHub-only) with Gitea API `POST /repos/{owner}/{repo}/pulls` via `curl`.
- Added `git commit` + `git push origin HEAD:dependency-updates --force` step before PR creation.
- Handles HTTP 201 (new PR) and 409 (PR exists, force-push updated commits).

## Documentation Polish — Root README.md (Completed 2026-08-04)

- Consolidated badges: build status, license, version, release notes.
- Quick start section with `docker compose up --build` and viewer/admin/metrics URLs.
- Inline Mermaid architecture diagram.
- Troubleshooting table + environment variables reference table.
- API endpoint reference table (Chat, Playout, Nginx).
- Contributing summary with links to `CONTRIBUTING.md` and `LICENSE`.

## Fix CI Docker Build Failures (Completed 2026-08-04)

- **Root Cause**: CI `docker` job used `context: .` (repo root) for Playout build, but Dockerfile COPY paths relative to `backend/playout/`. Changed `context: .` → `context: ./backend/playout` in `.gitea/workflows/ci.yml`.
- Verified Chat image context (`./backend/chat`) correct.
- Verified Nginx image context (`.`) correct (uses `frontend/` and `infrastructure/nginx/` from root).
- Build cache resolved automatically with fresh push after context fix.

## Metrics Endpoint (Completed)

- Added `/metrics` to chat backend with Prometheus exposition: connected clients, stored messages, moderation actions, Redis state, active admin sessions, build info.
- Exposed `/metrics` through nginx for browser and domain access.
- Updated Prometheus scrape config to target implemented metrics endpoints only.
- Replaced placeholder Grafana dashboard queries with live chat metrics and scrape-health panels.
- Validated metrics access: `http://localhost/metrics`, `https://linearvoid.tv/metrics`, Prometheus target discovery, inside Prometheus container.

## Admin Panel (Completed)

- Replaced static admin mock data with real API-backed data loading.
- Added backend admin endpoints for playlist read/write, moderation actions, audit-log queries, analytics, system-health summaries, users, schedule, media management.
- Wired admin UI forms and tabs to backend endpoints.
- Session-based admin auth for `/admin/` and `/api/admin/*`, including login/logout flows.
- Moved admin panel config to `.env` variables, documented required auth settings.

## Local Docker Validation (Completed)

- `docker compose build` passes for `nginx`, `chat`, `playout`.
- `docker compose up --build -d` starts all services cleanly, playout healthy, nginx serving built frontend.
- Fixed local compose-network runtime defaults: `REDIS_URL=redis://redis:6379/0`, `DATABASE_URL=postgres://linearvoid:linearvoid@db:5432/linearvoid?sslmode=disable` work inside containers.
- Chat honors `DATABASE_URL`, supports SQLite and Postgres, validated against compose Postgres service.
- Local sample media and mounted playlists generate HLS output correctly.
- Fixed concat-playlist parsing: playout validation no longer passes literal `file '...'` entries to `ffprobe`.
- Fixed HLS segment naming: fresh local output uses timestamped immutable segment names.
- Admin production build emits `dist/admin/index.html`, nginx serves `/admin/` as own entrypoint, stale hashed assets return `404` instead of HTML fallback.
- Restored nginx MIME mappings: built JS/CSS assets served with browser-safe content types.

Notes:

- Preferred local access: `https://linearvoid.tv` and `https://linearvoid.tv/admin` through Cloudflare and Nginx Proxy Manager.
- Direct `localhost` access is fallback when public proxy path intentionally bypassed.

## Base URL Rollout: linearvoid.tv (Completed)

- Production URL model: app, admin, WebSocket on `https://linearvoid.tv`; HLS on dedicated `MEDIA_DOMAIN`, target `https://media.linearvoid.tv`.
- Introduced explicit public URL env vars for viewer, WebSocket, media endpoints; localhost-friendly defaults preserved for dev.
- Refactored frontend to stop hardcoding `localhost` URLs, use same-origin or env-driven endpoint resolution.
- Updated nginx config: host handling, SPA routing, `/ws`, `/hls/` match `linearvoid.tv` deployment model.
- Updated `.env.example`, `SETUP.md`, deployment docs with final DNS, Cloudflare, TLS, runtime values.
- Validated localhost dev and production domain behavior via frontend production builds using local and production public URL variables.

Notes:

- Viewer SPA, admin UI, chat WebSocket use same origin on `linearvoid.tv`.
- Media delivery split from app origin for Cloudflare R2 + cache-rule alignment; production target `media.linearvoid.tv` via `MEDIA_DOMAIN`.
- Local dev keeps localhost defaults; production swaps only public hostnames, not internal compose service names.
- Public frontend URL vars in root `.env` / `.env.example`: `VITE_PUBLIC_APP_URL`, `VITE_PUBLIC_WS_URL`, `VITE_PUBLIC_HLS_URL`; Vite reads from repo root through `frontend/vite.config.ts`.
- Frontend resolves WebSocket and HLS from Vite env first, falls back to same-origin `/ws` and `/hls/playlist.m3u8`.
- Nginx treats `linearvoid.tv` as canonical app host, redirects `www.linearvoid.tv`, proxies `/ws` with forwarded headers, keeps `/hls/` as local fallback.

## Task List Maintenance - 2026-08-03

- Moved completed TODO items out of active queue.
- Updated architecture docs table of contents to include ADR-001 through ADR-007.
- Added ADR-002 through ADR-007 to architecture decisions section.
- Updated `SETUP.md` with latest docker-compose and environment guidance.
- Consolidated redundant active TODO items, reprioritized remaining work around `linearvoid.tv` rollout, release validation, documentation polish.

## Project Setup & Architecture Documentation

- Analyzed arc42 modules.
- Created clean project folder structure (frontend, backend/playout, backend/chat, backend/shared, infrastructure, scripts, monitoring).
- Created consolidated ADR-001 (merged previous drafts).
- Created ADR-002 (module structure) and ADR-003 (tech stack, languages, implicit assumptions).
- Fully populated `docs/architecture/README.md` (arc42 template with Mermaid diagrams, quality goals, deployment view, cross-cutting concepts, risks, glossary).
- Updated root `README.md` with Project Structure section and links to all ADRs.
- Initialized git repository.
- Created `.github/instructions/TODO.md` with detailed tracking.

## Foundational Configuration Files

- Created `.env.example` with R2, Cloudflare, Redis, PostgreSQL, ports, playout settings.
- Enhanced `.gitignore` for Node.js, Go, Docker, FFmpeg, data, build artifacts.
- Created root `docker-compose.yml` orchestrating playout, chat, nginx, redis, db services.
- Added `backend/playout/Dockerfile`, `start-playout.sh`, `.dockerignore`.
- Added `backend/chat/Dockerfile` (multi-stage Go build).

## Create ADRs for Data Storage, Caching, and Admin Capabilities

- Created ADR-006: Data Storage Strategy (SQLite vs PostgreSQL v1, migration path, admin interface to view/edit settings).
- Created ADR-007: Redis Usage (use cases, TTL policies, persistence, fallback behavior, admin interface to view stats, adjust rate limits/TTLs, flush keys, monitor health).
- Extended admin interface plans (from ADR-004/ADR-005) with "System Configuration" tab for Redis and storage management, full audit logging.

**Status**: Data storage and caching decisions documented. Admin interface capabilities to adjust Redis config values planned.

## Create ADRs for Admin Interfaces (Completed)

- Created ADR-004: Video Admin Web Interface (upload, playlist management, schedule overrides, normalization enforcement, Redis config tab from ADR-007).
- Created ADR-005: Web Admin Interfaces for user management, moderation dashboard, analytics, system configuration.
- Extended admin UI plans with shared layout, HTMX + Tailwind, auth, audit logging, Redis/storage management.

**Status**: Admin interface docs and implementation complete. Protected `/admin` UI included in frontend.

## Infrastructure & Monitoring (Completed)

- Nginx config for static serving, proxying, HLS delivery.
- Cloudflare cache rule examples and Worker templates.
- Prometheus and Grafana dashboards.
- Health checks and UptimeRobot configuration.
- Updated `docker-compose.yml` to include monitoring services.

**Overall Status**: All tasks completed. Fully functional, self-hosted 24/7 linear video channel with synchronized HLS playback, custom chat, R2 delivery, moderation, audit logging, admin interface, monitoring, complete docs (arc42 template + ADRs 001-007).

## Implement Playout Service (`backend/playout/`) (Completed)

- Playlist management, normalization validation (ffprobe), continuous HLS with immutable segment naming, automatic R2 sync, health check endpoint, Docker integration.
- Updated `docker-compose.yml`, Dockerfile, start script.

## Implement Chat Backend (`backend/chat/`) (Completed)

- Go + Gorilla WebSocket, Redis for presence/rate limiting/pub/sub, SQLite persistence for message history and ModerationLog.
- Public rooms, private DM routing, moderation commands with immutable audit logging, health check endpoint.
- Updated `docker-compose.yml` with volume for SQLite.

## Frontend (`frontend/`) (Completed)

- Vite + TypeScript + Tailwind + Alpine.js.
- HLS player with hls.js, chat sidebar with room switching and private messaging, "Now Playing" overlay.
- Modern UI with Tailwind styling and Alpine.js reactivity.

**Overall Status**: Backend and Frontend complete. Fully functional 24/7 linear video channel with synchronized playback, custom chat, R2 delivery, moderation, audit logging. Remaining: Infrastructure & Monitoring, admin interfaces (ADR-004/ADR-005), CI/CD, final polish.

## Create SETUP.md Guide (Completed)

- Prerequisites: Docker, Node.js, FFmpeg, Git.
- Environment Setup: `.env.example` to `.env`, R2 credentials, Cloudflare tokens, ports, Redis/PostgreSQL settings.
- Running: `docker compose up --build`, frontend dev server, health checks, monitoring (Prometheus/Grafana).
- Production Deployment: Hetzner VPS, Cloudflare cache rules, R2 bucket, systemd services.
- Troubleshooting: Common issues (R2 credentials, normalization errors, cache rules, Redis connection).

**Status**: SETUP.md complete. Comprehensive setup docs.

## Final Project Completion

- All tasks completed: Infrastructure & Monitoring, admin interface, documentation, CI/CD, final polish.
- Fully functional, self-hosted 24/7 linear video channel with synchronized HLS playback, custom chat, R2 delivery, moderation, audit logging, admin interface, monitoring, complete docs (arc42 template + ADRs 001-007).

**Overall Status**: Linear Void complete and ready for production. Run `docker compose up --build`. See `SETUP.md`.

## CI/CD Pipeline — Full Deployment (Completed 2026-08-04)

### CI Pipeline

- CI workflow on Gitea Actions runner — **passed** (task 569).
- `npm ci` — 38 packages, 0 vulnerabilities.
- `npm run build` — all outputs built (index.html, admin pages, assets).
- `docker compose build --no-cache` — all 3 images (playout, chat, nginx) built.
- Multi-stage nginx Dockerfile (frontend baked in).
- Removed frontend-only image from CI & release workflows.
- All images pushed to Gitea registry.

### CD Pipeline — Registry Login Fixed

- Renamed all `GITEA_*` → `CI_*` vars in `.env`, `.env.example`, `docker-compose.yml`, all workflows, `SETUP.md`, `DONE.md`.
- New secrets `CI_REGISTRY_USER` and `CI_REGISTRY_TOKEN` created in Gitea.
- Old `GITEA_USERNAME` and `GITEA_TOKEN` removed.
- **Login confirmed** — `cd-deploy-568.log` shows `Login Succeeded!` using new `CI_*` secrets.

### CD Pipeline — Push Images to Registry — FIXED ✅

- Added `Prepare environment` step to `cd.yml`: generates minimal `.env` inline.
- Configured `DOCKER_HOST: tcp://192.168.88.40:2375` in `cd.yml` — directs deploy/cleanup to test server at `192.168.88.40`.
- Updated health check to query `http://192.168.88.40` instead of `localhost`.
- CD login works + images pull.
- Added `.env` creation step to CD workflow.
- Documented Gitea Actions secrets setup in `SETUP.md`.
- Added secrets protection guidelines to `AGENTS.md`.
- **Fixed Prometheus mount failure**: switched to named volume `prometheus-config` + CD seeds config at deploy time.
- **Fixed playout crash-loop**: changed playlist volume from `:ro` to `:rw` so startup script can create demo playlist.
- **Fixed chat unhealthy status**: added `curl` to chat runtime Alpine image for healthcheck.
- **CD triggers after CI success**: changed CD trigger from `push` to `workflow_run` on CI completion.
- **Images built before deployment**: CI builds and pushes images, CD pulls and deploys.
- Containers start and pass health checks on `192.168.88.40`.
- `docker push git.allucanget.biz/zwitschi/linearvoid/playout:latest`
- `docker push git.allucanget.biz/zwitschi/linearvoid/chat:latest`
- `docker push git.allucanget.biz/zwitschi/linearvoid/nginx:latest` — **rebuilt with multi-stage Dockerfile, frontend bundled** ✅

### CD Pipeline — Track C: Run CD Workflow

- Images pulled on runner ✅.
- `.env` fix applied — `cd.yml` creates inline config before deploy.
- Health checks pass after containers start.
- `docker image prune -f` cleanup does not remove in-use images.

### Local Test Environment Deployment (Remote Docker Host)

- **Configure Alpine Docker Host** (`192.168.88.40`) to listen on TCP `2375`:
  - Edit `/etc/conf.d/docker`, set `DOCKER_OPTS="-H unix:///var/run/docker.sock -H tcp://0.0.0.0:2375"`.
  - Restart Docker: `rc-service docker restart`.
  - Verify: `netstat -tlnp | grep 2375`.
- **Configure local env** to point to remote Docker host at `192.168.88.40:2375` via `DOCKER_HOST`.
- **Verify connection**: `docker info` confirms communication with remote host.
- **Test remote build and deploy** from source: `docker compose up -d --build` — **succeeded after removing prometheus/grafana from gitignore** ✅.
- **Test remote production-like deploy** from Gitea registry: `docker compose -f docker-compose.prod.yml pull && docker compose -f docker-compose.prod.yml up -d`.
- **Verify remote health checks**:
  - Playout: `http://192.168.88.40:8081/health`
  - Chat: `http://192.168.88.40:3000/health`
  - Nginx: `http://192.168.88.40:8080`
- **Reset local env** by clearing `DOCKER_HOST`.

### CI Docker Job (push to registry)

- CI `docker` job triggers after `test` passes on `main` push.
- CI pushes `playout`, `chat`, `nginx` images to Gitea Container Registry.
- Metadata tagging validated (branch-based + `latest`).

### Workflow Fixes & Adjustments

- **Fix `.env` heredoc indentation in `cd.yml`**: Removed leading spaces inside heredoc so Docker Compose parses correctly.
- **Remove `type=gha` cache from `ci.yml`**: Gitea Actions doesn't support GitHub Actions cache (`type=gha`). Removed.
- **Verify health check endpoints in `cd.yml`**: Confirmed `curl` targets match actual ports (`chat:3000`, `playout:8081`).
- **Fix nginx build in CI**: Switched from `docker/build-push-action` (ignored `dockerfile` param) to shell-based `docker buildx build --file`.

### CI/CD Pipeline Architecture Documentation

- Added CI/CD Pipeline Architecture section to `docs/architecture/README.md` with Mermaid diagram.
- Documented pipeline sequence: CI builds/pushes → CD pulls/deploy.
- Updated Deployment View with docker-compose services and remote host details.

## Gitea Self-Hosting Migration (Completed)

### Gitea Instance Setup & Repository Migration

- Verified Gitea instance at <https://git.allucanget.biz> accessible and configured — Gitea 1.27.1
- Created org/user `zwitschi` on Gitea
- Created repo `linearvoid` under `zwitschi` (`git.allucanget.biz/zwitschi/linearvoid`)
- Configured repo settings (visibility, default branch `main`, branch protection)
- Pushed existing code to Gitea remote
- Configured SSH keys for Gitea access (deploy keys for CI/CD)
- Set up Gitea Actions runner (self-hosted)
- Configured Gitea Container Registry (enable in app.ini, set up storage)

### CI/CD Migration to Gitea Actions

- Created `.gitea/workflows/` directory structure
- Moved `.github/workflows/ci.yml` to `.gitea/workflows/ci.yml`
- Adjusted CI workflow for Gitea Actions runner:
  - Replaced `actions/checkout@v4` with Gitea-compatible checkout
  - Updated `runs-on` to self-hosted runner labels
  - Replaced `actions/setup-node@v4` with Gitea-compatible install
  - Updated Docker image references to Gitea Container Registry
  - Configured registry auth for Gitea Container Registry
  - Added Gitea-specific env vars (CI_SERVER_URL, CI_REGISTRY_TOKEN)
- Created separate workflows for CI, CD, Release, Dependency updates

### Gitea Container Registry Setup

- Enabled Container Registry in Gitea config (app.ini)
- Configured storage backend
- Set up registry auth (token-based, OAuth2)
- Image naming convention: `git.allucanget.biz/zwitschi/linearvoid/<service>:<tag>`
- Image retention policies (keep last N tags, cleanup old images)
- Registry cleanup cron job

### Repository & Documentation Updates

- Updated all GitHub references in docs to Gitea URLs (README.md, CONTRIBUTING.md, SETUP.md, docs/architecture/README.md)
- Updated docker-compose.yml to use Gitea registry images
- Updated .env.example with Gitea registry variables
- Created .gitea/ directory for workflows and config
- Updated docker-compose.prod.yml for production with Gitea registry
- Removed .github/workflows/ after migration complete

### Infrastructure & Runner Setup

- Provisioned self-hosted Gitea Actions runner (VM or container)
- Installed Docker, Node.js, Go, FFmpeg on runner
- Registered runner with Gitea instance
- Configured runner labels (linux)
- Set up runner auto-scaling (optional)
- Configured runner logging and monitoring