- Updated README.md to reflect changes in load testing commands and added usage examples for Bash scripts. - Converted PowerShell load testing scripts to Bash equivalents for better cross-platform compatibility. - Added new Bash scripts: run-loadtests.sh, summarize-artillery.sh, and summarize-k6.sh to mirror existing PowerShell functionality. - Updated admin-api.js and ws-chat.yml for consistency and improved performance metrics reporting. - Enhanced error handling and output formatting in summary scripts for better readability.
8.5 KiB
Deployment Guide
Environments
| Environment | Host | Docker Target | Purpose |
|---|---|---|---|
| Test Lab | 192.168.88.40 (internal) |
tcp://192.168.88.40:2375 |
Integration testing, staging, CI/CD validation |
| Production | Hetzner VPS (TBD) | tcp://<hetzner-ip>:2375 |
Public linearvoid.tv |
Current state (2026-08-06): Test lab is active and receives automated CD deploys. Production Hetzner VPS is not yet provisioned. cd.yml and docker-compose.prod.yml target the test lab; switching to production only requires changing DOCKER_HOST.
Production Target (Hetzner)
Status: Not provisioned. Plan documented and pipeline ready; only the VPS needs creation.
Planned Infrastructure
- Provider: Hetzner Cloud, Region: Ashburn, VA (us-east)
- Instance: CX22 (2 vCPU, 4 GB RAM, 40 GB SSD) or equivalent
- OS: Ubuntu 24.04 LTS or Debian 12
- Bandwidth: 20 TB included traffic/month; Estimated cost: ~€7–10/month base
Cutover Plan
- Provision Hetzner VPS
- Add production secrets to Gitea (
CI_PROD_*prefix) - Update
DOCKER_HOSTin CD workflow to Hetzner public IP - Trigger deploy → validate health checks
- Update Cloudflare A record
linearvoid.tv→ Hetzner IP - Enable HTTPS, WAF, cache rules on production zone
- Decommission test lab as public target
Production Preflight (minimum readiness gate)
- Verify Hetzner host reachable over SSH and Docker remote API TLS works for the Gitea runner
- Confirm all production secrets exist under
CI_PROD_*prefix - Confirm
linearvoid.tv,www.linearvoid.tv,media.linearvoid.tvDNS targets match intended phase - Complete manual Cloudflare/UptimeRobot checklist (see
docs/operations/cloudflare-uptimerobot-checklist.md) - Run production smoke path end-to-end: viewer SPA, HLS playback, chat connect, admin login, upload, playlist save, playout health
- Verify rollback possible before final DNS switch
Firewall / Security Group
| Port | Source | Purpose |
|---|---|---|
| 22 | Admin IPs | SSH management |
| 80 | 0.0.0.0/0 | HTTP (redirect to HTTPS) |
| 443 | 0.0.0.0/0 | HTTPS (nginx) |
| 2375 | Gitea runner IP only | Docker remote API (TLS) |
| 9090 | Admin IPs or internal | Prometheus (or tunnel via SSH) |
| 3001 | Admin IPs or internal | Grafana (or tunnel via SSH) |
Pipeline Readiness
| Workflow | Trigger | Purpose |
|---|---|---|
ci.yml |
Push to main |
Lint, build frontend, vet Go, build & push images (:latest, :main) |
cd.yml |
Push to main |
Deploy to test lab (192.168.88.40:2375) — runs in parallel with CI |
release.yml |
Tag push *.*.* |
Build & push versioned images (:X.Y.Z, :X.Y, :latest), create Gitea Release |
workers-deploy.yml |
After CI succeeds | Deploy frontend to Cloudflare Workers |
CI and CD run simultaneously on every push to main. Deploy pulls whatever :latest images are available at that moment. cd.yml uses DOCKER_HOST to target the host:
# Test lab (current)
DOCKER_HOST=tcp://192.168.88.40:2375
# Production (when provisioned)
DOCKER_HOST=tcp://<hetzner-public-ip>:2375
No pipeline code changes needed to switch targets.
Target Topology
- Host: Linux server with Docker Engine
- Reverse proxy: nginx container (linearvoid-nginx)
- Core services: playout, chat, redis, postgres
- Observability: prometheus, grafana
- CDN/edge: Cloudflare
Prerequisites
- Docker Engine + Docker Compose plugin
- Network access to git.allucanget.biz
- Valid
.envfile with production values - DNS configured for
linearvoid.tv(andmedia.linearvoid.tvif split media domain used)
Environment File
.env requires at minimum:
DATABASE_URL— PostgreSQL connection stringREDIS_URL— Redis connection stringADMIN_USERNAME/ADMIN_PASSWORD— admin dashboard credentialsADMIN_SESSION_TTL_HOURS— session expiry (default 12)VITE_PUBLIC_APP_URL/VITE_PUBLIC_WS_URL/VITE_PUBLIC_HLS_URL— frontend URLs
Cloudflare R2 (required for HLS segment sync to edge):
R2_ACCESS_KEY_ID/R2_SECRET_ACCESS_KEYR2_BUCKET_NAME(e.g.linearvoid-media)R2_ENDPOINT— S3-compatible endpoint URLR2_REGION— set toauto
Cloudflare (optional, for Workers):
CLOUDFLARE_API_TOKEN/CLOUDFLARE_ACCOUNT_IDMEDIA_DOMAIN— split media domain (e.g.media.linearvoid.tv)
See .env.example and SETUP.md for full variable reference.
Gitea Secrets
cd.yml injects sensitive values from Gitea secrets into .env at deploy time. Configure in the Gitea repository:
DATABASE_URL, ADMIN_USERNAME, ADMIN_PASSWORD, R2_ACCESS_KEY_ID, R2_SECRET_ACCESS_KEY, R2_BUCKET_NAME, R2_ENDPOINT, R2_ADMIN_ACCESS_KEY_ID (optional), R2_ADMIN_SECRET_ACCESS_KEY (optional), R2_ADMIN_ENDPOINT (optional), CLOUDFLARE_API_TOKEN (optional), CLOUDFLARE_ACCOUNT_ID (optional), CI_REGISTRY_USER, CI_REGISTRY_TOKEN
Pull-and-Run Deployment
Images are built/pushed automatically by CI and release workflows. Every push to main triggers CI (:main, :latest) and CD (deploys :latest to test lab) in parallel. Semantic version tags (*.*.*, e.g. 1.0.9) trigger the release workflow (:X.Y.Z, :X.Y, :latest + Gitea Release).
docker login git.allucanget.biz
docker compose -f docker-compose.prod.yml pull
docker compose -f docker-compose.prod.yml up -d --remove-orphans
First-Time or Recovery Provisioning
mkdir -p data/chat data/playlist data/media data/hls
docker volume ls
docker network ls
docker compose -f docker-compose.prod.yml up -d
Health Verification
docker compose -f docker-compose.prod.yml ps
Endpoint checks:
- App:
http://{host}:8080/ - Nginx health:
http://{host}:8080/health - Chat health:
http://{host}:3000/health - Playout health:
http://{host}:8081/health - Prometheus:
http://{host}:9090 - Grafana:
http://{host}:3001
Domain checks:
https://linearvoid.tvhttps://linearvoid.tv/adminhttps://linearvoid.tv/metrics
Zero/Low-Downtime Update Pattern
- Pull newest images.
- Recreate changed services only:
docker compose -f docker-compose.prod.yml pull
docker compose -f docker-compose.prod.yml up -d nginx chat playout
- Re-run health checks.
Rollback
If new images are unhealthy:
- Pin previous tags in
docker-compose.prod.yml. - Pull pinned tags.
- Restart impacted services:
docker compose -f docker-compose.prod.yml pull
docker compose -f docker-compose.prod.yml up -d nginx chat playout
Operational Tasks
# View logs
docker compose -f docker-compose.prod.yml logs -f nginx chat playout
# Restart one service
docker compose -f docker-compose.prod.yml restart chat
# Stop stack (keep data)
docker compose -f docker-compose.prod.yml down
# Stop stack — destructive (remove data)
docker compose -f docker-compose.prod.yml down -v
Nginx and Proxy Considerations
- If an external proxy (e.g. Nginx Proxy Manager) fronts the stack, align request body limits with internal nginx (see
infrastructure/nginx/nginx.conf). - For resumable admin uploads, ensure
client_max_body_sizesupports chunk requests in both proxy layers.
Security Baseline
- Use a strong
ADMIN_PASSWORDand rotate regularly. - Keep Docker host access restricted.
- Prefer HTTPS/WSS through Cloudflare.
- Restrict exposed ports if only proxy access is needed.
Common Deployment Pitfalls
- Missing frontend build artifacts in local-only compose workflows.
- Wrong registry credentials or missing login.
- Misaligned domain env vars (app/ws/hls) causing mixed-origin errors.
Troubleshooting
| Symptom | Check | Fix |
|---|---|---|
| Service not starting | docker logs linearvoid-<svc> --tail 50 |
Check .env vars, volume mounts, port conflicts |
| Playout stall / slate mode | curl http://<host>:8081/health — mode=slate |
Check playlist file, media paths, disk space |
| Chat not connecting | curl http://<host>:3000/health |
Check Redis: docker exec linearvoid-redis redis-cli ping |
| Admin 401 loop | Cookie not sent | Verify X-Forwarded-Proto: https, nginx auth_request headers |
| Upload fails (413) | File too large | Check nginx client_max_body_size (should be 64m+) |
| HLS not playing | curl http://<host>/hls/playlist.m3u8 |
Check playout health, HLS dir permissions |
| Prometheus no targets | curl http://<host>:9090/targets |
Verify container network, scrape config |
| Grafana no data | Check datasource | Verify Prometheus URL http://prometheus:9090 provisioned |
Additional notes: proxy-layer body size limits can return 413 on upload; health checks may fail before service warmup completes.