linearvoid/SETUP.md

# Linear Void Setup Guide

This guide covers prerequisites, environment setup, running the application locally and in production, and troubleshooting. It references the [architecture documentation](./docs/architecture/README.md)(`docs/architecture/README.md`).

## Prerequisites

- **Docker & Docker Compose**: Required for running the full stack (playout, chat, nginx, redis, db, prometheus, grafana). Install from [docker.com](https://www.docker.com).
- **Node.js 20+**: Required for frontend development and production builds (`cd frontend && npm install && npm run build`).
- **npm**: Installed with Node.js and used for the frontend dependency installation and Vite workflow.
- **Go 1.23+**: Recommended when working on the Go services outside Docker (`backend/chat`, `backend/playout`).
- **FFmpeg**: Used by the playout service for HLS generation. Install via `apt install ffmpeg` (Linux), Homebrew (macOS), or Winget (Windows).
- **Git**: For cloning and version control.
- **Cloudflare Account**: For R2 storage and CDN (free tier sufficient for most use cases).
- **Gitea Account**: Required if you want to push images to the self-hosted registry or use the repository's CI/CD flow.
- **Hetzner Cloud Account** (optional for production): For the US-East VPS (Ashburn, VA recommended per ADR-001).

## Development Prerequisites

- **Frontend dependencies**: Run `cd frontend && npm install` before `npm run dev` or `npm run build`.
- **Docker build support**: Local development expects Docker to be able to build the `chat`, `playout`, and `nginx` images referenced in `docker-compose.yml`.
- **Data directories**: Create `data/playlist`, `data/media`, `data/hls`, and `data/chat` before the first run so bind mounts resolve cleanly.
- **Database choice**: The stack defaults to PostgreSQL in `docker-compose.yml`. If you want SQLite for local-only development, update `DATABASE_URL` in `.env` accordingly.
- **Redis availability**: Chat presence, rate limiting, and pub/sub require Redis. The compose stack provides it automatically.
- **Video inputs**: Source videos should be normalized before playout. If the playlist contains invalid files, the playout container will fail its validation flow.

## Environment Setup

1. **Clone the Repository** (if not already done):

   ```bash
   git clone https://git.allucanget.biz/zwitschi/linearvoid.git
   cd linearvoid
   ```

2. **Copy Environment File**:

   ```bash
   cp .env.example .env
   ```

3. **Configure `.env`** (edit with your values):

   ```env
   # Cloudflare R2 (required for video storage and $0 egress)
   R2_ACCESS_KEY_ID=your-r2-access-key
   R2_SECRET_ACCESS_KEY=your-r2-secret-key
   R2_BUCKET_NAME=linearvoid-media
   R2_ENDPOINT=https://your-account-id.r2.cloudflarestorage.com
   R2_REGION=auto

   # Cloudflare (for custom domain and cache rules)
   CLOUDFLARE_API_TOKEN=your-cloudflare-api-token
   CLOUDFLARE_ACCOUNT_ID=your-account-id
   MEDIA_DOMAIN=media.linearvoid.tv

   # Public frontend URLs
   # Preferred local setup uses the real public domains through Cloudflare and Nginx Proxy Manager.
   VITE_PUBLIC_APP_URL=https://linearvoid.tv
   VITE_PUBLIC_WS_URL=wss://linearvoid.tv/ws
   VITE_PUBLIC_HLS_URL=https://media.linearvoid.tv/hls/playlist.m3u8
   # Optional direct-local fallback when bypassing the public proxy:
   # VITE_PUBLIC_APP_URL=http://localhost
   # VITE_PUBLIC_WS_URL=ws://localhost/ws
   # VITE_PUBLIC_HLS_URL=http://localhost/hls/playlist.m3u8

   # Gitea Container Registry (for self-hosted Docker images)
   GITEA_REGISTRY_URL=git.allucanget.biz
   GITEA_USERNAME=your-gitea-username
   GITEA_TOKEN=your-gitea-token
   GITEA_REGISTRY_NAMESPACE=zwitschi/linearvoid

   # Chat / Database
   DATABASE_URL=postgres://linearvoid:linearvoid@db:5432/linearvoid?sslmode=disable
   # or for SQLite-only local development:
   # DATABASE_URL=sqlite://./data/chat.db
   REDIS_URL=redis://redis:6379/0

   # Ports (for local development)
   PLAYOUT_PORT=8080
   CHAT_PORT=3000
   NGINX_PORT=80

   # Playout Configuration
   PLAYLIST_PATH=/app/playlist.txt
   HLS_SEGMENT_DURATION=6
   HLS_WINDOW_SIZE=24

   # Logging / Monitoring
   LOG_LEVEL=info
   PROMETHEUS_PORT=9090
   GRAFANA_PORT=3001

   # Development flags
   NODE_ENV=development
   GO_ENV=development

   # Gitea API access (optional, for CI/release automation)
   GITEA_SERVER_URL=https://git.allucanget.biz
   GITEA_TOKEN=your-gitea-token
   GITEA_USERNAME=your-gitea-username
   ```

4. **Check local value overrides for Docker Compose**:
   - `docker-compose.yml` builds images locally and maps the frontend build output from `frontend/dist` into nginx.
   - `docker-compose.prod.yml` pulls prebuilt images from the Gitea registry.
   - Internal container-to-container URLs should use compose hostnames, for example `REDIS_URL=redis://redis:6379/0` and `DATABASE_URL=postgres://linearvoid:<password>@db:5432/linearvoid?sslmode=disable`.
   - For production public routing, set `VITE_PUBLIC_APP_URL=https://linearvoid.tv`, `VITE_PUBLIC_WS_URL=wss://linearvoid.tv/ws`, and `VITE_PUBLIC_HLS_URL=https://media.linearvoid.tv/hls/playlist.m3u8`.

5. **Create Data Directories**:

   ```bash
   mkdir -p data/playlist data/media data/hls data/chat
   ```

6. **Prepare Sample Videos** (optional):
   - Place normalized MP4 files in `data/media/`.
   - Create a `data/playlist/playlist.txt` with paths (see ADR-001 for normalization requirements).

## Cloudflare R2 Setup

1. **Create an R2 bucket**:
   - Create a bucket such as `linearvoid-media` in Cloudflare R2.
   - Keep the bucket name aligned with `R2_BUCKET_NAME` in `.env`.

2. **Generate R2 API credentials**:
   - Create an access key pair with read/write permissions for the bucket.
   - Set `R2_ACCESS_KEY_ID`, `R2_SECRET_ACCESS_KEY`, and `R2_REGION=auto` in `.env`.

3. **Find the S3 endpoint**:
   - Use the account-scoped endpoint format `https://<account-id>.r2.cloudflarestorage.com`.
   - Store the account ID in both `R2_ENDPOINT` and `CLOUDFLARE_ACCOUNT_ID` where applicable.

4. **Bind a custom media domain**:
   - Attach a domain such as `media.linearvoid.tv` to the bucket.
   - Set `MEDIA_DOMAIN` to that hostname.

5. **Configure app-domain DNS and TLS**:
   - Point `linearvoid.tv` to the public entrypoint serving nginx.
   - Point `www.linearvoid.tv` to the same target and redirect it to `https://linearvoid.tv`.
   - Keep `media.linearvoid.tv` on the R2 custom domain path so HLS delivery stays split from the app origin.
   - Use Cloudflare proxied DNS with Full (strict) TLS so browser traffic uses `https://linearvoid.tv`, `wss://linearvoid.tv/ws`, and `https://media.linearvoid.tv`.

6. **Apply cache behavior**:
   - Configure the `.ts` and `.m3u8` cache rules from `infrastructure/cloudflare/cache-rules.md`.
   - Enable Smart Tiered Cache for the R2-backed hostname.
   - If you need header normalization for immutable segment caching, deploy `infrastructure/cloudflare/worker.js` to the same zone.

7. **Verify access**:
   - Confirm uploads with an S3-compatible client such as `aws s3 ls --endpoint-url <R2_ENDPOINT>`.
   - After deployment, check Cloudflare analytics for cache hit ratio and origin request volume.

## Gitea Setup

1. **Create or verify a Gitea user**:
   - The user should have access to the repository and permission to read from the container registry.

2. **Create an access token**:
   - Generate a token with the scopes needed for package/registry access and, if applicable, repository automation.
   - Store it as `GITEA_TOKEN` in `.env`.

3. **Configure registry variables**:
   - Set `GITEA_REGISTRY_URL` to the Gitea host.
   - Set `GITEA_REGISTRY_NAMESPACE` to the image namespace, for example `zwitschi/linearvoid`.
   - Set `GITEA_USERNAME` to the account that will push or pull images.

4. **Authenticate Docker to the registry**:

   ```bash
   docker login git.allucanget.biz
   ```

5. **Build and publish images when needed**:
   - Local development uses `docker-compose.yml`, which builds from source.
   - Production uses `docker-compose.prod.yml`, which expects the `playout`, `chat`, and `nginx` images to exist in the configured Gitea registry namespace.

6. **Validate image availability**:
   - Before a production rollout, verify that the expected `:latest` tags can be pulled from the registry on the target server.

## Running the Application

### Local Development

1. **Start All Services**:

   ```bash
   docker compose up --build
   ```

2. **Access the Application**:
   - Preferred local access with the documented proxy setup: <https://linearvoid.tv>
   - Admin Interface: <https://linearvoid.tv/admin>
   - Chat WebSocket: `wss://linearvoid.tv/ws`
   - Direct-local fallback when bypassing Cloudflare/Nginx Proxy Manager: <http://localhost>, <http://localhost/admin>, and `ws://localhost/ws`
   - Health Checks:
     - Playout: <http://localhost:8081/health>
     - Chat: <http://localhost:3000/health>
   - Monitoring:
     - Prometheus: <http://localhost:9090>
     - Grafana: <http://localhost:3001> (default login: admin/admin)

3. **Frontend Development** (optional, for UI changes):

   ```bash
   cd frontend
   npm install
   npm run dev
   ```

   The dev server runs on <http://localhost:5173>. It proxies API calls to the backend services.

4. **Build frontend assets for nginx-backed local runs**:

   ```bash
   cd frontend
   npm install
   npm run build
   ```

   The local nginx container serves `frontend/dist`, so rebuilding the frontend is required before validating production-like static output.

5. **Stop Services**:

   ```bash
   docker compose down
   ```

### Production Setup Guide

1. **Prepare the target host**:
   - Provision a Linux VPS with Docker Engine, Docker Compose, Git, and enough disk for media, HLS output, PostgreSQL, Redis persistence, and monitoring data.
   - Open the ports required for HTTP/HTTPS and any restricted monitoring access you intend to expose.

2. **Clone the repository and create the environment file**:

   ```bash
   git clone https://git.allucanget.biz/zwitschi/linearvoid.git
   cd linearvoid
   cp .env.example .env
   ```

3. **Set production-safe `.env` values**:
   - Replace placeholder secrets.
   - Use compose-network hostnames for internal services, for example `redis` and `db`.
   - Set strong values for `POSTGRES_PASSWORD` and `GRAFANA_PASSWORD`.
   - Keep `MEDIA_DOMAIN`, R2 credentials, and Gitea registry values consistent with the external infrastructure.
   - Set `MEDIA_DOMAIN=media.linearvoid.tv`.
   - Set `VITE_PUBLIC_APP_URL=https://linearvoid.tv`, `VITE_PUBLIC_WS_URL=wss://linearvoid.tv/ws`, and `VITE_PUBLIC_HLS_URL=https://media.linearvoid.tv/hls/playlist.m3u8`.
   - If you intentionally bypass the public proxy for local-only testing, switch those values to `http://localhost`, `ws://localhost/ws`, and `http://localhost/hls/playlist.m3u8`.

4. **Create runtime directories on the server**:

   ```bash
   mkdir -p data/playlist data/media data/hls data/chat
   ```

5. **Publish or verify required images in Gitea**:
   - Ensure the `playout`, `chat`, and `nginx` images exist under `${GITEA_REGISTRY_URL}/${GITEA_REGISTRY_NAMESPACE}`.
   - Log in to the registry from the VPS before pulling.

6. **Deploy the production stack**:

   ```bash
   docker compose -f docker-compose.prod.yml pull
   docker compose -f docker-compose.prod.yml up -d
   ```

7. **Run post-deploy verification**:
   - Check `docker compose -f docker-compose.prod.yml ps`.
   - Confirm nginx, chat, and playout health endpoints respond.
   - Verify that HLS output is being written to `data/hls` and that the media domain serves manifests and segments through Cloudflare.

8. **Harden and operate the deployment**:
   - Put HTTPS in front of nginx using Cloudflare or another reverse proxy, with `linearvoid.tv` as canonical app host and `www.linearvoid.tv` redirected.
   - Restrict Grafana and Prometheus exposure to trusted IPs or a private network.
   - Run the stack under systemd or another service manager if you need auto-start behavior outside Docker restart policies.

### Production Deployment (Hetzner VPS + Cloudflare)

1. **Provision VPS**: Use Hetzner Cloud CX22 in Ashburn, VA (per ADR-001).
2. **Configure Cloudflare**:
   - Create R2 bucket with `enam` location hint.
   - Set up custom media domain (`media.linearvoid.tv`) with the cache rules from `infrastructure/cloudflare/cache-rules.md`.
   - Point `linearvoid.tv` and `www.linearvoid.tv` at the app entrypoint, then redirect `www` to the apex domain.
   - Use Full (strict) TLS for both app and media hostnames.
   - Apply the Worker from `infrastructure/cloudflare/worker.js` for advanced header management.
3. **Deploy**:
   - Copy the project to the VPS.
   - Use production compose: `docker compose -f docker-compose.prod.yml up -d`
   - Or pull from Gitea registry: `docker compose -f docker-compose.prod.yml pull && docker compose -f docker-compose.prod.yml up -d`
   - Configure systemd to manage the Docker Compose stack.
4. **Monitoring**: Access Grafana at the exposed port for dashboards (included in `monitoring/grafana/dashboard.json`).

See `docs/architecture/README.md` for the full arc42 documentation and the ADRs for architectural decisions.

## Troubleshooting

- **R2 Upload Fails**: Check credentials in `.env` and ensure the bucket exists. Verify with `aws s3 ls --endpoint-url=$R2_ENDPOINT`.
- **Normalization Errors**: Run `backend/playout/start-playout.sh` manually to see ffprobe output. All videos must be H.264/AAC, 30 fps constant, 2s keyframes (ADR-001).
- **Chat Not Connecting**: Check Redis connection and WebSocket URL. The health check at `/health` should return 200.
- **Cache Not Working**: Verify Cloudflare cache rules (`.ts` = long TTL immutable, `.m3u8` = short TTL). Check cache hit ratio in Cloudflare dashboard.
- **Docker Issues**: Run `docker compose logs` for service-specific errors. Ensure ports are not in use.
- **Admin Interface**: Ensure the protected route is configured in nginx and authentication is set up (default is open for local development).
- **Production Pull Fails**: Confirm `docker login` to Gitea succeeded on the server and the image tags referenced by `docker-compose.prod.yml` exist.

## Further Reading

- Architecture: `docs/architecture/README.md` (arc42 template with diagrams).
- Decisions: `docs/architecture/adr/` (ADR-001 to ADR-007).
- Contributing: `CONTRIBUTING.md`.
- Project Structure: Root `README.md`.

The system is ready for use. For production, review ADR-001 for cost estimates and scaling guidance.

**Last Updated**: 2026-08-03