linearvoid/README.md

# Linear Void

[Linear Void](https://linearvoid.tv) is a self-managed **24/7 linear video channel** for a primarily US audience. It runs a continuous playlist of self-owned videos with true/near-true synchronization, a custom lightweight viewer SPA (hls.js player + chat), and full independence from third-party platforms (YouTube, Twitch, etc.).

**Repository**: [git.allucanget.biz/zwitschi/linearvoid](https://git.allucanget.biz/zwitschi/linearvoid)

[![Build Status](https://git.allucanget.biz/zwitschi/linearvoid/actions/workflows/ci.yml/badge.svg)](https://git.allucanget.biz/zwitschi/linearvoid/actions)
[![Gitea](https://img.shields.io/badge/Gitea-1.27.1-blue)](https://git.allucanget.biz)
[![License](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![Version](https://img.shields.io/badge/version-1.0.12-blue)](https://git.allucanget.biz/zwitschi/linearvoid/releases)
[![Release Notes](https://img.shields.io/badge/release-notes-green)](CHANGELOG.md)

## Architecture

The architecture (detailed in [`docs/architecture/README.md`](docs/architecture/README.md)) uses:

- Hetzner Ashburn VPS for playout (FFmpeg + MediaMTX/SRS) and chat _(planned production; currently test lab at 192.168.88.40)_.
- Cloudflare R2 + CDN with aggressive immutable caching of `.m4s` segments ($0 egress).
- Lightweight static frontend and WebSocket chat backend.
- Gitea self-hosted CI/CD (Actions) and Container Registry.

See [`docs/architecture/adr/`](docs/architecture/adr/) for the full set of decision records (ADR-001 through ADR-007) covering architecture, storage calculations, cache rules, cost model, module structure, tech stack, admin interfaces, data storage, and caching strategy.

## Key Features

- Continuous synchronized playback from a playlist (standard HLS, 15–35s latency).
- Self-hosted custom SPA with embedded chat (public rooms + private 1:1 DMs, moderation).
- Extreme cost efficiency via Cloudflare R2 edge caching.
- Video normalization, monitoring, fallback slate, and easy playlist management.
- US-centric performance (Ashburn origin + Cloudflare POPs).

## Architecture Diagram

```mermaid
flowchart TD
    V[US Viewers] -->|HTTPS| CF[Cloudflare CDN]
    CF -->|Cache HIT| R2[Cloudflare R2]
    CF -->|Cache MISS| N[Nginx :80]
    N -->|/ws| CH[Chat :3000]
    N -->|/admin| CH
    N -->|/hls| P[Playout :8081]
    N -->|/*| FE[Frontend SPA]
    CH --> R[(Redis)]
    CH --> DB[(PostgreSQL)]
    P -->|ffmpeg| HLS[/data/hls/]
    P -->|sync| R2
```

A detailed description of the Streaming architecture, playout, and delivery is available in [`docs/STREAMING.md`](docs/STREAMING.md).

## Project Structure

This structure directly reflects the building blocks, runtime flows, deployment view, and cross-cutting concepts defined in the arc42 documentation.

```text
linearvoid/
├── docs/                   # see Documentation section below
├── backend/                # see Backend section below
├── frontend/               # see Frontend section below
├── infrastructure/         # see Infrastructure section below
├── monitoring/             # see Monitoring section below
├── scripts/                # see Scripts section below
├── .editorconfig           # EditorConfig for consistent coding styles
├── .env.example            # Example environment variables for local development
├── .gitignore              # Ignore node_modules, build artifacts, etc.
├── .prettierignore         # Ignore files for Prettier formatting
├── CHANGELOG.md            # Release notes and version history
├── CONTRIBUTING.md         # Contribution guidelines
├── docker-compose.prod.yml # Production deployment (Hetzner + Cloudflare)
├── docker-compose.yml      # Top-level for dev (playout + chat + nginx)
└── README.md               # This file
```

### Documentation

- `docs/`: Folder containing all documentation for the project, including architecture, ADRs, operational procedures, and setup guides.
- `docs/architecture/`: Architecture documentation, including high-level system architecture, component interactions, and decision records (ADRs).

see [README.md](docs/README.md) for full architecture, ADRs, and operational documentation.

### Backend

- `backend/playout/`: Core linear engine. FFmpeg consumes a normalized playlist and outputs continuous HLS. Includes normalization validation.
- `backend/chat/`: Independent WebSocket service (Node.js/Socket.io or Go). Handles public rooms, private DMs, presence (Redis), persistence (PostgreSQL/SQLite), rate limiting, and moderation.
- `backend/shared/`: Reusable code (types, utils, video validation).

```text
backend/
├── chat/                               # WebSocket chat service (Node.js or Go)
│   ├── src/                            # Chat service source code (rooms, DMs, moderation)
|   |   ├── auth/                       # Admin authentication and session management
|   |   |   ├── auth_test.go            # Unit tests for authentication
|   |   |   └── auth.go                 # Authentication logic (login, session validation)
|   |   ├── config/                     # Configuration files for chat service
|   |   |   ├── config_test.go          # Unit tests for configuration
|   |   |   └── config.go               # Configuration loading and validation
|   |   ├── db/                         # Database access and models
|   |   |   ├── db_test.go              # Unit tests for database access
|   |   |   ├── db.go                   # Database connection and query logic
|   |   |   ├── models.go               # Database models (users, messages, rooms)
|   |   |   └── redis.go                # Redis connection and presence management
|   |   ├── handlers/                   # WebSocket message handlers
|   |   |   ├── handlers_test.go        # Unit tests for WebSocket message handlers
|   |   |   └── handlers.go             # WebSocket message handling logic
|   |   ├── media/                      # Media-related utilities (e.g., video validation)
|   |   |   ├── media_test.go           # Unit tests for media-related utilities
|   |   |   └── media.go                # Media-related utility functions
|   |   ├── metrics/                    # Prometheus metrics endpoints
|   |   |   ├── metrics_test.go         # Unit tests for Prometheus metrics
|   |   |   └── metrics.go              # Prometheus metrics collection and exposition
|   |   ├── ratelimit/                  # Rate limiting logic (Redis-based)
|   |   |   ├── ratelimit_test.go       # Unit tests for rate limiting
|   |   |   └── ratelimit.go            # Rate limiting logic using Redis
|   |   ├── ws/                         # WebSocket handlers
|   |   |   ├── ws_test.go              # Unit tests for WebSocket handlers
|   |   |   └── ws.go                   # WebSocket connection and message handling
|   |   └── main.go                     # Entry point for chat service
|   ├── Dockerfile                      # Dockerfile for chat service
|   ├── go.mod                          # Go module file for chat service
|   └── go.sum                          # Go checksum file for chat service
└── playout/                            # Continuous HLS generation from playlist (Go)
    ├── demodata/                       # Sample media and playlist files for testing
    |   ├── media/                      # Sample media files for testing
    |   └── playlist/                   # Sample playlist files for testing
    ├── src/                            # Playout service source code (FFmpeg orchestration)
    |   ├── go.mod                      # Go module file for playout service
    |   ├── go.sum                      # Go checksum file for playout service
    |   ├── playlist_test.go            # Unit tests for playlist management
    |   └── playlist.go                 # Playlist management logic
    └── testdata/                       # Test data for playout service (sample playlists, media)
    |   ├── comments_only.txt           # Sample playlist with comments only
    |   ├── empty_playlist.txt          # Sample empty playlist
    |   ├── invalid_24fps.mp4           # Sample invalid 24fps video for testing
    |   ├── invalid_h265.mp4            # Sample invalid H.265 video for testing
    |   ├── mixed_valid_invalid.txt     # Sample playlist with mixed valid and invalid entries
    |   ├── valid_720p30.mp4            # Sample valid 720p30 video for testing
    |   └── valid_playlist.txt          # Sample valid playlist for testing
    ├── Dockerfile                      # Dockerfile for the playout service
    └── start-playout.sh                # Script to start the playout service
```

### Frontend

The viewer experience — minimal, modern SPA with custom CSS + vanilla JS. Uses `hls.js` for playback and a WebSocket client for chat.

```text
frontend/
├── admin/                  # Admin SPA (playlist management, moderation, analytics)
│   ├── login/              # Login page for admin authentication
|   |  └── index.html       # Admin login page
|   └── index.html          # Admin SPA
├── public/                 # Static assets (favicon, images, etc.)
│   ├── admin/              # Admin static assets
|   |  ├── analytics.html   # Analytics dashboard page
|   |  ├── chat-log.html    # Chat log viewer page
|   |  ├── dashboard.html   # Admin dashboard page
|   |  ├── moderation.html  # User moderation page
|   |  ├── playlist.html    # Playlist management page
|   |  ├── schedule.html    # Schedule overrides page
|   |  ├── system.html      # System configuration page
|   |  ├── upload.html      # Video upload page
|   |  └── users.html       # User management page
│   ├── favicon.ico         # Favicon for the SPA
|   └── icons.svg           # SVG icons for the SPA
├── src/                    # Source code for the SPA
│   ├── assets/             # Images, fonts, and other static assets
|   |  ├── hero.png         # Hero image for the SPA
|   |  ├── typescript.svg   # TypeScript logo for the SPA
|   |  └── vite.svg         # Vite logo for the SPA
│   ├── admin.test.ts       # Admin frontend tests (Vitest)
│   ├── admin.ts            # Admin SPA logic (playlist, moderation, analytics)
│   ├── counter.ts          # Counter component for testing
│   ├── main.test.ts        # Main frontend tests (Vitest)
│   ├── main.ts             # Main SPA logic (hls.js player, WebSocket chat)
│   └── style.css           # Custom CSS for the SPA
├── .gitignore              # Ignore node_modules, build artifacts, etc.
├── index.html              # Main SPA entry point
├── package-lock.json       # Lockfile for npm dependencies
├── package.json            # Build tools (Vite recommended)
├── tsconfig.json           # TypeScript configuration
├── vite.config.ts          # Vite configuration
└── vitest.config.ts        # Vitest configuration
```

### Infrastructure

Everything needed to deploy and configure the stack (Docker, Nginx, Cloudflare rules, future Terraform).

- Cloudflare
  - [worker.js](infrastructure/cloudflare/worker.js) — adds immutable cache headers to HLS segments
- Nginx
  - [Dockerfile](infrastructure/nginx/Dockerfile) — builds Nginx with custom config
  - [nginx.conf](infrastructure/nginx/nginx.conf) — reverse proxy, static SPA, HLS delivery

### Monitoring

Grafana dashboards, Prometheus rules, logging configuration.

- Grafana
  - dashboards
    - [dashboard.json](monitoring//grafana/dashboards/dashboard.json) — operational dashboards for playout, chat, and Redis
    - [dashboard.yml](monitoring//grafana/dashboards/dashboard.yml) — operational dashboards for playout, chat, and Redis
  - datasources
    - [datasource.yml](monitoring/grafana/datasources/datasource.yml) — Prometheus datasource configuration
  - [dashboard.json](monitoring/grafana/dashboard.json) — operational dashboards for playout, chat, and Redis
- Prometheus
  - [alerts.yml](monitoring/prometheus/alerts.yml) — alerting rules for playout, chat, Redis, and system health
  - [prometheus.yml](monitoring/prometheus/prometheus.yml) — scrape targets for playout, chat, Redis, and system metrics

### Scripts

One-off and CI-friendly scripts for video preparation, R2 sync, backups, and playlist management.

- loadtest/
  - [admin-api.js](scripts/loadtest/admin-api.js) — admin API load test script
  - [run-loadtests.ps1](scripts/loadtest/run-loadtests.ps1) — PowerShell script to run load tests
  - [summarize-artillery.ps1](scripts/loadtest/summarize-artillery.ps1) — PowerShell script to summarize Artillery load test results
  - [summarize-k6.ps1](scripts/loadtest/summarize-k6.ps1) — PowerShell script to summarize k6 load test results
  - [ws-chat.yml](scripts/loadtest/ws-chat.yml) — WebSocket chat load test configuration

## Getting Started

> **Note:** The `docker compose up` below targets local development. The test lab (192.168.88.40) receives automated CD deploys. Production Hetzner VPS is not yet provisioned — see `DEPLOYMENT.md`.

```bash
git clone https://git.allucanget.biz/zwitschi/linearvoid.git
cd linearvoid
cp .env.example .env
# Edit .env with your R2 credentials and domain settings
docker compose up --build
```

Then open:

- **Viewer**: [https://linearvoid.tv](https://linearvoid.tv) (or `http://localhost`)
- **Admin**: [https://linearvoid.tv/admin](https://linearvoid.tv/admin) (or `http://localhost/admin`)
- **Metrics**: [https://linearvoid.tv/metrics](https://linearvoid.tv/metrics)

Full setup guide: [`SETUP.md`](SETUP.md).

See the [architecture docs](docs/architecture/README.md) for detailed architecture and ADRs.

## Development

- Use Docker Compose for local development (playout + chat + frontend).
- Test lab host: `192.168.88.40` (internal network, receives CD deploys). Production Hetzner VPS TBD.
- Public production host model: `https://linearvoid.tv` for viewer/admin/chat and `https://media.linearvoid.tv` for HLS delivery.
- Normalize all source videos before adding to playlist (see ADR-001 for exact codec/fps/keyframe requirements).
- Monitor cache hit ratio and R2 operations closely — they determine real cost.

### Environment Variables

| Variable               | Default                                         | Description                      |
| ---------------------- | ----------------------------------------------- | -------------------------------- |
| `R2_ACCESS_KEY_ID`     | —                                               | Cloudflare R2 access key         |
| `R2_SECRET_ACCESS_KEY` | —                                               | Cloudflare R2 secret key         |
| `R2_BUCKET_NAME`       | `linearvoid-media`                              | R2 bucket for HLS segments       |
| `DATABASE_URL`         | `postgres://...@db:5432/...`                    | Chat DB (Postgres or SQLite)     |
| `REDIS_URL`            | `redis://redis:6379/0`                          | Redis for presence/rate limiting |
| `VITE_PUBLIC_APP_URL`  | `https://linearvoid.tv`                         | Public viewer URL                |
| `VITE_PUBLIC_WS_URL`   | `wss://linearvoid.tv/ws`                        | WebSocket endpoint               |
| `VITE_PUBLIC_HLS_URL`  | `https://media.linearvoid.tv/hls/playlist.m3u8` | HLS playlist URL                 |
| `ADMIN_USERNAME`       | `admin`                                         | Admin panel username             |
| `ADMIN_PASSWORD`       | —                                               | Admin panel password             |
| `HLS_SEGMENT_DURATION` | `6`                                             | HLS segment duration in seconds  |
| `HLS_WINDOW_SIZE`      | `24`                                            | HLS playlist window size         |

Full reference: [`.env.example`](.env.example).

### Troubleshooting

| Problem                  | Check                                                                                 |
| ------------------------ | ------------------------------------------------------------------------------------- |
| **Playout not starting** | Verify playlist exists in `data/playlist/playlist.txt` and media files are accessible |
| **HLS not playing**      | Check `docker compose logs playout` for ffmpeg errors; verify `data/hls/` permissions |
| **Chat not connecting**  | Ensure Redis is running: `docker compose logs redis`; check `DATABASE_URL`            |
| **Admin panel 401**      | Verify `ADMIN_USERNAME`/`ADMIN_PASSWORD` in `.env` match login credentials            |
| **Docker build fails**   | Run `docker compose build --no-cache`; check `backend/playout/.dockerignore`          |
| **R2 upload failures**   | Verify `R2_ACCESS_KEY_ID` and `R2_SECRET_ACCESS_KEY`; check bucket CORS settings      |

### API Endpoints

| Service     | Endpoint                 | Description                |
| ----------- | ------------------------ | -------------------------- |
| **Chat**    | `GET /health`            | Health check               |
| **Chat**    | `GET /metrics`           | Prometheus metrics         |
| **Chat**    | `GET /ws`                | WebSocket upgrade for chat |
| **Chat**    | `GET /api/admin/*`       | Admin API (auth required)  |
| **Chat**    | `POST /api/admin/login`  | Admin login                |
| **Playout** | `GET /health`            | Health check (port 8081)   |
| **Nginx**   | `GET /`                  | Viewer SPA                 |
| **Nginx**   | `GET /admin/`            | Admin panel                |
| **Nginx**   | `GET /hls/playlist.m3u8` | HLS manifest               |

## Architecture Documentation

The architecture is fully documented using the [arc42 template](docs/architecture/README.md) (with populated sections, Mermaid context and building-block diagrams, deployment view, cross-cutting concepts, quality goals, risks, and glossary).

**Decision Records** (in `docs/architecture/adr/`):

- **[ADR-001](docs/architecture/adr/ADR-001.md)**: Overall system architecture, HLS + Cloudflare R2 caching strategy, storage calculations, cost model, risks, and implementation plan.
- **[ADR-002](docs/architecture/adr/ADR-002-module-structure.md)**: Chosen project folder/module structure (frontend, backend/playout, backend/chat, infrastructure, scripts, monitoring) and mapping to arc42 building blocks.
- **[ADR-003](docs/architecture/adr/ADR-003-tech-stack.md)**: Technology stack, language choices per module (TypeScript for frontend, Go preferred for chat and playout wrapper, Bash/Python for scripts), Docker-first approach, and all implicit assumptions.
- **[ADR-004](docs/architecture/adr/ADR-004-admin-video.md)**: Video admin web interface (upload, playlist management, schedule overrides, normalization enforcement).
- **[ADR-005](docs/architecture/adr/ADR-005-admin-moderation-analytics.md)**: Web admin interfaces for user management, moderation dashboard, analytics, and system configuration.
- **[ADR-006](docs/architecture/adr/ADR-006-data-storage.md)**: Data storage strategy (SQLite vs PostgreSQL for v1, migration path, admin interface capabilities).
- **[ADR-007](docs/architecture/adr/ADR-007-redis-usage.md)**: Redis usage (exact use cases, TTL policies, persistence, fallback behavior, admin configuration).

See the [architecture README](docs/architecture/README.md) for the big-picture view and the individual ADR files for detailed rationale, alternatives, and consequences.

Contributions, issues, and further evolution (LL-HLS, ABR, multi-channel, E2EE) should reference the architecture documentation.

## Resources

- [CHANGELOG.md](CHANGELOG.md) - Version history and completed features.
- [CONTRIBUTING.md](CONTRIBUTING.md) - Guidelines for contributing to the project.
- [SETUP.md](docs/SETUP.md) - Complete setup guide with prerequisites, environment configuration, running the application, and troubleshooting.
- [API.md](docs/API.md) - Implemented WebSocket and REST API reference.
- [DEPLOYMENT.md](docs/DEPLOYMENT.md) - Production deployment and operations guide.
- [MONITORING.md](docs/MONITORING.md) - Prometheus and Grafana setup and troubleshooting.
- [SECURITY.md](docs/SECURITY.md) - Security controls, hardening checklist, and incident baseline.
- [Scaling Playbook](docs/operations/scaling-performance-playbook.md) - Performance test method, thresholds, and scaling decision flow.
- [Architecture Documentation](docs/architecture/README.md) - Full arc42 template with diagrams and decisions.
- [ADRs](docs/architecture/adr/) - Detailed decision records (ADR-001 to ADR-007).

**CI/CD**: [View workflows](https://git.allucanget.biz/zwitschi/linearvoid/actions) on Gitea Actions.

**Container Registry**: `git.allucanget.biz/zwitschi/linearvoid/<service>:latest`

## Contributing

Contributions are welcome! See [`CONTRIBUTING.md`](CONTRIBUTING.md) for guidelines. Quick overview:

1. Read the [architecture docs](docs/architecture/README.md) and ADRs first.
2. Code against the defined module structure (frontend, backend/playout, backend/chat, infrastructure, scripts, monitoring).
3. Open PRs on [Gitea](https://git.allucanget.biz/zwitschi/linearvoid).
4. CI validates builds and tests automatically.

Since this is a linear project, we also rely on a linear development and release model: only `main` branch is used for releases, and all features are merged into `main` after review and CI validation. Releases use semantic version tags in `*.*.*` format (for example `1.0.9`) and are automatically built/pushed to the container registry. Version numbers are declared in `frontend/package.json`, `CHANGELOG.md`, and the version badge in this README.

[MIT License](LICENSE)

The project is ready for use. See [`SETUP.md`](docs/SETUP.md) for detailed installation instructions.