zwitschi/linearvoid
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 9 Wiki Activity

Initial commit: Project scaffolding + architecture documentation

Browse Source 
- arc42 template fully populated in docs/architecture/README.md with Mermaid diagrams, quality goals, deployment view, cross-cutting concepts, risks and glossary
- Consolidated ADR-001 (merged all previous root ADR drafts)
- ADR-002: Module/folder structure decision and rationale
- ADR-003: Technology stack, language choices per module (TypeScript, Go, Bash/Python), Docker-first approach, and explicit implicit assumptions
- Updated root README.md with comprehensive Project Structure section and links to all ADRs
- Created .github/instructions/TODO.md with detailed tracking (architecture phase marked complete)
- Folder structure created matching ADR-002 (frontend/, backend/playout/, backend/chat/, backend/shared/, infrastructure/, scripts/, monitoring/)
- Git repository initialized with main branch

This establishes the foundation defined in the arc42 documentation and the three ADRs.
This commit is contained in:
zwitschi committed 2026-08-02 15:31:33 +02:00
commit dafcec1303
8 files changed
+1158
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/architecture/README.md
+478
View File
@@ -0,0 +1,478 @@
# Architecture Documentation for Linear Void
> The architecture documentation for the Linear Void project is structured according to the [arc42 template](https://arc42.org/). This template provides a comprehensive framework for documenting software architecture, ensuring that all relevant aspects of the system are covered.
## Table of Contents
- [Introduction and Goals](#introduction-and-goals)
- [Architecture Constraints](#architecture-constraints)
- [Context and Scope](#context-and-scope)
- [Solution Strategy](#solution-strategy)
- [Building Block View](#building-block-view)
- [Runtime View](#runtime-view)
- [Deployment View](#deployment-view)
- [Cross-cutting Concepts](#cross-cutting-concepts)
- [Architecture Decisions](#architecture-decisions)
- [Quality Requirements](#quality-requirements)
- [Risks and Technical Debts](#risks-and-technical-debts)
- [Glossary](#glossary)
## Introduction and Goals
### Requirements Overview
Linear Void is an independent **24/7 linear video channel** broadcasting a playlist of self-owned pre-recorded videos. Viewers access it via a custom lightweight HTML/CSS/JavaScript SPA with synchronized HLS playback, schedule metadata, public chat rooms, and private 1:1 messaging. Complete independence from third-party video platforms (YouTube, Twitch, etc.).
**Core Requirements** (merged from all ADRs):
- Continuous synchronized playback from playlist (live-style HLS, not VOD).
- Self-hosted source media, frontend, schedule, and chat data.
- Primary US audience/origin (East Coast preferred for low latency/peering).
- Extreme cost efficiency (bandwidth dominant; target near-zero egress via Cloudflare R2 + edge caching).
- Custom chat with rooms, DMs, presence, moderation, persistence.
- Storage planning for 48h of 720p30/1080p30 content.
- Browser-compatible playback (`hls.js` + native HLS).
- Operational simplicity and scalability.
Full decision rationale, alternatives, and consequences in [`adr/ADR-001.md`](adr/ADR-001.md).
### Quality Goals
| Priority | Quality Attribute | Goal | Scenario |
| -------- | ------------------------- | -------------------------------------------------------- | -------------------------------------------------------------------------------- |
| 1 | **Cost Efficiency** | <$25/month for moderate scale (100s concurrent) | R2 $0 egress + immutable `.ts` edge caching eliminates per-viewer bandwidth cost |
| 2 | **Performance / Latency** | Low US latency; acceptable HLS delay | Hetzner Ashburn origin + Cloudflare US POPs (15–35s end-to-end) |
| 3 | **Availability** | 24/7 with minimal downtime | FFmpeg/MediaMTX auto-restart loop, fallback slate, monitoring |
| 4 | **Scalability** | Grow from 10s to 1000s viewers without proportional cost | Edge caching + R2; only request volume limits |
| 5 | **Maintainability** | Lightweight, clear configs, low ops overhead | Static SPA, Go/Node chat, SQLite/PostgreSQL+Redis |
| 6 | **Security / Usability** | Secure transport, moderation tools, good UX | HTTPS/WSS, Cloudflare WAF, rate limits, intuitive chat/player |
### Stakeholders
| Role/Name | Contact | Expectations |
| ------------------- | ------- | -------------------------------------------------------------------------------- |
| Product Owner | - | Cost-effective independent 24/7 channel with synchronized viewing and rich chat |
| Engineering Team | - | Simple, evolvable stack (FFmpeg, HLS, Cloudflare R2, lightweight WebSocket chat) |
| US Viewers | - | Reliable low-latency playback, responsive synchronized chat experience |
| Moderators / Admins | - | Easy playlist management, moderation dashboard, monitoring/alerting |
| Operations | - | Low TCO, clear cost model, automated failover, backups |
(See `adr/ADR-001.md` for detailed forces, decision drivers, and rationale.)
## Architecture Constraints
- Independence from video hosting platforms (no YouTube, Twitch, paid Cloudflare Stream).
- US-centric (origin + audience).
- Cost must remain low even at scale (bandwidth dominant).
- Standard browser playback only (no plugins).
- Cloudflare ToS compliance for video delivery (use R2 as origin for large immutable files).
- HLS as primary protocol (standard, not LL-HLS initially).
- Self-hosted playlist normalization and continuous playout required.
(See `adr/ADR-001.md` for full forces and trade-offs.)
## Context and Scope
### Business Context
Linear Void provides a continuously running scheduled video program (playlist-driven) to a primarily US audience. The custom viewer site replaces traditional platforms, giving full control over content, schedule, UI, and real-time chat (public rooms + private DMs).
### High-Level Context Diagram (Mermaid)
```mermaid
flowchart TD
subgraph "External Users"
Viewers[US Viewers<br/>Browser SPA + hls.js + Chat UI]
Admins[Admins / Moderators<br/>Playlist & Schedule UI]
end
subgraph "Cloudflare Global Edge"
CDN[Cloudflare CDN<br/>Cache Rules + WAF + TLS]
Pages[Cloudflare Pages<br/>Static SPA Hosting]
R2[Cloudflare R2<br/>HLS Segments + Source MP4s<br/>enam location hint]
end
subgraph "US-East Origin (Hetzner Ashburn VPS)"
VPS[VPS CX22<br/>Nginx Proxy + Monitoring]
Playout[Playout Service<br/>FFmpeg + MediaMTX/SRS<br/>Continuous HLS from Playlist]
Chat[Chat Backend<br/>WebSockets + Redis + DB<br/>Public rooms + Private DMs]
end
Admins -->|HTTPS - Schedule/Playlist| Playout
Playout -->|S3-compatible Upload| R2
R2 -->|Cached HLS| CDN
Viewers -->|HTTPS| Pages
Viewers -->|HLS Requests| CDN
Viewers -->|WSS Chat| Chat
Chat --> VPS
VPS -->|Proxy / Static| Pages
CDN -->|Origin Pull on Miss| R2
classDef cloud fill:#f0f0ff,stroke:#4263eb
classDef vps fill:#f0fff0,stroke:#2e7d32
classDef user fill:#fff0f0,stroke:#c62828
class Cloudflare,CDN,Pages,R2 cloud
class VPS,Playout,Chat vps
class Viewers,Admins user
```
**External Interfaces**:
- **Cloudflare**: CDN (cache rules for `.ts` vs `.m3u8`), R2 (object storage, $0 egress), Pages (static hosting), WAF/TLS.
- **Hetzner VPS**: Compute for playout and chat (low-cost US-East with good peering).
- **Browsers**: Native HLS (`<video>` + hls.js) and WebSocket APIs.
- **Source**: Pre-normalized MP4 files uploaded to R2.
This diagram synthesizes the context views from all original ADRs. Full decision details, cache rules, storage calculations, and rationale are in [`adr/ADR-001.md`](adr/ADR-001.md).
### Technical Context
- **Playout**: FFmpeg consuming normalized playlist → continuous HLS.
- **Packaging/Delivery**: MediaMTX/SRS/Nginx → immutable `.ts` segments to R2; Cloudflare edge cache.
- **Frontend**: Static SPA with hls.js.
- **Chat**: Lightweight WebSocket service (Node/Go) + Redis (presence/rate limits) + PostgreSQL/SQLite (history/moderation).
- **Normalization**: All source videos must be H.264/AAC, 30 fps constant, 2s keyframes.
**Mapping**:
- Input: Normalized MP4s in R2 + playlist.
- Output: Live `master.m3u8` + segments served via Cloudflare; chat via WSS.
## Solution Strategy
**Core Strategy** (synthesized & accepted from all ADRs):
1. **US-East Playout on Hetzner Ashburn VPS** (`CX22` or equivalent): FFmpeg + MediaMTX/SRS for continuous playlist-to-HLS conversion (`-c copy` where possible).
2. **Immutable HLS to Cloudflare R2**: Segments use unique names; R2 acts as origin with $0 egress.
3. **Aggressive Edge Caching**: Long TTL for `.ts` (immutable), short TTL (1–10s) for live `.m3u8`. Tiered Cache enabled. This makes viewer delivery essentially free.
4. **Lightweight Custom Frontend**: Static HTML/JS SPA + hls.js player + embedded chat sidebar.
5. **Separate but co-located Chat Service**: WebSockets with persistence and moderation (low bandwidth).
6. **Normalization & Monitoring**: Pre-process all videos; use dashboards + alerts for playout health, cache hit ratio, R2 ops.
This eliminates egress economics, ensures synchronization via live playlist, keeps ops simple, and scales well. See `adr/ADR-001.md` for detailed cache rules, storage tables, cost model, configs (FFmpeg/Nginx examples), and phase-by-phase implementation plan.
**Key Trade-off Accepted**: Standard HLS latency (15–35s) is acceptable for linear programming/chat. LL-HLS/WebRTC deferred.
## Building Block View
### Whitebox Overall System
Overview (Mermaid diagram):
```mermaid
graph TD
subgraph Cloudflare
CDN[CDN + Cache Rules<br/>.ts immutable long TTL<br/>.m3u8 short TTL]
R2[R2 - HLS + Source]
Pages[Pages - Static SPA]
end
VPS[VPS Hetzner Ashburn]
Playout[FFmpeg + MediaMTX/SRS<br/>Playlist → HLS]
Chat[WebSocket Chat<br/>Node/Go + Redis + DB]
Viewer[US Viewers]
Admin[Admin UI]
Admin --> Playout
Playout --> R2
R2 --> CDN
Viewer --> Pages
Viewer --> CDN
Viewer --> Chat
Chat --> VPS
VPS --> Nginx[Nginx Proxy]
```
**Motivation**: Decouple playout (continuous linear), delivery (cached immutable segments), and interaction (chat). Cloudflare R2 + edge caching solves the bandwidth cost problem while maintaining synchronization and independence.
**Contained Building Blocks** (high-level):
- **Playout**: FFmpeg + MediaMTX/SRS (continuous HLS from playlist).
- **Storage/Delivery**: Cloudflare R2 + CDN (immutable segments).
- **Frontend**: Static SPA (hls.js player + chat UI).
- **Chat Service**: WebSocket backend with persistence/moderation.
- **Proxy/Static**: Nginx on VPS.
**Important Interfaces**: HTTP(S) for HLS, WSS for chat, S3-compatible for R2 uploads.
Detailed black-box descriptions, level-2/3 views, and code locations are in `adr/ADR-001.md` and future implementation ADRs.
### Level 2 & 3
Deferred to implementation (see ADR for component responsibilities, interfaces, quality attributes, and risks). Playout enforces normalization; Delivery layer focuses on cache rules and immutable naming.
## Runtime View
### Playout & Delivery Scenario
1. Admin adds normalized MP4 to R2 + updates playlist.
2. Playout worker (FFmpeg loop with concat demuxer) reads next file, packages as HLS segments (4-6s).
3. Segments uploaded to R2 with unique immutable names; live `.m3u8` updated.
4. Viewer loads SPA → requests master/live manifest → hls.js player follows current edge.
5. Cloudflare serves `.ts` from nearest edge cache (origin/R2 hit only once per POP/region).
6. Parallel WebSocket connection to chat service for synchronized interaction.
Notable aspects: Manifest freshness ensures all viewers stay on same timeline; cache hit ratio directly impacts cost and latency. Fallback slate activates on playout failure. Full cache policy, example configs (FFmpeg bash loop, Nginx RTMP HLS), and storage tables in `adr/ADR-001.md`.
(Additional scenarios: moderation events, scale spikes, failover covered in the ADR.)
## Deployment View
### Infrastructure Level 1
**Overview Diagram**: See Mermaid in Building Block section (VPS + Cloudflare R2/Pages/CDN).
**Motivation**: Low-cost US-East origin with global edge caching. Hetzner provides excellent price/performance and included traffic; Cloudflare eliminates egress fees.
**Quality/Performance Features**: High cache-hit ratio for video, low US latency, automatic scaling at edge, built-in WAF/DDoS protection.
**Mapping of Building Blocks to Infrastructure**:
- Playout + Chat + Nginx → Hetzner Cloud CX22 (Ashburn, VA).
- HLS storage, caching, static SPA → Cloudflare R2 + CDN + Pages (custom domains, `enam` location hint).
- All public traffic routed through Cloudflare.
### Infrastructure Level 2
**Hetzner VPS**: Runs playout (FFmpeg/MediaMTX), chat backend, Nginx. Monitoring via dashboards + UptimeRobot. Backups to R2.
**Cloudflare Layer**:
- R2 bucket for media.
- Cache Rules (`.ts`: Cache Everything, immutable, long TTL; `.m3u8`: short TTL/must-revalidate).
- WAF, rate limiting, Tiered Cache.
- Pages for SPA hosting.
Future evolution: Docker Compose, managed DB, secondary regions (detailed in subsequent ADRs).
## Cross-cutting Concepts
### Caching & Cost Optimization
Immutable segment naming + targeted Cloudflare Cache Rules + R2 ($0 egress) = near-zero viewer bandwidth cost. Long TTL on `.ts`, short on live manifests. Monitor hit ratio and R2 Class B ops.
### Video Normalization & Playout
All source assets must be pre-normalized (H.264/AVC, AAC-LC, constant 30 fps, 2s keyframe interval, CRF). Prevents glitches during continuous concat. FFmpeg `-c copy` preferred for efficiency.
### Monitoring & Resilience
Alert on no new segment (>30–60s), low cache hits, high R2 ops, WebSocket errors. Maintain emergency slate, auto-restart, weekly backups, incident runbook.
### Security & Moderation
HTTPS/WSS only, strict CORS, Cloudflare WAF, authenticated admin paths, rate limiting, immutable moderation log. Private DMs not E2EE by default.
### Chat
WebSockets + Redis (presence, pub/sub, rate limits) + DB (history, audit). Public rooms + on-demand private chats. Scales independently of video.
Full details, cache rule table, bitrate/storage calculations, cost model, example configs, ToS notes, and implementation phases are in [`adr/ADR-001.md`](adr/ADR-001.md).
## Architecture Decisions
The primary decisions are captured in the following consolidated ADRs (all located in `adr/`):
- **[ADR-001](./adr/ADR-001.md)**: Overall architecture, HLS + Cloudflare R2 caching strategy, storage calculations, cost model, risks.
- **[ADR-002](./adr/ADR-002-module-structure.md)**: Chosen project folder/module structure (frontend, backend/playout, backend/chat, infrastructure, scripts, monitoring) and rationale.
- **[ADR-003](./adr/ADR-003-tech-stack.md)**: Technology stack, programming language choices per module (TypeScript frontend, Go for chat/playout wrapper, Bash/Python scripts), Docker-first approach, and explicit documentation of all implicit assumptions (normalization requirement, Cloudflare-centric delivery, monitoring focus, etc.).
These ADRs together form the canonical decision log. Future ADRs (e.g., Docker Compose layout, API contracts, monitoring strategy) will be added to the same folder and referenced here.
**Summary of Key Decisions** (from ADR-001/002/003):
- Self-managed US-East playout (Hetzner Ashburn) with FFmpeg + MediaMTX/SRS → HLS → Cloudflare R2 + edge caching.
- Module structure that directly reflects the arc42 building blocks and context diagram.
- Go preferred for performance-critical services; TypeScript for frontend; Docker for all deployment.
See the individual ADR files for full context, alternatives considered, consequences, and references.
## Quality Requirements
See **Quality Goals** table in the Introduction section. Primary scenarios focus on cost (near-zero egress), availability (auto-restart + fallback), scalability (edge caching), and usability (synchronized player + responsive chat).
## Risks and Technical Debts
- Cloudflare ToS risk for heavy video on free tier (use R2 path; fallback to BunnyCDN if challenged).
- HLS latency (15–35s) may affect perceived chat synchronization.
- Single VPS as SPOF (mitigate with DNS failover in Phase 3).
- Upfront effort for video normalization and cache tuning.
- Moderation/abuse prevention operational load at scale.
- R2 request costs on cache misses (optimize with Tiered Cache; measure in prod).
- Deferred items: LL-HLS, ABR, E2EE, paid access controls, full Docker Compose.
**Technical Debt**: Detailed service configs, CI/CD, and advanced resilience are left for implementation ADRs.
## Glossary
| Term | Definition |
| ----------------- | ------------------------------------------------------------------------------------------ |
| HLS | HTTP Live Streaming – protocol using `.m3u8` manifests and media segments (`.ts` or fMP4). |
| Immutable Segment | HLS chunk with unique, unchanging name enabling long-lived caching. |
| R2 | Cloudflare object storage ($0 egress, ideal origin for cached HLS). |
| Edge Cache / POP | Cloudflare points-of-presence that cache and serve content close to viewers. |
| Playout | Continuous generation of a scheduled linear program from a playlist. |
| Normalization | Pre-processing videos to consistent codec/fps/keyframe settings for seamless playback. |
| Cache Rule | Cloudflare configuration defining caching behavior, TTL, and headers per URL pattern. |
| SPOF | Single Point of Failure (current single VPS origin). |
| Tiered Cache | Cloudflare feature reducing origin requests by checking upper-tier caches first. |
**References**: Cloudflare R2/Cache docs, MediaMTX/SRS/FFmpeg HLS guides, Hetzner pricing, hls.js documentation, standard video bitrate calculators. All original root `ADR-001-*.md` files have been consolidated and can be archived/deleted.
### Whitebox Overall System
**_\<Overview Diagram\>_**
Motivation
: _\<text explanation\>_
Contained Building Blocks
: _\<Description of contained building block (black boxes)\>_
Important Interfaces
: _\<Description of important interfaces\>_
#### \<Name black box 1\>
_\<Purpose/Responsibility\>_
_\<Interface(s)\>_
_\<(Optional) Quality/Performance Characteristics\>_
_\<(Optional) Directory/File Location\>_
_\<(Optional) Fulfilled Requirements\>_
_\<(optional) Open Issues/Problems/Risks\>_
#### \<Name black box 2\>
_\<black box template\>_
#### \<Name black box n\>
_\<black box template\>_
#### \<Name interface 1\>
...​
#### \<Name interface m\>
### Level 2
#### White Box _\<building block 1\>_
_\<white box template\>_
#### White Box _\<building block 2\>_
_\<white box template\>_
...​
#### White Box _\<building block m\>_
_\<white box template\>_
### Level 3
#### White Box \<\_building block x.1\_\>
_\<white box template\>_
#### White Box \<\_building block x.2\_\>
_\<white box template\>_
#### White Box \<\_building block y.1\_\>
_\<white box template\>_
## Runtime View
### \<Runtime Scenario 1\>
- _\<insert runtime diagram or textual description of the scenario\>_
- _\<insert description of the notable aspects of the interactions
between the building block instances depicted in this diagram.\>_
### \<Runtime Scenario 2\>
### ...​
### \<Runtime Scenario n\>
## Deployment View
### Infrastructure Level 1
**_\<Overview Diagram\>_**
Motivation
: _\<explanation in text form\>_
Quality and/or Performance Features
: _\<explanation in text form\>_
Mapping of Building Blocks to Infrastructure
: _\<description of the mapping\>_
### Infrastructure Level 2
#### _\<Infrastructure Element 1\>_
_\<diagram + explanation\>_
#### _\<Infrastructure Element 2\>_
_\<diagram + explanation\>_
...​
#### _\<Infrastructure Element n\>_
_\<diagram + explanation\>_
## Cross-cutting Concepts
### _\<Concept 1\>_
_\<explanation\>_
### _\<Concept 2\>_
_\<explanation\>_
...​
### _\<Concept n\>_
_\<explanation\>_
## Architecture Decisions
## Quality Requirements
### Quality Requirements Overview
### Quality Scenarios
## Risks and Technical Debts
## Glossary
+----------------------+-----------------------------------------------+
| Term | Definition |
+======================+===============================================+
| _\<Term-1\>_ | _\<definition-1\>_ |
+----------------------+-----------------------------------------------+
| _\<Term-2\>_ | _\<definition-2\>_ |
+----------------------+-----------------------------------------------+
docs/architecture/adr/ADR-001.md
+170
View File
@@ -0,0 +1,170 @@
# ADR-001: Cost-Efficient Self-Hosted 24/7 Synchronized Linear Video Streaming with Custom Chat (US-Centric, Cloudflare + R2 Optimized)
**Status:** Accepted
**Date:** 2026-08-02
**Authors:** Synthesized from Grok, GPT-Terra, DeepSeek, Gemini (research-assisted)
**Decision Owners:** Product & Engineering
**Scope:** One 24/7 linear video channel, custom viewer website, public/private chat, playlist-driven playout. Related ADRs: authentication, moderation, payments, multi-channel.
## Context
We require a completely independent 24/7 linear video channel that broadcasts a continuous playlist of self-owned pre-recorded videos. Viewers consume it via a lightweight custom HTML/CSS/JavaScript single-page application (no reliance on YouTube, Twitch, Vimeo, or paid Cloudflare Stream tiers). The experience must include:
- True (or near-true) synchronized playback — all viewers see the same program timeline.
- Embedded custom chat with public rooms, private 1:1/direct messages, presence, moderation (mute/ban/report), rate limiting.
- “Now playing” schedule/metadata.
- Full control over source media, frontend, schedule, and chat data.
- Primary audience and operational origin in the **United States** (East/Central preferred for uploads and low latency).
**Key distinctions**:
- **Playout**: Continuous scheduled program from playlist.
- **Packaging**: FFmpeg → standard HLS (`.m3u8` + `.ts` segments, 4–6s duration).
- **Delivery**: Immutable segments served via Cloudflare edge caching + R2 (zero egress cost).
- **Interaction**: Separate lightweight WebSocket service for chat.
Bandwidth (not storage or compute) is the dominant cost driver. A single viewer at ~4 Mbps consumes ~1.3–1.8 TB/month; 100 concurrent viewers = ~130–180 TB/month. Direct VPS delivery is only viable for PoC/small audiences.
## Decision Drivers
1. Minimize monthly cost (target <$25 for moderate scale).
2. Full independence and ownership (no video platform lock-in).
3. Strong US performance and low latency.
4. True/near-true synchronization via live-style HLS (not pure VOD).
5. Aggressive immutable caching of `.ts` segments to achieve near-zero egress.
6. Browser-native playback (`hls.js` + native HLS).
7. Lightweight, maintainable stack with modest operational complexity.
8. Support for custom multi-room + private chat with persistence and moderation.
9. Scalability from tens to thousands of concurrent viewers without linear cost growth.
10. Compliance with Cloudflare ToS (use R2 as origin for large video files).
## Decision
Adopt a **self-managed US-East playout + Cloudflare R2 + edge-cached HLS** architecture:
### 1. Video Pipeline & Playout (US-East VPS)
- **Location**: Hetzner Cloud CX22 (or equivalent) in **Ashburn, VA** (~$4–8/mo, 2 vCPU/4 GB, 40 GB NVMe, 20 TB included traffic). Excellent peering.
- **Tools**:
- FFmpeg (concat demuxer + `while true` loop for auto-restart) for playlist playout and normalization (`-c copy` where possible, CRF encoding).
- MediaMTX (preferred), SRS, or Nginx-RTMP as lightweight media server/packager for continuous HLS output.
- Sliding window HLS: 4–6s segments, playlist length ~24–60s.
- **Source Normalization** (mandatory before scheduling):
- Container: MP4
- Video: H.264/AVC, yuv420p, constant 30 fps, keyframe interval 2s
- Audio: AAC-LC, 48 kHz, stereo, 128–192 kbps
- Bitrate targets: 720p30 (2.5–5 Mbps), 1080p30 (5–8 Mbps)
- Dashboard for playlist management, current file, health.
### 2. Storage & Delivery (Cloudflare R2 + CDN)
- HLS segments (`.ts`) and manifests uploaded to **Cloudflare R2** (Eastern North America hint `enam`).
- Custom domain (`media.example.com`) fronted by Cloudflare with **Cache Rules**:
- `*.ts` (and init `.mp4` if using fMP4 later): `Cache Everything`, immutable, Edge TTL 24h–1yr (`Cache-Control: public, max-age=31536000, immutable`).
- `*.m3u8`: Short TTL (1–10s) or `must-revalidate` (`Cache-Control: public, max-age=1..10, must-revalidate`).
- Master playlist: TTL ~30s.
- Tiered Cache enabled where possible.
- **R2 as origin** satisfies ToS for large video; $0 egress, free 10 GB, then ~$0.015/GB-mo (Standard).
- Viewer player hits Cloudflare edge → vast majority of traffic served from cache (origin/R2 hit only once per region).
### 3. Frontend
- Static SPA (HTML/CSS/JS + Tailwind + Alpine.js or vanilla) hosted on Cloudflare Pages or same VPS/Nginx.
- Video player: `hls.js` (Chromium/Firefox) or native HLS (Safari). Single live `master.m3u8`.
- Embedded chat sidebar with room switcher, private DMs, now-playing overlay.
### 4. Custom Chat
- Lightweight backend (Go, Node.js + Socket.io, or Elixir) on same VPS.
- WebSockets (`wss://`), Redis (presence, pub/sub, rate limits), PostgreSQL/SQLite (message history, moderation log, users).
- Features: public rooms, private 1:1, presence, rate limiting, mute/timeout/ban/report, basic search.
- Persistence with retention policy. Moderation audit log. No E2EE by default (separate ADR if required).
### 5. Infrastructure Summary & Cost Model
- **Primary**: Hetzner Ashburn VPS (playout + chat + Nginx) + Cloudflare R2 + Pages.
- **Monitoring**: MediaMTX/SRS dashboard, UptimeRobot, systemd logs, Cloudflare analytics.
- **Backup**: Weekly R2/S3-compatible snapshots of playlist, DB, config.
- **Fallback**: Emergency slate/loop playlist, automated restart.
- **Monthly Cost (early production, moderate US audience)**: $5–25 (VPS $4–8 + R2 $1–3 + negligible requests). Scales to hundreds/thousands of viewers with near-zero additional egress.
**Initial Targets**:
- Protocol: Standard HLS (`.ts` segments). Defer LL-HLS/CMAF/WebRTC.
- Max rendition: 720p30 initially (upgrade to 1080p30 or ABR later).
- Latency: 15–35s (acceptable for linear programming/chat).
## Storage for 48 Hours of Source Footage
Formula: `GB ≈ (bitrate_Mbps × duration_seconds) / 8 / 1024` (+5–10% overhead).
| Resolution | Video Bitrate | Total w/ Audio | 48h Raw | Recommended Allocation | R2 Cost/mo (Standard) |
| ---------- | ------------- | -------------- | ---------- | ---------------------- | --------------------- |
| 720p30 | 2.5–4 Mbps | ~2.6–4.2 Mbps | 57–90 GB | ~80–110 GB | ~$1.20–$1.65 |
| 1080p30 | 5–8 Mbps | ~5.1–8.2 Mbps | 110–175 GB | ~130–180 GB | ~$1.95–$2.70 |
Plan 2–3× headroom for variants, backups, artwork, processing. Use R2 Infrequent Access if latency acceptable.
## Rationale
- **Hetzner Ashburn + Cloudflare R2** delivers unmatched price/performance for US traffic. R2’s $0 egress + edge caching eliminates the dominant bandwidth cost.
- HLS is the most compatible, cache-friendly, operationally simple protocol for one-to-many linear broadcast.
- FFmpeg + MediaMTX/SRS provides reliable playlist-driven continuous playout with low CPU.
- Immutable segment naming + targeted Cache Rules ensure high hit ratios while keeping live manifests fresh.
- Lightweight static frontend + separate lightweight chat service keeps complexity low.
- Starts simple (single VPS) and evolves (ABR, secondary origins, managed DBs, LL-HLS) without breaking changes.
This hybrid synthesizes the strongest elements from all evaluated ADRs while resolving minor variances (Hetzner vs generic VPS, direct origin vs R2, SQLite vs PostgreSQL+Redis, exact bitrates).
## Considered Alternatives
- **Pure VPS origin (no R2/CDN)**: Rejected for production (20 TB cap limits ~12–20 concurrent HD viewers).
- **Managed platforms (YouTube/Twitch/Cloudflare Stream)**: Rejected (loss of control, higher cost at scale, ToS restrictions).
- **WebRTC / P2P**: High complexity, poor caching, not suited for passive linear broadcast.
- **Owncast**: Good for turnkey but less flexible for custom playlist/schedule/chat.
- **LL-HLS or CMAF initially**: Deferred (added complexity; standard HLS sufficient for v1).
## Consequences
**Positive**:
- Near-zero egress cost → unlimited viewer scaling.
- Full ownership, US performance, synchronized linear experience.
- Extremely low TCO (~$5–25/mo).
- Cacheable, maintainable, evolvable architecture.
**Negative / Risks & Technical Debt**:
- Cloudflare ToS scrutiny for heavy video (mitigated by R2 path; monitor & have BunnyCDN fallback).
- HLS latency (15–35s) affects real-time chat feel.
- Single-origin SPOF (mitigate with DNS failover/secondary region later).
- Video normalization and cache-rule tuning required upfront.
- Chat moderation and abuse prevention operational burden.
- R2 request costs on cache misses (measure in production; Tiered Cache helps).
- No built-in E2EE for private chats (future ADR).
**Operational Requirements**:
- Monitor playout health, segment generation, cache hit ratio, R2 ops, WebSocket connections.
- Maintain fallback slate, emergency playlist, backup/restore procedures.
- Strict CORS, HTTPS, WAF, rate limiting, authenticated admin paths.
## Implementation Plan
**Phase 1 (Prototype)**: Hetzner VPS, normalized playlist, FFmpeg+MediaMTX → HLS to R2, basic hls.js page + Socket.io chat, cache rules, load test.
**Phase 2 (MVP)**: Scheduler/metadata, PostgreSQL+Redis, full moderation, dashboards/alerts, emergency fallback.
**Phase 3 (Scale)**: ABR, multi-region, LL-HLS evaluation, paid access controls.
## References
- Cloudflare R2 pricing, custom domains, cache rules, Tiered Cache.
- MediaMTX / SRS / FFmpeg HLS documentation.
- Hetzner Cloud Ashburn pricing (2026).
- hls.js, Video.js, standard HLS best practices.
- Video bitrate/storage calculators.
---
_This consolidated ADR merges all prior drafts. Future ADRs will detail Docker Compose, exact cache rules, playlist normalization scripts, chat schema, monitoring, and access control._
docs/architecture/adr/ADR-002-module-structure.md
+94
View File
@@ -0,0 +1,94 @@
# ADR-002: Project Module / Folder Structure
**Status:** Accepted
**Date:** 2026-08-02
**Author:** AI Assistant (based on arc42 analysis)
**Related ADRs:** [ADR-001](./ADR-001.md), [ADR-003](./ADR-003-tech-stack.md)
## Context
The arc42 documentation in `docs/architecture/README.md` defines several clear building blocks:
- Playout Service (FFmpeg + MediaMTX/SRS, playlist handling, normalization)
- Chat Backend (WebSocket server, rooms, DMs, moderation, Redis/DB)
- Frontend SPA (hls.js player, chat UI, static assets)
- Infrastructure / Delivery (Nginx, Cloudflare R2/CDN rules, Docker, IaC)
- Supporting concerns (scripts for normalization/sync/backup, monitoring, shared utilities)
We need a folder layout that makes these modules obvious, supports independent development/deployment (Docker), keeps the root clean, and aligns with the high-level context and deployment diagrams.
## Decision Drivers
1. Clear mapping to arc42 building blocks and runtime/deployment views.
2. Easy separation for frontend vs backend services.
3. Docker / container friendliness (one Dockerfile per major service).
4. Support for scripts, infrastructure-as-code, and monitoring configs.
5. Minimal cognitive load for new contributors (standard `frontend/`, `backend/`, `infrastructure/` layout).
6. Future extensibility (multi-channel, ABR, LL-HLS, paid features).
## Decision
Adopt the following structure (created in the workspace):
```text
linearvoid/
├── frontend/ # Static SPA (HTML/CSS/JS + hls.js + chat UI)
├── backend/
│ ├── playout/ # Continuous HLS playout engine (FFmpeg + MediaMTX/SRS)
│ ├── chat/ # WebSocket chat service (Go/Node + Redis + DB)
│ └── shared/ # Common utilities, types, normalization logic
├── infrastructure/ # Docker Compose, Nginx, Cloudflare rules, IaC
├── scripts/ # Normalization, R2 sync, backup, deployment scripts
├── monitoring/ # Dashboards, Prometheus rules, Grafana
├── docs/architecture/
│ ├── README.md # arc42 (populated)
│ └── adr/
│ ├── ADR-001.md
│ ├── ADR-002-module-structure.md
│ └── ADR-003-tech-stack.md
├── .env.example
├── docker-compose.yml
├── README.md
└── .gitignore
```
**Key conventions**:
- `backend/<service>/` contains its own `src/`, `config/`, and `Dockerfile`.
- `infrastructure/` owns all deployment and cloud configuration.
- `scripts/` holds automation that is not part of a running service.
- All documentation lives under `docs/architecture/`.
## Rationale
- **Alignment with arc42**: Each top-level folder corresponds to a major building block or cross-cutting concern shown in the context and deployment diagrams.
- **Developer Experience**: Clear separation reduces merge conflicts and makes it obvious where to add player logic vs chat logic vs cache rules.
- **Operational Simplicity**: Docker Compose can easily orchestrate `playout`, `chat`, and `nginx` services from the root.
- **Scalability**: Services can be split into separate repositories or Kubernetes deployments later without major refactoring.
- **Alternatives Considered**:
- Flat `src/` structure → rejected (hard to see architectural boundaries).
- Monorepo with `apps/` and `packages/` (Turborepo style) → overkill for current scale.
- All-in-one `services/` folder → less explicit than `backend/playout` vs `backend/chat`.
This structure was created via the `create_directory` tool and is reflected in the updated root and architecture READMEs.
## Consequences
**Positive**:
- Excellent traceability from architecture docs → code.
- Easy to add new modules (e.g., `backend/scheduler` later).
- Supports the normalization-first and caching-first principles from ADR-001.
**Negative / Risks**:
- Slightly more directories than a minimal prototype (acceptable trade-off).
- Need to keep documentation in sync when structure evolves (mitigated by ADRs).
**Future**: Next ADRs will document Docker Compose layout (ADR-004) and API contracts between services.
## References
- `docs/architecture/README.md` (Building Block View, Deployment View, Context Diagram)
- ADR-001 (overall architecture and technology constraints)
- ADR-003 (language and runtime choices per module)
docs/architecture/adr/ADR-003-tech-stack.md
+87
View File
@@ -0,0 +1,87 @@
# ADR-003: Technology Stack, Languages, and Implicit Assumptions
**Status:** Accepted
**Date:** 2026-08-02
**Author:** AI Assistant (based on arc42 and ADR-001)
**Related ADRs:** [ADR-001](./ADR-001.md), [ADR-002](./ADR-002-module-structure.md)
## Context
From ADR-001 and the arc42 documentation we have the following constraints and preferences:
- Lightweight, cost-efficient, maintainable stack.
- Strong preference for US-East hosting (Hetzner), Cloudflare R2/CDN, HLS.
- Playout must be reliable and low-CPU (FFmpeg + MediaMTX/SRS).
- Chat needs real-time WebSockets, persistence, moderation, and presence.
- Frontend must be a minimal static SPA with good browser compatibility.
- Operational simplicity and easy monitoring are critical.
- Implicit assumptions include Docker for deployment, normalization of all video assets, heavy reliance on Cloudflare caching, and a focus on immutable infrastructure.
We must explicitly document language/runtime choices per module and the assumptions that were made.
## Decision
**Chosen Stack:**
| Layer | Technology | Language | Rationale |
| ------------------- | ------------------------------------------ | ----------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| **Frontend SPA** | HTML + Tailwind + Alpine.js + hls.js | TypeScript / JavaScript | Minimal bundle, excellent HLS support, fast iteration. Vite for build. |
| **Playout Service** | FFmpeg + MediaMTX (or SRS) | Bash + Go (wrapper) | FFmpeg is the proven tool for continuous playlist HLS. Go wrapper for reliability and monitoring. |
| **Chat Backend** | Go (preferred) or Node.js + Socket.io | Go | High-performance WebSockets, low memory, excellent Redis/PostgreSQL clients. Node.js acceptable for rapid prototyping. |
| **Shared Code** | TypeScript definitions + Go structs | TS + Go | Clear contracts between frontend and backend. |
| **Infrastructure** | Docker, Docker Compose, Nginx, Cloudflare | YAML + Bash | Standard, reproducible, Cloudflare-native. |
| **Scripts** | Bash (with Python helpers where needed) | Bash/Python | Simple, no extra runtime for normalization and R2 sync. |
| **Monitoring** | Prometheus + Grafana + UptimeRobot | - | Standard open-source stack that integrates well with Docker. |
| **Storage** | Cloudflare R2 (primary), SQLite/PostgreSQL | - | $0 egress, simple for chat history. |
**Implicit Assumptions** (now made explicit):
1. All video assets **must** be normalized before being added to any playlist (H.264, AAC, 30 fps constant, 2s keyframes, CRF encoding).
2. Deployment is **Docker-first**; services will be containerized.
3. Cloudflare is the primary CDN and will be configured with specific Cache Rules (`.ts` immutable long TTL, `.m3u8` short TTL).
4. Chat does **not** require end-to-end encryption in v1 (separate ADR if needed).
5. Single VPS origin initially; multi-region is a later evolution.
6. Monitoring focuses on playout health, cache hit ratio, and R2 operation counts.
7. Development environment uses `docker compose` for local testing of playout + chat + frontend.
## Decision Drivers
- Match the low-resource, high-reliability requirements from ADR-001.
- Minimize new language overhead (Go for performance-critical services, JS/TS for frontend and rapid scripting).
- Leverage mature tools (FFmpeg, hls.js, Redis) rather than reinventing.
- Keep operational complexity low (Docker + Bash scripts).
## Alternatives Considered
- **Full Node.js stack**: Simpler for small team but higher memory usage for WebSockets and less efficient HLS packaging.
- **Python for everything**: Excellent for scripting but poor real-time WebSocket performance.
- **Rust for chat/playout**: Overkill for initial velocity.
- **Separate repositories per service**: Considered but rejected for early stage (monorepo with clear folders is better).
Go was chosen for the chat backend because it offers excellent concurrency, small binaries, and native performance for WebSockets and Redis.
## Consequences
**Positive**:
- Clear, justified choices that align with cost, performance, and maintainability goals.
- Easy onboarding — developers know exactly which language belongs where.
- Explicit documentation of assumptions prevents future architectural drift.
**Risks**:
- Go learning curve if team is primarily JS (mitigated by allowing Node.js fallback for chat).
- Assumption that normalization scripts will be followed (enforce via CI/validation in playout service).
**Future ADRs**:
- ADR-004: Docker Compose and deployment pipeline.
- ADR-005: Monitoring and alerting strategy.
- ADR-006: API contracts between playout, chat, and frontend.
## References
- ADR-001 (overall architecture, HLS, Cloudflare caching, normalization requirements)
- ADR-002 (folder structure that this stack maps into)
- arc42 documentation (`docs/architecture/README.md`) – Building Block View, Deployment View, Cross-cutting Concepts
- MediaMTX, hls.js, Go WebSocket best practices, Cloudflare R2 caching documentation
docs/architecture/arc42-template-EN.md
+215
View File
@@ -0,0 +1,215 @@
---
date: July 2025
title: "![arc42](images/arc42-logo.png) Template"
---
#
**About arc42**
arc42, the template for documentation of software and system
architecture.
Template Version 9.0-EN. (based upon AsciiDoc version), July 2025
Created, maintained and © by Dr. Peter Hruschka, Dr. Gernot Starke and
contributors. See <https://arc42.org>.
# Introduction and Goals {#section-introduction-and-goals}
## Requirements Overview {#_requirements_overview}
## Quality Goals {#_quality_goals}
## Stakeholders {#_stakeholders}
+-------------+---------------------------+---------------------------+
| Role/Name | Contact | Expectations |
+=============+===========================+===========================+
| * | *\<Contact-1\>* | *\<Expectation-1\>* |
| \<Role-1\>* | | |
+-------------+---------------------------+---------------------------+
| * | *\<Contact-2\>* | *\<Expectation-2\>* |
| \<Role-2\>* | | |
+-------------+---------------------------+---------------------------+
# Architecture Constraints {#section-architecture-constraints}
# Context and Scope {#section-context-and-scope}
## Business Context {#_business_context}
**\<Diagram or Table\>**
**\<optionally: Explanation of external domain interfaces\>**
## Technical Context {#_technical_context}
**\<Diagram or Table\>**
**\<optionally: Explanation of technical interfaces\>**
**\<Mapping Input/Output to Channels\>**
# Solution Strategy {#section-solution-strategy}
# Building Block View {#section-building-block-view}
## Whitebox Overall System {#_whitebox_overall_system}
***\<Overview Diagram\>***
Motivation
: *\<text explanation\>*
Contained Building Blocks
: *\<Description of contained building block (black boxes)\>*
Important Interfaces
: *\<Description of important interfaces\>*
### \<Name black box 1\> {#_name_black_box_1}
*\<Purpose/Responsibility\>*
*\<Interface(s)\>*
*\<(Optional) Quality/Performance Characteristics\>*
*\<(Optional) Directory/File Location\>*
*\<(Optional) Fulfilled Requirements\>*
*\<(optional) Open Issues/Problems/Risks\>*
### \<Name black box 2\> {#_name_black_box_2}
*\<black box template\>*
### \<Name black box n\> {#_name_black_box_n}
*\<black box template\>*
### \<Name interface 1\> {#_name_interface_1}
...​
### \<Name interface m\> {#_name_interface_m}
## Level 2 {#_level_2}
### White Box *\<building block 1\>* {#_white_box_building_block_1}
*\<white box template\>*
### White Box *\<building block 2\>* {#_white_box_building_block_2}
*\<white box template\>*
...​
### White Box *\<building block m\>* {#_white_box_building_block_m}
*\<white box template\>*
## Level 3 {#_level_3}
### White Box \<\_building block x.1\_\> {#_white_box_building_block_x_1}
*\<white box template\>*
### White Box \<\_building block x.2\_\> {#_white_box_building_block_x_2}
*\<white box template\>*
### White Box \<\_building block y.1\_\> {#_white_box_building_block_y_1}
*\<white box template\>*
# Runtime View {#section-runtime-view}
## \<Runtime Scenario 1\> {#_runtime_scenario_1}
- *\<insert runtime diagram or textual description of the scenario\>*
- *\<insert description of the notable aspects of the interactions
between the building block instances depicted in this diagram.\>*
## \<Runtime Scenario 2\> {#_runtime_scenario_2}
## ...​
## \<Runtime Scenario n\> {#_runtime_scenario_n}
# Deployment View {#section-deployment-view}
## Infrastructure Level 1 {#_infrastructure_level_1}
***\<Overview Diagram\>***
Motivation
: *\<explanation in text form\>*
Quality and/or Performance Features
: *\<explanation in text form\>*
Mapping of Building Blocks to Infrastructure
: *\<description of the mapping\>*
## Infrastructure Level 2 {#_infrastructure_level_2}
### *\<Infrastructure Element 1\>* {#_infrastructure_element_1}
*\<diagram + explanation\>*
### *\<Infrastructure Element 2\>* {#_infrastructure_element_2}
*\<diagram + explanation\>*
...​
### *\<Infrastructure Element n\>* {#_infrastructure_element_n}
*\<diagram + explanation\>*
# Cross-cutting Concepts {#section-concepts}
## *\<Concept 1\>* {#_concept_1}
*\<explanation\>*
## *\<Concept 2\>* {#_concept_2}
*\<explanation\>*
...​
## *\<Concept n\>* {#_concept_n}
*\<explanation\>*
# Architecture Decisions {#section-design-decisions}
# Quality Requirements {#section-quality-scenarios}
## Quality Requirements Overview {#_quality_requirements_overview}
## Quality Scenarios {#_quality_scenarios}
# Risks and Technical Debts {#section-technical-risks}
# Glossary {#section-glossary}
+----------------------+-----------------------------------------------+
| Term | Definition |
+======================+===============================================+
| *\<Term-1\>* | *\<definition-1\>* |
+----------------------+-----------------------------------------------+
| *\<Term-2\>* | *\<definition-2\>* |
+----------------------+-----------------------------------------------+