zwitschi/linearvoid
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 9 Wiki Activity

Initial commit: Project scaffolding + architecture documentation

Browse Source 
- arc42 template fully populated in docs/architecture/README.md with Mermaid diagrams, quality goals, deployment view, cross-cutting concepts, risks and glossary
- Consolidated ADR-001 (merged all previous root ADR drafts)
- ADR-002: Module/folder structure decision and rationale
- ADR-003: Technology stack, language choices per module (TypeScript, Go, Bash/Python), Docker-first approach, and explicit implicit assumptions
- Updated root README.md with comprehensive Project Structure section and links to all ADRs
- Created .github/instructions/TODO.md with detailed tracking (architecture phase marked complete)
- Folder structure created matching ADR-002 (frontend/, backend/playout/, backend/chat/, backend/shared/, infrastructure/, scripts/, monitoring/)
- Git repository initialized with main branch

This establishes the foundation defined in the arc42 documentation and the three ADRs.
This commit is contained in:
zwitschi committed 2026-08-02 15:31:33 +02:00
commit dafcec1303
8 files changed
+1158
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/architecture/README.md
+478
View File
@@ -0,0 +1,478 @@
# Architecture Documentation for Linear Void
> The architecture documentation for the Linear Void project is structured according to the [arc42 template](https://arc42.org/). This template provides a comprehensive framework for documenting software architecture, ensuring that all relevant aspects of the system are covered.
## Table of Contents
- [Introduction and Goals](#introduction-and-goals)
- [Architecture Constraints](#architecture-constraints)
- [Context and Scope](#context-and-scope)
- [Solution Strategy](#solution-strategy)
- [Building Block View](#building-block-view)
- [Runtime View](#runtime-view)
- [Deployment View](#deployment-view)
- [Cross-cutting Concepts](#cross-cutting-concepts)
- [Architecture Decisions](#architecture-decisions)
- [Quality Requirements](#quality-requirements)
- [Risks and Technical Debts](#risks-and-technical-debts)
- [Glossary](#glossary)
## Introduction and Goals
### Requirements Overview
Linear Void is an independent **24/7 linear video channel** broadcasting a playlist of self-owned pre-recorded videos. Viewers access it via a custom lightweight HTML/CSS/JavaScript SPA with synchronized HLS playback, schedule metadata, public chat rooms, and private 1:1 messaging. Complete independence from third-party video platforms (YouTube, Twitch, etc.).
**Core Requirements** (merged from all ADRs):
- Continuous synchronized playback from playlist (live-style HLS, not VOD).
- Self-hosted source media, frontend, schedule, and chat data.
- Primary US audience/origin (East Coast preferred for low latency/peering).
- Extreme cost efficiency (bandwidth dominant; target near-zero egress via Cloudflare R2 + edge caching).
- Custom chat with rooms, DMs, presence, moderation, persistence.
- Storage planning for 48h of 720p30/1080p30 content.
- Browser-compatible playback (`hls.js` + native HLS).
- Operational simplicity and scalability.
Full decision rationale, alternatives, and consequences in [`adr/ADR-001.md`](adr/ADR-001.md).
### Quality Goals
| Priority | Quality Attribute | Goal | Scenario |
| -------- | ------------------------- | -------------------------------------------------------- | -------------------------------------------------------------------------------- |
| 1 | **Cost Efficiency** | <$25/month for moderate scale (100s concurrent) | R2 $0 egress + immutable `.ts` edge caching eliminates per-viewer bandwidth cost |
| 2 | **Performance / Latency** | Low US latency; acceptable HLS delay | Hetzner Ashburn origin + Cloudflare US POPs (15–35s end-to-end) |
| 3 | **Availability** | 24/7 with minimal downtime | FFmpeg/MediaMTX auto-restart loop, fallback slate, monitoring |
| 4 | **Scalability** | Grow from 10s to 1000s viewers without proportional cost | Edge caching + R2; only request volume limits |
| 5 | **Maintainability** | Lightweight, clear configs, low ops overhead | Static SPA, Go/Node chat, SQLite/PostgreSQL+Redis |
| 6 | **Security / Usability** | Secure transport, moderation tools, good UX | HTTPS/WSS, Cloudflare WAF, rate limits, intuitive chat/player |
### Stakeholders
| Role/Name | Contact | Expectations |
| ------------------- | ------- | -------------------------------------------------------------------------------- |
| Product Owner | - | Cost-effective independent 24/7 channel with synchronized viewing and rich chat |
| Engineering Team | - | Simple, evolvable stack (FFmpeg, HLS, Cloudflare R2, lightweight WebSocket chat) |
| US Viewers | - | Reliable low-latency playback, responsive synchronized chat experience |
| Moderators / Admins | - | Easy playlist management, moderation dashboard, monitoring/alerting |
| Operations | - | Low TCO, clear cost model, automated failover, backups |
(See `adr/ADR-001.md` for detailed forces, decision drivers, and rationale.)
## Architecture Constraints
- Independence from video hosting platforms (no YouTube, Twitch, paid Cloudflare Stream).
- US-centric (origin + audience).
- Cost must remain low even at scale (bandwidth dominant).
- Standard browser playback only (no plugins).
- Cloudflare ToS compliance for video delivery (use R2 as origin for large immutable files).
- HLS as primary protocol (standard, not LL-HLS initially).
- Self-hosted playlist normalization and continuous playout required.
(See `adr/ADR-001.md` for full forces and trade-offs.)
## Context and Scope
### Business Context
Linear Void provides a continuously running scheduled video program (playlist-driven) to a primarily US audience. The custom viewer site replaces traditional platforms, giving full control over content, schedule, UI, and real-time chat (public rooms + private DMs).
### High-Level Context Diagram (Mermaid)
```mermaid
flowchart TD
subgraph "External Users"
Viewers[US Viewers<br/>Browser SPA + hls.js + Chat UI]
Admins[Admins / Moderators<br/>Playlist & Schedule UI]
end
subgraph "Cloudflare Global Edge"
CDN[Cloudflare CDN<br/>Cache Rules + WAF + TLS]
Pages[Cloudflare Pages<br/>Static SPA Hosting]
R2[Cloudflare R2<br/>HLS Segments + Source MP4s<br/>enam location hint]
end
subgraph "US-East Origin (Hetzner Ashburn VPS)"
VPS[VPS CX22<br/>Nginx Proxy + Monitoring]
Playout[Playout Service<br/>FFmpeg + MediaMTX/SRS<br/>Continuous HLS from Playlist]
Chat[Chat Backend<br/>WebSockets + Redis + DB<br/>Public rooms + Private DMs]
end
Admins -->|HTTPS - Schedule/Playlist| Playout
Playout -->|S3-compatible Upload| R2
R2 -->|Cached HLS| CDN
Viewers -->|HTTPS| Pages
Viewers -->|HLS Requests| CDN
Viewers -->|WSS Chat| Chat
Chat --> VPS
VPS -->|Proxy / Static| Pages
CDN -->|Origin Pull on Miss| R2
classDef cloud fill:#f0f0ff,stroke:#4263eb
classDef vps fill:#f0fff0,stroke:#2e7d32
classDef user fill:#fff0f0,stroke:#c62828
class Cloudflare,CDN,Pages,R2 cloud
class VPS,Playout,Chat vps
class Viewers,Admins user
```
**External Interfaces**:
- **Cloudflare**: CDN (cache rules for `.ts` vs `.m3u8`), R2 (object storage, $0 egress), Pages (static hosting), WAF/TLS.
- **Hetzner VPS**: Compute for playout and chat (low-cost US-East with good peering).
- **Browsers**: Native HLS (`<video>` + hls.js) and WebSocket APIs.
- **Source**: Pre-normalized MP4 files uploaded to R2.
This diagram synthesizes the context views from all original ADRs. Full decision details, cache rules, storage calculations, and rationale are in [`adr/ADR-001.md`](adr/ADR-001.md).
### Technical Context
- **Playout**: FFmpeg consuming normalized playlist → continuous HLS.
- **Packaging/Delivery**: MediaMTX/SRS/Nginx → immutable `.ts` segments to R2; Cloudflare edge cache.
- **Frontend**: Static SPA with hls.js.
- **Chat**: Lightweight WebSocket service (Node/Go) + Redis (presence/rate limits) + PostgreSQL/SQLite (history/moderation).
- **Normalization**: All source videos must be H.264/AAC, 30 fps constant, 2s keyframes.
**Mapping**:
- Input: Normalized MP4s in R2 + playlist.
- Output: Live `master.m3u8` + segments served via Cloudflare; chat via WSS.
## Solution Strategy
**Core Strategy** (synthesized & accepted from all ADRs):
1. **US-East Playout on Hetzner Ashburn VPS** (`CX22` or equivalent): FFmpeg + MediaMTX/SRS for continuous playlist-to-HLS conversion (`-c copy` where possible).
2. **Immutable HLS to Cloudflare R2**: Segments use unique names; R2 acts as origin with $0 egress.
3. **Aggressive Edge Caching**: Long TTL for `.ts` (immutable), short TTL (1–10s) for live `.m3u8`. Tiered Cache enabled. This makes viewer delivery essentially free.
4. **Lightweight Custom Frontend**: Static HTML/JS SPA + hls.js player + embedded chat sidebar.
5. **Separate but co-located Chat Service**: WebSockets with persistence and moderation (low bandwidth).
6. **Normalization & Monitoring**: Pre-process all videos; use dashboards + alerts for playout health, cache hit ratio, R2 ops.
This eliminates egress economics, ensures synchronization via live playlist, keeps ops simple, and scales well. See `adr/ADR-001.md` for detailed cache rules, storage tables, cost model, configs (FFmpeg/Nginx examples), and phase-by-phase implementation plan.
**Key Trade-off Accepted**: Standard HLS latency (15–35s) is acceptable for linear programming/chat. LL-HLS/WebRTC deferred.
## Building Block View
### Whitebox Overall System
Overview (Mermaid diagram):
```mermaid
graph TD
subgraph Cloudflare
CDN[CDN + Cache Rules<br/>.ts immutable long TTL<br/>.m3u8 short TTL]
R2[R2 - HLS + Source]
Pages[Pages - Static SPA]
end
VPS[VPS Hetzner Ashburn]
Playout[FFmpeg + MediaMTX/SRS<br/>Playlist → HLS]
Chat[WebSocket Chat<br/>Node/Go + Redis + DB]
Viewer[US Viewers]
Admin[Admin UI]
Admin --> Playout
Playout --> R2
R2 --> CDN
Viewer --> Pages
Viewer --> CDN
Viewer --> Chat
Chat --> VPS
VPS --> Nginx[Nginx Proxy]
```
**Motivation**: Decouple playout (continuous linear), delivery (cached immutable segments), and interaction (chat). Cloudflare R2 + edge caching solves the bandwidth cost problem while maintaining synchronization and independence.
**Contained Building Blocks** (high-level):
- **Playout**: FFmpeg + MediaMTX/SRS (continuous HLS from playlist).
- **Storage/Delivery**: Cloudflare R2 + CDN (immutable segments).
- **Frontend**: Static SPA (hls.js player + chat UI).
- **Chat Service**: WebSocket backend with persistence/moderation.
- **Proxy/Static**: Nginx on VPS.
**Important Interfaces**: HTTP(S) for HLS, WSS for chat, S3-compatible for R2 uploads.
Detailed black-box descriptions, level-2/3 views, and code locations are in `adr/ADR-001.md` and future implementation ADRs.
### Level 2 & 3
Deferred to implementation (see ADR for component responsibilities, interfaces, quality attributes, and risks). Playout enforces normalization; Delivery layer focuses on cache rules and immutable naming.
## Runtime View
### Playout & Delivery Scenario
1. Admin adds normalized MP4 to R2 + updates playlist.
2. Playout worker (FFmpeg loop with concat demuxer) reads next file, packages as HLS segments (4-6s).
3. Segments uploaded to R2 with unique immutable names; live `.m3u8` updated.
4. Viewer loads SPA → requests master/live manifest → hls.js player follows current edge.
5. Cloudflare serves `.ts` from nearest edge cache (origin/R2 hit only once per POP/region).
6. Parallel WebSocket connection to chat service for synchronized interaction.
Notable aspects: Manifest freshness ensures all viewers stay on same timeline; cache hit ratio directly impacts cost and latency. Fallback slate activates on playout failure. Full cache policy, example configs (FFmpeg bash loop, Nginx RTMP HLS), and storage tables in `adr/ADR-001.md`.
(Additional scenarios: moderation events, scale spikes, failover covered in the ADR.)
## Deployment View
### Infrastructure Level 1
**Overview Diagram**: See Mermaid in Building Block section (VPS + Cloudflare R2/Pages/CDN).
**Motivation**: Low-cost US-East origin with global edge caching. Hetzner provides excellent price/performance and included traffic; Cloudflare eliminates egress fees.
**Quality/Performance Features**: High cache-hit ratio for video, low US latency, automatic scaling at edge, built-in WAF/DDoS protection.
**Mapping of Building Blocks to Infrastructure**:
- Playout + Chat + Nginx → Hetzner Cloud CX22 (Ashburn, VA).
- HLS storage, caching, static SPA → Cloudflare R2 + CDN + Pages (custom domains, `enam` location hint).
- All public traffic routed through Cloudflare.
### Infrastructure Level 2
**Hetzner VPS**: Runs playout (FFmpeg/MediaMTX), chat backend, Nginx. Monitoring via dashboards + UptimeRobot. Backups to R2.
**Cloudflare Layer**:
- R2 bucket for media.
- Cache Rules (`.ts`: Cache Everything, immutable, long TTL; `.m3u8`: short TTL/must-revalidate).
- WAF, rate limiting, Tiered Cache.
- Pages for SPA hosting.
Future evolution: Docker Compose, managed DB, secondary regions (detailed in subsequent ADRs).
## Cross-cutting Concepts
### Caching & Cost Optimization
Immutable segment naming + targeted Cloudflare Cache Rules + R2 ($0 egress) = near-zero viewer bandwidth cost. Long TTL on `.ts`, short on live manifests. Monitor hit ratio and R2 Class B ops.
### Video Normalization & Playout
All source assets must be pre-normalized (H.264/AVC, AAC-LC, constant 30 fps, 2s keyframe interval, CRF). Prevents glitches during continuous concat. FFmpeg `-c copy` preferred for efficiency.
### Monitoring & Resilience
Alert on no new segment (>30–60s), low cache hits, high R2 ops, WebSocket errors. Maintain emergency slate, auto-restart, weekly backups, incident runbook.
### Security & Moderation
HTTPS/WSS only, strict CORS, Cloudflare WAF, authenticated admin paths, rate limiting, immutable moderation log. Private DMs not E2EE by default.
### Chat
WebSockets + Redis (presence, pub/sub, rate limits) + DB (history, audit). Public rooms + on-demand private chats. Scales independently of video.
Full details, cache rule table, bitrate/storage calculations, cost model, example configs, ToS notes, and implementation phases are in [`adr/ADR-001.md`](adr/ADR-001.md).
## Architecture Decisions
The primary decisions are captured in the following consolidated ADRs (all located in `adr/`):
- **[ADR-001](./adr/ADR-001.md)**: Overall architecture, HLS + Cloudflare R2 caching strategy, storage calculations, cost model, risks.
- **[ADR-002](./adr/ADR-002-module-structure.md)**: Chosen project folder/module structure (frontend, backend/playout, backend/chat, infrastructure, scripts, monitoring) and rationale.
- **[ADR-003](./adr/ADR-003-tech-stack.md)**: Technology stack, programming language choices per module (TypeScript frontend, Go for chat/playout wrapper, Bash/Python scripts), Docker-first approach, and explicit documentation of all implicit assumptions (normalization requirement, Cloudflare-centric delivery, monitoring focus, etc.).
These ADRs together form the canonical decision log. Future ADRs (e.g., Docker Compose layout, API contracts, monitoring strategy) will be added to the same folder and referenced here.
**Summary of Key Decisions** (from ADR-001/002/003):
- Self-managed US-East playout (Hetzner Ashburn) with FFmpeg + MediaMTX/SRS → HLS → Cloudflare R2 + edge caching.
- Module structure that directly reflects the arc42 building blocks and context diagram.
- Go preferred for performance-critical services; TypeScript for frontend; Docker for all deployment.
See the individual ADR files for full context, alternatives considered, consequences, and references.
## Quality Requirements
See **Quality Goals** table in the Introduction section. Primary scenarios focus on cost (near-zero egress), availability (auto-restart + fallback), scalability (edge caching), and usability (synchronized player + responsive chat).
## Risks and Technical Debts
- Cloudflare ToS risk for heavy video on free tier (use R2 path; fallback to BunnyCDN if challenged).
- HLS latency (15–35s) may affect perceived chat synchronization.
- Single VPS as SPOF (mitigate with DNS failover in Phase 3).
- Upfront effort for video normalization and cache tuning.
- Moderation/abuse prevention operational load at scale.
- R2 request costs on cache misses (optimize with Tiered Cache; measure in prod).
- Deferred items: LL-HLS, ABR, E2EE, paid access controls, full Docker Compose.
**Technical Debt**: Detailed service configs, CI/CD, and advanced resilience are left for implementation ADRs.
## Glossary
| Term | Definition |
| ----------------- | ------------------------------------------------------------------------------------------ |
| HLS | HTTP Live Streaming – protocol using `.m3u8` manifests and media segments (`.ts` or fMP4). |
| Immutable Segment | HLS chunk with unique, unchanging name enabling long-lived caching. |
| R2 | Cloudflare object storage ($0 egress, ideal origin for cached HLS). |
| Edge Cache / POP | Cloudflare points-of-presence that cache and serve content close to viewers. |
| Playout | Continuous generation of a scheduled linear program from a playlist. |
| Normalization | Pre-processing videos to consistent codec/fps/keyframe settings for seamless playback. |
| Cache Rule | Cloudflare configuration defining caching behavior, TTL, and headers per URL pattern. |
| SPOF | Single Point of Failure (current single VPS origin). |
| Tiered Cache | Cloudflare feature reducing origin requests by checking upper-tier caches first. |
**References**: Cloudflare R2/Cache docs, MediaMTX/SRS/FFmpeg HLS guides, Hetzner pricing, hls.js documentation, standard video bitrate calculators. All original root `ADR-001-*.md` files have been consolidated and can be archived/deleted.
### Whitebox Overall System
**_\<Overview Diagram\>_**
Motivation
: _\<text explanation\>_
Contained Building Blocks
: _\<Description of contained building block (black boxes)\>_
Important Interfaces
: _\<Description of important interfaces\>_
#### \<Name black box 1\>
_\<Purpose/Responsibility\>_
_\<Interface(s)\>_
_\<(Optional) Quality/Performance Characteristics\>_
_\<(Optional) Directory/File Location\>_
_\<(Optional) Fulfilled Requirements\>_
_\<(optional) Open Issues/Problems/Risks\>_
#### \<Name black box 2\>
_\<black box template\>_
#### \<Name black box n\>
_\<black box template\>_
#### \<Name interface 1\>
...​
#### \<Name interface m\>
### Level 2
#### White Box _\<building block 1\>_
_\<white box template\>_
#### White Box _\<building block 2\>_
_\<white box template\>_
...​
#### White Box _\<building block m\>_
_\<white box template\>_
### Level 3
#### White Box \<\_building block x.1\_\>
_\<white box template\>_
#### White Box \<\_building block x.2\_\>
_\<white box template\>_
#### White Box \<\_building block y.1\_\>
_\<white box template\>_
## Runtime View
### \<Runtime Scenario 1\>
- _\<insert runtime diagram or textual description of the scenario\>_
- _\<insert description of the notable aspects of the interactions
between the building block instances depicted in this diagram.\>_
### \<Runtime Scenario 2\>
### ...​
### \<Runtime Scenario n\>
## Deployment View
### Infrastructure Level 1
**_\<Overview Diagram\>_**
Motivation
: _\<explanation in text form\>_
Quality and/or Performance Features
: _\<explanation in text form\>_
Mapping of Building Blocks to Infrastructure
: _\<description of the mapping\>_
### Infrastructure Level 2
#### _\<Infrastructure Element 1\>_
_\<diagram + explanation\>_
#### _\<Infrastructure Element 2\>_
_\<diagram + explanation\>_
...​
#### _\<Infrastructure Element n\>_
_\<diagram + explanation\>_
## Cross-cutting Concepts
### _\<Concept 1\>_
_\<explanation\>_
### _\<Concept 2\>_
_\<explanation\>_
...​
### _\<Concept n\>_
_\<explanation\>_
## Architecture Decisions
## Quality Requirements
### Quality Requirements Overview
### Quality Scenarios
## Risks and Technical Debts
## Glossary
+----------------------+-----------------------------------------------+
| Term | Definition |
+======================+===============================================+
| _\<Term-1\>_ | _\<definition-1\>_ |
+----------------------+-----------------------------------------------+
| _\<Term-2\>_ | _\<definition-2\>_ |
+----------------------+-----------------------------------------------+