From dafcec130391156114e80dc2cbaf286be890735a Mon Sep 17 00:00:00 2001 From: zwitschi Date: Sun, 2 Aug 2026 15:31:33 +0200 Subject: [PATCH] Initial commit: Project scaffolding + architecture documentation - arc42 template fully populated in docs/architecture/README.md with Mermaid diagrams, quality goals, deployment view, cross-cutting concepts, risks and glossary - Consolidated ADR-001 (merged all previous root ADR drafts) - ADR-002: Module/folder structure decision and rationale - ADR-003: Technology stack, language choices per module (TypeScript, Go, Bash/Python), Docker-first approach, and explicit implicit assumptions - Updated root README.md with comprehensive Project Structure section and links to all ADRs - Created .github/instructions/TODO.md with detailed tracking (architecture phase marked complete) - Folder structure created matching ADR-002 (frontend/, backend/playout/, backend/chat/, backend/shared/, infrastructure/, scripts/, monitoring/) - Git repository initialized with main branch This establishes the foundation defined in the arc42 documentation and the three ADRs. --- .gitignore | 2 + CHANGELOG.md | 5 + README.md | 107 ++++ docs/architecture/README.md | 478 ++++++++++++++++++ docs/architecture/adr/ADR-001.md | 170 +++++++ .../adr/ADR-002-module-structure.md | 94 ++++ docs/architecture/adr/ADR-003-tech-stack.md | 87 ++++ docs/architecture/arc42-template-EN.md | 215 ++++++++ 8 files changed, 1158 insertions(+) create mode 100644 .gitignore create mode 100644 CHANGELOG.md create mode 100644 README.md create mode 100644 docs/architecture/README.md create mode 100644 docs/architecture/adr/ADR-001.md create mode 100644 docs/architecture/adr/ADR-002-module-structure.md create mode 100644 docs/architecture/adr/ADR-003-tech-stack.md create mode 100644 docs/architecture/arc42-template-EN.md diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..ff794dd --- /dev/null +++ b/.gitignore @@ -0,0 +1,2 @@ +# ignore agentic files +.github/instructions/ \ No newline at end of file diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..fd0c67c --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,5 @@ +# Change Log for linearvoid + +## [Unreleased] + +- project structure and architecture documentation finalized (arc42 template, ADRs for module structure and tech stack). diff --git a/README.md b/README.md new file mode 100644 index 0000000..8693c49 --- /dev/null +++ b/README.md @@ -0,0 +1,107 @@ +# Linear Void + +A self-managed **24/7 linear video channel** for a primarily US audience. It runs a continuous playlist of self-owned videos with true/near-true synchronization, a custom lightweight viewer SPA (hls.js player + chat), and full independence from third-party platforms (YouTube, Twitch, etc.). + +The architecture (detailed in [`docs/architecture/README.md`](docs/architecture/README.md)) uses: + +- Hetzner Ashburn VPS for playout (FFmpeg + MediaMTX/SRS) and chat. +- Cloudflare R2 + CDN with aggressive immutable caching of HLS `.ts` segments ($0 egress). +- Lightweight static frontend and WebSocket chat backend. + +See [`docs/architecture/adr/ADR-001.md`](docs/architecture/adr/ADR-001.md) for the consolidated decision record, storage calculations, cache rules, cost model, and implementation plan. + +## Key Features + +- Continuous synchronized playback from a playlist (standard HLS, 15–35s latency). +- Self-hosted custom SPA with embedded chat (public rooms + private 1:1 DMs, moderation). +- Extreme cost efficiency via Cloudflare R2 edge caching. +- Video normalization, monitoring, fallback slate, and easy playlist management. +- US-centric performance (Ashburn origin + Cloudflare POPs). + +## Project Structure + +```text +linearvoid/ +├── frontend/ # Static Single-Page Application (HTML/CSS/JS) +│ ├── src/ +│ │ ├── components/ # Video player, chat sidebar, now-playing overlay +│ │ ├── js/ # hls.js integration, WebSocket client, UI logic +│ │ └── styles/ # Tailwind / custom CSS +│ ├── index.html +│ └── package.json # Build tools (Vite recommended) +├── backend/ +│ ├── playout/ # Continuous HLS generation from playlist +│ │ ├── src/ # FFmpeg orchestration, playlist manager +│ │ ├── config/ +│ │ └── Dockerfile +│ ├── chat/ # Real-time WebSocket service +│ │ ├── src/ # Rooms, DMs, moderation, presence (Node/Go) +│ │ ├── config/ +│ │ └── Dockerfile +│ └── shared/ # Common utilities, types, normalization helpers +├── infrastructure/ # Deployment, IaC, Cloudflare configs +│ ├── docker-compose.yml # Local/dev orchestration +│ ├── nginx/ # Proxy, static serving, HLS delivery +│ ├── cloudflare/ # Cache rule examples, Workers +│ └── terraform/ # (optional) provisioning +├── scripts/ # Automation scripts +│ ├── normalize-video.sh # Pre-process MP4s (codec, keyframes, CRF) +│ ├── sync-to-r2.sh # Upload segments/manifests to R2 +│ └── backup.sh +├── monitoring/ # Dashboards, alerts, logging +│ ├── grafana/ +│ └── prometheus/ +├── docs/ +│ └── architecture/ +│ ├── README.md # arc42 template (fully populated) +│ └── adr/ +│ └── ADR-001.md # Consolidated decision record +├── .env.example +├── .gitignore +├── docker-compose.yml # Top-level for dev (playout + chat + nginx) +├── README.md +└── LICENSE +``` + +### Folder Explanations + +- **`frontend/`**: The viewer experience — minimal, modern SPA with Tailwind + Alpine.js (or vanilla). Uses `hls.js` for playback and a WebSocket client for chat. +- **`backend/playout/`**: Core linear engine. FFmpeg consumes a normalized playlist and outputs continuous HLS. Includes normalization validation. +- **`backend/chat/`**: Independent WebSocket service (Node.js/Socket.io or Go). Handles public rooms, private DMs, presence (Redis), persistence (PostgreSQL/SQLite), rate limiting, and moderation. +- **`backend/shared/`**: Reusable code (types, utils, video validation). +- **`infrastructure/`**: Everything needed to deploy and configure the stack (Docker, Nginx, Cloudflare rules, future Terraform). +- **`scripts/`**: One-off and CI-friendly scripts for video preparation, R2 sync, backups, and playlist management. +- **`monitoring/`**: Grafana dashboards, Prometheus rules, logging configuration. +- **`docs/architecture/`**: Complete arc42 documentation + ADR-001 (the single source of truth for all architectural decisions). + +This structure directly reflects the building blocks, runtime flows, deployment view, and cross-cutting concepts defined in the arc42 documentation. + +## Getting Started + +1. `cp .env.example .env` and configure secrets (R2 keys, Cloudflare, etc.). +2. Prepare normalized videos (`scripts/normalize-video.sh`). +3. Run locally: `docker compose up` (or individual services). +4. Deploy: Provision Hetzner VPS, push to R2, configure Cloudflare cache rules (see `infrastructure/cloudflare/` and ADR-001). +5. Full setup and operational runbooks are in the [architecture docs](docs/architecture/README.md). + +## Development + +- Use Docker Compose for local development (playout + chat + frontend). +- Normalize all source videos before adding to playlist (see ADR-001 for exact codec/fps/keyframe requirements). +- Monitor cache hit ratio and R2 operations closely — they determine real cost. + +## Architecture Documentation + +The architecture is fully documented using the [arc42 template](docs/architecture/README.md) (with populated sections, Mermaid context and building-block diagrams, deployment view, cross-cutting concepts, quality goals, risks, and glossary). + +**Decision Records** (in `docs/architecture/adr/`): + +- **[ADR-001](docs/architecture/adr/ADR-001.md)**: Overall system architecture, HLS + Cloudflare R2 caching strategy, storage calculations, cost model, risks, and implementation plan. +- **[ADR-002](docs/architecture/adr/ADR-002-module-structure.md)**: Chosen project folder/module structure (frontend, backend/playout, backend/chat, infrastructure, scripts, monitoring) and mapping to arc42 building blocks. +- **[ADR-003](docs/architecture/adr/ADR-003-tech-stack.md)**: Technology stack, language choices per module (TypeScript for frontend, Go preferred for chat and playout wrapper, Bash/Python for scripts), Docker-first approach, and all implicit assumptions (mandatory video normalization, Cloudflare-centric delivery, monitoring focus, etc.). + +These three ADRs together provide complete traceability for all major design decisions. Future ADRs will cover Docker Compose layout, API contracts, monitoring strategy, and access control. + +See the [architecture README](docs/architecture/README.md) for the big-picture view and the individual ADR files for detailed rationale, alternatives, and consequences. + +Contributions, issues, and further evolution (LL-HLS, ABR, multi-channel, E2EE) should reference the architecture documentation. diff --git a/docs/architecture/README.md b/docs/architecture/README.md new file mode 100644 index 0000000..2d10985 --- /dev/null +++ b/docs/architecture/README.md @@ -0,0 +1,478 @@ +# Architecture Documentation for Linear Void + +> The architecture documentation for the Linear Void project is structured according to the [arc42 template](https://arc42.org/). This template provides a comprehensive framework for documenting software architecture, ensuring that all relevant aspects of the system are covered. + +## Table of Contents + +- [Introduction and Goals](#introduction-and-goals) +- [Architecture Constraints](#architecture-constraints) +- [Context and Scope](#context-and-scope) +- [Solution Strategy](#solution-strategy) +- [Building Block View](#building-block-view) +- [Runtime View](#runtime-view) +- [Deployment View](#deployment-view) +- [Cross-cutting Concepts](#cross-cutting-concepts) +- [Architecture Decisions](#architecture-decisions) +- [Quality Requirements](#quality-requirements) +- [Risks and Technical Debts](#risks-and-technical-debts) +- [Glossary](#glossary) + +## Introduction and Goals + +### Requirements Overview + +Linear Void is an independent **24/7 linear video channel** broadcasting a playlist of self-owned pre-recorded videos. Viewers access it via a custom lightweight HTML/CSS/JavaScript SPA with synchronized HLS playback, schedule metadata, public chat rooms, and private 1:1 messaging. Complete independence from third-party video platforms (YouTube, Twitch, etc.). + +**Core Requirements** (merged from all ADRs): + +- Continuous synchronized playback from playlist (live-style HLS, not VOD). +- Self-hosted source media, frontend, schedule, and chat data. +- Primary US audience/origin (East Coast preferred for low latency/peering). +- Extreme cost efficiency (bandwidth dominant; target near-zero egress via Cloudflare R2 + edge caching). +- Custom chat with rooms, DMs, presence, moderation, persistence. +- Storage planning for 48h of 720p30/1080p30 content. +- Browser-compatible playback (`hls.js` + native HLS). +- Operational simplicity and scalability. + +Full decision rationale, alternatives, and consequences in [`adr/ADR-001.md`](adr/ADR-001.md). + +### Quality Goals + +| Priority | Quality Attribute | Goal | Scenario | +| -------- | ------------------------- | -------------------------------------------------------- | -------------------------------------------------------------------------------- | +| 1 | **Cost Efficiency** | <$25/month for moderate scale (100s concurrent) | R2 $0 egress + immutable `.ts` edge caching eliminates per-viewer bandwidth cost | +| 2 | **Performance / Latency** | Low US latency; acceptable HLS delay | Hetzner Ashburn origin + Cloudflare US POPs (15–35s end-to-end) | +| 3 | **Availability** | 24/7 with minimal downtime | FFmpeg/MediaMTX auto-restart loop, fallback slate, monitoring | +| 4 | **Scalability** | Grow from 10s to 1000s viewers without proportional cost | Edge caching + R2; only request volume limits | +| 5 | **Maintainability** | Lightweight, clear configs, low ops overhead | Static SPA, Go/Node chat, SQLite/PostgreSQL+Redis | +| 6 | **Security / Usability** | Secure transport, moderation tools, good UX | HTTPS/WSS, Cloudflare WAF, rate limits, intuitive chat/player | + +### Stakeholders + +| Role/Name | Contact | Expectations | +| ------------------- | ------- | -------------------------------------------------------------------------------- | +| Product Owner | - | Cost-effective independent 24/7 channel with synchronized viewing and rich chat | +| Engineering Team | - | Simple, evolvable stack (FFmpeg, HLS, Cloudflare R2, lightweight WebSocket chat) | +| US Viewers | - | Reliable low-latency playback, responsive synchronized chat experience | +| Moderators / Admins | - | Easy playlist management, moderation dashboard, monitoring/alerting | +| Operations | - | Low TCO, clear cost model, automated failover, backups | + +(See `adr/ADR-001.md` for detailed forces, decision drivers, and rationale.) + +## Architecture Constraints + +- Independence from video hosting platforms (no YouTube, Twitch, paid Cloudflare Stream). +- US-centric (origin + audience). +- Cost must remain low even at scale (bandwidth dominant). +- Standard browser playback only (no plugins). +- Cloudflare ToS compliance for video delivery (use R2 as origin for large immutable files). +- HLS as primary protocol (standard, not LL-HLS initially). +- Self-hosted playlist normalization and continuous playout required. + +(See `adr/ADR-001.md` for full forces and trade-offs.) + +## Context and Scope + +### Business Context + +Linear Void provides a continuously running scheduled video program (playlist-driven) to a primarily US audience. The custom viewer site replaces traditional platforms, giving full control over content, schedule, UI, and real-time chat (public rooms + private DMs). + +### High-Level Context Diagram (Mermaid) + +```mermaid +flowchart TD + subgraph "External Users" + Viewers[US Viewers
Browser SPA + hls.js + Chat UI] + Admins[Admins / Moderators
Playlist & Schedule UI] + end + + subgraph "Cloudflare Global Edge" + CDN[Cloudflare CDN
Cache Rules + WAF + TLS] + Pages[Cloudflare Pages
Static SPA Hosting] + R2[Cloudflare R2
HLS Segments + Source MP4s
enam location hint] + end + + subgraph "US-East Origin (Hetzner Ashburn VPS)" + VPS[VPS CX22
Nginx Proxy + Monitoring] + Playout[Playout Service
FFmpeg + MediaMTX/SRS
Continuous HLS from Playlist] + Chat[Chat Backend
WebSockets + Redis + DB
Public rooms + Private DMs] + end + + Admins -->|HTTPS - Schedule/Playlist| Playout + Playout -->|S3-compatible Upload| R2 + R2 -->|Cached HLS| CDN + Viewers -->|HTTPS| Pages + Viewers -->|HLS Requests| CDN + Viewers -->|WSS Chat| Chat + Chat --> VPS + VPS -->|Proxy / Static| Pages + CDN -->|Origin Pull on Miss| R2 + + classDef cloud fill:#f0f0ff,stroke:#4263eb + classDef vps fill:#f0fff0,stroke:#2e7d32 + classDef user fill:#fff0f0,stroke:#c62828 + class Cloudflare,CDN,Pages,R2 cloud + class VPS,Playout,Chat vps + class Viewers,Admins user +``` + +**External Interfaces**: + +- **Cloudflare**: CDN (cache rules for `.ts` vs `.m3u8`), R2 (object storage, $0 egress), Pages (static hosting), WAF/TLS. +- **Hetzner VPS**: Compute for playout and chat (low-cost US-East with good peering). +- **Browsers**: Native HLS (`