zwitschi/linearvoid
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 9 Wiki Activity

refactor: modularize chat backend, update docs for .m4s segments

Browse Source
This commit is contained in:
zwitschi committed 2026-08-07 13:16:44 +02:00
1 parent 46e878bef6
commit b3a10ba3ef
24 files changed
+3771 -2781
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/architecture/README.md
+27 -27
View File
@@ -113,14 +113,14 @@ Full decision rationale, alternatives, and consequences in [`adr/ADR-001.md`](ad
### Quality Goals
| Priority | Quality Attribute | Goal | Scenario |
| -------- | ------------------------- | -------------------------------------------------------- | -------------------------------------------------------------------------------- |
| 1 | **Cost Efficiency** | <$25/month for moderate scale (100s concurrent) | R2 $0 egress + immutable `.ts` edge caching eliminates per-viewer bandwidth cost |
| 2 | **Performance / Latency** | Low US latency; acceptable HLS delay | Hetzner Ashburn origin + Cloudflare US POPs (15–35s end-to-end) |
| 3 | **Availability** | 24/7 with minimal downtime | FFmpeg/MediaMTX auto-restart loop, fallback slate, monitoring |
| 4 | **Scalability** | Grow from 10s to 1000s viewers without proportional cost | Edge caching + R2; only request volume limits |
| 5 | **Maintainability** | Lightweight, clear configs, low ops overhead | Static SPA, Go/Node chat, SQLite/PostgreSQL+Redis |
| 6 | **Security / Usability** | Secure transport, moderation tools, good UX | HTTPS/WSS, Cloudflare WAF, rate limits, intuitive chat/player |
| Priority | Quality Attribute | Goal | Scenario |
| -------- | ------------------------- | -------------------------------------------------------- | ------------------------------------------------------------------------------------------- |
| 1 | **Cost Efficiency** | <$25/month for moderate scale (100s concurrent) | R2 $0 egress + immutable `.m4s` and `.ts` edge caching eliminates per-viewer bandwidth cost |
| 2 | **Performance / Latency** | Low US latency; acceptable HLS delay | Hetzner Ashburn origin + Cloudflare US POPs (15–35s end-to-end) |
| 3 | **Availability** | 24/7 with minimal downtime | FFmpeg/MediaMTX auto-restart loop, fallback slate, monitoring |
| 4 | **Scalability** | Grow from 10s to 1000s viewers without proportional cost | Edge caching + R2; only request volume limits |
| 5 | **Maintainability** | Lightweight, clear configs, low ops overhead | Static SPA, Go/Node chat, SQLite/PostgreSQL+Redis |
| 6 | **Security / Usability** | Secure transport, moderation tools, good UX | HTTPS/WSS, Cloudflare WAF, rate limits, intuitive chat/player |
### Stakeholders
@@ -186,7 +186,7 @@ flowchart TD
**External Interfaces**:
- **Cloudflare**: CDN (cache rules for `.ts` vs `.m3u8`), R2 (object storage, $0 egress), Pages (static hosting), WAF/TLS.
- **Cloudflare**: CDN (cache rules for `.m4s` and `.ts` vs `.m3u8`), R2 (object storage, $0 egress), Pages (static hosting), WAF/TLS.
- **Hetzner VPS**: Compute for playout and chat (low-cost US-East with good peering).
- **Browsers**: Native HLS (`<video>` + hls.js) and WebSocket APIs.
- **Source**: Pre-normalized MP4 files uploaded to R2.
@@ -196,7 +196,7 @@ This diagram synthesizes the context views from all original ADRs. Full decision
### Technical Context
- **Playout**: FFmpeg consuming normalized playlist → continuous HLS.
- **Packaging/Delivery**: MediaMTX/SRS/Nginx → immutable `.ts` segments to R2; Cloudflare edge cache.
- **Packaging/Delivery**: MediaMTX/SRS/Nginx → immutable `.m4s` and `.ts` segments to R2; Cloudflare edge cache.
- **Frontend**: Static SPA with hls.js.
- **Chat**: Lightweight WebSocket service (Node/Go) + Redis (presence/rate limits) + PostgreSQL/SQLite (history/moderation).
- **Normalization**: All source videos must be H.264/AAC, 30 fps constant, 2s keyframes.
@@ -212,7 +212,7 @@ This diagram synthesizes the context views from all original ADRs. Full decision
1. **US-East Playout on Hetzner Ashburn VPS** (`CX22` or equivalent): FFmpeg + MediaMTX/SRS for continuous playlist-to-HLS conversion (`-c copy` where possible).
2. **Immutable HLS to Cloudflare R2**: Segments use unique names; R2 acts as origin with $0 egress.
3. **Aggressive Edge Caching**: Long TTL for `.ts` (immutable), short TTL (1–10s) for live `.m3u8`. Tiered Cache enabled. This makes viewer delivery essentially free.
3. **Aggressive Edge Caching**: Long TTL for `.m4s` and `.ts` (immutable), short TTL (1–10s) for live `.m3u8`. Tiered Cache enabled. This makes viewer delivery essentially free.
4. **Lightweight Custom Frontend**: Static HTML/JS SPA + hls.js player + embedded chat sidebar.
5. **Separate but co-located Chat Service**: WebSockets with persistence and moderation (low bandwidth).
6. **Normalization & Monitoring**: Pre-process all videos; use dashboards + alerts for playout health, cache hit ratio, R2 ops.
@@ -230,7 +230,7 @@ Overview (Mermaid diagram):
```mermaid
graph TD
subgraph Cloudflare
CDN[CDN + Cache Rules<br/>.ts immutable long TTL<br/>.m3u8 short TTL]
CDN[CDN + Cache Rules<br/>.m4s and .ts immutable long TTL<br/>.m3u8 short TTL]
R2[R2 - HLS + Source]
Pages[Pages - Static SPA]
end
@@ -306,7 +306,7 @@ graph TB
- **Role**: Continuous HLS encoding loop using FFmpeg concat demuxer
- **Input**: `data/playlist/playlist.txt` — ordered list of MP4 file paths
- **Output**: HLS segments (`.ts`) + live manifest (`.m3u8`) written to `data/hls/`, mirrored to R2
- **Output**: HLS segments (`.m4s` and `.ts`) + live manifest (`.m3u8`) written to `data/hls/`, mirrored to R2
- **Resilience**: Restart counter (>5/60s → activate slate), Prometheus metrics on port 8081
- **Validation**: ffprobe gate on all ingested files (codec, fps, resolution, keyframe interval)
@@ -383,7 +383,7 @@ Module-level breakdown per service.
2. Playout worker (FFmpeg loop with concat demuxer) reads next file, packages as HLS segments (4-6s).
3. Segments uploaded to R2 with unique immutable names; live `.m3u8` updated.
4. Viewer loads SPA → requests master/live manifest → hls.js player follows current edge.
5. Cloudflare serves `.ts` from nearest edge cache (origin/R2 hit only once per POP/region).
5. Cloudflare serves `.m4s` and `.ts` from nearest edge cache (origin/R2 hit only once per POP/region).
6. Parallel WebSocket connection to chat service for synchronized interaction.
Notable aspects: Manifest freshness ensures all viewers stay on same timeline; cache hit ratio directly impacts cost and latency. Fallback slate activates on playout failure. Full cache policy, example configs (FFmpeg bash loop, Nginx RTMP HLS), and storage tables in `adr/ADR-001.md`.
@@ -469,7 +469,7 @@ flowchart LR
**Cloudflare Layer**:
- R2 bucket for media.
- Cache Rules (`.ts`: Cache Everything, immutable, long TTL; `.m3u8`: short TTL/must-revalidate).
- Cache Rules (`.m4s` and `.ts`: Cache Everything, immutable, long TTL; `.m3u8`: short TTL/must-revalidate).
- WAF, rate limiting, Tiered Cache.
- Pages for SPA hosting.
@@ -479,7 +479,7 @@ Future evolution: Docker Compose, managed DB, secondary regions (detailed in sub
### Caching & Cost Optimization
Immutable segment naming + targeted Cloudflare Cache Rules + R2 ($0 egress) = near-zero viewer bandwidth cost. Long TTL on `.ts`, short on live manifests. Monitor hit ratio and R2 Class B ops.
Immutable segment naming + targeted Cloudflare Cache Rules + R2 ($0 egress) = near-zero viewer bandwidth cost. Long TTL on `.m4s` and `.ts`, short on live manifests. Monitor hit ratio and R2 Class B ops.
### Video Normalization & Playout
@@ -888,16 +888,16 @@ Document and rehearse this runbook quarterly; recovery that is not exercised is
## Glossary
| Term | Definition |
| ----------------- | ------------------------------------------------------------------------------------------ |
| HLS | HTTP Live Streaming – protocol using `.m3u8` manifests and media segments (`.ts` or fMP4). |
| Immutable Segment | HLS chunk with unique, unchanging name enabling long-lived caching. |
| R2 | Cloudflare object storage ($0 egress, ideal origin for cached HLS). |
| Edge Cache / POP | Cloudflare points-of-presence that cache and serve content close to viewers. |
| Playout | Continuous generation of a scheduled linear program from a playlist. |
| Normalization | Pre-processing videos to consistent codec/fps/keyframe settings for seamless playback. |
| Cache Rule | Cloudflare configuration defining caching behavior, TTL, and headers per URL pattern. |
| SPOF | Single Point of Failure (current single VPS origin). |
| Tiered Cache | Cloudflare feature reducing origin requests by checking upper-tier caches first. |
| Term | Definition |
| ----------------- | -------------------------------------------------------------------------------------- |
| HLS | HTTP Live Streaming – protocol using `.m3u8` manifests and media segments (fMP4). |
| Immutable Segment | HLS chunk with unique, unchanging name enabling long-lived caching. |
| R2 | Cloudflare object storage ($0 egress, ideal origin for cached HLS). |
| Edge Cache / POP | Cloudflare points-of-presence that cache and serve content close to viewers. |
| Playout | Continuous generation of a scheduled linear program from a playlist. |
| Normalization | Pre-processing videos to consistent codec/fps/keyframe settings for seamless playback. |
| Cache Rule | Cloudflare configuration defining caching behavior, TTL, and headers per URL pattern. |
| SPOF | Single Point of Failure (current single VPS origin). |
| Tiered Cache | Cloudflare feature reducing origin requests by checking upper-tier caches first. |
**References**: Cloudflare R2/Cache docs, MediaMTX/SRS/FFmpeg HLS guides, Hetzner pricing, hls.js documentation, standard video bitrate calculators. All original root `ADR-001-*.md` files have been consolidated and can be archived/deleted.