5 Commits
Author SHA1 Message Date
zwitschi d8cf853842 chore(release): 1.1.0
Release / release (push) Successful in 1m15s
CI / test (push) Successful in 2m53s
CI / docker (push) Successful in 2m0s
- playout: run-scoped R2 manifest URIs (candidate a rewrite via rewriteManifestToRun)
- tests: playlist_rewrite_test.go (rewrite + publish integration + slate guard)
- docs: STREAMING, cloudflare-workers-setup, architecture (incl. Level-2 diagram)
- release housekeeping: TODO->DONE, CHANGELOG 1.1.0, version bump
2026-09-03 09:27:38 +02:00
zwitschi 8c5e70959b feat(cloudflare): add linearvoid-media edge worker for R2 HLS delivery
Replace the ad-hoc `worker.js` with a self-contained `linearvoid-media`
worker under `infrastructure/cloudflare/` that owns `media.linearvoid.tv`.
It serves HLS objects from the R2 bucket, enforcing immutable Cache-Control
on segments, a short TTL on live manifests, plus HLS CORS, OPTIONS
preflight, Range support, and a `/health` probe. Includes wrapper config
with production domain, Vitest unit tests, Miniflare R2 integration tests,
automated deploy workflow, and docs updates.
2026-09-03 08:52:31 +02:00
zwitschi 5f1f796a20 chore(cd): publish nginx on port 8082 to avoid Coolify port conflicts
CI / test (push) Successful in 3m4s
CI / docker (push) Successful in 34s
2026-09-02 21:07:06 +02:00
zwitschi 11d48c8e0d chore(env): update PLAYOUT_PORT to 8081 and normalize line endings
CI / test (push) Successful in 3m3s
CI / docker (push) Successful in 24s
2026-09-02 20:59:57 +02:00
zwitschi 242fc8da67 chore(env): update PLAYOUT_PORT to 8081 and normalize line endings 2026-09-02 20:38:22 +02:00
29 changed files with 3804 additions and 117 deletions

No files matched your search

+3 -2
View File
@@ -49,9 +49,10 @@ DATABASE_URL=postgres://linearvoid:linearvoid@db:5432/linearvoid?sslmode=disable
REDIS_URL=redis://redis:6379/0 REDIS_URL=redis://redis:6379/0
# Service Ports (for local docker-compose) # Service Ports (for local docker-compose)
PLAYOUT_PORT=8080 # Coolify env: 80/443 (Traefik proxy), 8000 (dashboard/Portainer), 8080 taken.
PLAYOUT_PORT=8081
CHAT_PORT=3000 CHAT_PORT=3000
NGINX_PORT=80 NGINX_PORT=8082
# Playout configuration # Playout configuration
PLAYLIST_PATH=/app/playlist/playlist.txt PLAYLIST_PATH=/app/playlist/playlist.txt
+2 -1
View File
@@ -69,7 +69,8 @@ jobs:
ADMIN_SESSION_TTL_HOURS=12 ADMIN_SESSION_TTL_HOURS=12
# Nginx # Nginx
NGINX_PORT=80 # Host port 80/443 (Coolify Traefik) and 8080 are taken; nginx publishes on 8082.
NGINX_PORT=8082
EOF EOF
- name: Seed Prometheus config on remote host - name: Seed Prometheus config on remote host
+35
View File
@@ -0,0 +1,35 @@
name: Deploy Media Worker to Cloudflare
on:
workflow_run:
workflows: [CI]
types: [completed]
workflow_dispatch: {}
jobs:
deploy:
# Production deploy: only on pushes to main (via CI success) or manual main runs.
if: ${{ (github.event_name == 'workflow_dispatch' && github.ref == 'refs/heads/main') || (github.event_name == 'workflow_run' && github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.head_branch == 'main') }}
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: 22
- name: Install dependencies
run: cd infrastructure/cloudflare && npm ci
- name: Test worker
run: cd infrastructure/cloudflare && npm test
- name: Deploy to Cloudflare Workers (production)
env:
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
run: |
cd infrastructure/cloudflare
npx wrangler deploy --env production
+39
View File
@@ -2,6 +2,13 @@
> Completed tasks from [TODO](./TODO.md). Product, feature, and release history lives in [CHANGELOG](./CHANGELOG.md); this file keeps only engineering and ops notes not captured there. > Completed tasks from [TODO](./TODO.md). Product, feature, and release history lives in [CHANGELOG](./CHANGELOG.md); this file keeps only engineering and ops notes not captured there.
## Manifest run-scoped rewrite — candidate (a) (2026-09-03)
- Playout now publishes the live R2 manifest with run-scoped media URIs. `rewriteManifestToRun` in `backend/playout/src/playlist.go` prefixes each bare relative URI (`seg-*.m4s` / `init.mp4`, plus `#EXT-X-MAP:URI`) with `r-{runID}/` before `publishManifest`'s atomic `.tmp` → `CopyObject` commit; skipped in slate mode. Added a stubbable `copyObject` hook for tests (consistent with `putObject`/`listObjects`/`deleteObjects`).
- Tests: `playlist_rewrite_test.go` — `TestRewriteManifestToRun`, `TestRewriteManifestToRun_AlreadyPrefixedUnchanged`, `TestRewriteManifestToRun_HandlesTSAndComments`, `TestRewriteManifestToRun_EmptyRunID`, `TestIsMediaURI`, `TestPublishManifest_RewritesToRunScopedURIs`, `TestPublishManifest_SlateNotRewritten`.
- Docs updated: `STREAMING.md`, `operations/cloudflare-workers-setup.md`, `architecture/README.md` (incl. Level-2 diagram showing the R2-bound `linearvoid-media` Worker and run-scoped layout).
- **Still pending (in TODO): live Validation** — deploy the updated playout to the test lab and confirm manifest/segment URIs resolve via `media.linearvoid.tv`.
## Test infrastructure & refactors (2026-08-07) ## Test infrastructure & refactors (2026-08-07)
- Chat service `main.go` split into 8 packages (config, auth, db, handlers, ws, media, ratelimit, metrics); `db.RedisClient` + `db.PubSub` interfaces added for mocking. - Chat service `main.go` split into 8 packages (config, auth, db, handlers, ws, media, ratelimit, metrics); `db.RedisClient` + `db.PubSub` interfaces added for mocking.
@@ -37,3 +44,35 @@
- Test in Firefox: live playback, rebuffer/recovery, chat sync; record quirks in `docs/STREAMING.md` - Test in Firefox: live playback, rebuffer/recovery, chat sync; record quirks in `docs/STREAMING.md`
- Rebuild + run playout; confirm `playlist.m3u8` lists ~12 segments at 2s each - Rebuild + run playout; confirm `playlist.m3u8` lists ~12 segments at 2s each
- Manual: verify viewer latency and cache-hit behavior after the change - Manual: verify viewer latency and cache-hit behavior after the change
## Chat sidebar (2026-08-17)
- [x] remove search bar from chat sidebar (not needed for single-room)
- [x] remove "user" toggle button from chat sidebar (hide for now, add back when user interface is stable)
- [x] make login/registration form a modal and only toggle it when user is not logged in and clicks "Login" or "Register"
- [x] when user is logged in, do not show login/registration form
- [x] make sure each message is only rendered once in the chat log (no duplicates on reconnect or initial load)
### Plan
Files: `frontend/index.html`, `frontend/src/main.ts`, `frontend/src/style.css`.
1. **Remove search bar** (single-room)
- `index.html`: delete the `<div id="dm-search-row">` block.
- `main.ts`: remove `searchDM()` and the `dm-search-btn` listener; drop the `#dm-search-row` show/hide logic in `switchRoom()` (the `searchRow.hidden = !isDM` lines). Keep `currentDMTarget` only if the "New DM" flow stays.
2. **Remove "Users" toggle button**
- `index.html`: delete the `<button id="users-toggle-btn">` button.
- `main.ts`: remove `toggleUserList()` and the `users-toggle-btn` listener; drop the now-unused `usersVisible` variable.
3. **Login/registration form as a modal**
- `index.html`: wrap the `#auth-panel` form in a modal structure (`.auth-modal` + backdrop) with a close button.
- `style.css`: add `.auth-modal` / `.auth-modal-backdrop` styles.
- `main.ts`: add `openAuthModal()` / `closeAuthModal()`; change the `auth-toggle-btn` handler to open the modal (only when not authenticated) instead of toggling `#auth-panel` inline; wire a close/backdrop-click to `closeAuthModal()`.
4. **Hide form when logged in**
- `main.ts` `updateAuthUI()`: when `authenticated` → hide the "Sign in" button and call `closeAuthModal()`; show the user label + "Sign out" button. When not authenticated → show "Sign in" and hide "Sign out".
5. **No duplicate messages**
- `main.ts`: add `clearMessages()` (empties `#messages`) and call it at the top of `connectWebSocket()` so each (re)connect renders history fresh instead of appending.
- Fix the initial double-connect: make the initial `connectWebSocket("main")` wait for `refreshUserSession()` (or connect once with the resolved username) so history isn't pushed twice on load.
+9 -27
View File
@@ -2,37 +2,19 @@
> Completed tasks moved to [DONE](./DONE.md). Remaining tasks in [BACKLOG](./BACKLOG.md). > Completed tasks moved to [DONE](./DONE.md). Remaining tasks in [BACKLOG](./BACKLOG.md).
## Chat sidebar ## Playout / R2 manifest–segment layout mismatch
- [x] remove search bar from chat sidebar (not needed for single-room) **Chosen approach: candidate (a)** — rewrite the live manifest so its media URIs point into the current run-scoped folder before the atomic R2 publish. Go change in `backend/playout/src/playlist.go`. The `linearvoid-media` Worker stays a faithful path→R2 mapper (no serve-time rewrite).
- [x] remove "user" toggle button from chat sidebar (hide for now, add back when user interface is stable)
- [x] make login/registration form a modal and only toggle it when user is not logged in and clicks "Login" or "Register"
- [x] when user is logged in, do not show login/registration form
- [x] make sure each message is only rendered once in the chat log (no duplicates on reconnect or initial load)
### Plan ### Remaining — Validation
Files: `frontend/index.html`, `frontend/src/main.ts`, `frontend/src/style.css`. - [ ] Run playout (local or test lab), then:
- `curl https://media.linearvoid.tv/hls/playlist.m3u8` → 200, short TTL.
- Every URI in the manifest resolves `200` with `Cache-Control: public, max-age=31536000, immutable` (via the `linearvoid-media` Worker).
- Confirm manifest refresh + `CF-Cache-Status` progression and no 404 on segment fetch across a run loop.
- [ ] Verify slate fallback still serves correctly after the rewrite change.
1. **Remove search bar** (single-room) > Implementation (candidate a), Go unit tests, and documentation are complete — see [DONE.md](./DONE.md).
- `index.html`: delete the `<div id="dm-search-row">` block.
- `main.ts`: remove `searchDM()` and the `dm-search-btn` listener; drop the `#dm-search-row` show/hide logic in `switchRoom()` (the `searchRow.hidden = !isDM` lines). Keep `currentDMTarget` only if the "New DM" flow stays.
2. **Remove "Users" toggle button**
- `index.html`: delete the `<button id="users-toggle-btn">` button.
- `main.ts`: remove `toggleUserList()` and the `users-toggle-btn` listener; drop the now-unused `usersVisible` variable.
3. **Login/registration form as a modal**
- `index.html`: wrap the `#auth-panel` form in a modal structure (`.auth-modal` + backdrop) with a close button.
- `style.css`: add `.auth-modal` / `.auth-modal-backdrop` styles.
- `main.ts`: add `openAuthModal()` / `closeAuthModal()`; change the `auth-toggle-btn` handler to open the modal (only when not authenticated) instead of toggling `#auth-panel` inline; wire a close/backdrop-click to `closeAuthModal()`.
4. **Hide form when logged in**
- `main.ts` `updateAuthUI()`: when `authenticated` → hide the "Sign in" button and call `closeAuthModal()`; show the user label + "Sign out" button. When not authenticated → show "Sign in" and hide "Sign out".
5. **No duplicate messages**
- `main.ts`: add `clearMessages()` (empties `#messages`) and call it at the top of `connectWebSocket()` so each (re)connect renders history fresh instead of appending.
- Fix the initial double-connect: make the initial `connectWebSocket("main")` wait for `refreshUserSession()` (or connect once with the resolved username) so history isn't pushed twice on load.
## Frontend Tests ## Frontend Tests
+17 -20
View File
@@ -2,38 +2,35 @@
<!-- markdownlint-disable MD024 --> <!-- markdownlint-disable MD024 -->
## [Unreleased] ## [1.1.0] - 2026-09-03
### Added ### Added
- **Bash equivalents of Windows scripts**: Added `scripts/cache-hit-ratio.sh`, `scripts/check-env.sh`, `scripts/check-tokens.sh`, and `scripts/loadtest/run-loadtests.sh`, `scripts/loadtest/summarize-artillery.sh`, `scripts/loadtest/summarize-k6.sh` — Linux/macOS ports of the existing PowerShell scripts (the `.ps1` files are retained). - **Cloudflare media edge worker (`linearvoid-media`)**: new self-contained project under `infrastructure/cloudflare/` — an R2-bound Worker owning `media.linearvoid.tv` that enforces immutable segment caching (`public, max-age=31536000, immutable`) and short-TTL live `.m3u8`, with HLS CORS/OPTIONS, byte-Range, and a `/health` probe. Includes Vitest unit tests + Miniflare/workerd R2 integration tests and production deploy via `.gitea/workflows/workers-media-deploy.yml`. Replaces the ad-hoc `worker.js`.
- **Transient presence flash**: Join/leave announcements now appear as a 3-second flash at the top of the chat panel (`#chat-flash`) with a fade-out, instead of permanent chat rows. Presence indicators still update from the same events. - **Bash script ports**: Linux/macOS equivalents of the Windows ops/load-test scripts (`cache-hit-ratio`, `check-env`, `check-tokens`, loadtest run/summarize); `.ps1` files retained.
- **Transient presence flash**: join/leave events show as a 3-second `#chat-flash` instead of permanent chat rows.
### Changed ### Changed
- **Docs condensed and de-Windows'd**: Removed Windows/PowerShell-specific instructions from the docs and condensed the longest documents into tighter, bash-first guides — `SETUP.md`, `DEPLOYMENT.md`, `API.md`, `MONITORING.md`, `STREAMING.md`, `COOLIFY.md`, `incident-runbook.md`, `operations/scaling-performance-playbook.md`, `architecture/README.md`, and the Prometheus metrics inventory. `scripts/loadtest/README.md` now documents the bash load-test helpers. - **Cloudflare Workers hosting**: frontend moved from Cloudflare Pages to the `linearvoid-frontend` static-assets Worker; docs now describe `linearvoid-frontend` (SPA) and `linearvoid-media` (edge HLS), correcting stale `worker.js` and R2 custom-domain wording.
- **System messages no longer persisted**: The chat backend no longer stores `system`-type messages; chat history and the admin chat-log table only contain real chat messages (moderation actions remain in the audit log). - **Docs de-Windows'd & condensed**: bash-first guides for `SETUP`, `DEPLOYMENT`, `API`, `MONITORING`, `STREAMING`, `COOLIFY`, incident runbook, scaling playbook, architecture README, and Prometheus metrics inventory.
- **Reconnect grace period**: Disconnects schedule a delayed "left" announcement (5s); a reconnect within 5s cancels it and skips "joined", eliminating join/leave spam on brief reconnects. - **Compose port de-collision**: playout on host port `8081`, nginx moved to `8082` (`NGINX_PORT` default updated) to avoid Coolify-reserved ports.
- **HLS segment duration 2s**: Default `HLS_SEGMENT_DURATION` changed from 6 to 2 and `HLS_WINDOW_SIZE` from 24 to 12 (2s × 12 = 24s live window), reducing end-to-end latency; the 2s GOP was already in place. - **Playout**: R2 retention now by run count + byte cap (`R2_KEEP_RUNS` default 2, `R2_MAX_RETENTION_BYTES` default 10 GiB, new gauges) replacing the 24h time-based cleanup; default `HLS_SEGMENT_DURATION` 2s / `HLS_WINDOW_SIZE` 12; deduplicated, concurrent startup validation.
- **Chat font sizing**: `.message-text` set to `0.95rem` (slightly smaller than the default, larger than the `0.83rem` system/flash text). - **Chat/UX**: 5s reconnect grace window (no join/leave spam); `system` messages no longer persisted; smaller `.message-text`; sidebar cleanup (removed DM search + Users toggle, login/register now a modal).
- **R2 retention by run count + size**: Playout now keeps the newest `R2_KEEP_RUNS` run folders (default 2) and enforces an `R2_MAX_RETENTION_BYTES` cap (default 10 GiB), replacing the old 24h time-based `R2_SEGMENT_RETENTION_HOURS` cleanup. Cleanup runs after each new run plus an hourly backstop, and never deletes the live manifest. Adds `linearvoid_playout_r2_runs_retained` and `linearvoid_playout_r2_bytes_retained` gauges. - **Auth**: viewer session cookie uses `SameSite=None; Secure` over HTTPS (cross-site SPA) and `Lax` over plain HTTP.
- **Cloudflare Workers frontend hosting**: Replaced Cloudflare Pages with a Cloudflare Worker (`linearvoid-frontend`) that serves the static SPA via Workers Static Assets. Added `frontend/wrangler.toml` and renamed the deploy workflow to `.gitea/workflows/workers-deploy.yml` (builds the SPA and runs `npx wrangler deploy`). Architecture, setup/deployment, and ops documentation updated from Pages to Workers.
- **Viewer session cookie cross-site**: The `linearvoid_user_session` cookie now uses `SameSite=None; Secure` over HTTPS (so it is sent cross-site from the Workers-served SPA) and `SameSite=Lax` over plain HTTP (local dev). The admin session cookie remains `SameSite=Strict`.
- **Parallel playout startup validation**: `ValidatePlaylist` now deduplicates repeated paths and probes unique files concurrently through a bounded worker pool (`PLAYOUT_VALIDATE_WORKERS`, default `runtime.NumCPU()`); `ValidateVideo` runs its stream/format and keyframe `ffprobe` probes concurrently; and `main()` performs the strict validation once (removing the redundant `-validate` pre-step in `start-playout.sh`). This sharply cuts startup time on large or duplicated playlists.
- **Chat sidebar cleanup**: Removed the DM search bar and the "Users" toggle button, and converted the login/register form into a modal that opens from "Sign in" only when signed out and is hidden once logged in.
### Fixed ### Fixed
- **Login/register cross-origin API URLs**: `submitAuth` in `frontend/src/main.ts` now prefixes `/api/auth/login` and `/api/auth/register` with `VITE_PUBLIC_APP_URL` (falling back to same-origin), matching the other API calls so the Workers-served SPA reaches the app origin instead of a same-origin path. - **Run-scoped R2 manifest URIs (playout)**: `publishManifest` now rewrites bare relative media URIs to the current run folder (`r-{runID}/…`) via `rewriteManifestToRun` before the atomic `.tmp` → `CopyObject` publish, so the live `hls/playlist.m3u8` references resolve against the run-scoped segment keys on `media.linearvoid.tv`. Skipped in slate mode. (Live validation pending — see `TODO.md`.)
- **Duplicate chat history on reconnect/initial load**: `connectWebSocket` now clears `#messages` before each (re)connect, and the initial connect waits for session resolution, so room history renders once instead of being appended repeatedly. - **Cross-origin auth**: login/register API calls now prefix `VITE_PUBLIC_APP_URL` so the Workers-served SPA reaches the app origin.
- **Duplicate chat history**: `connectWebSocket` clears messages on each (re)connect and the initial connect waits for session resolution.
### Tests ### Tests
- WebSocket: `TestHub_Broadcast_SystemNotPersisted`, `TestHub_ReconnectGracePeriod`. - WebSocket (`TestHub_Broadcast_SystemNotPersisted`, `TestHub_ReconnectGracePeriod`), handlers (`TestHandleAdminChatLog_ExcludesSystem`), frontend presence flash.
- Handlers: `TestHandleAdminChatLog_ExcludesSystem`. - Playout: `TestCleanupOldRuns(_SizeCap)`, `TestR2KeepRuns`, `TestR2MaxRetentionBytes`, `TestR2RunFromKey`, `TestValidateWorkersConfig`, `TestValidatePlaylist_Dedup`.
- Frontend: `presence flash` test. - Playout (run-scoped manifest rewrite): `TestRewriteManifestToRun`, `TestRewriteManifestToRun_AlreadyPrefixedUnchanged`, `TestRewriteManifestToRun_HandlesTSAndComments`, `TestRewriteManifestToRun_EmptyRunID`, `TestIsMediaURI`, `TestPublishManifest_RewritesToRunScopedURIs`, `TestPublishManifest_SlateNotRewritten`.
- Playout: `TestCleanupOldRuns`, `TestCleanupOldRuns_SizeCap`, `TestR2KeepRuns`, `TestR2MaxRetentionBytes`, `TestR2RunFromKey`. - Cloudflare media worker: Vitest unit (`cacheControlForPath`, `contentTypeForPath`, `parseByteRange`) + Miniflare R2 integration (immutable segment, short-TTL manifest, Range `206`, `404`, OPTIONS, method guard).
- Playout: `TestValidateWorkersConfig`, `TestValidatePlaylist_Dedup`.
## [1.0.13] - 2026-08-13 ## [1.0.13] - 2026-08-13
+3 -2
View File
@@ -7,7 +7,7 @@
[![Build Status](https://git.allucanget.biz/zwitschi/linearvoid/actions/workflows/ci.yml/badge.svg)](https://git.allucanget.biz/zwitschi/linearvoid/actions) [![Build Status](https://git.allucanget.biz/zwitschi/linearvoid/actions/workflows/ci.yml/badge.svg)](https://git.allucanget.biz/zwitschi/linearvoid/actions)
[![Gitea](https://img.shields.io/badge/Gitea-1.27.1-blue)](https://git.allucanget.biz) [![Gitea](https://img.shields.io/badge/Gitea-1.27.1-blue)](https://git.allucanget.biz)
[![License](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE) [![License](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![Version](https://img.shields.io/badge/version-1.0.13-blue)](https://git.allucanget.biz/zwitschi/linearvoid/releases) [![Version](https://img.shields.io/badge/version-1.1.0-blue)](https://git.allucanget.biz/zwitschi/linearvoid/releases)
[![Release Notes](https://img.shields.io/badge/release-notes-green)](CHANGELOG.md) [![Release Notes](https://img.shields.io/badge/release-notes-green)](CHANGELOG.md)
## Architecture ## Architecture
@@ -187,7 +187,8 @@ frontend/
Everything needed to deploy and configure the stack (Docker, Nginx, Cloudflare rules, future Terraform). Everything needed to deploy and configure the stack (Docker, Nginx, Cloudflare rules, future Terraform).
- Cloudflare - Cloudflare
- [worker.js](infrastructure/cloudflare/worker.js) — Cloudflare Worker serving the static SPA and adding immutable cache headers to HLS segments - [linearvoid-media worker](infrastructure/cloudflare/) — R2-bound edge Worker serving HLS on `media.linearvoid.tv` (immutable segment caching, CORS, Range)
- [linearvoid-frontend worker](frontend/wrangler.toml) — serves the static SPA on `linearvoid.tv`
- Nginx - Nginx
- [Dockerfile](infrastructure/nginx/Dockerfile) — builds Nginx with custom config - [Dockerfile](infrastructure/nginx/Dockerfile) — builds Nginx with custom config
- [nginx.conf](infrastructure/nginx/nginx.conf) — reverse proxy, static SPA, HLS delivery - [nginx.conf](infrastructure/nginx/nginx.conf) — reverse proxy, static SPA, HLS delivery
+76 -2
View File
@@ -44,6 +44,9 @@ var (
deleteObjects = func(client *s3.Client, input *s3.DeleteObjectsInput) (*s3.DeleteObjectsOutput, error) { deleteObjects = func(client *s3.Client, input *s3.DeleteObjectsInput) (*s3.DeleteObjectsOutput, error) {
return client.DeleteObjects(context.TODO(), input) return client.DeleteObjects(context.TODO(), input)
} }
copyObject = func(client *s3.Client, input *s3.CopyObjectInput) (*s3.CopyObjectOutput, error) {
return client.CopyObject(context.TODO(), input)
}
bucket = os.Getenv("R2_BUCKET_NAME") bucket = os.Getenv("R2_BUCKET_NAME")
playoutMode = "starting" // "live", "slate", "degraded" playoutMode = "starting" // "live", "slate", "degraded"
restartCount = 0 restartCount = 0
@@ -185,6 +188,8 @@ func watchAndPublishSegments(ctx context.Context, hlsOutputDir string) {
} }
// publishManifest uploads the manifest atomically: temp key first, then copy to final key (STREAMING.md contract). // publishManifest uploads the manifest atomically: temp key first, then copy to final key (STREAMING.md contract).
// Bare media URIs in the manifest are rewritten to the run-scoped R2 layout
// (r-{runID}/…) so that every referenced segment/init resolves on the media host.
func publishManifest(localManifestPath string) error { func publishManifest(localManifestPath string) error {
if r2Client == nil { if r2Client == nil {
if err := initR2Client(); err != nil { if err := initR2Client(); err != nil {
@@ -192,13 +197,33 @@ func publishManifest(localManifestPath string) error {
} }
} }
data, err := os.ReadFile(localManifestPath)
if err != nil {
return fmt.Errorf("read manifest failed: %w", err)
}
// Rewrite bare relative media URIs to the current run folder so the manifest
// matches the run-scoped segment keys uploaded by watchAndPublishSegments /
// syncLatestToR2 (candidate (a)). Slate mode is skipped: its manifest references
// slate assets that are not published under the live run prefix.
if playoutMode != "slate" {
data = rewriteManifestToRun(runID, data)
}
// Stage the (rewritten) manifest locally so the existing SyncToR2 path is reused.
staging := localManifestPath + ".r2staging"
if err := os.WriteFile(staging, data, 0o644); err != nil {
return fmt.Errorf("write staging manifest failed: %w", err)
}
defer os.Remove(staging)
// 1. Upload to temp key // 1. Upload to temp key
if err := SyncToR2(localManifestPath, "hls/playlist.m3u8.tmp"); err != nil { if err := SyncToR2(staging, "hls/playlist.m3u8.tmp"); err != nil {
return fmt.Errorf("temp manifest upload failed: %w", err) return fmt.Errorf("temp manifest upload failed: %w", err)
} }
// 2. Atomic copy to final key // 2. Atomic copy to final key
_, err := r2Client.CopyObject(context.TODO(), &s3.CopyObjectInput{ _, err = copyObject(r2Client, &s3.CopyObjectInput{
Bucket: aws.String(bucket), Bucket: aws.String(bucket),
CopySource: aws.String(bucket + "/hls/playlist.m3u8.tmp"), CopySource: aws.String(bucket + "/hls/playlist.m3u8.tmp"),
Key: aws.String("hls/playlist.m3u8"), Key: aws.String("hls/playlist.m3u8"),
@@ -210,6 +235,55 @@ func publishManifest(localManifestPath string) error {
return nil return nil
} }
// isMediaURI reports whether s names an HLS media file (a segment or init file).
func isMediaURI(s string) bool {
name := s
if i := strings.IndexByte(name, '?'); i >= 0 {
name = name[:i]
}
switch strings.ToLower(filepath.Ext(name)) {
case ".m4s", ".ts", ".mp4", ".m4a":
return true
}
return false
}
// rewriteManifestToRun prefixes bare relative media URIs in an HLS manifest with the
// run folder, e.g. "seg-000001.m4s" -> "r-20260903T…/seg-000001.m4s" and
// #EXT-X-MAP:URI="init.mp4" -> URI="r-20260903T…/init.mp4". This keeps the published
// manifest consistent with the run-scoped R2 layout (STREAMING.md contract). Directives,
// comments, blank lines, and URIs that already contain a path are left unchanged.
// Returns data unchanged when runID is empty.
func rewriteManifestToRun(runID string, data []byte) []byte {
if runID == "" {
return data
}
prefix := runID + "/"
lines := strings.Split(string(data), "\n")
out := make([]string, 0, len(lines))
for _, line := range lines {
trimmed := strings.TrimSpace(line)
switch {
case strings.HasPrefix(trimmed, "#EXT-X-MAP:URI="):
if open := strings.Index(trimmed, `URI="`); open >= 0 {
rest := trimmed[open+len(`URI="`):]
if closeIdx := strings.IndexByte(rest, '"'); closeIdx > 0 {
uri := rest[:closeIdx]
if uri != "" && !strings.Contains(uri, "/") {
line = strings.Replace(line, `URI="`+uri+`"`, `URI="`+prefix+uri+`"`, 1)
}
}
}
out = append(out, line)
case trimmed != "" && !strings.HasPrefix(trimmed, "#") && !strings.Contains(trimmed, "/") && isMediaURI(trimmed):
out = append(out, prefix+trimmed)
default:
out = append(out, line)
}
}
return []byte(strings.Join(out, "\n"))
}
// LoadPlaylist reads a playlist file and returns a list of VideoFile entries // LoadPlaylist reads a playlist file and returns a list of VideoFile entries
func LoadPlaylist(playlistPath string) ([]VideoFile, error) { func LoadPlaylist(playlistPath string) ([]VideoFile, error) {
file, err := os.Open(playlistPath) file, err := os.Open(playlistPath)
@@ -0,0 +1,227 @@
package main
import (
"io"
"os"
"path/filepath"
"testing"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/service/s3"
)
const sampleManifest = `#EXTM3U
#EXT-X-VERSION:7
#EXT-X-TARGETDURATION:2
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-MAP:URI="init.mp4"
#EXTINF:2.000000,
seg-000000.m4s
#EXTINF:2.000000,
seg-000001.m4s
#EXT-X-ENDLIST
`
func TestRewriteManifestToRun(t *testing.T) {
runID := "r-20260903T120000Z"
out := string(rewriteManifestToRun(runID, []byte(sampleManifest)))
wants := []string{
`#EXT-X-MAP:URI="r-20260903T120000Z/init.mp4"`,
"r-20260903T120000Z/seg-000000.m4s",
"r-20260903T120000Z/seg-000001.m4s",
"#EXTM3U",
"#EXT-X-VERSION:7",
"#EXT-X-TARGETDURATION:2",
"#EXT-X-MEDIA-SEQUENCE:0",
"#EXT-X-INDEPENDENT-SEGMENTS",
"#EXT-X-ENDLIST",
"#EXTINF:2.000000,",
}
for _, want := range wants {
if !contains(out, want) {
t.Errorf("output missing %q\noutput:\n%s", want, out)
}
}
// Media URIs must be prefixed (no bare segments left).
for _, bare := range []string{"\nseg-000000.m4s\n", "\nseg-000001.m4s\n", `URI="init.mp4"`} {
if contains(out, bare) {
t.Errorf("output still contains bare %q\noutput:\n%s", bare, out)
}
}
}
func TestRewriteManifestToRun_AlreadyPrefixedUnchanged(t *testing.T) {
manifest := "#EXTM3U\n#EXT-X-MAP:URI=\"r-old/init.mp4\"\nseg-000000.m4s\nr-old/seg-000001.m4s\n#EXT-X-ENDLIST\n"
out := string(rewriteManifestToRun("r-new", []byte(manifest)))
if !contains(out, `#EXT-X-MAP:URI="r-old/init.mp4"`) {
t.Errorf("EXT-X-MAP already-prefixed URI was rewritten: %s", out)
}
// A URI line that already contains a path is left alone.
if !contains(out, "r-old/seg-000001.m4s") {
t.Errorf("already-prefixed segment URI was changed:\n%s", out)
}
// Bare URI still gets the run prefix.
if !contains(out, "r-new/seg-000000.m4s") {
t.Errorf("bare segment not prefixed:\n%s", out)
}
}
func TestRewriteManifestToRun_HandlesTSAndComments(t *testing.T) {
manifest := "#EXTM3U\n# a comment\nvideo-1.ts\ninit.mp4\n#EXT-X-ENDLIST\n"
out := string(rewriteManifestToRun("r-x", []byte(manifest)))
for _, want := range []string{"r-x/video-1.ts", "r-x/init.mp4", "# a comment", "#EXTM3U", "#EXT-X-ENDLIST"} {
if !contains(out, want) {
t.Errorf("output missing %q\noutput:\n%s", want, out)
}
}
}
func TestRewriteManifestToRun_EmptyRunID(t *testing.T) {
out := rewriteManifestToRun("", []byte(sampleManifest))
if string(out) != sampleManifest {
t.Errorf("empty runID should return input unchanged")
}
}
func TestIsMediaURI(t *testing.T) {
for _, ok := range []string{"seg-000001.m4s", "init.mp4", "a.ts", "x.m4a"} {
if !isMediaURI(ok) {
t.Errorf("isMediaURI(%q) = false, want true", ok)
}
}
for _, notOK := range []string{"playlist.m3u8", "readme.txt", "seg-000001.m4s?x=1"} {
// Note: query strings are stripped; '?x=1' on m4s should still be media.
if notOK == "seg-000001.m4s?x=1" {
continue
}
if isMediaURI(notOK) {
t.Errorf("isMediaURI(%q) = true, want false", notOK)
}
}
if !isMediaURI("seg-000001.m4s?token=abc") {
t.Errorf("isMediaURI with query should strip query and be true")
}
}
func TestPublishManifest_RewritesToRunScopedURIs(t *testing.T) {
prevClient, prevBucket := r2Client, bucket
prevRunID, prevMode := runID, playoutMode
prevPut, prevCopy := putObject, copyObject
defer func() {
r2Client, bucket = prevClient, prevBucket
runID, playoutMode = prevRunID, prevMode
putObject, copyObject = prevPut, prevCopy
}()
tmpDir := t.TempDir()
mp := filepath.Join(tmpDir, "playlist.m3u8")
if err := os.WriteFile(mp, []byte(sampleManifest), 0o644); err != nil {
t.Fatalf("write manifest: %v", err)
}
bucket = "linearvoid-media"
runID = "r-20260903T120000Z"
playoutMode = "live"
r2Client = &s3.Client{}
var gotKey, gotCopySource, gotCopyKey string
var gotBody []byte
putObject = func(_ *s3.Client, in *s3.PutObjectInput) (*s3.PutObjectOutput, error) {
gotKey = aws.ToString(in.Key)
b, err := io.ReadAll(in.Body)
if err != nil {
return nil, err
}
gotBody = b
return &s3.PutObjectOutput{}, nil
}
copyObject = func(_ *s3.Client, in *s3.CopyObjectInput) (*s3.CopyObjectOutput, error) {
gotCopySource = aws.ToString(in.CopySource)
gotCopyKey = aws.ToString(in.Key)
return &s3.CopyObjectOutput{}, nil
}
if err := publishManifest(mp); err != nil {
t.Fatalf("publishManifest error: %v", err)
}
if gotKey != "hls/playlist.m3u8.tmp" {
t.Errorf("temp upload key = %q, want hls/playlist.m3u8.tmp", gotKey)
}
if gotCopySource != "linearvoid-media/hls/playlist.m3u8.tmp" {
t.Errorf("CopySource = %q, want linearvoid-media/hls/playlist.m3u8.tmp", gotCopySource)
}
if gotCopyKey != "hls/playlist.m3u8" {
t.Errorf("final key = %q, want hls/playlist.m3u8", gotCopyKey)
}
body := string(gotBody)
for _, want := range []string{
"r-20260903T120000Z/seg-000000.m4s",
"r-20260903T120000Z/seg-000001.m4s",
`#EXT-X-MAP:URI="r-20260903T120000Z/init.mp4"`,
} {
if !contains(body, want) {
t.Errorf("published manifest missing %q:\n%s", want, body)
}
}
for _, bare := range []string{"\nseg-000000.m4s\n", "\nseg-000001.m4s\n"} {
if contains(body, bare) {
t.Errorf("published manifest still contains bare %q:\n%s", bare, body)
}
}
}
func TestPublishManifest_SlateNotRewritten(t *testing.T) {
prevClient, prevBucket := r2Client, bucket
prevRunID, prevMode := runID, playoutMode
prevPut, prevCopy := putObject, copyObject
defer func() {
r2Client, bucket = prevClient, prevBucket
runID, playoutMode = prevRunID, prevMode
putObject, copyObject = prevPut, prevCopy
}()
tmpDir := t.TempDir()
mp := filepath.Join(tmpDir, "playlist.m3u8")
if err := os.WriteFile(mp, []byte(sampleManifest), 0o644); err != nil {
t.Fatalf("write manifest: %v", err)
}
bucket = "linearvoid-media"
runID = "r-20260903T120000Z"
playoutMode = "slate"
r2Client = &s3.Client{}
var gotBody []byte
putObject = func(_ *s3.Client, in *s3.PutObjectInput) (*s3.PutObjectOutput, error) {
b, _ := io.ReadAll(in.Body)
gotBody = b
return &s3.PutObjectOutput{}, nil
}
copyObject = func(_ *s3.Client, _ *s3.CopyObjectInput) (*s3.CopyObjectOutput, error) {
return &s3.CopyObjectOutput{}, nil
}
if err := publishManifest(mp); err != nil {
t.Fatalf("publishManifest error: %v", err)
}
if string(gotBody) != sampleManifest {
t.Errorf("slate manifest should be published unmodified; got:\n%s", string(gotBody))
}
}
func contains(s, sub string) bool {
for i := 0; i+len(sub) <= len(s); i++ {
if s[i:i+len(sub)] == sub {
return true
}
}
return false
}
+3 -1
View File
@@ -11,6 +11,8 @@ services:
restart: unless-stopped restart: unless-stopped
env_file: env_file:
- .env - .env
ports:
- "${PLAYOUT_PORT:-8081}:8081"
volumes: volumes:
- ./data/playlist:/app/playlist:rw - ./data/playlist:/app/playlist:rw
- ./data/media:/app/media:ro - ./data/media:/app/media:ro
@@ -56,7 +58,7 @@ services:
container_name: linearvoid-nginx container_name: linearvoid-nginx
restart: unless-stopped restart: unless-stopped
ports: ports:
- "${NGINX_PORT:-8080}:80" - "${NGINX_PORT:-8082}:80"
volumes: volumes:
- ./data/hls:/srv/hls:ro - ./data/hls:/srv/hls:ro
networks: networks:
+3 -1
View File
@@ -13,6 +13,8 @@ services:
restart: unless-stopped restart: unless-stopped
env_file: env_file:
- .env - .env
ports:
- "${PLAYOUT_PORT:-8081}:8081"
volumes: volumes:
- ./data/playlist:/app/playlist:ro - ./data/playlist:/app/playlist:ro
- ./data/media:/app/media:ro - ./data/media:/app/media:ro
@@ -64,7 +66,7 @@ services:
container_name: linearvoid-nginx container_name: linearvoid-nginx
restart: unless-stopped restart: unless-stopped
ports: ports:
- "${NGINX_PORT:-8080}:80" - "${NGINX_PORT:-8082}:80"
volumes: volumes:
- ./frontend/dist:/srv/frontend:ro - ./frontend/dist:/srv/frontend:ro
- ./data/hls:/srv/hls:ro - ./data/hls:/srv/hls:ro
+1 -1
View File
@@ -109,7 +109,7 @@ mkdir -p data/playlist data/media data/hls data/chat
### Domains and proxy ### Domains and proxy
1. Assign the **nginx** service a public domain (e.g. `linearvoid.tv`); Traefik terminates TLS and forwards to `nginx`. 1. Assign the **nginx** service a public domain (e.g. `linearvoid.tv`); Traefik terminates TLS and forwards to `nginx`.
2. Set **Ports Exposes** so Traefik targets the nginx port (`80` inside container, mapped from `${NGINX_PORT:-8080}:80`). 2. Set **Ports Exposes** so Traefik targets the nginx port (`80` inside container, mapped from `${NGINX_PORT:-8082}:80`).
3. Do **not** publicly expose playout (`8081`) or chat (`3000`) health ports — keep them internal and route through nginx. 3. Do **not** publicly expose playout (`8081`) or chat (`3000`) health ports — keep them internal and route through nginx.
4. For `media.linearvoid.tv`, keep serving HLS from the R2 custom domain (per SETUP.md) rather than routing through Coolify, so segments stay on Cloudflare's edge cache. 4. For `media.linearvoid.tv`, keep serving HLS from the R2 custom domain (per SETUP.md) rather than routing through Coolify, so segments stay on Cloudflare's edge cache.
+35 -8
View File
@@ -3,7 +3,7 @@
## Environments ## Environments
| Environment | Host | Docker Target | Purpose | | Environment | Host | Docker Target | Purpose |
|---|---|---|---| | -------------- | -------------------------- | -------------------------- | ---------------------------------------------- |
| **Test Lab** | `192.168.88.40` (internal) | `tcp://192.168.88.40:2375` | Integration testing, staging, CI/CD validation | | **Test Lab** | `192.168.88.40` (internal) | `tcp://192.168.88.40:2375` | Integration testing, staging, CI/CD validation |
| **Production** | Hetzner VPS (TBD) | `tcp://<hetzner-ip>:2375` | Public linearvoid.tv | | **Production** | Hetzner VPS (TBD) | `tcp://<hetzner-ip>:2375` | Public linearvoid.tv |
@@ -14,12 +14,14 @@
**Status: Not provisioned.** Plan documented and pipeline ready; only the VPS needs creation. **Status: Not provisioned.** Plan documented and pipeline ready; only the VPS needs creation.
### Planned Infrastructure ### Planned Infrastructure
- Provider: Hetzner Cloud, Region: Ashburn, VA (us-east) - Provider: Hetzner Cloud, Region: Ashburn, VA (us-east)
- Instance: CX22 (2 vCPU, 4 GB RAM, 40 GB SSD) or equivalent - Instance: CX22 (2 vCPU, 4 GB RAM, 40 GB SSD) or equivalent
- OS: Ubuntu 24.04 LTS or Debian 12 - OS: Ubuntu 24.04 LTS or Debian 12
- Bandwidth: 20 TB included traffic/month; Estimated cost: ~€7–10/month base - Bandwidth: 20 TB included traffic/month; Estimated cost: ~€7–10/month base
### Cutover Plan ### Cutover Plan
1. Provision Hetzner VPS 1. Provision Hetzner VPS
2. Add production secrets to Gitea (`CI_PROD_*` prefix) 2. Add production secrets to Gitea (`CI_PROD_*` prefix)
3. Update `DOCKER_HOST` in CD workflow to Hetzner public IP 3. Update `DOCKER_HOST` in CD workflow to Hetzner public IP
@@ -29,6 +31,7 @@
7. Decommission test lab as public target 7. Decommission test lab as public target
### Production Preflight (minimum readiness gate) ### Production Preflight (minimum readiness gate)
1. Verify Hetzner host reachable over SSH and Docker remote API TLS works for the Gitea runner 1. Verify Hetzner host reachable over SSH and Docker remote API TLS works for the Gitea runner
2. Confirm all production secrets exist under `CI_PROD_*` prefix 2. Confirm all production secrets exist under `CI_PROD_*` prefix
3. Confirm `linearvoid.tv`, `www.linearvoid.tv`, `media.linearvoid.tv` DNS targets match intended phase 3. Confirm `linearvoid.tv`, `www.linearvoid.tv`, `media.linearvoid.tv` DNS targets match intended phase
@@ -37,8 +40,9 @@
6. Verify rollback possible before final DNS switch 6. Verify rollback possible before final DNS switch
### Firewall / Security Group ### Firewall / Security Group
| Port | Source | Purpose | | Port | Source | Purpose |
|---|---|---| | ---- | --------------------- | ------------------------------ |
| 22 | Admin IPs | SSH management | | 22 | Admin IPs | SSH management |
| 80 | 0.0.0.0/0 | HTTP (redirect to HTTPS) | | 80 | 0.0.0.0/0 | HTTP (redirect to HTTPS) |
| 443 | 0.0.0.0/0 | HTTPS (nginx) | | 443 | 0.0.0.0/0 | HTTPS (nginx) |
@@ -47,8 +51,9 @@
| 3001 | Admin IPs or internal | Grafana (or tunnel via SSH) | | 3001 | Admin IPs or internal | Grafana (or tunnel via SSH) |
### Pipeline Readiness ### Pipeline Readiness
| Workflow | Trigger | Purpose | | Workflow | Trigger | Purpose |
|---|---|---| | -------------------- | ----------------- | --------------------------------------------------------------------------------- |
| `ci.yml` | Push to `main` | Lint, build frontend, vet Go, build & push images (`:latest`, `:main`) | | `ci.yml` | Push to `main` | Lint, build frontend, vet Go, build & push images (`:latest`, `:main`) |
| `cd.yml` | Push to `main` | Deploy to test lab (`192.168.88.40:2375`) — runs **in parallel** with CI | | `cd.yml` | Push to `main` | Deploy to test lab (`192.168.88.40:2375`) — runs **in parallel** with CI |
| `release.yml` | Tag push `*.*.*` | Build & push versioned images (`:X.Y.Z`, `:X.Y`, `:latest`), create Gitea Release | | `release.yml` | Tag push `*.*.*` | Build & push versioned images (`:X.Y.Z`, `:X.Y`, `:latest`), create Gitea Release |
@@ -67,6 +72,7 @@ DOCKER_HOST=tcp://<hetzner-public-ip>:2375
No pipeline code changes needed to switch targets. No pipeline code changes needed to switch targets.
## Target Topology ## Target Topology
- Host: Linux server with Docker Engine - Host: Linux server with Docker Engine
- Reverse proxy: nginx container (linearvoid-nginx) - Reverse proxy: nginx container (linearvoid-nginx)
- Core services: playout, chat, redis, postgres - Core services: playout, chat, redis, postgres
@@ -74,6 +80,7 @@ No pipeline code changes needed to switch targets.
- CDN/edge: Cloudflare - CDN/edge: Cloudflare
## Prerequisites ## Prerequisites
- Docker Engine + Docker Compose plugin - Docker Engine + Docker Compose plugin
- Network access to git.allucanget.biz - Network access to git.allucanget.biz
- Valid `.env` file with production values - Valid `.env` file with production values
@@ -82,6 +89,7 @@ No pipeline code changes needed to switch targets.
## Environment File ## Environment File
`.env` requires at minimum: `.env` requires at minimum:
- `DATABASE_URL` — PostgreSQL connection string - `DATABASE_URL` — PostgreSQL connection string
- `REDIS_URL` — Redis connection string - `REDIS_URL` — Redis connection string
- `ADMIN_USERNAME` / `ADMIN_PASSWORD` — admin dashboard credentials - `ADMIN_USERNAME` / `ADMIN_PASSWORD` — admin dashboard credentials
@@ -89,18 +97,21 @@ No pipeline code changes needed to switch targets.
- `VITE_PUBLIC_APP_URL` / `VITE_PUBLIC_WS_URL` / `VITE_PUBLIC_HLS_URL` — frontend URLs - `VITE_PUBLIC_APP_URL` / `VITE_PUBLIC_WS_URL` / `VITE_PUBLIC_HLS_URL` — frontend URLs
**Cloudflare R2 (required for HLS segment sync to edge):** **Cloudflare R2 (required for HLS segment sync to edge):**
- `R2_ACCESS_KEY_ID` / `R2_SECRET_ACCESS_KEY` - `R2_ACCESS_KEY_ID` / `R2_SECRET_ACCESS_KEY`
- `R2_BUCKET_NAME` (e.g. `linearvoid-media`) - `R2_BUCKET_NAME` (e.g. `linearvoid-media`)
- `R2_ENDPOINT` — S3-compatible endpoint URL - `R2_ENDPOINT` — S3-compatible endpoint URL
- `R2_REGION` — set to `auto` - `R2_REGION` — set to `auto`
**Cloudflare (optional, for Workers):** **Cloudflare (optional, for Workers):**
- `CLOUDFLARE_API_TOKEN` / `CLOUDFLARE_ACCOUNT_ID` - `CLOUDFLARE_API_TOKEN` / `CLOUDFLARE_ACCOUNT_ID`
- `MEDIA_DOMAIN` — split media domain (e.g. `media.linearvoid.tv`) - `MEDIA_DOMAIN` — split media domain (e.g. `media.linearvoid.tv`)
See `.env.example` and `SETUP.md` for full variable reference. See `.env.example` and `SETUP.md` for full variable reference.
### Gitea Secrets ### Gitea Secrets
`cd.yml` injects sensitive values from Gitea secrets into `.env` at deploy time. Configure in the Gitea repository: `cd.yml` injects sensitive values from Gitea secrets into `.env` at deploy time. Configure in the Gitea repository:
`DATABASE_URL`, `ADMIN_USERNAME`, `ADMIN_PASSWORD`, `R2_ACCESS_KEY_ID`, `R2_SECRET_ACCESS_KEY`, `R2_BUCKET_NAME`, `R2_ENDPOINT`, `R2_ADMIN_ACCESS_KEY_ID` (optional), `R2_ADMIN_SECRET_ACCESS_KEY` (optional), `R2_ADMIN_ENDPOINT` (optional), `CLOUDFLARE_API_TOKEN` (optional), `CLOUDFLARE_ACCOUNT_ID` (optional), `CI_REGISTRY_USER`, `CI_REGISTRY_TOKEN` `DATABASE_URL`, `ADMIN_USERNAME`, `ADMIN_PASSWORD`, `R2_ACCESS_KEY_ID`, `R2_SECRET_ACCESS_KEY`, `R2_BUCKET_NAME`, `R2_ENDPOINT`, `R2_ADMIN_ACCESS_KEY_ID` (optional), `R2_ADMIN_SECRET_ACCESS_KEY` (optional), `R2_ADMIN_ENDPOINT` (optional), `CLOUDFLARE_API_TOKEN` (optional), `CLOUDFLARE_ACCOUNT_ID` (optional), `CI_REGISTRY_USER`, `CI_REGISTRY_TOKEN`
@@ -116,6 +127,7 @@ docker compose -f docker-compose.prod.yml up -d --remove-orphans
``` ```
## First-Time or Recovery Provisioning ## First-Time or Recovery Provisioning
```bash ```bash
mkdir -p data/chat data/playlist data/media data/hls mkdir -p data/chat data/playlist data/media data/hls
docker volume ls docker volume ls
@@ -124,10 +136,13 @@ docker compose -f docker-compose.prod.yml up -d
``` ```
## Health Verification ## Health Verification
```bash ```bash
docker compose -f docker-compose.prod.yml ps docker compose -f docker-compose.prod.yml ps
``` ```
Endpoint checks: Endpoint checks:
- App: `http://{host}:8080/` - App: `http://{host}:8080/`
- Nginx health: `http://{host}:8080/health` - Nginx health: `http://{host}:8080/health`
- Chat health: `http://{host}:3000/health` - Chat health: `http://{host}:3000/health`
@@ -136,30 +151,38 @@ Endpoint checks:
- Grafana: `http://{host}:3001` - Grafana: `http://{host}:3001`
Domain checks: Domain checks:
- `https://linearvoid.tv` - `https://linearvoid.tv`
- `https://linearvoid.tv/admin` - `https://linearvoid.tv/admin`
- `https://linearvoid.tv/metrics` - `https://linearvoid.tv/metrics`
## Zero/Low-Downtime Update Pattern ## Zero/Low-Downtime Update Pattern
1. Pull newest images. 1. Pull newest images.
2. Recreate changed services only: 2. Recreate changed services only:
```bash
docker compose -f docker-compose.prod.yml pull ```bash
docker compose -f docker-compose.prod.yml up -d nginx chat playout docker compose -f docker-compose.prod.yml pull
``` docker compose -f docker-compose.prod.yml up -d nginx chat playout
```
3. Re-run health checks. 3. Re-run health checks.
## Rollback ## Rollback
If new images are unhealthy: If new images are unhealthy:
1. Pin previous tags in `docker-compose.prod.yml`. 1. Pin previous tags in `docker-compose.prod.yml`.
2. Pull pinned tags. 2. Pull pinned tags.
3. Restart impacted services: 3. Restart impacted services:
```bash ```bash
docker compose -f docker-compose.prod.yml pull docker compose -f docker-compose.prod.yml pull
docker compose -f docker-compose.prod.yml up -d nginx chat playout docker compose -f docker-compose.prod.yml up -d nginx chat playout
``` ```
## Operational Tasks ## Operational Tasks
```bash ```bash
# View logs # View logs
docker compose -f docker-compose.prod.yml logs -f nginx chat playout docker compose -f docker-compose.prod.yml logs -f nginx chat playout
@@ -175,23 +198,27 @@ docker compose -f docker-compose.prod.yml down -v
``` ```
## Nginx and Proxy Considerations ## Nginx and Proxy Considerations
- If an external proxy (e.g. Nginx Proxy Manager) fronts the stack, align request body limits with internal nginx (see `infrastructure/nginx/nginx.conf`). - If an external proxy (e.g. Nginx Proxy Manager) fronts the stack, align request body limits with internal nginx (see `infrastructure/nginx/nginx.conf`).
- For resumable admin uploads, ensure `client_max_body_size` supports chunk requests in both proxy layers. - For resumable admin uploads, ensure `client_max_body_size` supports chunk requests in both proxy layers.
## Security Baseline ## Security Baseline
- Use a strong `ADMIN_PASSWORD` and rotate regularly. - Use a strong `ADMIN_PASSWORD` and rotate regularly.
- Keep Docker host access restricted. - Keep Docker host access restricted.
- Prefer HTTPS/WSS through Cloudflare. - Prefer HTTPS/WSS through Cloudflare.
- Restrict exposed ports if only proxy access is needed. - Restrict exposed ports if only proxy access is needed.
## Common Deployment Pitfalls ## Common Deployment Pitfalls
- Missing frontend build artifacts in local-only compose workflows. - Missing frontend build artifacts in local-only compose workflows.
- Wrong registry credentials or missing login. - Wrong registry credentials or missing login.
- Misaligned domain env vars (app/ws/hls) causing mixed-origin errors. - Misaligned domain env vars (app/ws/hls) causing mixed-origin errors.
## Troubleshooting ## Troubleshooting
| Symptom | Check | Fix | | Symptom | Check | Fix |
|---|---|---| | -------------------------- | --------------------------------------------- | ------------------------------------------------------------- |
| Service not starting | `docker logs linearvoid-<svc> --tail 50` | Check `.env` vars, volume mounts, port conflicts | | Service not starting | `docker logs linearvoid-<svc> --tail 50` | Check `.env` vars, volume mounts, port conflicts |
| Playout stall / slate mode | `curl http://<host>:8081/health` — mode=slate | Check playlist file, media paths, disk space | | Playout stall / slate mode | `curl http://<host>:8081/health` — mode=slate | Check playlist file, media paths, disk space |
| Chat not connecting | `curl http://<host>:3000/health` | Check Redis: `docker exec linearvoid-redis redis-cli ping` | | Chat not connecting | `curl http://<host>:3000/health` | Check Redis: `docker exec linearvoid-redis redis-cli ping` |
+2 -2
View File
@@ -70,7 +70,7 @@ flowchart LR
2. S3 endpoint: `https://<account-id>.r2.cloudflarestorage.com` → `R2_ENDPOINT`. 2. S3 endpoint: `https://<account-id>.r2.cloudflarestorage.com` → `R2_ENDPOINT`.
3. Attach `media.linearvoid.tv` as custom media domain → `MEDIA_DOMAIN`; add CORS allowing `https://linearvoid.tv` GET for HLS delivery. 3. Attach `media.linearvoid.tv` as custom media domain → `MEDIA_DOMAIN`; add CORS allowing `https://linearvoid.tv` GET for HLS delivery.
4. Point `linearvoid.tv` (+ `www` redirect) at the app entrypoint; keep `media.*` on R2. Proxied DNS, TLS **Full (strict)**. 4. Point `linearvoid.tv` (+ `www` redirect) at the app entrypoint; keep `media.*` on R2. Proxied DNS, TLS **Full (strict)**.
5. Apply the `.m4s`/`.m3u8` cache rules from [`cache-rules.md`](./infrastructure/cloudflare/cache-rules.md); enable Smart Tiered Cache; optionally deploy `infrastructure/cloudflare/worker.js` for header normalization. 5. Apply the `.m4s`/`.m3u8` cache rules from [`cache-rules.md`](./infrastructure/cloudflare/cache-rules.md); enable Smart Tiered Cache; deploy the [`linearvoid-media` worker](../infrastructure/cloudflare/) on `media.linearvoid.tv` (replace the R2 bucket's direct custom-domain binding on that host) to enforce immutable segment caching + HLS CORS.
6. Verify: `aws s3 ls --endpoint-url $R2_ENDPOINT`; check cache hit ratio after serving traffic. 6. Verify: `aws s3 ls --endpoint-url $R2_ENDPOINT`; check cache hit ratio after serving traffic.
## Running the Application ## Running the Application
@@ -129,7 +129,7 @@ Coolify replaces the Gitea CD `DOCKER_HOST` orchestration (Cloudflare DNS/R2/cac
### Production (Hetzner VPS + Cloudflare) ### Production (Hetzner VPS + Cloudflare)
1. Provision CX22 in Ashburn, VA (ADR-001). 1. Provision CX22 in Ashburn, VA (ADR-001).
2. Cloudflare: R2 bucket (`enam` hint), `media.linearvoid.tv` custom domain + cache rules, `linearvoid.tv`/`www` at the app entrypoint with `www`→apex redirect, Full (strict) TLS, deploy `infrastructure/cloudflare/worker.js`. 2. Cloudflare: R2 bucket (`enam` hint), deploy the [`linearvoid-media` worker](../infrastructure/cloudflare/) on `media.linearvoid.tv` + cache rules, `linearvoid.tv`/`www` at the app entrypoint with `www`→apex redirect, Full (strict) TLS.
3. Deploy from the project: `docker compose -f docker-compose.prod.yml up -d` (or pull from Gitea registry); manage under systemd. 3. Deploy from the project: `docker compose -f docker-compose.prod.yml up -d` (or pull from Gitea registry); manage under systemd.
4. Monitoring: Grafana dashboards from `monitoring/grafana/`. 4. Monitoring: Grafana dashboards from `monitoring/grafana/`.
+14 -4
View File
@@ -27,7 +27,7 @@ Nginx `/hls/` is a local/test fallback, not the production origin. The SPA, admi
Only validated files may enter the production playlist. Only validated files may enter the production playlist.
| Property | Required profile | | Property | Required profile |
| ---------- | ------------------------------------------------------------------------ | | ---------- | ----------------------------------------------------------------------- |
| Container | MP4 | | Container | MP4 |
| Video | H.264/AVC, `yuv420p` | | Video | H.264/AVC, `yuv420p` |
| Resolution | 1280x720 or 1920x1080; avoid resolution changes within a playlist block | | Resolution | 1280x720 or 1920x1080; avoid resolution changes within a playlist block |
@@ -63,7 +63,7 @@ ffmpeg -i input.mov \
The playout service wraps FFmpeg, tracks the active item/revision, restarts failed processes, and exposes `/health` plus Prometheus metrics. The playout service wraps FFmpeg, tracks the active item/revision, restarts failed processes, and exposes `/health` plus Prometheus metrics.
| Setting | Value | | Setting | Value |
| -------------- | ---------------------------------------------------------------------------------- | | ---------------- | ------------------------------------------------------------------------ |
| Protocol | Standard live HLS, not LL-HLS | | Protocol | Standard live HLS, not LL-HLS |
| Segment format | fMP4 (`.m4s`) with init segment; `.ts` may be supported during migration | | Segment format | fMP4 (`.m4s`) with init segment; `.ts` may be supported during migration |
| Segment target | 2 seconds (`HLS_SEGMENT_DURATION=2`) | | Segment target | 2 seconds (`HLS_SEGMENT_DURATION=2`) |
@@ -96,12 +96,22 @@ Order prevents a manifest referencing an unavailable segment:
Never upload an actively written segment or publish the manifest first. Use a single publisher (or fencing/leader election) so two workers can't update the live manifest concurrently. Never upload an actively written segment or publish the manifest first. Use a single publisher (or fencing/leader election) so two workers can't update the live manifest concurrently.
**Run-scoped manifest URIs.** Segments and the init file are stored run-scoped under `hls/r-{runID}/…`, so the live `hls/playlist.m3u8` must reference them with that run prefix. Before the final commit step, the publisher rewrites each bare relative media URI (segments plus `#EXT-X-MAP:URI`) to `r-{runID}/<name>`. `publishManifest` in `backend/playout/src/playlist.go` does this via `rewriteManifestToRun`, using the same `runID` under which the segments were uploaded; the rewrite is skipped in slate mode (its manifest is served as-is). The published manifest therefore contains, e.g.:
```m3u8
#EXT-X-MAP:URI="r-20260807T170412Z/init.mp4"
#EXTINF:2.000000,
r-20260807T170412Z/seg-000001234.m4s
```
The `linearvoid-media` Worker on `media.linearvoid.tv` is a faithful path→R2 mapper and serves these run-scoped paths directly; it does **not** rewrite URIs at serve time.
`data/hls/` is regenerable staging. Source media, playlist revisions, and schedule state are authoritative; generated HLS is not backed up as application state. `data/hls/` is regenerable staging. Source media, playlist revisions, and schedule state are authoritative; generated HLS is not backed up as application state.
## Cache contract ## Cache contract
| Objects | Edge policy | Browser policy | Required headers/behavior | | Objects | Edge policy | Browser policy | Required headers/behavior |
| -------------------------- | --------------------------------- | --------------------------------------- | ---------------------------------------------------------------------------- | | -------------------------- | --------------------------- | --------------------------------------- | ---------------------------------------------------------------------------- |
| `.m4s`, `.ts`, init `.mp4` | Cache eligible; TTL 1 year | TTL 1 year | `Cache-Control: public, max-age=31536000, immutable`; full path in cache key | | `.m4s`, `.ts`, init `.mp4` | Cache eligible; TTL 1 year | TTL 1 year | `Cache-Control: public, max-age=31536000, immutable`; full path in cache key |
| Live `.m3u8` | Cache eligible; edge TTL 2s | Do not retain beyond freshness interval | `Cache-Control: public, max-age=1, s-maxage=2, must-revalidate` | | Live `.m3u8` | Cache eligible; edge TTL 2s | Do not retain beyond freshness interval | `Cache-Control: public, max-age=1, s-maxage=2, must-revalidate` |
| Versioned JS/CSS | Cache eligible; TTL 1 year | TTL 1 year | Content-hashed filenames and `immutable` | | Versioned JS/CSS | Cache eligible; TTL 1 year | TTL 1 year | Content-hashed filenames and `immutable` |
@@ -160,7 +170,7 @@ A release is streaming-ready only when: one full playlist boundary plays without
## Required configuration ## Required configuration
| Variable | Purpose | | Variable | Purpose |
| ------------------------------------------ | -------------------------------------------------- | | ------------------------------------------ | ------------------------------------- |
| `PLAYLIST_PATH` | Active playlist file | | `PLAYLIST_PATH` | Active playlist file |
| `HLS_SEGMENT_DURATION` | Target segment duration; baseline `2` | | `HLS_SEGMENT_DURATION` | Target segment duration; baseline `2` |
| `HLS_WINDOW_SIZE` | Manifest window; baseline `12` | | `HLS_WINDOW_SIZE` | Manifest window; baseline `12` |
+26 -9
View File
@@ -4,6 +4,20 @@ Linear Void architecture is documented using the [arc42 template](https://arc42.
> Companion docs: see [Documentation README](../README.md) and `adr/` for full decision rationale, alternatives, and consequences. > Companion docs: see [Documentation README](../README.md) and `adr/` for full decision rationale, alternatives, and consequences.
- [Architecture Documentation for Linear Void](#architecture-documentation-for-linear-void)
- [Introduction and Goals](#introduction-and-goals)
- [Architecture Constraints](#architecture-constraints)
- [Context and Scope](#context-and-scope)
- [Solution Strategy](#solution-strategy)
- [Building Block View](#building-block-view)
- [Runtime View](#runtime-view)
- [Deployment View](#deployment-view)
- [Cross-cutting Concepts](#cross-cutting-concepts)
- [Architecture Decisions](#architecture-decisions)
- [Quality Requirements](#quality-requirements)
- [Risks and Technical Debts](#risks-and-technical-debts)
- [Glossary](#glossary)
## Introduction and Goals ## Introduction and Goals
**Requirements Overview** — Linear Void is an independent **24/7 linear video channel** broadcasting self-owned pre-recorded videos to a primarily US audience via a custom lightweight HTML/CSS/JS SPA with synchronized HLS playback, schedule metadata, public chat rooms, and private 1:1 messaging. Fully independent of third-party video platforms (YouTube, Twitch, etc.). **Requirements Overview** — Linear Void is an independent **24/7 linear video channel** broadcasting self-owned pre-recorded videos to a primarily US audience via a custom lightweight HTML/CSS/JS SPA with synchronized HLS playback, schedule metadata, public chat rooms, and private 1:1 messaging. Fully independent of third-party video platforms (YouTube, Twitch, etc.).
@@ -142,11 +156,12 @@ graph TD
graph TB graph TB
subgraph "Cloudflare Edge" subgraph "Cloudflare Edge"
CF[CDN + WAF<br/>Cache Rules] CF[CDN + WAF<br/>Cache Rules]
R2[R2 Bucket<br/>media + HLS] R2["R2 Bucket<br/>media + HLS<br/>hls/r-{runID}/… run-scoped"]
WORKERS[Workers<br/>SPA hosting] MEDIA[linearvoid-media Worker<br/>R2-bound · media.linearvoid.tv]
WORKERS[linearvoid-frontend Worker<br/>SPA hosting]
end end
subgraph "Docker Host (Hetzner CX22)" subgraph "Docker Host"
NGINX[nginx<br/>:80/:443<br/>proxy + auth] NGINX[nginx<br/>:80/:443<br/>proxy + auth]
CHAT[chat service<br/>:3000<br/>Go + gorilla/websocket] CHAT[chat service<br/>:3000<br/>Go + gorilla/websocket]
PLAYOUT[playout service<br/>:8081<br/>Go + FFmpeg] PLAYOUT[playout service<br/>:8081<br/>Go + FFmpeg]
@@ -159,13 +174,14 @@ graph TB
ADMIN[Admin Browser<br/>SPA + uploads] ADMIN[Admin Browser<br/>SPA + uploads]
end end
VIEWER -->|HTTPS| CF VIEWER -->|HTTPS SPA| CF
VIEWER -->|WSS| NGINX VIEWER -->|WSS Chat| NGINX
VIEWER -->|HTTPS /hls| MEDIA
ADMIN -->|HTTPS| CF ADMIN -->|HTTPS| CF
CF --> NGINX CF --> NGINX
MEDIA --> R2
NGINX --> CHAT NGINX --> CHAT
NGINX --> PLAYOUT NGINX --> PLAYOUT
NGINX -->|/hls/*| R2
CHAT --> REDIS CHAT --> REDIS
CHAT --> DB CHAT --> DB
PLAYOUT --> R2 PLAYOUT --> R2
@@ -175,7 +191,8 @@ graph TB
**Playout Service** (`backend/playout/src/playlist.go`) **Playout Service** (`backend/playout/src/playlist.go`)
- Role: continuous HLS encoding loop via FFmpeg concat demuxer. - Role: continuous HLS encoding loop via FFmpeg concat demuxer.
- Input: `data/playlist/playlist.txt` (ordered MP4 paths); Output: HLS segments (`.m4s`, `.ts`) + live `.m3u8` to `data/hls/`, mirrored to R2. - Input: `data/playlist/playlist.txt` (ordered MP4 paths); Output: HLS segments (`.m4s`, `.ts`) + live `.m3u8` to `data/hls/`, mirrored to R2 run-scoped under `hls/r-{runID}/…`.
- Run-scoped manifest: before publishing the live `hls/playlist.m3u8`, `publishManifest` rewrites bare media URIs to the current run folder (`rewriteManifestToRun`), so manifest → segment links resolve on `media.linearvoid.tv`. See [`STREAMING.md`](../STREAMING.md).
- Resilience: restart counter (>5/60s → slate); Prometheus on 8081. - Resilience: restart counter (>5/60s → slate); Prometheus on 8081.
- Validation: ffprobe gate (codec/fps/resolution/keyframe); `-validate` mode checks playlist without running. - Validation: ffprobe gate (codec/fps/resolution/keyframe); `-validate` mode checks playlist without running.
- Seamless looping: resets HLS only when playlist fingerprint (SHA-256) changes, crash/slate recovery, or first run; else continues via `-start_number`. - Seamless looping: resets HLS only when playlist fingerprint (SHA-256) changes, crash/slate recovery, or first run; else continues via `-start_number`.
@@ -253,7 +270,7 @@ graph TB
1. Admin adds normalized MP4 to R2 + updates playlist. 1. Admin adds normalized MP4 to R2 + updates playlist.
2. Playout worker (FFmpeg concat loop) reads next file, packages HLS segments (2s). 2. Playout worker (FFmpeg concat loop) reads next file, packages HLS segments (2s).
3. Segments uploaded to R2 with unique immutable names; live `.m3u8` updated. 3. Segments uploaded to R2 with unique immutable, run-scoped names (`hls/r-{runID}/…`); the live `.m3u8` is published referencing those run-prefixed URIs (see `STREAMING.md`).
4. Viewer loads SPA → requests manifest → hls.js follows current edge. 4. Viewer loads SPA → requests manifest → hls.js follows current edge.
5. Cloudflare serves `.m4s`/`.ts` from nearest edge cache (origin hit once per POP/region). 5. Cloudflare serves `.m4s`/`.ts` from nearest edge cache (origin hit once per POP/region).
6. Parallel WebSocket to chat service for synchronized interaction. 6. Parallel WebSocket to chat service for synchronized interaction.
@@ -386,7 +403,7 @@ _Cache rules baseline_: `.ts`/`.m4s` Cache Everything, long TTL, immutable; `.m3
_Edge security/controls_: enforce HTTPS/WSS + redirect HTTP; WAF baseline; rate limits on abusive patterns (auth/admin); minimize origin exposure. _Edge security/controls_: enforce HTTPS/WSS + redirect HTTP; WAF baseline; rate limits on abusive patterns (auth/admin); minimize origin exposure.
_Worker & header policy_: Worker logic for immutable cache headers/request normalization where cache rules insufficient (`infrastructure/cloudflare/worker.js`); explicit CORS for HLS. _Worker & header policy_: the `linearvoid-media` Worker ([`infrastructure/cloudflare/`](../../infrastructure/cloudflare/), R2-bound) owns `media.linearvoid.tv` and enforces immutable caching for segments, short TTL for live manifests, plus HLS CORS/Range; it is a faithful path→R2 mapper and does **not** rewrite manifest URIs at serve time (playout rewrites them to run-scoped paths at publish — see [`STREAMING.md`](../STREAMING.md)). The `linearvoid-frontend` Worker ([`frontend/wrangler.toml`](../../frontend/wrangler.toml)) serves the static SPA on `linearvoid.tv`. Automated deploys: `workers-deploy.yml` (SPA) and `workers-media-deploy.yml` (media). Ops detail: [`cloudflare-workers-setup.md`](../operations/cloudflare-workers-setup.md).
_Operations & validation_: verify cache status progression (MISS→HIT), manifest freshness under active playout; track hit ratio, request/error rates, ToS/cost posture. _Operations & validation_: verify cache status progression (MISS→HIT), manifest freshness under active playout; track hit ratio, request/error rates, ToS/cost posture.
+17 -12
View File
@@ -3,31 +3,36 @@
Linear Void uses Cloudflare Workers for two purposes: Linear Void uses Cloudflare Workers for two purposes:
| Product | Purpose | What | | Product | Purpose | What |
| ----------- | ------------------ | ------------------------------------------------------------------------------------ | | ----------- | ------------------ | -------------------------------------------------------------------------------------------- |
| **Workers** | Static SPA hosting | Vite-built frontend (`frontend/dist`) served as Worker static assets | | **Workers** | Static SPA hosting | `linearvoid-frontend` (`frontend/wrangler.toml`) — Vite SPA (`frontend/dist`) static assets |
| **Workers** | Edge cache logic | `infrastructure/cloudflare/worker.js` — adds immutable cache headers to HLS segments | | **Workers** | Edge HLS delivery | `linearvoid-media` (`infrastructure/cloudflare/`) — R2-bound Worker on `media.linearvoid.tv` |
Both are managed under **Workers & Pages** in the Cloudflare dashboard. Both are managed under **Workers & Pages** in the Cloudflare dashboard.
## Dashboard URLs ## Dashboard URLs
| Resource | URL | | Resource | URL |
| ------------------------ | ------------------------------------------------------------------------------------------------------------------- | | ------------------------ | ------------------------------------------------------------------------------------------------------------------------- |
| Workers & Pages overview | `https://dash.cloudflare.com/deb11491c54ec53a24772e98ac6d8128/workers-and-pages` | | Workers & Pages overview | `https://dash.cloudflare.com/deb11491c54ec53a24772e98ac6d8128/workers-and-pages` |
| Worker: linearvoid | `https://dash.cloudflare.com/deb11491c54ec53a24772e98ac6d8128/workers/services/view/linearvoid/production/settings` | | Worker: linearvoid-media | `https://dash.cloudflare.com/deb11491c54ec53a24772e98ac6d8128/workers/services/view/linearvoid-media/production/settings` |
| API tokens | <https://dash.cloudflare.com/profile/api-tokens> | | API tokens | <https://dash.cloudflare.com/profile/api-tokens> |
## Worker: `linearvoid` (edge cache headers) ## Worker: `linearvoid-media` (edge HLS delivery)
The Worker at `infrastructure/cloudflare/worker.js` runs on the `linearvoid` Workers service. It intercepts requests and adds immutable `Cache-Control` headers to `.m4s` segment files. The Worker in [`infrastructure/cloudflare/`](../../infrastructure/cloudflare/) runs on the `linearvoid-media` Workers service. It owns the `media.linearvoid.tv` host, serves HLS objects from the R2 bucket (bound as `MEDIA_BUCKET`), and stamps immutable `Cache-Control` on segments plus a short TTL on live manifests (CORS + Range supported). See its [README](../../infrastructure/cloudflare/README.md).
Deploy via Wrangler CLI: **Run-scoped layout.** R2 objects are stored run-scoped (`hls/r-{runID}/seg-*.m4s`, `…/init.mp4`), and the live `hls/playlist.m3u8` references them with the run prefix (the playout rewrites bare URIs before publishing — see [`docs/STREAMING.md`](../STREAMING.md)). The Worker is a faithful path→R2 mapper: it serves whatever key the request path resolves to (e.g. `/hls/r-…/seg-*.m4s`), adds the cache headers by file type, and does not rewrite manifest URIs at serve time.
Test / deploy:
```bash ```bash
npx wrangler deploy infrastructure/cloudflare/worker.js cd infrastructure/cloudflare
npm install && npm test # unit + Miniflare R2 tests
npx wrangler deploy # preview/test target
npx wrangler deploy --env production # attach media.linearvoid.tv
``` ```
Or paste into the Workers dashboard at the URL above (Quick Edit). Attach the production custom domain only after removing `media.linearvoid.tv` from the R2 bucket's direct custom-domain binding. Automated production deploy: [`.gitea/workflows/workers-media-deploy.yml`](../../.gitea/workflows/workers-media-deploy.yml).
## Worker: static frontend ## Worker: static frontend
@@ -56,9 +61,9 @@ Backend services (playout, chat, nginx) read their own runtime variables (`ADMIN
## Custom domains ## Custom domains
- `linearvoid.tv` → Worker (static SPA) - `linearvoid.tv` → Worker (static SPA, `linearvoid-frontend`)
- `www.linearvoid.tv` → redirect to `linearvoid.tv` - `www.linearvoid.tv` → redirect to `linearvoid.tv`
- `media.linearvoid.tv` → R2 bucket (separate; not a Worker) - `media.linearvoid.tv` → Worker (`linearvoid-media`, R2-bound) serving the `linearvoid-media` bucket
## Related ## Related
+1 -1
View File
@@ -16,7 +16,7 @@ Deploy the Linear Void frontend (Vite + TypeScript SPA) from Gitea to Cloudflare
3. **Account ID** from Cloudflare dashboard overview → store as `CLOUDFLARE_ACCOUNT_ID` 3. **Account ID** from Cloudflare dashboard overview → store as `CLOUDFLARE_ACCOUNT_ID`
4. **Wrangler config** (`frontend/wrangler.toml`) binding the `frontend/dist` output as static assets. 4. **Wrangler config** (`frontend/wrangler.toml`) binding the `frontend/dist` output as static assets.
> **Naming conflict:** Workers share a namespace under "Workers & Pages". The edge-cache Worker (`infrastructure/cloudflare/worker.js`) already uses the `linearvoid` name, so the SPA Worker is named `linearvoid-frontend` (see `frontend/wrangler.toml`). > **Naming:** Workers share a namespace under "Workers & Pages". The media edge Worker is `linearvoid-media` ([`infrastructure/cloudflare/`](../../infrastructure/cloudflare/wrangler.toml), deployed via `.gitea/workflows/workers-media-deploy.yml`) and the SPA Worker is `linearvoid-frontend` (`frontend/wrangler.toml`, deployed via `.gitea/workflows/workers-deploy.yml`).
## Project-specific values ## Project-specific values
+2 -2
View File
@@ -1,12 +1,12 @@
{ {
"name": "frontend", "name": "frontend",
"version": "1.0.11", "version": "1.1.0",
"lockfileVersion": 3, "lockfileVersion": 3,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "frontend", "name": "frontend",
"version": "1.0.11", "version": "1.1.0",
"dependencies": { "dependencies": {
"hls.js": "^1.6.16" "hls.js": "^1.6.16"
}, },
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "frontend", "name": "frontend",
"private": true, "private": true,
"version": "1.0.13", "version": "1.1.0",
"type": "module", "type": "module",
"engines": { "engines": {
"node": ">=22.0.0" "node": ">=22.0.0"
+5
View File
@@ -0,0 +1,5 @@
node_modules/
.wrangler/
dist/
coverage/
*.log
+65
View File
@@ -0,0 +1,65 @@
# linearvoid-media Worker
Cloudflare Worker that owns **`media.linearvoid.tv`** and serves the HLS objects
stored in the `linearvoid-media` R2 bucket, applying the edge cache policy in code:
| Object | Cache-Control |
| ------------------------------------------- | ------------------------------------------------ |
| `.m4s` / `.ts` / `.mp4` / `.m4a` (segments) | `public, max-age=31536000, immutable` |
| `.m3u8` (live manifest) | `public, max-age=1, s-maxage=2, must-revalidate` |
It also adds HLS CORS (`Access-Control-Allow-Origin: *`), OPTIONS preflight, a
`/health` probe, and Range-request support. This worker replaces direct R2
custom-domain serving on the media host so cache headers/CORS are controlled in
code (cache rules on the same host remain the origin baseline).
## Layout
```text
infrastructure/cloudflare/
├── wrangler.toml # name/binding + production custom-domain route
├── package.json
├── src/worker.js # worker module (exports default.fetch + pure helpers)
└── test/ # Vitest unit tests + Miniflare (workerd) R2 tests
```
## Local dev / test
```bash
cd infrastructure/cloudflare
npm install
npm test # unit + Miniflare R2 integration tests
npx wrangler dev # run locally against your account's R2 bucket
```
## Deploy
**Preview / test target** (default environment → `*.workers.dev`):
```bash
cd infrastructure/cloudflare
npx wrangler deploy
```
**Production** (attaches `media.linearvoid.tv` as a custom domain on this worker):
```bash
cd infrastructure/cloudflare
npx wrangler deploy --env production
```
Both need `CLOUDFLARE_API_TOKEN` (Workers Scripts/Routes edit) and
`CLOUDFLARE_ACCOUNT_ID` in the environment. The media host must first be removed
from the R2 bucket's direct custom-domain binding, otherwise the route collides.
Automated production deploys run via
[`.gitea/workflows/workers-media-deploy.yml`](../../.gitea/workflows/workers-media-deploy.yml)
after CI succeeds on `main`.
## Verify
```bash
curl -I https://media.linearvoid.tv/hls/playlist.m3u8 # short TTL
curl -I https://media.linearvoid.tv/hls/<segment>.m4s # immutable + CORS
curl https://media.linearvoid.tv/health # {"status":"ok",...}
```
File diff suppressed because it is too large. Load diff
+20
View File
@@ -0,0 +1,20 @@
{
"name": "linearvoid-media-worker",
"version": "1.0.0",
"private": true,
"description": "Linear Void media edge worker: serves HLS from R2 on media.linearvoid.tv with immutable segment caching.",
"type": "module",
"scripts": {
"dev": "wrangler dev",
"start": "wrangler dev",
"deploy": "wrangler deploy",
"deploy:production": "wrangler deploy --env production",
"test": "vitest run",
"test:watch": "vitest"
},
"devDependencies": {
"miniflare": "^4.0.0",
"vitest": "^4.1.10",
"wrangler": "^4.0.0"
}
}
+144
View File
@@ -0,0 +1,144 @@
// Linear Void media edge worker (linearvoid-media).
//
// Owns the media.linearvoid.tv host and serves HLS objects from the R2 bucket
// bound as MEDIA_BUCKET. Its purpose (arc42 quality goal #1, cost efficiency) is
// to guarantee immutable, long-lived caching of media segments at the edge while
// keeping live manifests fresh — the caching baseline is also configured as
// Cloudflare cache rules on the same host, but this worker enforces it in code.
//
// Cache policy (matches docs/infrastructure/cloudflare/cache-rules.md + STREAMING.md):
// .m4s / .ts / .mp4 / .m4a -> public, max-age=31536000, immutable
// .m3u8 (live manifest) -> public, max-age=1, s-maxage=2, must-revalidate
const IMMUTABLE_CACHE = "public, max-age=31536000, immutable";
const MANIFEST_CACHE = "public, max-age=1, s-maxage=2, must-revalidate";
const DEFAULT_CACHE = "public, max-age=60";
const IMMUTABLE_SUFFIXES = [".m4s", ".ts", ".mp4", ".m4a"];
const MANIFEST_SUFFIXES = [".m3u8"];
const CONTENT_TYPES = {
".m3u8": "application/vnd.apple.mpegurl",
".m4s": "video/mp4",
".mp4": "video/mp4",
".ts": "video/MP2T",
".m4a": "audio/mp4",
};
/** Map a request path to its edge Cache-Control value. Pure + testable. */
export function cacheControlForPath(pathname) {
const lower = pathname.toLowerCase();
for (const suffix of MANIFEST_SUFFIXES) {
if (lower.endsWith(suffix)) return MANIFEST_CACHE;
}
for (const suffix of IMMUTABLE_SUFFIXES) {
if (lower.endsWith(suffix)) return IMMUTABLE_CACHE;
}
return DEFAULT_CACHE;
}
/** Map a request path to a browser-friendly Content-Type. Pure + testable. */
export function contentTypeForPath(pathname) {
const lower = pathname.toLowerCase();
for (const [suffix, type] of Object.entries(CONTENT_TYPES)) {
if (lower.endsWith(suffix)) return type;
}
return "application/octet-stream";
}
const CORS_HEADERS = {
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Methods": "GET, HEAD, OPTIONS",
"Access-Control-Allow-Headers": "Range, If-None-Match, If-Modified-Since",
"Access-Control-Max-Age": "86400",
};
function json(body, status = 200) {
return new Response(JSON.stringify(body), {
status,
headers: { "Content-Type": "application/json", ...CORS_HEADERS },
});
}
// Parse a single, closed byte range ("bytes=start-end") into an R2 range object.
// Returns null for open/suffix/multi/invalid ranges — the caller then serves the
// full object (HTTP 200), which is valid per RFC 7233 (a server may ignore Range).
export function parseByteRange(header) {
if (!header) return null;
const m = /^bytes=(\d*)-(\d*)$/.exec(header.trim());
if (!m) return null;
const start = m[1];
const end = m[2];
if (start === "" || end === "") return null; // open or suffix range
const offset = Number(start);
const length = Number(end) - offset + 1;
if (!Number.isInteger(offset) || !Number.isInteger(length) || offset < 0 || length < 1) {
return null;
}
return { offset, length };
}
export default {
async fetch(request, env) {
const url = new URL(request.url);
// CORS preflight for HLS media.
if (request.method === "OPTIONS") {
return new Response(null, { status: 204, headers: CORS_HEADERS });
}
// Liveness probe used by uptime/monitoring (see cloudflare-uptimerobot-checklist).
if (url.pathname === "/health") {
return json({ status: "ok", service: "linearvoid-media" });
}
if (request.method !== "GET" && request.method !== "HEAD") {
return json({ error: "method not allowed" }, 405);
}
const key = decodeURIComponent(url.pathname).replace(/^\/+/, "");
if (!key) {
return json({ error: "not found" }, 404);
}
if (!env.MEDIA_BUCKET) {
return json({ error: "media bucket not configured" }, 500);
}
// Serve from R2, honouring single closed Range requests (used by some hls.js
// / player paths for byte-range fetches). Unsupported ranges fall back to a
// full-body 200, which clients accept.
const rangeHeader = request.headers.get("range");
const parsedRange = parseByteRange(rangeHeader);
const object = parsedRange
? await env.MEDIA_BUCKET.get(key, { range: parsedRange })
: await env.MEDIA_BUCKET.get(key);
if (object === null) {
return json({ error: "not found" }, 404);
}
const headers = new Headers();
object.writeHttpMetadata(headers);
headers.set("Cache-Control", cacheControlForPath(url.pathname));
headers.set("Content-Type", contentTypeForPath(url.pathname));
headers.set("ETag", object.httpEtag);
headers.set("Accept-Ranges", "bytes");
headers.set("Access-Control-Allow-Origin", "*");
let status = 200;
if (parsedRange && object.range) {
status = 206;
const { offset, length } = object.range;
headers.set(
"Content-Range",
`bytes ${offset}-${offset + length - 1}/${object.size}`
);
}
return new Response(request.method === "HEAD" ? null : object.body, {
status,
headers,
});
},
};
@@ -0,0 +1,77 @@
import { Miniflare } from "miniflare";
import { beforeAll, afterAll, describe, it, expect } from "vitest";
const IMMUTABLE = "public, max-age=31536000, immutable";
const MANIFEST = "public, max-age=1, s-maxage=2, must-revalidate";
// Real-workerd integration test: boots the worker with an in-memory R2 bucket and
// dispatches requests through the fetch handler.
let mf;
beforeAll(async () => {
mf = new Miniflare({
modules: true,
scriptPath: "src/worker.js",
r2Buckets: { MEDIA_BUCKET: "linearvoid-media" },
});
const bucket = await mf.getR2Bucket("MEDIA_BUCKET");
await bucket.put("hls/playlist.m3u8", "#EXTM3U\n#EXT-X-ENDLIST\n", {
httpMetadata: { contentType: "application/vnd.apple.mpegurl" },
});
await bucket.put("hls/seg-000001.m4s", new Uint8Array([0, 1, 2, 3, 4]));
await bucket.put("hls/init.mp4", new Uint8Array([9, 9, 9]));
});
afterAll(async () => {
await mf?.dispose();
});
const base = "http://media.linearvoid.tv";
describe("linearvoid-media worker (R2)", () => {
it("serves a media segment with immutable cache headers", async () => {
const res = await mf.dispatchFetch(`${base}/hls/seg-000001.m4s`);
expect(res.status).toBe(200);
expect(res.headers.get("cache-control")).toBe(IMMUTABLE);
expect(res.headers.get("content-type")).toBe("video/mp4");
expect(res.headers.get("access-control-allow-origin")).toBe("*");
expect(await res.arrayBuffer()).toEqual(new Uint8Array([0, 1, 2, 3, 4]).buffer);
});
it("serves a live manifest with a short TTL", async () => {
const res = await mf.dispatchFetch(`${base}/hls/playlist.m3u8`);
expect(res.status).toBe(200);
expect(res.headers.get("cache-control")).toBe(MANIFEST);
expect(res.headers.get("content-type")).toBe("application/vnd.apple.mpegurl");
});
it("honours Range requests with 206 + Content-Range", async () => {
const res = await mf.dispatchFetch(`${base}/hls/seg-000001.m4s`, {
headers: { Range: "bytes=0-2" },
});
expect(res.status).toBe(206);
expect(res.headers.get("content-range")).toBe("bytes 0-2/5");
const bytes = new Uint8Array(await res.arrayBuffer());
expect([...bytes]).toEqual([0, 1, 2]);
});
it("returns 404 for a missing object", async () => {
const res = await mf.dispatchFetch(`${base}/hls/does-not-exist.m4s`);
expect(res.status).toBe(404);
});
it("answers OPTIONS preflight with CORS headers", async () => {
const res = await mf.dispatchFetch(`${base}/hls/seg-000001.m4s`, {
method: "OPTIONS",
});
expect(res.status).toBe(204);
expect(res.headers.get("access-control-allow-origin")).toBe("*");
});
it("rejects non-GET/HEAD methods", async () => {
const res = await mf.dispatchFetch(`${base}/hls/seg-000001.m4s`, {
method: "POST",
});
expect(res.status).toBe(405);
});
});
@@ -0,0 +1,54 @@
import { describe, it, expect } from "vitest";
import { cacheControlForPath, contentTypeForPath, parseByteRange } from "../src/worker.js";
const IMMUTABLE = "public, max-age=31536000, immutable";
const MANIFEST = "public, max-age=1, s-maxage=2, must-revalidate";
const DEFAULT = "public, max-age=60";
describe("cacheControlForPath", () => {
it("marks media segments as immutable", () => {
expect(cacheControlForPath("/hls/seg-000001.m4s")).toBe(IMMUTABLE);
expect(cacheControlForPath("/hls/init.mp4")).toBe(IMMUTABLE);
expect(cacheControlForPath("/hls/seg-000001.ts")).toBe(IMMUTABLE);
expect(cacheControlForPath("/media/audio.m4a")).toBe(IMMUTABLE);
});
it("keeps live manifests short-lived", () => {
expect(cacheControlForPath("/hls/playlist.m3u8")).toBe(MANIFEST);
expect(cacheControlForPath("/hls/master.m3u8")).toBe(MANIFEST);
});
it("falls back to a sane default for other paths", () => {
expect(cacheControlForPath("/hls/unknown.txt")).toBe(DEFAULT);
});
});
describe("contentTypeForPath", () => {
it("maps known media types", () => {
expect(contentTypeForPath("/hls/playlist.m3u8")).toBe("application/vnd.apple.mpegurl");
expect(contentTypeForPath("/hls/seg-000001.m4s")).toBe("video/mp4");
expect(contentTypeForPath("/hls/init.mp4")).toBe("video/mp4");
expect(contentTypeForPath("/hls/seg-000001.ts")).toBe("video/MP2T");
expect(contentTypeForPath("/media/audio.m4a")).toBe("audio/mp4");
});
it("falls back to octet-stream", () => {
expect(contentTypeForPath("/hls/unknown.bin")).toBe("application/octet-stream");
});
});
describe("parseByteRange", () => {
it("parses a closed byte range", () => {
expect(parseByteRange("bytes=0-2")).toEqual({ offset: 0, length: 3 });
expect(parseByteRange("bytes=10-19")).toEqual({ offset: 10, length: 10 });
});
it("rejects open, suffix, multi, or malformed ranges", () => {
expect(parseByteRange("bytes=0-")).toBeNull();
expect(parseByteRange("bytes=-500")).toBeNull();
expect(parseByteRange("bytes=0-1,3-4")).toBeNull();
expect(parseByteRange("items=0-2")).toBeNull();
expect(parseByteRange("")).toBeNull();
expect(parseByteRange(null)).toBeNull();
});
});
-21
View File
@@ -1,21 +0,0 @@
// Cloudflare Worker for advanced cache rules and WAF (ADR-001)
addEventListener("fetch", (event) => {
event.respondWith(handleRequest(event.request));
});
async function handleRequest(request) {
const url = new URL(request.url);
// Add immutable cache headers for .m4s segments
if (url.pathname.endsWith(".m4s")) {
const response = await fetch(request);
const headers = new Headers(response.headers);
headers.set("Cache-Control", "public, max-age=31536000, immutable");
return new Response(response.body, {
status: response.status,
headers,
});
}
return fetch(request);
}
+31
View File
@@ -0,0 +1,31 @@
# Cloudflare Worker: linearvoid-media (edge HLS delivery)
# Serves HLS objects from the `linearvoid-media` R2 bucket on media.linearvoid.tv.
# Applies immutable caching to media segments and a short TTL to live manifests.
# This worker replaces direct R2 custom-domain serving on the media host so cache
# headers/CORS are controlled in code (cache rules remain as the origin baseline).
name = "linearvoid-media"
main = "src/worker.js"
compatibility_date = "2025-08-01"
# R2 bucket holding the HLS objects. The request path is mirrored onto the bucket,
# e.g. GET /hls/playlist.m3u8 -> object key `hls/playlist.m3u8`.
[[r2_buckets]]
binding = "MEDIA_BUCKET"
bucket_name = "linearvoid-media"
# Default environment: deploy to the preview *.workers.dev URL (test target).
# npx wrangler deploy
#
# Production environment: attach the media host as a custom domain on this worker.
# Note: bindings are NOT inherited by environments, so r2_buckets is redeclared here.
# npx wrangler deploy --env production
[env.production]
routes = [
{ pattern = "media.linearvoid.tv", custom_domain = true }
]
[[env.production.r2_buckets]]
binding = "MEDIA_BUCKET"
bucket_name = "linearvoid-media"