2 Commits
Author SHA1 Message Date
zwitschi d8cf853842 chore(release): 1.1.0
Release / release (push) Successful in 1m15s
CI / test (push) Successful in 2m53s
CI / docker (push) Successful in 2m0s
- playout: run-scoped R2 manifest URIs (candidate a rewrite via rewriteManifestToRun)
- tests: playlist_rewrite_test.go (rewrite + publish integration + slate guard)
- docs: STREAMING, cloudflare-workers-setup, architecture (incl. Level-2 diagram)
- release housekeeping: TODO->DONE, CHANGELOG 1.1.0, version bump
2026-09-03 09:27:38 +02:00
zwitschi 8c5e70959b feat(cloudflare): add linearvoid-media edge worker for R2 HLS delivery
Replace the ad-hoc `worker.js` with a self-contained `linearvoid-media`
worker under `infrastructure/cloudflare/` that owns `media.linearvoid.tv`.
It serves HLS objects from the R2 bucket, enforcing immutable Cache-Control
on segments, a short TTL on live manifests, plus HLS CORS, OPTIONS
preflight, Range support, and a `/health` probe. Includes wrapper config
with production domain, Vitest unit tests, Miniflare R2 integration tests,
automated deploy workflow, and docs updates.
2026-09-03 08:52:31 +02:00
24 changed files with 3881 additions and 202 deletions

No files matched your search

+35
View File
@@ -0,0 +1,35 @@
name: Deploy Media Worker to Cloudflare
on:
workflow_run:
workflows: [CI]
types: [completed]
workflow_dispatch: {}
jobs:
deploy:
# Production deploy: only on pushes to main (via CI success) or manual main runs.
if: ${{ (github.event_name == 'workflow_dispatch' && github.ref == 'refs/heads/main') || (github.event_name == 'workflow_run' && github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.head_branch == 'main') }}
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: 22
- name: Install dependencies
run: cd infrastructure/cloudflare && npm ci
- name: Test worker
run: cd infrastructure/cloudflare && npm test
- name: Deploy to Cloudflare Workers (production)
env:
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
run: |
cd infrastructure/cloudflare
npx wrangler deploy --env production
+39
View File
@@ -2,6 +2,13 @@
> Completed tasks from [TODO](./TODO.md). Product, feature, and release history lives in [CHANGELOG](./CHANGELOG.md); this file keeps only engineering and ops notes not captured there.
## Manifest run-scoped rewrite — candidate (a) (2026-09-03)
- Playout now publishes the live R2 manifest with run-scoped media URIs. `rewriteManifestToRun` in `backend/playout/src/playlist.go` prefixes each bare relative URI (`seg-*.m4s` / `init.mp4`, plus `#EXT-X-MAP:URI`) with `r-{runID}/` before `publishManifest`'s atomic `.tmp` → `CopyObject` commit; skipped in slate mode. Added a stubbable `copyObject` hook for tests (consistent with `putObject`/`listObjects`/`deleteObjects`).
- Tests: `playlist_rewrite_test.go` — `TestRewriteManifestToRun`, `TestRewriteManifestToRun_AlreadyPrefixedUnchanged`, `TestRewriteManifestToRun_HandlesTSAndComments`, `TestRewriteManifestToRun_EmptyRunID`, `TestIsMediaURI`, `TestPublishManifest_RewritesToRunScopedURIs`, `TestPublishManifest_SlateNotRewritten`.
- Docs updated: `STREAMING.md`, `operations/cloudflare-workers-setup.md`, `architecture/README.md` (incl. Level-2 diagram showing the R2-bound `linearvoid-media` Worker and run-scoped layout).
- **Still pending (in TODO): live Validation** — deploy the updated playout to the test lab and confirm manifest/segment URIs resolve via `media.linearvoid.tv`.
## Test infrastructure & refactors (2026-08-07)
- Chat service `main.go` split into 8 packages (config, auth, db, handlers, ws, media, ratelimit, metrics); `db.RedisClient` + `db.PubSub` interfaces added for mocking.
@@ -37,3 +44,35 @@
- Test in Firefox: live playback, rebuffer/recovery, chat sync; record quirks in `docs/STREAMING.md`
- Rebuild + run playout; confirm `playlist.m3u8` lists ~12 segments at 2s each
- Manual: verify viewer latency and cache-hit behavior after the change
## Chat sidebar (2026-08-17)
- [x] remove search bar from chat sidebar (not needed for single-room)
- [x] remove "user" toggle button from chat sidebar (hide for now, add back when user interface is stable)
- [x] make login/registration form a modal and only toggle it when user is not logged in and clicks "Login" or "Register"
- [x] when user is logged in, do not show login/registration form
- [x] make sure each message is only rendered once in the chat log (no duplicates on reconnect or initial load)
### Plan
Files: `frontend/index.html`, `frontend/src/main.ts`, `frontend/src/style.css`.
1. **Remove search bar** (single-room)
- `index.html`: delete the `<div id="dm-search-row">` block.
- `main.ts`: remove `searchDM()` and the `dm-search-btn` listener; drop the `#dm-search-row` show/hide logic in `switchRoom()` (the `searchRow.hidden = !isDM` lines). Keep `currentDMTarget` only if the "New DM" flow stays.
2. **Remove "Users" toggle button**
- `index.html`: delete the `<button id="users-toggle-btn">` button.
- `main.ts`: remove `toggleUserList()` and the `users-toggle-btn` listener; drop the now-unused `usersVisible` variable.
3. **Login/registration form as a modal**
- `index.html`: wrap the `#auth-panel` form in a modal structure (`.auth-modal` + backdrop) with a close button.
- `style.css`: add `.auth-modal` / `.auth-modal-backdrop` styles.
- `main.ts`: add `openAuthModal()` / `closeAuthModal()`; change the `auth-toggle-btn` handler to open the modal (only when not authenticated) instead of toggling `#auth-panel` inline; wire a close/backdrop-click to `closeAuthModal()`.
4. **Hide form when logged in**
- `main.ts` `updateAuthUI()`: when `authenticated` → hide the "Sign in" button and call `closeAuthModal()`; show the user label + "Sign out" button. When not authenticated → show "Sign in" and hide "Sign out".
5. **No duplicate messages**
- `main.ts`: add `clearMessages()` (empties `#messages`) and call it at the top of `connectWebSocket()` so each (re)connect renders history fresh instead of appending.
- Fix the initial double-connect: make the initial `connectWebSocket("main")` wait for `refreshUserSession()` (or connect once with the resolved username) so history isn't pushed twice on load.
+39 -57
View File
@@ -1,57 +1,39 @@
# TODO
> Completed tasks moved to [DONE](./DONE.md). Remaining tasks in [BACKLOG](./BACKLOG.md).
## Chat sidebar
- [x] remove search bar from chat sidebar (not needed for single-room)
- [x] remove "user" toggle button from chat sidebar (hide for now, add back when user interface is stable)
- [x] make login/registration form a modal and only toggle it when user is not logged in and clicks "Login" or "Register"
- [x] when user is logged in, do not show login/registration form
- [x] make sure each message is only rendered once in the chat log (no duplicates on reconnect or initial load)
### Plan
Files: `frontend/index.html`, `frontend/src/main.ts`, `frontend/src/style.css`.
1. **Remove search bar** (single-room)
- `index.html`: delete the `<div id="dm-search-row">` block.
- `main.ts`: remove `searchDM()` and the `dm-search-btn` listener; drop the `#dm-search-row` show/hide logic in `switchRoom()` (the `searchRow.hidden = !isDM` lines). Keep `currentDMTarget` only if the "New DM" flow stays.
2. **Remove "Users" toggle button**
- `index.html`: delete the `<button id="users-toggle-btn">` button.
- `main.ts`: remove `toggleUserList()` and the `users-toggle-btn` listener; drop the now-unused `usersVisible` variable.
3. **Login/registration form as a modal**
- `index.html`: wrap the `#auth-panel` form in a modal structure (`.auth-modal` + backdrop) with a close button.
- `style.css`: add `.auth-modal` / `.auth-modal-backdrop` styles.
- `main.ts`: add `openAuthModal()` / `closeAuthModal()`; change the `auth-toggle-btn` handler to open the modal (only when not authenticated) instead of toggling `#auth-panel` inline; wire a close/backdrop-click to `closeAuthModal()`.
4. **Hide form when logged in**
- `main.ts` `updateAuthUI()`: when `authenticated` → hide the "Sign in" button and call `closeAuthModal()`; show the user label + "Sign out" button. When not authenticated → show "Sign in" and hide "Sign out".
5. **No duplicate messages**
- `main.ts`: add `clearMessages()` (empties `#messages`) and call it at the top of `connectWebSocket()` so each (re)connect renders history fresh instead of appending.
- Fix the initial double-connect: make the initial `connectWebSocket("main")` wait for `refreshUserSession()` (or connect once with the resolved username) so history isn't pushed twice on load.
## Frontend Tests
### User Frontend
- [ ] Test coverage for all critical frontend components and interactions
- [ ] Add code coverage reporting and enforce minimum coverage threshold in CI/CD pipeline
- [ ] Document test cases and expected outcomes
### Admin Frontend
- [ ] Test coverage for all critical admin UI components and interactions
- [ ] Add code coverage reporting and enforce minimum coverage threshold in CI/CD pipeline
- [ ] Document test cases and expected outcomes
## Backend Integration Tests
### Cloudflare R2
- [ ] Test retrieval of segments and manifests from R2 via Cloudflare edge
- [ ] Validate cache hit/miss behavior and performance metrics
- [ ] Validate edge cache hit/miss behavior with repeat requests — script `scripts/validate-edge-cache.sh` created
# TODO
> Completed tasks moved to [DONE](./DONE.md). Remaining tasks in [BACKLOG](./BACKLOG.md).
## Playout / R2 manifest–segment layout mismatch
**Chosen approach: candidate (a)** — rewrite the live manifest so its media URIs point into the current run-scoped folder before the atomic R2 publish. Go change in `backend/playout/src/playlist.go`. The `linearvoid-media` Worker stays a faithful path→R2 mapper (no serve-time rewrite).
### Remaining — Validation
- [ ] Run playout (local or test lab), then:
- `curl https://media.linearvoid.tv/hls/playlist.m3u8` → 200, short TTL.
- Every URI in the manifest resolves `200` with `Cache-Control: public, max-age=31536000, immutable` (via the `linearvoid-media` Worker).
- Confirm manifest refresh + `CF-Cache-Status` progression and no 404 on segment fetch across a run loop.
- [ ] Verify slate fallback still serves correctly after the rewrite change.
> Implementation (candidate a), Go unit tests, and documentation are complete — see [DONE.md](./DONE.md).
## Frontend Tests
### User Frontend
- [ ] Test coverage for all critical frontend components and interactions
- [ ] Add code coverage reporting and enforce minimum coverage threshold in CI/CD pipeline
- [ ] Document test cases and expected outcomes
### Admin Frontend
- [ ] Test coverage for all critical admin UI components and interactions
- [ ] Add code coverage reporting and enforce minimum coverage threshold in CI/CD pipeline
- [ ] Document test cases and expected outcomes
## Backend Integration Tests
### Cloudflare R2
- [ ] Test retrieval of segments and manifests from R2 via Cloudflare edge
- [ ] Validate cache hit/miss behavior and performance metrics
- [ ] Validate edge cache hit/miss behavior with repeat requests — script `scripts/validate-edge-cache.sh` created
+17 -22
View File
@@ -2,40 +2,35 @@
<!-- markdownlint-disable MD024 -->
## [Unreleased]
## [1.1.0] - 2026-09-03
### Added
- **Bash equivalents of Windows scripts**: Added `scripts/cache-hit-ratio.sh`, `scripts/check-env.sh`, `scripts/check-tokens.sh`, and `scripts/loadtest/run-loadtests.sh`, `scripts/loadtest/summarize-artillery.sh`, `scripts/loadtest/summarize-k6.sh` — Linux/macOS ports of the existing PowerShell scripts (the `.ps1` files are retained).
- **Transient presence flash**: Join/leave announcements now appear as a 3-second flash at the top of the chat panel (`#chat-flash`) with a fade-out, instead of permanent chat rows. Presence indicators still update from the same events.
- **Cloudflare media edge worker (`linearvoid-media`)**: new self-contained project under `infrastructure/cloudflare/` — an R2-bound Worker owning `media.linearvoid.tv` that enforces immutable segment caching (`public, max-age=31536000, immutable`) and short-TTL live `.m3u8`, with HLS CORS/OPTIONS, byte-Range, and a `/health` probe. Includes Vitest unit tests + Miniflare/workerd R2 integration tests and production deploy via `.gitea/workflows/workers-media-deploy.yml`. Replaces the ad-hoc `worker.js`.
- **Bash script ports**: Linux/macOS equivalents of the Windows ops/load-test scripts (`cache-hit-ratio`, `check-env`, `check-tokens`, loadtest run/summarize); `.ps1` files retained.
- **Transient presence flash**: join/leave events show as a 3-second `#chat-flash` instead of permanent chat rows.
### Changed
- **Coolify host-port de-collision**: Published playout on host port `8081` (`${PLAYOUT_PORT:-8081}:8081`) and moved the nginx host port from `8080`/`80` to a free `8082` (`${NGINX_PORT:-8082}:80`) in both `docker-compose.yml` and `docker-compose.prod.yml`, so no service binds ports reserved by the Coolify environment (Traefik proxy `80`/`443`, dashboard/Portainer `8000`, and the taken `8080`). `NGINX_PORT` default in `.env.example` updated from `80` to `8082`.
- **Docs condensed and de-Windows'd**: Removed Windows/PowerShell-specific instructions from the docs and condensed the longest documents into tighter, bash-first guides — `SETUP.md`, `DEPLOYMENT.md`, `API.md`, `MONITORING.md`, `STREAMING.md`, `COOLIFY.md`, `incident-runbook.md`, `operations/scaling-performance-playbook.md`, `architecture/README.md`, and the Prometheus metrics inventory. `scripts/loadtest/README.md` now documents the bash load-test helpers.
- **System messages no longer persisted**: The chat backend no longer stores `system`-type messages; chat history and the admin chat-log table only contain real chat messages (moderation actions remain in the audit log).
- **Reconnect grace period**: Disconnects schedule a delayed "left" announcement (5s); a reconnect within 5s cancels it and skips "joined", eliminating join/leave spam on brief reconnects.
- **HLS segment duration 2s**: Default `HLS_SEGMENT_DURATION` changed from 6 to 2 and `HLS_WINDOW_SIZE` from 24 to 12 (2s × 12 = 24s live window), reducing end-to-end latency; the 2s GOP was already in place.
- **Chat font sizing**: `.message-text` set to `0.95rem` (slightly smaller than the default, larger than the `0.83rem` system/flash text).
- **R2 retention by run count + size**: Playout now keeps the newest `R2_KEEP_RUNS` run folders (default 2) and enforces an `R2_MAX_RETENTION_BYTES` cap (default 10 GiB), replacing the old 24h time-based `R2_SEGMENT_RETENTION_HOURS` cleanup. Cleanup runs after each new run plus an hourly backstop, and never deletes the live manifest. Adds `linearvoid_playout_r2_runs_retained` and `linearvoid_playout_r2_bytes_retained` gauges.
- **Cloudflare Workers frontend hosting**: Replaced Cloudflare Pages with a Cloudflare Worker (`linearvoid-frontend`) that serves the static SPA via Workers Static Assets. Added `frontend/wrangler.toml` and renamed the deploy workflow to `.gitea/workflows/workers-deploy.yml` (builds the SPA and runs `npx wrangler deploy`). Architecture, setup/deployment, and ops documentation updated from Pages to Workers.
- **Viewer session cookie cross-site**: The `linearvoid_user_session` cookie now uses `SameSite=None; Secure` over HTTPS (so it is sent cross-site from the Workers-served SPA) and `SameSite=Lax` over plain HTTP (local dev). The admin session cookie remains `SameSite=Strict`.
- **Parallel playout startup validation**: `ValidatePlaylist` now deduplicates repeated paths and probes unique files concurrently through a bounded worker pool (`PLAYOUT_VALIDATE_WORKERS`, default `runtime.NumCPU()`); `ValidateVideo` runs its stream/format and keyframe `ffprobe` probes concurrently; and `main()` performs the strict validation once (removing the redundant `-validate` pre-step in `start-playout.sh`). This sharply cuts startup time on large or duplicated playlists.
- **Chat sidebar cleanup**: Removed the DM search bar and the "Users" toggle button, and converted the login/register form into a modal that opens from "Sign in" only when signed out and is hidden once logged in.
- **Cloudflare Workers hosting**: frontend moved from Cloudflare Pages to the `linearvoid-frontend` static-assets Worker; docs now describe `linearvoid-frontend` (SPA) and `linearvoid-media` (edge HLS), correcting stale `worker.js` and R2 custom-domain wording.
- **Docs de-Windows'd & condensed**: bash-first guides for `SETUP`, `DEPLOYMENT`, `API`, `MONITORING`, `STREAMING`, `COOLIFY`, incident runbook, scaling playbook, architecture README, and Prometheus metrics inventory.
- **Compose port de-collision**: playout on host port `8081`, nginx moved to `8082` (`NGINX_PORT` default updated) to avoid Coolify-reserved ports.
- **Playout**: R2 retention now by run count + byte cap (`R2_KEEP_RUNS` default 2, `R2_MAX_RETENTION_BYTES` default 10 GiB, new gauges) replacing the 24h time-based cleanup; default `HLS_SEGMENT_DURATION` 2s / `HLS_WINDOW_SIZE` 12; deduplicated, concurrent startup validation.
- **Chat/UX**: 5s reconnect grace window (no join/leave spam); `system` messages no longer persisted; smaller `.message-text`; sidebar cleanup (removed DM search + Users toggle, login/register now a modal).
- **Auth**: viewer session cookie uses `SameSite=None; Secure` over HTTPS (cross-site SPA) and `Lax` over plain HTTP.
### Fixed
- **Login/register cross-origin API URLs**: `submitAuth` in `frontend/src/main.ts` now prefixes `/api/auth/login` and `/api/auth/register` with `VITE_PUBLIC_APP_URL` (falling back to same-origin), matching the other API calls so the Workers-served SPA reaches the app origin instead of a same-origin path.
- **Duplicate chat history on reconnect/initial load**: `connectWebSocket` now clears `#messages` before each (re)connect, and the initial connect waits for session resolution, so room history renders once instead of being appended repeatedly.
- **Run-scoped R2 manifest URIs (playout)**: `publishManifest` now rewrites bare relative media URIs to the current run folder (`r-{runID}/…`) via `rewriteManifestToRun` before the atomic `.tmp` → `CopyObject` publish, so the live `hls/playlist.m3u8` references resolve against the run-scoped segment keys on `media.linearvoid.tv`. Skipped in slate mode. (Live validation pending — see `TODO.md`.)
- **Cross-origin auth**: login/register API calls now prefix `VITE_PUBLIC_APP_URL` so the Workers-served SPA reaches the app origin.
- **Duplicate chat history**: `connectWebSocket` clears messages on each (re)connect and the initial connect waits for session resolution.
### Tests
- WebSocket: `TestHub_Broadcast_SystemNotPersisted`, `TestHub_ReconnectGracePeriod`.
- Handlers: `TestHandleAdminChatLog_ExcludesSystem`.
- Frontend: `presence flash` test.
- Playout: `TestCleanupOldRuns`, `TestCleanupOldRuns_SizeCap`, `TestR2KeepRuns`, `TestR2MaxRetentionBytes`, `TestR2RunFromKey`.
- Playout: `TestValidateWorkersConfig`, `TestValidatePlaylist_Dedup`.
- WebSocket (`TestHub_Broadcast_SystemNotPersisted`, `TestHub_ReconnectGracePeriod`), handlers (`TestHandleAdminChatLog_ExcludesSystem`), frontend presence flash.
- Playout: `TestCleanupOldRuns(_SizeCap)`, `TestR2KeepRuns`, `TestR2MaxRetentionBytes`, `TestR2RunFromKey`, `TestValidateWorkersConfig`, `TestValidatePlaylist_Dedup`.
- Playout (run-scoped manifest rewrite): `TestRewriteManifestToRun`, `TestRewriteManifestToRun_AlreadyPrefixedUnchanged`, `TestRewriteManifestToRun_HandlesTSAndComments`, `TestRewriteManifestToRun_EmptyRunID`, `TestIsMediaURI`, `TestPublishManifest_RewritesToRunScopedURIs`, `TestPublishManifest_SlateNotRewritten`.
- Cloudflare media worker: Vitest unit (`cacheControlForPath`, `contentTypeForPath`, `parseByteRange`) + Miniflare R2 integration (immutable segment, short-TTL manifest, Range `206`, `404`, OPTIONS, method guard).
## [1.0.13] - 2026-08-13
+3 -2
View File
@@ -7,7 +7,7 @@
[![Build Status](https://git.allucanget.biz/zwitschi/linearvoid/actions/workflows/ci.yml/badge.svg)](https://git.allucanget.biz/zwitschi/linearvoid/actions)
[![Gitea](https://img.shields.io/badge/Gitea-1.27.1-blue)](https://git.allucanget.biz)
[![License](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![Version](https://img.shields.io/badge/version-1.0.13-blue)](https://git.allucanget.biz/zwitschi/linearvoid/releases)
[![Version](https://img.shields.io/badge/version-1.1.0-blue)](https://git.allucanget.biz/zwitschi/linearvoid/releases)
[![Release Notes](https://img.shields.io/badge/release-notes-green)](CHANGELOG.md)
## Architecture
@@ -187,7 +187,8 @@ frontend/
Everything needed to deploy and configure the stack (Docker, Nginx, Cloudflare rules, future Terraform).
- Cloudflare
- [worker.js](infrastructure/cloudflare/worker.js) — Cloudflare Worker serving the static SPA and adding immutable cache headers to HLS segments
- [linearvoid-media worker](infrastructure/cloudflare/) — R2-bound edge Worker serving HLS on `media.linearvoid.tv` (immutable segment caching, CORS, Range)
- [linearvoid-frontend worker](frontend/wrangler.toml) — serves the static SPA on `linearvoid.tv`
- Nginx
- [Dockerfile](infrastructure/nginx/Dockerfile) — builds Nginx with custom config
- [nginx.conf](infrastructure/nginx/nginx.conf) — reverse proxy, static SPA, HLS delivery
+77 -3
View File
@@ -44,6 +44,9 @@ var (
deleteObjects = func(client *s3.Client, input *s3.DeleteObjectsInput) (*s3.DeleteObjectsOutput, error) {
return client.DeleteObjects(context.TODO(), input)
}
copyObject = func(client *s3.Client, input *s3.CopyObjectInput) (*s3.CopyObjectOutput, error) {
return client.CopyObject(context.TODO(), input)
}
bucket = os.Getenv("R2_BUCKET_NAME")
playoutMode = "starting" // "live", "slate", "degraded"
restartCount = 0
@@ -185,6 +188,8 @@ func watchAndPublishSegments(ctx context.Context, hlsOutputDir string) {
}
// publishManifest uploads the manifest atomically: temp key first, then copy to final key (STREAMING.md contract).
// Bare media URIs in the manifest are rewritten to the run-scoped R2 layout
// (r-{runID}/…) so that every referenced segment/init resolves on the media host.
func publishManifest(localManifestPath string) error {
if r2Client == nil {
if err := initR2Client(); err != nil {
@@ -192,13 +197,33 @@ func publishManifest(localManifestPath string) error {
}
}
data, err := os.ReadFile(localManifestPath)
if err != nil {
return fmt.Errorf("read manifest failed: %w", err)
}
// Rewrite bare relative media URIs to the current run folder so the manifest
// matches the run-scoped segment keys uploaded by watchAndPublishSegments /
// syncLatestToR2 (candidate (a)). Slate mode is skipped: its manifest references
// slate assets that are not published under the live run prefix.
if playoutMode != "slate" {
data = rewriteManifestToRun(runID, data)
}
// Stage the (rewritten) manifest locally so the existing SyncToR2 path is reused.
staging := localManifestPath + ".r2staging"
if err := os.WriteFile(staging, data, 0o644); err != nil {
return fmt.Errorf("write staging manifest failed: %w", err)
}
defer os.Remove(staging)
// 1. Upload to temp key
if err := SyncToR2(localManifestPath, "hls/playlist.m3u8.tmp"); err != nil {
if err := SyncToR2(staging, "hls/playlist.m3u8.tmp"); err != nil {
return fmt.Errorf("temp manifest upload failed: %w", err)
}
// 2. Atomic copy to final key
_, err := r2Client.CopyObject(context.TODO(), &s3.CopyObjectInput{
_, err = copyObject(r2Client, &s3.CopyObjectInput{
Bucket: aws.String(bucket),
CopySource: aws.String(bucket + "/hls/playlist.m3u8.tmp"),
Key: aws.String("hls/playlist.m3u8"),
@@ -210,6 +235,55 @@ func publishManifest(localManifestPath string) error {
return nil
}
// isMediaURI reports whether s names an HLS media file (a segment or init file).
func isMediaURI(s string) bool {
name := s
if i := strings.IndexByte(name, '?'); i >= 0 {
name = name[:i]
}
switch strings.ToLower(filepath.Ext(name)) {
case ".m4s", ".ts", ".mp4", ".m4a":
return true
}
return false
}
// rewriteManifestToRun prefixes bare relative media URIs in an HLS manifest with the
// run folder, e.g. "seg-000001.m4s" -> "r-20260903T…/seg-000001.m4s" and
// #EXT-X-MAP:URI="init.mp4" -> URI="r-20260903T…/init.mp4". This keeps the published
// manifest consistent with the run-scoped R2 layout (STREAMING.md contract). Directives,
// comments, blank lines, and URIs that already contain a path are left unchanged.
// Returns data unchanged when runID is empty.
func rewriteManifestToRun(runID string, data []byte) []byte {
if runID == "" {
return data
}
prefix := runID + "/"
lines := strings.Split(string(data), "\n")
out := make([]string, 0, len(lines))
for _, line := range lines {
trimmed := strings.TrimSpace(line)
switch {
case strings.HasPrefix(trimmed, "#EXT-X-MAP:URI="):
if open := strings.Index(trimmed, `URI="`); open >= 0 {
rest := trimmed[open+len(`URI="`):]
if closeIdx := strings.IndexByte(rest, '"'); closeIdx > 0 {
uri := rest[:closeIdx]
if uri != "" && !strings.Contains(uri, "/") {
line = strings.Replace(line, `URI="`+uri+`"`, `URI="`+prefix+uri+`"`, 1)
}
}
}
out = append(out, line)
case trimmed != "" && !strings.HasPrefix(trimmed, "#") && !strings.Contains(trimmed, "/") && isMediaURI(trimmed):
out = append(out, prefix+trimmed)
default:
out = append(out, line)
}
}
return []byte(strings.Join(out, "\n"))
}
// LoadPlaylist reads a playlist file and returns a list of VideoFile entries
func LoadPlaylist(playlistPath string) ([]VideoFile, error) {
file, err := os.Open(playlistPath)
@@ -1218,4 +1292,4 @@ func loadSharedHLSConfig() {
}
}
fmt.Printf("Shared HLS config: segment=%ss window=%s\n", hlsSegmentDuration, hlsWindowSize)
}
}
@@ -0,0 +1,227 @@
package main
import (
"io"
"os"
"path/filepath"
"testing"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/service/s3"
)
const sampleManifest = `#EXTM3U
#EXT-X-VERSION:7
#EXT-X-TARGETDURATION:2
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-MAP:URI="init.mp4"
#EXTINF:2.000000,
seg-000000.m4s
#EXTINF:2.000000,
seg-000001.m4s
#EXT-X-ENDLIST
`
func TestRewriteManifestToRun(t *testing.T) {
runID := "r-20260903T120000Z"
out := string(rewriteManifestToRun(runID, []byte(sampleManifest)))
wants := []string{
`#EXT-X-MAP:URI="r-20260903T120000Z/init.mp4"`,
"r-20260903T120000Z/seg-000000.m4s",
"r-20260903T120000Z/seg-000001.m4s",
"#EXTM3U",
"#EXT-X-VERSION:7",
"#EXT-X-TARGETDURATION:2",
"#EXT-X-MEDIA-SEQUENCE:0",
"#EXT-X-INDEPENDENT-SEGMENTS",
"#EXT-X-ENDLIST",
"#EXTINF:2.000000,",
}
for _, want := range wants {
if !contains(out, want) {
t.Errorf("output missing %q\noutput:\n%s", want, out)
}
}
// Media URIs must be prefixed (no bare segments left).
for _, bare := range []string{"\nseg-000000.m4s\n", "\nseg-000001.m4s\n", `URI="init.mp4"`} {
if contains(out, bare) {
t.Errorf("output still contains bare %q\noutput:\n%s", bare, out)
}
}
}
func TestRewriteManifestToRun_AlreadyPrefixedUnchanged(t *testing.T) {
manifest := "#EXTM3U\n#EXT-X-MAP:URI=\"r-old/init.mp4\"\nseg-000000.m4s\nr-old/seg-000001.m4s\n#EXT-X-ENDLIST\n"
out := string(rewriteManifestToRun("r-new", []byte(manifest)))
if !contains(out, `#EXT-X-MAP:URI="r-old/init.mp4"`) {
t.Errorf("EXT-X-MAP already-prefixed URI was rewritten: %s", out)
}
// A URI line that already contains a path is left alone.
if !contains(out, "r-old/seg-000001.m4s") {
t.Errorf("already-prefixed segment URI was changed:\n%s", out)
}
// Bare URI still gets the run prefix.
if !contains(out, "r-new/seg-000000.m4s") {
t.Errorf("bare segment not prefixed:\n%s", out)
}
}
func TestRewriteManifestToRun_HandlesTSAndComments(t *testing.T) {
manifest := "#EXTM3U\n# a comment\nvideo-1.ts\ninit.mp4\n#EXT-X-ENDLIST\n"
out := string(rewriteManifestToRun("r-x", []byte(manifest)))
for _, want := range []string{"r-x/video-1.ts", "r-x/init.mp4", "# a comment", "#EXTM3U", "#EXT-X-ENDLIST"} {
if !contains(out, want) {
t.Errorf("output missing %q\noutput:\n%s", want, out)
}
}
}
func TestRewriteManifestToRun_EmptyRunID(t *testing.T) {
out := rewriteManifestToRun("", []byte(sampleManifest))
if string(out) != sampleManifest {
t.Errorf("empty runID should return input unchanged")
}
}
func TestIsMediaURI(t *testing.T) {
for _, ok := range []string{"seg-000001.m4s", "init.mp4", "a.ts", "x.m4a"} {
if !isMediaURI(ok) {
t.Errorf("isMediaURI(%q) = false, want true", ok)
}
}
for _, notOK := range []string{"playlist.m3u8", "readme.txt", "seg-000001.m4s?x=1"} {
// Note: query strings are stripped; '?x=1' on m4s should still be media.
if notOK == "seg-000001.m4s?x=1" {
continue
}
if isMediaURI(notOK) {
t.Errorf("isMediaURI(%q) = true, want false", notOK)
}
}
if !isMediaURI("seg-000001.m4s?token=abc") {
t.Errorf("isMediaURI with query should strip query and be true")
}
}
func TestPublishManifest_RewritesToRunScopedURIs(t *testing.T) {
prevClient, prevBucket := r2Client, bucket
prevRunID, prevMode := runID, playoutMode
prevPut, prevCopy := putObject, copyObject
defer func() {
r2Client, bucket = prevClient, prevBucket
runID, playoutMode = prevRunID, prevMode
putObject, copyObject = prevPut, prevCopy
}()
tmpDir := t.TempDir()
mp := filepath.Join(tmpDir, "playlist.m3u8")
if err := os.WriteFile(mp, []byte(sampleManifest), 0o644); err != nil {
t.Fatalf("write manifest: %v", err)
}
bucket = "linearvoid-media"
runID = "r-20260903T120000Z"
playoutMode = "live"
r2Client = &s3.Client{}
var gotKey, gotCopySource, gotCopyKey string
var gotBody []byte
putObject = func(_ *s3.Client, in *s3.PutObjectInput) (*s3.PutObjectOutput, error) {
gotKey = aws.ToString(in.Key)
b, err := io.ReadAll(in.Body)
if err != nil {
return nil, err
}
gotBody = b
return &s3.PutObjectOutput{}, nil
}
copyObject = func(_ *s3.Client, in *s3.CopyObjectInput) (*s3.CopyObjectOutput, error) {
gotCopySource = aws.ToString(in.CopySource)
gotCopyKey = aws.ToString(in.Key)
return &s3.CopyObjectOutput{}, nil
}
if err := publishManifest(mp); err != nil {
t.Fatalf("publishManifest error: %v", err)
}
if gotKey != "hls/playlist.m3u8.tmp" {
t.Errorf("temp upload key = %q, want hls/playlist.m3u8.tmp", gotKey)
}
if gotCopySource != "linearvoid-media/hls/playlist.m3u8.tmp" {
t.Errorf("CopySource = %q, want linearvoid-media/hls/playlist.m3u8.tmp", gotCopySource)
}
if gotCopyKey != "hls/playlist.m3u8" {
t.Errorf("final key = %q, want hls/playlist.m3u8", gotCopyKey)
}
body := string(gotBody)
for _, want := range []string{
"r-20260903T120000Z/seg-000000.m4s",
"r-20260903T120000Z/seg-000001.m4s",
`#EXT-X-MAP:URI="r-20260903T120000Z/init.mp4"`,
} {
if !contains(body, want) {
t.Errorf("published manifest missing %q:\n%s", want, body)
}
}
for _, bare := range []string{"\nseg-000000.m4s\n", "\nseg-000001.m4s\n"} {
if contains(body, bare) {
t.Errorf("published manifest still contains bare %q:\n%s", bare, body)
}
}
}
func TestPublishManifest_SlateNotRewritten(t *testing.T) {
prevClient, prevBucket := r2Client, bucket
prevRunID, prevMode := runID, playoutMode
prevPut, prevCopy := putObject, copyObject
defer func() {
r2Client, bucket = prevClient, prevBucket
runID, playoutMode = prevRunID, prevMode
putObject, copyObject = prevPut, prevCopy
}()
tmpDir := t.TempDir()
mp := filepath.Join(tmpDir, "playlist.m3u8")
if err := os.WriteFile(mp, []byte(sampleManifest), 0o644); err != nil {
t.Fatalf("write manifest: %v", err)
}
bucket = "linearvoid-media"
runID = "r-20260903T120000Z"
playoutMode = "slate"
r2Client = &s3.Client{}
var gotBody []byte
putObject = func(_ *s3.Client, in *s3.PutObjectInput) (*s3.PutObjectOutput, error) {
b, _ := io.ReadAll(in.Body)
gotBody = b
return &s3.PutObjectOutput{}, nil
}
copyObject = func(_ *s3.Client, _ *s3.CopyObjectInput) (*s3.CopyObjectOutput, error) {
return &s3.CopyObjectOutput{}, nil
}
if err := publishManifest(mp); err != nil {
t.Fatalf("publishManifest error: %v", err)
}
if string(gotBody) != sampleManifest {
t.Errorf("slate manifest should be published unmodified; got:\n%s", string(gotBody))
}
}
func contains(s, sub string) bool {
for i := 0; i+len(sub) <= len(s); i++ {
if s[i:i+len(sub)] == sub {
return true
}
}
return false
}
+59 -32
View File
@@ -2,10 +2,10 @@
## Environments
| Environment | Host | Docker Target | Purpose |
|---|---|---|---|
| **Test Lab** | `192.168.88.40` (internal) | `tcp://192.168.88.40:2375` | Integration testing, staging, CI/CD validation |
| **Production** | Hetzner VPS (TBD) | `tcp://<hetzner-ip>:2375` | Public linearvoid.tv |
| Environment | Host | Docker Target | Purpose |
| -------------- | -------------------------- | -------------------------- | ---------------------------------------------- |
| **Test Lab** | `192.168.88.40` (internal) | `tcp://192.168.88.40:2375` | Integration testing, staging, CI/CD validation |
| **Production** | Hetzner VPS (TBD) | `tcp://<hetzner-ip>:2375` | Public linearvoid.tv |
**Current state (2026-08-06):** Test lab is active and receives automated CD deploys. Production Hetzner VPS is **not yet provisioned**. `cd.yml` and `docker-compose.prod.yml` target the test lab; switching to production only requires changing `DOCKER_HOST`.
@@ -14,12 +14,14 @@
**Status: Not provisioned.** Plan documented and pipeline ready; only the VPS needs creation.
### Planned Infrastructure
- Provider: Hetzner Cloud, Region: Ashburn, VA (us-east)
- Instance: CX22 (2 vCPU, 4 GB RAM, 40 GB SSD) or equivalent
- OS: Ubuntu 24.04 LTS or Debian 12
- Bandwidth: 20 TB included traffic/month; Estimated cost: ~€7–10/month base
### Cutover Plan
1. Provision Hetzner VPS
2. Add production secrets to Gitea (`CI_PROD_*` prefix)
3. Update `DOCKER_HOST` in CD workflow to Hetzner public IP
@@ -29,6 +31,7 @@
7. Decommission test lab as public target
### Production Preflight (minimum readiness gate)
1. Verify Hetzner host reachable over SSH and Docker remote API TLS works for the Gitea runner
2. Confirm all production secrets exist under `CI_PROD_*` prefix
3. Confirm `linearvoid.tv`, `www.linearvoid.tv`, `media.linearvoid.tv` DNS targets match intended phase
@@ -37,22 +40,24 @@
6. Verify rollback possible before final DNS switch
### Firewall / Security Group
| Port | Source | Purpose |
|---|---|---|
| 22 | Admin IPs | SSH management |
| 80 | 0.0.0.0/0 | HTTP (redirect to HTTPS) |
| 443 | 0.0.0.0/0 | HTTPS (nginx) |
| 2375 | Gitea runner IP only | Docker remote API (TLS) |
| Port | Source | Purpose |
| ---- | --------------------- | ------------------------------ |
| 22 | Admin IPs | SSH management |
| 80 | 0.0.0.0/0 | HTTP (redirect to HTTPS) |
| 443 | 0.0.0.0/0 | HTTPS (nginx) |
| 2375 | Gitea runner IP only | Docker remote API (TLS) |
| 9090 | Admin IPs or internal | Prometheus (or tunnel via SSH) |
| 3001 | Admin IPs or internal | Grafana (or tunnel via SSH) |
| 3001 | Admin IPs or internal | Grafana (or tunnel via SSH) |
### Pipeline Readiness
| Workflow | Trigger | Purpose |
|---|---|---|
| `ci.yml` | Push to `main` | Lint, build frontend, vet Go, build & push images (`:latest`, `:main`) |
| `cd.yml` | Push to `main` | Deploy to test lab (`192.168.88.40:2375`) — runs **in parallel** with CI |
| `release.yml` | Tag push `*.*.*` | Build & push versioned images (`:X.Y.Z`, `:X.Y`, `:latest`), create Gitea Release |
| `workers-deploy.yml` | After CI succeeds | Deploy frontend to Cloudflare Workers |
| Workflow | Trigger | Purpose |
| -------------------- | ----------------- | --------------------------------------------------------------------------------- |
| `ci.yml` | Push to `main` | Lint, build frontend, vet Go, build & push images (`:latest`, `:main`) |
| `cd.yml` | Push to `main` | Deploy to test lab (`192.168.88.40:2375`) — runs **in parallel** with CI |
| `release.yml` | Tag push `*.*.*` | Build & push versioned images (`:X.Y.Z`, `:X.Y`, `:latest`), create Gitea Release |
| `workers-deploy.yml` | After CI succeeds | Deploy frontend to Cloudflare Workers |
CI and CD run simultaneously on every push to `main`. Deploy pulls whatever `:latest` images are available at that moment. `cd.yml` uses `DOCKER_HOST` to target the host:
@@ -67,6 +72,7 @@ DOCKER_HOST=tcp://<hetzner-public-ip>:2375
No pipeline code changes needed to switch targets.
## Target Topology
- Host: Linux server with Docker Engine
- Reverse proxy: nginx container (linearvoid-nginx)
- Core services: playout, chat, redis, postgres
@@ -74,6 +80,7 @@ No pipeline code changes needed to switch targets.
- CDN/edge: Cloudflare
## Prerequisites
- Docker Engine + Docker Compose plugin
- Network access to git.allucanget.biz
- Valid `.env` file with production values
@@ -82,6 +89,7 @@ No pipeline code changes needed to switch targets.
## Environment File
`.env` requires at minimum:
- `DATABASE_URL` — PostgreSQL connection string
- `REDIS_URL` — Redis connection string
- `ADMIN_USERNAME` / `ADMIN_PASSWORD` — admin dashboard credentials
@@ -89,18 +97,21 @@ No pipeline code changes needed to switch targets.
- `VITE_PUBLIC_APP_URL` / `VITE_PUBLIC_WS_URL` / `VITE_PUBLIC_HLS_URL` — frontend URLs
**Cloudflare R2 (required for HLS segment sync to edge):**
- `R2_ACCESS_KEY_ID` / `R2_SECRET_ACCESS_KEY`
- `R2_BUCKET_NAME` (e.g. `linearvoid-media`)
- `R2_ENDPOINT` — S3-compatible endpoint URL
- `R2_REGION` — set to `auto`
**Cloudflare (optional, for Workers):**
- `CLOUDFLARE_API_TOKEN` / `CLOUDFLARE_ACCOUNT_ID`
- `MEDIA_DOMAIN` — split media domain (e.g. `media.linearvoid.tv`)
See `.env.example` and `SETUP.md` for full variable reference.
### Gitea Secrets
`cd.yml` injects sensitive values from Gitea secrets into `.env` at deploy time. Configure in the Gitea repository:
`DATABASE_URL`, `ADMIN_USERNAME`, `ADMIN_PASSWORD`, `R2_ACCESS_KEY_ID`, `R2_SECRET_ACCESS_KEY`, `R2_BUCKET_NAME`, `R2_ENDPOINT`, `R2_ADMIN_ACCESS_KEY_ID` (optional), `R2_ADMIN_SECRET_ACCESS_KEY` (optional), `R2_ADMIN_ENDPOINT` (optional), `CLOUDFLARE_API_TOKEN` (optional), `CLOUDFLARE_ACCOUNT_ID` (optional), `CI_REGISTRY_USER`, `CI_REGISTRY_TOKEN`
@@ -116,6 +127,7 @@ docker compose -f docker-compose.prod.yml up -d --remove-orphans
```
## First-Time or Recovery Provisioning
```bash
mkdir -p data/chat data/playlist data/media data/hls
docker volume ls
@@ -124,10 +136,13 @@ docker compose -f docker-compose.prod.yml up -d
```
## Health Verification
```bash
docker compose -f docker-compose.prod.yml ps
```
Endpoint checks:
- App: `http://{host}:8080/`
- Nginx health: `http://{host}:8080/health`
- Chat health: `http://{host}:3000/health`
@@ -136,30 +151,38 @@ Endpoint checks:
- Grafana: `http://{host}:3001`
Domain checks:
- `https://linearvoid.tv`
- `https://linearvoid.tv/admin`
- `https://linearvoid.tv/metrics`
## Zero/Low-Downtime Update Pattern
1. Pull newest images.
2. Recreate changed services only:
```bash
docker compose -f docker-compose.prod.yml pull
docker compose -f docker-compose.prod.yml up -d nginx chat playout
```
```bash
docker compose -f docker-compose.prod.yml pull
docker compose -f docker-compose.prod.yml up -d nginx chat playout
```
3. Re-run health checks.
## Rollback
If new images are unhealthy:
1. Pin previous tags in `docker-compose.prod.yml`.
2. Pull pinned tags.
3. Restart impacted services:
```bash
docker compose -f docker-compose.prod.yml pull
docker compose -f docker-compose.prod.yml up -d nginx chat playout
```
## Operational Tasks
```bash
# View logs
docker compose -f docker-compose.prod.yml logs -f nginx chat playout
@@ -175,30 +198,34 @@ docker compose -f docker-compose.prod.yml down -v
```
## Nginx and Proxy Considerations
- If an external proxy (e.g. Nginx Proxy Manager) fronts the stack, align request body limits with internal nginx (see `infrastructure/nginx/nginx.conf`).
- For resumable admin uploads, ensure `client_max_body_size` supports chunk requests in both proxy layers.
## Security Baseline
- Use a strong `ADMIN_PASSWORD` and rotate regularly.
- Keep Docker host access restricted.
- Prefer HTTPS/WSS through Cloudflare.
- Restrict exposed ports if only proxy access is needed.
## Common Deployment Pitfalls
- Missing frontend build artifacts in local-only compose workflows.
- Wrong registry credentials or missing login.
- Misaligned domain env vars (app/ws/hls) causing mixed-origin errors.
## Troubleshooting
| Symptom | Check | Fix |
|---|---|---|
| Service not starting | `docker logs linearvoid-<svc> --tail 50` | Check `.env` vars, volume mounts, port conflicts |
| Playout stall / slate mode | `curl http://<host>:8081/health` — mode=slate | Check playlist file, media paths, disk space |
| Chat not connecting | `curl http://<host>:3000/health` | Check Redis: `docker exec linearvoid-redis redis-cli ping` |
| Admin 401 loop | Cookie not sent | Verify `X-Forwarded-Proto: https`, nginx auth_request headers |
| Upload fails (413) | File too large | Check nginx `client_max_body_size` (should be 64m+) |
| HLS not playing | `curl http://<host>/hls/playlist.m3u8` | Check playout health, HLS dir permissions |
| Prometheus no targets | `curl http://<host>:9090/targets` | Verify container network, scrape config |
| Grafana no data | Check datasource | Verify Prometheus URL `http://prometheus:9090` provisioned |
Additional notes: proxy-layer body size limits can return 413 on upload; health checks may fail before service warmup completes.
| Symptom | Check | Fix |
| -------------------------- | --------------------------------------------- | ------------------------------------------------------------- |
| Service not starting | `docker logs linearvoid-<svc> --tail 50` | Check `.env` vars, volume mounts, port conflicts |
| Playout stall / slate mode | `curl http://<host>:8081/health` — mode=slate | Check playlist file, media paths, disk space |
| Chat not connecting | `curl http://<host>:3000/health` | Check Redis: `docker exec linearvoid-redis redis-cli ping` |
| Admin 401 loop | Cookie not sent | Verify `X-Forwarded-Proto: https`, nginx auth_request headers |
| Upload fails (413) | File too large | Check nginx `client_max_body_size` (should be 64m+) |
| HLS not playing | `curl http://<host>/hls/playlist.m3u8` | Check playout health, HLS dir permissions |
| Prometheus no targets | `curl http://<host>:9090/targets` | Verify container network, scrape config |
| Grafana no data | Check datasource | Verify Prometheus URL `http://prometheus:9090` provisioned |
Additional notes: proxy-layer body size limits can return 413 on upload; health checks may fail before service warmup completes.
+2 -2
View File
@@ -70,7 +70,7 @@ flowchart LR
2. S3 endpoint: `https://<account-id>.r2.cloudflarestorage.com` → `R2_ENDPOINT`.
3. Attach `media.linearvoid.tv` as custom media domain → `MEDIA_DOMAIN`; add CORS allowing `https://linearvoid.tv` GET for HLS delivery.
4. Point `linearvoid.tv` (+ `www` redirect) at the app entrypoint; keep `media.*` on R2. Proxied DNS, TLS **Full (strict)**.
5. Apply the `.m4s`/`.m3u8` cache rules from [`cache-rules.md`](./infrastructure/cloudflare/cache-rules.md); enable Smart Tiered Cache; optionally deploy `infrastructure/cloudflare/worker.js` for header normalization.
5. Apply the `.m4s`/`.m3u8` cache rules from [`cache-rules.md`](./infrastructure/cloudflare/cache-rules.md); enable Smart Tiered Cache; deploy the [`linearvoid-media` worker](../infrastructure/cloudflare/) on `media.linearvoid.tv` (replace the R2 bucket's direct custom-domain binding on that host) to enforce immutable segment caching + HLS CORS.
6. Verify: `aws s3 ls --endpoint-url $R2_ENDPOINT`; check cache hit ratio after serving traffic.
## Running the Application
@@ -129,7 +129,7 @@ Coolify replaces the Gitea CD `DOCKER_HOST` orchestration (Cloudflare DNS/R2/cac
### Production (Hetzner VPS + Cloudflare)
1. Provision CX22 in Ashburn, VA (ADR-001).
2. Cloudflare: R2 bucket (`enam` hint), `media.linearvoid.tv` custom domain + cache rules, `linearvoid.tv`/`www` at the app entrypoint with `www`→apex redirect, Full (strict) TLS, deploy `infrastructure/cloudflare/worker.js`.
2. Cloudflare: R2 bucket (`enam` hint), deploy the [`linearvoid-media` worker](../infrastructure/cloudflare/) on `media.linearvoid.tv` + cache rules, `linearvoid.tv`/`www` at the app entrypoint with `www`→apex redirect, Full (strict) TLS.
3. Deploy from the project: `docker compose -f docker-compose.prod.yml up -d` (or pull from Gitea registry); manage under systemd.
4. Monitoring: Grafana dashboards from `monitoring/grafana/`.
+44 -34
View File
@@ -26,14 +26,14 @@ Nginx `/hls/` is a local/test fallback, not the production origin. The SPA, admi
Only validated files may enter the production playlist.
| Property | Required profile |
| ---------- | ------------------------------------------------------------------------ |
| Container | MP4 |
| Video | H.264/AVC, `yuv420p` |
| Resolution | 1280x720 or 1920x1080; avoid resolution changes within a playlist block |
| Frame rate | Constant 30 fps |
| GOP | Closed, fixed 2 seconds: `-g 60 -keyint_min 60 -sc_threshold 0` |
| Audio | AAC-LC, stereo, 48 kHz, 128–192 kbit/s |
| Property | Required profile |
| ---------- | ----------------------------------------------------------------------- |
| Container | MP4 |
| Video | H.264/AVC, `yuv420p` |
| Resolution | 1280x720 or 1920x1080; avoid resolution changes within a playlist block |
| Frame rate | Constant 30 fps |
| GOP | Closed, fixed 2 seconds: `-g 60 -keyint_min 60 -sc_threshold 0` |
| Audio | AAC-LC, stereo, 48 kHz, 128–192 kbit/s |
The ingest gate uses `ffprobe` to reject incompatible codecs, variable frame rate, missing audio, invalid dimensions, or wrong keyframe cadence. Assets are immutable after acceptance; corrections create a new asset ID/object key.
@@ -62,14 +62,14 @@ ffmpeg -i input.mov \
The playout service wraps FFmpeg, tracks the active item/revision, restarts failed processes, and exposes `/health` plus Prometheus metrics.
| Setting | Value |
| -------------- | ---------------------------------------------------------------------------------- |
| Protocol | Standard live HLS, not LL-HLS |
| Segment format | fMP4 (`.m4s`) with init segment; `.ts` may be supported during migration |
| Segment target | 2 seconds (`HLS_SEGMENT_DURATION=2`) |
| Live window | 12 segments (`HLS_WINDOW_SIZE=12`), ~24 seconds |
| Codec path | Stream copy for conforming assets; no per-viewer transcoding |
| Expected latency | 15–35 seconds |
| Setting | Value |
| ---------------- | ------------------------------------------------------------------------ |
| Protocol | Standard live HLS, not LL-HLS |
| Segment format | fMP4 (`.m4s`) with init segment; `.ts` may be supported during migration |
| Segment target | 2 seconds (`HLS_SEGMENT_DURATION=2`) |
| Live window | 12 segments (`HLS_WINDOW_SIZE=12`), ~24 seconds |
| Codec path | Stream copy for conforming assets; no per-viewer transcoding |
| Expected latency | 15–35 seconds |
Segment boundaries must align with keyframes; timestamps must stay monotonic across item boundaries. If FFmpeg cannot preserve a seamless timeline, insert `#EXT-X-DISCONTINUITY` and publish the corresponding new init segment.
@@ -96,15 +96,25 @@ Order prevents a manifest referencing an unavailable segment:
Never upload an actively written segment or publish the manifest first. Use a single publisher (or fencing/leader election) so two workers can't update the live manifest concurrently.
**Run-scoped manifest URIs.** Segments and the init file are stored run-scoped under `hls/r-{runID}/…`, so the live `hls/playlist.m3u8` must reference them with that run prefix. Before the final commit step, the publisher rewrites each bare relative media URI (segments plus `#EXT-X-MAP:URI`) to `r-{runID}/<name>`. `publishManifest` in `backend/playout/src/playlist.go` does this via `rewriteManifestToRun`, using the same `runID` under which the segments were uploaded; the rewrite is skipped in slate mode (its manifest is served as-is). The published manifest therefore contains, e.g.:
```m3u8
#EXT-X-MAP:URI="r-20260807T170412Z/init.mp4"
#EXTINF:2.000000,
r-20260807T170412Z/seg-000001234.m4s
```
The `linearvoid-media` Worker on `media.linearvoid.tv` is a faithful path→R2 mapper and serves these run-scoped paths directly; it does **not** rewrite URIs at serve time.
`data/hls/` is regenerable staging. Source media, playlist revisions, and schedule state are authoritative; generated HLS is not backed up as application state.
## Cache contract
| Objects | Edge policy | Browser policy | Required headers/behavior |
| -------------------------- | --------------------------------- | --------------------------------------- | ---------------------------------------------------------------------------- |
| `.m4s`, `.ts`, init `.mp4` | Cache eligible; TTL 1 year | TTL 1 year | `Cache-Control: public, max-age=31536000, immutable`; full path in cache key |
| Live `.m3u8` | Cache eligible; edge TTL 2s | Do not retain beyond freshness interval | `Cache-Control: public, max-age=1, s-maxage=2, must-revalidate` |
| Versioned JS/CSS | Cache eligible; TTL 1 year | TTL 1 year | Content-hashed filenames and `immutable` |
| Objects | Edge policy | Browser policy | Required headers/behavior |
| -------------------------- | --------------------------- | --------------------------------------- | ---------------------------------------------------------------------------- |
| `.m4s`, `.ts`, init `.mp4` | Cache eligible; TTL 1 year | TTL 1 year | `Cache-Control: public, max-age=31536000, immutable`; full path in cache key |
| Live `.m3u8` | Cache eligible; edge TTL 2s | Do not retain beyond freshness interval | `Cache-Control: public, max-age=1, s-maxage=2, must-revalidate` |
| Versioned JS/CSS | Cache eligible; TTL 1 year | TTL 1 year | Content-hashed filenames and `immutable` |
Further rules:
@@ -159,21 +169,21 @@ A release is streaming-ready only when: one full playlist boundary plays without
## Required configuration
| Variable | Purpose |
| ------------------------------------------ | -------------------------------------------------- |
| `PLAYLIST_PATH` | Active playlist file |
| `HLS_SEGMENT_DURATION` | Target segment duration; baseline `2` |
| `HLS_WINDOW_SIZE` | Manifest window; baseline `12` |
| `R2_ENDPOINT`, `R2_REGION` | S3-compatible publication endpoint |
| `R2_BUCKET_NAME` | Media bucket |
| `R2_ACCESS_KEY_ID`, `R2_SECRET_ACCESS_KEY` | Write credentials, publisher only |
| `R2_KEEP_RUNS` | Run folders to retain (default `2`) |
| `R2_MAX_RETENTION_BYTES` | Max bytes across run folders (10 GiB) |
| `MEDIA_DOMAIN` | Public R2 custom domain |
| `VITE_PUBLIC_HLS_URL` | Public live manifest URL |
| Variable | Purpose |
| ------------------------------------------ | ------------------------------------- |
| `PLAYLIST_PATH` | Active playlist file |
| `HLS_SEGMENT_DURATION` | Target segment duration; baseline `2` |
| `HLS_WINDOW_SIZE` | Manifest window; baseline `12` |
| `R2_ENDPOINT`, `R2_REGION` | S3-compatible publication endpoint |
| `R2_BUCKET_NAME` | Media bucket |
| `R2_ACCESS_KEY_ID`, `R2_SECRET_ACCESS_KEY` | Write credentials, publisher only |
| `R2_KEEP_RUNS` | Run folders to retain (default `2`) |
| `R2_MAX_RETENTION_BYTES` | Max bytes across run folders (10 GiB) |
| `MEDIA_DOMAIN` | Public R2 custom domain |
| `VITE_PUBLIC_HLS_URL` | Public live manifest URL |
Production values: `MEDIA_DOMAIN=media.linearvoid.tv`, `VITE_PUBLIC_HLS_URL=https://media.linearvoid.tv/hls/playlist.m3u8`. Local fallback may use `http://localhost/hls/playlist.m3u8`.
## Explicitly deferred
ABR ladders, LL-HLS, DVR/catch-up, multi-origin playout, and regional chat affinity are outside the v1 contract. Adding any requires an ADR and an update to this document.
ABR ladders, LL-HLS, DVR/catch-up, multi-origin playout, and regional chat affinity are outside the v1 contract. Adding any requires an ADR and an update to this document.
+26 -9
View File
@@ -4,6 +4,20 @@ Linear Void architecture is documented using the [arc42 template](https://arc42.
> Companion docs: see [Documentation README](../README.md) and `adr/` for full decision rationale, alternatives, and consequences.
- [Architecture Documentation for Linear Void](#architecture-documentation-for-linear-void)
- [Introduction and Goals](#introduction-and-goals)
- [Architecture Constraints](#architecture-constraints)
- [Context and Scope](#context-and-scope)
- [Solution Strategy](#solution-strategy)
- [Building Block View](#building-block-view)
- [Runtime View](#runtime-view)
- [Deployment View](#deployment-view)
- [Cross-cutting Concepts](#cross-cutting-concepts)
- [Architecture Decisions](#architecture-decisions)
- [Quality Requirements](#quality-requirements)
- [Risks and Technical Debts](#risks-and-technical-debts)
- [Glossary](#glossary)
## Introduction and Goals
**Requirements Overview** — Linear Void is an independent **24/7 linear video channel** broadcasting self-owned pre-recorded videos to a primarily US audience via a custom lightweight HTML/CSS/JS SPA with synchronized HLS playback, schedule metadata, public chat rooms, and private 1:1 messaging. Fully independent of third-party video platforms (YouTube, Twitch, etc.).
@@ -142,11 +156,12 @@ graph TD
graph TB
subgraph "Cloudflare Edge"
CF[CDN + WAF<br/>Cache Rules]
R2[R2 Bucket<br/>media + HLS]
WORKERS[Workers<br/>SPA hosting]
R2["R2 Bucket<br/>media + HLS<br/>hls/r-{runID}/… run-scoped"]
MEDIA[linearvoid-media Worker<br/>R2-bound · media.linearvoid.tv]
WORKERS[linearvoid-frontend Worker<br/>SPA hosting]
end
subgraph "Docker Host (Hetzner CX22)"
subgraph "Docker Host"
NGINX[nginx<br/>:80/:443<br/>proxy + auth]
CHAT[chat service<br/>:3000<br/>Go + gorilla/websocket]
PLAYOUT[playout service<br/>:8081<br/>Go + FFmpeg]
@@ -159,13 +174,14 @@ graph TB
ADMIN[Admin Browser<br/>SPA + uploads]
end
VIEWER -->|HTTPS| CF
VIEWER -->|WSS| NGINX
VIEWER -->|HTTPS SPA| CF
VIEWER -->|WSS Chat| NGINX
VIEWER -->|HTTPS /hls| MEDIA
ADMIN -->|HTTPS| CF
CF --> NGINX
MEDIA --> R2
NGINX --> CHAT
NGINX --> PLAYOUT
NGINX -->|/hls/*| R2
CHAT --> REDIS
CHAT --> DB
PLAYOUT --> R2
@@ -175,7 +191,8 @@ graph TB
**Playout Service** (`backend/playout/src/playlist.go`)
- Role: continuous HLS encoding loop via FFmpeg concat demuxer.
- Input: `data/playlist/playlist.txt` (ordered MP4 paths); Output: HLS segments (`.m4s`, `.ts`) + live `.m3u8` to `data/hls/`, mirrored to R2.
- Input: `data/playlist/playlist.txt` (ordered MP4 paths); Output: HLS segments (`.m4s`, `.ts`) + live `.m3u8` to `data/hls/`, mirrored to R2 run-scoped under `hls/r-{runID}/…`.
- Run-scoped manifest: before publishing the live `hls/playlist.m3u8`, `publishManifest` rewrites bare media URIs to the current run folder (`rewriteManifestToRun`), so manifest → segment links resolve on `media.linearvoid.tv`. See [`STREAMING.md`](../STREAMING.md).
- Resilience: restart counter (>5/60s → slate); Prometheus on 8081.
- Validation: ffprobe gate (codec/fps/resolution/keyframe); `-validate` mode checks playlist without running.
- Seamless looping: resets HLS only when playlist fingerprint (SHA-256) changes, crash/slate recovery, or first run; else continues via `-start_number`.
@@ -253,7 +270,7 @@ graph TB
1. Admin adds normalized MP4 to R2 + updates playlist.
2. Playout worker (FFmpeg concat loop) reads next file, packages HLS segments (2s).
3. Segments uploaded to R2 with unique immutable names; live `.m3u8` updated.
3. Segments uploaded to R2 with unique immutable, run-scoped names (`hls/r-{runID}/…`); the live `.m3u8` is published referencing those run-prefixed URIs (see `STREAMING.md`).
4. Viewer loads SPA → requests manifest → hls.js follows current edge.
5. Cloudflare serves `.m4s`/`.ts` from nearest edge cache (origin hit once per POP/region).
6. Parallel WebSocket to chat service for synchronized interaction.
@@ -386,7 +403,7 @@ _Cache rules baseline_: `.ts`/`.m4s` Cache Everything, long TTL, immutable; `.m3
_Edge security/controls_: enforce HTTPS/WSS + redirect HTTP; WAF baseline; rate limits on abusive patterns (auth/admin); minimize origin exposure.
_Worker & header policy_: Worker logic for immutable cache headers/request normalization where cache rules insufficient (`infrastructure/cloudflare/worker.js`); explicit CORS for HLS.
_Worker & header policy_: the `linearvoid-media` Worker ([`infrastructure/cloudflare/`](../../infrastructure/cloudflare/), R2-bound) owns `media.linearvoid.tv` and enforces immutable caching for segments, short TTL for live manifests, plus HLS CORS/Range; it is a faithful path→R2 mapper and does **not** rewrite manifest URIs at serve time (playout rewrites them to run-scoped paths at publish — see [`STREAMING.md`](../STREAMING.md)). The `linearvoid-frontend` Worker ([`frontend/wrangler.toml`](../../frontend/wrangler.toml)) serves the static SPA on `linearvoid.tv`. Automated deploys: `workers-deploy.yml` (SPA) and `workers-media-deploy.yml` (media). Ops detail: [`cloudflare-workers-setup.md`](../operations/cloudflare-workers-setup.md).
_Operations & validation_: verify cache status progression (MISS→HIT), manifest freshness under active playout; track hit ratio, request/error rates, ToS/cost posture.
+21 -16
View File
@@ -2,32 +2,37 @@
Linear Void uses Cloudflare Workers for two purposes:
| Product | Purpose | What |
| ----------- | ------------------ | ------------------------------------------------------------------------------------ |
| **Workers** | Static SPA hosting | Vite-built frontend (`frontend/dist`) served as Worker static assets |
| **Workers** | Edge cache logic | `infrastructure/cloudflare/worker.js` — adds immutable cache headers to HLS segments |
| Product | Purpose | What |
| ----------- | ------------------ | -------------------------------------------------------------------------------------------- |
| **Workers** | Static SPA hosting | `linearvoid-frontend` (`frontend/wrangler.toml`) — Vite SPA (`frontend/dist`) static assets |
| **Workers** | Edge HLS delivery | `linearvoid-media` (`infrastructure/cloudflare/`) — R2-bound Worker on `media.linearvoid.tv` |
Both are managed under **Workers & Pages** in the Cloudflare dashboard.
## Dashboard URLs
| Resource | URL |
| ------------------------ | ------------------------------------------------------------------------------------------------------------------- |
| Workers & Pages overview | `https://dash.cloudflare.com/deb11491c54ec53a24772e98ac6d8128/workers-and-pages` |
| Worker: linearvoid | `https://dash.cloudflare.com/deb11491c54ec53a24772e98ac6d8128/workers/services/view/linearvoid/production/settings` |
| API tokens | <https://dash.cloudflare.com/profile/api-tokens> |
| Resource | URL |
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------- |
| Workers & Pages overview | `https://dash.cloudflare.com/deb11491c54ec53a24772e98ac6d8128/workers-and-pages` |
| Worker: linearvoid-media | `https://dash.cloudflare.com/deb11491c54ec53a24772e98ac6d8128/workers/services/view/linearvoid-media/production/settings` |
| API tokens | <https://dash.cloudflare.com/profile/api-tokens> |
## Worker: `linearvoid` (edge cache headers)
## Worker: `linearvoid-media` (edge HLS delivery)
The Worker at `infrastructure/cloudflare/worker.js` runs on the `linearvoid` Workers service. It intercepts requests and adds immutable `Cache-Control` headers to `.m4s` segment files.
The Worker in [`infrastructure/cloudflare/`](../../infrastructure/cloudflare/) runs on the `linearvoid-media` Workers service. It owns the `media.linearvoid.tv` host, serves HLS objects from the R2 bucket (bound as `MEDIA_BUCKET`), and stamps immutable `Cache-Control` on segments plus a short TTL on live manifests (CORS + Range supported). See its [README](../../infrastructure/cloudflare/README.md).
Deploy via Wrangler CLI:
**Run-scoped layout.** R2 objects are stored run-scoped (`hls/r-{runID}/seg-*.m4s`, `…/init.mp4`), and the live `hls/playlist.m3u8` references them with the run prefix (the playout rewrites bare URIs before publishing — see [`docs/STREAMING.md`](../STREAMING.md)). The Worker is a faithful path→R2 mapper: it serves whatever key the request path resolves to (e.g. `/hls/r-…/seg-*.m4s`), adds the cache headers by file type, and does not rewrite manifest URIs at serve time.
Test / deploy:
```bash
npx wrangler deploy infrastructure/cloudflare/worker.js
cd infrastructure/cloudflare
npm install && npm test # unit + Miniflare R2 tests
npx wrangler deploy # preview/test target
npx wrangler deploy --env production # attach media.linearvoid.tv
```
Or paste into the Workers dashboard at the URL above (Quick Edit).
Attach the production custom domain only after removing `media.linearvoid.tv` from the R2 bucket's direct custom-domain binding. Automated production deploy: [`.gitea/workflows/workers-media-deploy.yml`](../../.gitea/workflows/workers-media-deploy.yml).
## Worker: static frontend
@@ -56,9 +61,9 @@ Backend services (playout, chat, nginx) read their own runtime variables (`ADMIN
## Custom domains
- `linearvoid.tv` → Worker (static SPA)
- `linearvoid.tv` → Worker (static SPA, `linearvoid-frontend`)
- `www.linearvoid.tv` → redirect to `linearvoid.tv`
- `media.linearvoid.tv` → R2 bucket (separate; not a Worker)
- `media.linearvoid.tv` → Worker (`linearvoid-media`, R2-bound) serving the `linearvoid-media` bucket
## Related
+1 -1
View File
@@ -16,7 +16,7 @@ Deploy the Linear Void frontend (Vite + TypeScript SPA) from Gitea to Cloudflare
3. **Account ID** from Cloudflare dashboard overview → store as `CLOUDFLARE_ACCOUNT_ID`
4. **Wrangler config** (`frontend/wrangler.toml`) binding the `frontend/dist` output as static assets.
> **Naming conflict:** Workers share a namespace under "Workers & Pages". The edge-cache Worker (`infrastructure/cloudflare/worker.js`) already uses the `linearvoid` name, so the SPA Worker is named `linearvoid-frontend` (see `frontend/wrangler.toml`).
> **Naming:** Workers share a namespace under "Workers & Pages". The media edge Worker is `linearvoid-media` ([`infrastructure/cloudflare/`](../../infrastructure/cloudflare/wrangler.toml), deployed via `.gitea/workflows/workers-media-deploy.yml`) and the SPA Worker is `linearvoid-frontend` (`frontend/wrangler.toml`, deployed via `.gitea/workflows/workers-deploy.yml`).
## Project-specific values
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "frontend",
"version": "1.0.11",
"version": "1.1.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "frontend",
"version": "1.0.11",
"version": "1.1.0",
"dependencies": {
"hls.js": "^1.6.16"
},
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "frontend",
"private": true,
"version": "1.0.13",
"version": "1.1.0",
"type": "module",
"engines": {
"node": ">=22.0.0"
+5
View File
@@ -0,0 +1,5 @@
node_modules/
.wrangler/
dist/
coverage/
*.log
+65
View File
@@ -0,0 +1,65 @@
# linearvoid-media Worker
Cloudflare Worker that owns **`media.linearvoid.tv`** and serves the HLS objects
stored in the `linearvoid-media` R2 bucket, applying the edge cache policy in code:
| Object | Cache-Control |
| ------------------------------------------- | ------------------------------------------------ |
| `.m4s` / `.ts` / `.mp4` / `.m4a` (segments) | `public, max-age=31536000, immutable` |
| `.m3u8` (live manifest) | `public, max-age=1, s-maxage=2, must-revalidate` |
It also adds HLS CORS (`Access-Control-Allow-Origin: *`), OPTIONS preflight, a
`/health` probe, and Range-request support. This worker replaces direct R2
custom-domain serving on the media host so cache headers/CORS are controlled in
code (cache rules on the same host remain the origin baseline).
## Layout
```text
infrastructure/cloudflare/
├── wrangler.toml # name/binding + production custom-domain route
├── package.json
├── src/worker.js # worker module (exports default.fetch + pure helpers)
└── test/ # Vitest unit tests + Miniflare (workerd) R2 tests
```
## Local dev / test
```bash
cd infrastructure/cloudflare
npm install
npm test # unit + Miniflare R2 integration tests
npx wrangler dev # run locally against your account's R2 bucket
```
## Deploy
**Preview / test target** (default environment → `*.workers.dev`):
```bash
cd infrastructure/cloudflare
npx wrangler deploy
```
**Production** (attaches `media.linearvoid.tv` as a custom domain on this worker):
```bash
cd infrastructure/cloudflare
npx wrangler deploy --env production
```
Both need `CLOUDFLARE_API_TOKEN` (Workers Scripts/Routes edit) and
`CLOUDFLARE_ACCOUNT_ID` in the environment. The media host must first be removed
from the R2 bucket's direct custom-domain binding, otherwise the route collides.
Automated production deploys run via
[`.gitea/workflows/workers-media-deploy.yml`](../../.gitea/workflows/workers-media-deploy.yml)
after CI succeeds on `main`.
## Verify
```bash
curl -I https://media.linearvoid.tv/hls/playlist.m3u8 # short TTL
curl -I https://media.linearvoid.tv/hls/<segment>.m4s # immutable + CORS
curl https://media.linearvoid.tv/health # {"status":"ok",...}
```
File diff suppressed because it is too large. Load diff
+20
View File
@@ -0,0 +1,20 @@
{
"name": "linearvoid-media-worker",
"version": "1.0.0",
"private": true,
"description": "Linear Void media edge worker: serves HLS from R2 on media.linearvoid.tv with immutable segment caching.",
"type": "module",
"scripts": {
"dev": "wrangler dev",
"start": "wrangler dev",
"deploy": "wrangler deploy",
"deploy:production": "wrangler deploy --env production",
"test": "vitest run",
"test:watch": "vitest"
},
"devDependencies": {
"miniflare": "^4.0.0",
"vitest": "^4.1.10",
"wrangler": "^4.0.0"
}
}
+144
View File
@@ -0,0 +1,144 @@
// Linear Void media edge worker (linearvoid-media).
//
// Owns the media.linearvoid.tv host and serves HLS objects from the R2 bucket
// bound as MEDIA_BUCKET. Its purpose (arc42 quality goal #1, cost efficiency) is
// to guarantee immutable, long-lived caching of media segments at the edge while
// keeping live manifests fresh — the caching baseline is also configured as
// Cloudflare cache rules on the same host, but this worker enforces it in code.
//
// Cache policy (matches docs/infrastructure/cloudflare/cache-rules.md + STREAMING.md):
// .m4s / .ts / .mp4 / .m4a -> public, max-age=31536000, immutable
// .m3u8 (live manifest) -> public, max-age=1, s-maxage=2, must-revalidate
const IMMUTABLE_CACHE = "public, max-age=31536000, immutable";
const MANIFEST_CACHE = "public, max-age=1, s-maxage=2, must-revalidate";
const DEFAULT_CACHE = "public, max-age=60";
const IMMUTABLE_SUFFIXES = [".m4s", ".ts", ".mp4", ".m4a"];
const MANIFEST_SUFFIXES = [".m3u8"];
const CONTENT_TYPES = {
".m3u8": "application/vnd.apple.mpegurl",
".m4s": "video/mp4",
".mp4": "video/mp4",
".ts": "video/MP2T",
".m4a": "audio/mp4",
};
/** Map a request path to its edge Cache-Control value. Pure + testable. */
export function cacheControlForPath(pathname) {
const lower = pathname.toLowerCase();
for (const suffix of MANIFEST_SUFFIXES) {
if (lower.endsWith(suffix)) return MANIFEST_CACHE;
}
for (const suffix of IMMUTABLE_SUFFIXES) {
if (lower.endsWith(suffix)) return IMMUTABLE_CACHE;
}
return DEFAULT_CACHE;
}
/** Map a request path to a browser-friendly Content-Type. Pure + testable. */
export function contentTypeForPath(pathname) {
const lower = pathname.toLowerCase();
for (const [suffix, type] of Object.entries(CONTENT_TYPES)) {
if (lower.endsWith(suffix)) return type;
}
return "application/octet-stream";
}
const CORS_HEADERS = {
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Methods": "GET, HEAD, OPTIONS",
"Access-Control-Allow-Headers": "Range, If-None-Match, If-Modified-Since",
"Access-Control-Max-Age": "86400",
};
function json(body, status = 200) {
return new Response(JSON.stringify(body), {
status,
headers: { "Content-Type": "application/json", ...CORS_HEADERS },
});
}
// Parse a single, closed byte range ("bytes=start-end") into an R2 range object.
// Returns null for open/suffix/multi/invalid ranges — the caller then serves the
// full object (HTTP 200), which is valid per RFC 7233 (a server may ignore Range).
export function parseByteRange(header) {
if (!header) return null;
const m = /^bytes=(\d*)-(\d*)$/.exec(header.trim());
if (!m) return null;
const start = m[1];
const end = m[2];
if (start === "" || end === "") return null; // open or suffix range
const offset = Number(start);
const length = Number(end) - offset + 1;
if (!Number.isInteger(offset) || !Number.isInteger(length) || offset < 0 || length < 1) {
return null;
}
return { offset, length };
}
export default {
async fetch(request, env) {
const url = new URL(request.url);
// CORS preflight for HLS media.
if (request.method === "OPTIONS") {
return new Response(null, { status: 204, headers: CORS_HEADERS });
}
// Liveness probe used by uptime/monitoring (see cloudflare-uptimerobot-checklist).
if (url.pathname === "/health") {
return json({ status: "ok", service: "linearvoid-media" });
}
if (request.method !== "GET" && request.method !== "HEAD") {
return json({ error: "method not allowed" }, 405);
}
const key = decodeURIComponent(url.pathname).replace(/^\/+/, "");
if (!key) {
return json({ error: "not found" }, 404);
}
if (!env.MEDIA_BUCKET) {
return json({ error: "media bucket not configured" }, 500);
}
// Serve from R2, honouring single closed Range requests (used by some hls.js
// / player paths for byte-range fetches). Unsupported ranges fall back to a
// full-body 200, which clients accept.
const rangeHeader = request.headers.get("range");
const parsedRange = parseByteRange(rangeHeader);
const object = parsedRange
? await env.MEDIA_BUCKET.get(key, { range: parsedRange })
: await env.MEDIA_BUCKET.get(key);
if (object === null) {
return json({ error: "not found" }, 404);
}
const headers = new Headers();
object.writeHttpMetadata(headers);
headers.set("Cache-Control", cacheControlForPath(url.pathname));
headers.set("Content-Type", contentTypeForPath(url.pathname));
headers.set("ETag", object.httpEtag);
headers.set("Accept-Ranges", "bytes");
headers.set("Access-Control-Allow-Origin", "*");
let status = 200;
if (parsedRange && object.range) {
status = 206;
const { offset, length } = object.range;
headers.set(
"Content-Range",
`bytes ${offset}-${offset + length - 1}/${object.size}`
);
}
return new Response(request.method === "HEAD" ? null : object.body, {
status,
headers,
});
},
};
@@ -0,0 +1,77 @@
import { Miniflare } from "miniflare";
import { beforeAll, afterAll, describe, it, expect } from "vitest";
const IMMUTABLE = "public, max-age=31536000, immutable";
const MANIFEST = "public, max-age=1, s-maxage=2, must-revalidate";
// Real-workerd integration test: boots the worker with an in-memory R2 bucket and
// dispatches requests through the fetch handler.
let mf;
beforeAll(async () => {
mf = new Miniflare({
modules: true,
scriptPath: "src/worker.js",
r2Buckets: { MEDIA_BUCKET: "linearvoid-media" },
});
const bucket = await mf.getR2Bucket("MEDIA_BUCKET");
await bucket.put("hls/playlist.m3u8", "#EXTM3U\n#EXT-X-ENDLIST\n", {
httpMetadata: { contentType: "application/vnd.apple.mpegurl" },
});
await bucket.put("hls/seg-000001.m4s", new Uint8Array([0, 1, 2, 3, 4]));
await bucket.put("hls/init.mp4", new Uint8Array([9, 9, 9]));
});
afterAll(async () => {
await mf?.dispose();
});
const base = "http://media.linearvoid.tv";
describe("linearvoid-media worker (R2)", () => {
it("serves a media segment with immutable cache headers", async () => {
const res = await mf.dispatchFetch(`${base}/hls/seg-000001.m4s`);
expect(res.status).toBe(200);
expect(res.headers.get("cache-control")).toBe(IMMUTABLE);
expect(res.headers.get("content-type")).toBe("video/mp4");
expect(res.headers.get("access-control-allow-origin")).toBe("*");
expect(await res.arrayBuffer()).toEqual(new Uint8Array([0, 1, 2, 3, 4]).buffer);
});
it("serves a live manifest with a short TTL", async () => {
const res = await mf.dispatchFetch(`${base}/hls/playlist.m3u8`);
expect(res.status).toBe(200);
expect(res.headers.get("cache-control")).toBe(MANIFEST);
expect(res.headers.get("content-type")).toBe("application/vnd.apple.mpegurl");
});
it("honours Range requests with 206 + Content-Range", async () => {
const res = await mf.dispatchFetch(`${base}/hls/seg-000001.m4s`, {
headers: { Range: "bytes=0-2" },
});
expect(res.status).toBe(206);
expect(res.headers.get("content-range")).toBe("bytes 0-2/5");
const bytes = new Uint8Array(await res.arrayBuffer());
expect([...bytes]).toEqual([0, 1, 2]);
});
it("returns 404 for a missing object", async () => {
const res = await mf.dispatchFetch(`${base}/hls/does-not-exist.m4s`);
expect(res.status).toBe(404);
});
it("answers OPTIONS preflight with CORS headers", async () => {
const res = await mf.dispatchFetch(`${base}/hls/seg-000001.m4s`, {
method: "OPTIONS",
});
expect(res.status).toBe(204);
expect(res.headers.get("access-control-allow-origin")).toBe("*");
});
it("rejects non-GET/HEAD methods", async () => {
const res = await mf.dispatchFetch(`${base}/hls/seg-000001.m4s`, {
method: "POST",
});
expect(res.status).toBe(405);
});
});
@@ -0,0 +1,54 @@
import { describe, it, expect } from "vitest";
import { cacheControlForPath, contentTypeForPath, parseByteRange } from "../src/worker.js";
const IMMUTABLE = "public, max-age=31536000, immutable";
const MANIFEST = "public, max-age=1, s-maxage=2, must-revalidate";
const DEFAULT = "public, max-age=60";
describe("cacheControlForPath", () => {
it("marks media segments as immutable", () => {
expect(cacheControlForPath("/hls/seg-000001.m4s")).toBe(IMMUTABLE);
expect(cacheControlForPath("/hls/init.mp4")).toBe(IMMUTABLE);
expect(cacheControlForPath("/hls/seg-000001.ts")).toBe(IMMUTABLE);
expect(cacheControlForPath("/media/audio.m4a")).toBe(IMMUTABLE);
});
it("keeps live manifests short-lived", () => {
expect(cacheControlForPath("/hls/playlist.m3u8")).toBe(MANIFEST);
expect(cacheControlForPath("/hls/master.m3u8")).toBe(MANIFEST);
});
it("falls back to a sane default for other paths", () => {
expect(cacheControlForPath("/hls/unknown.txt")).toBe(DEFAULT);
});
});
describe("contentTypeForPath", () => {
it("maps known media types", () => {
expect(contentTypeForPath("/hls/playlist.m3u8")).toBe("application/vnd.apple.mpegurl");
expect(contentTypeForPath("/hls/seg-000001.m4s")).toBe("video/mp4");
expect(contentTypeForPath("/hls/init.mp4")).toBe("video/mp4");
expect(contentTypeForPath("/hls/seg-000001.ts")).toBe("video/MP2T");
expect(contentTypeForPath("/media/audio.m4a")).toBe("audio/mp4");
});
it("falls back to octet-stream", () => {
expect(contentTypeForPath("/hls/unknown.bin")).toBe("application/octet-stream");
});
});
describe("parseByteRange", () => {
it("parses a closed byte range", () => {
expect(parseByteRange("bytes=0-2")).toEqual({ offset: 0, length: 3 });
expect(parseByteRange("bytes=10-19")).toEqual({ offset: 10, length: 10 });
});
it("rejects open, suffix, multi, or malformed ranges", () => {
expect(parseByteRange("bytes=0-")).toBeNull();
expect(parseByteRange("bytes=-500")).toBeNull();
expect(parseByteRange("bytes=0-1,3-4")).toBeNull();
expect(parseByteRange("items=0-2")).toBeNull();
expect(parseByteRange("")).toBeNull();
expect(parseByteRange(null)).toBeNull();
});
});
-21
View File
@@ -1,21 +0,0 @@
// Cloudflare Worker for advanced cache rules and WAF (ADR-001)
addEventListener("fetch", (event) => {
event.respondWith(handleRequest(event.request));
});
async function handleRequest(request) {
const url = new URL(request.url);
// Add immutable cache headers for .m4s segments
if (url.pathname.endsWith(".m4s")) {
const response = await fetch(request);
const headers = new Headers(response.headers);
headers.set("Cache-Control", "public, max-age=31536000, immutable");
return new Response(response.body, {
status: response.status,
headers,
});
}
return fetch(request);
}
+31
View File
@@ -0,0 +1,31 @@
# Cloudflare Worker: linearvoid-media (edge HLS delivery)
# Serves HLS objects from the `linearvoid-media` R2 bucket on media.linearvoid.tv.
# Applies immutable caching to media segments and a short TTL to live manifests.
# This worker replaces direct R2 custom-domain serving on the media host so cache
# headers/CORS are controlled in code (cache rules remain as the origin baseline).
name = "linearvoid-media"
main = "src/worker.js"
compatibility_date = "2025-08-01"
# R2 bucket holding the HLS objects. The request path is mirrored onto the bucket,
# e.g. GET /hls/playlist.m3u8 -> object key `hls/playlist.m3u8`.
[[r2_buckets]]
binding = "MEDIA_BUCKET"
bucket_name = "linearvoid-media"
# Default environment: deploy to the preview *.workers.dev URL (test target).
# npx wrangler deploy
#
# Production environment: attach the media host as a custom domain on this worker.
# Note: bindings are NOT inherited by environments, so r2_buckets is redeclared here.
# npx wrangler deploy --env production
[env.production]
routes = [
{ pattern = "media.linearvoid.tv", custom_domain = true }
]
[[env.production.r2_buckets]]
binding = "MEDIA_BUCKET"
bucket_name = "linearvoid-media"