chore(release): 1.1.0
- playout: run-scoped R2 manifest URIs (candidate a rewrite via rewriteManifestToRun) - tests: playlist_rewrite_test.go (rewrite + publish integration + slate guard) - docs: STREAMING, cloudflare-workers-setup, architecture (incl. Level-2 diagram) - release housekeeping: TODO->DONE, CHANGELOG 1.1.0, version bump
This commit is contained in:
zwitschi committed
1 parent
8c5e70959b
commit
d8cf853842
13 files changed
+441
-79
No files matched your search
@@ -2,6 +2,13 @@
|
||||
|
||||
> Completed tasks from [TODO](./TODO.md). Product, feature, and release history lives in [CHANGELOG](./CHANGELOG.md); this file keeps only engineering and ops notes not captured there.
|
||||
|
||||
## Manifest run-scoped rewrite — candidate (a) (2026-09-03)
|
||||
|
||||
- Playout now publishes the live R2 manifest with run-scoped media URIs. `rewriteManifestToRun` in `backend/playout/src/playlist.go` prefixes each bare relative URI (`seg-*.m4s` / `init.mp4`, plus `#EXT-X-MAP:URI`) with `r-{runID}/` before `publishManifest`'s atomic `.tmp` → `CopyObject` commit; skipped in slate mode. Added a stubbable `copyObject` hook for tests (consistent with `putObject`/`listObjects`/`deleteObjects`).
|
||||
- Tests: `playlist_rewrite_test.go` — `TestRewriteManifestToRun`, `TestRewriteManifestToRun_AlreadyPrefixedUnchanged`, `TestRewriteManifestToRun_HandlesTSAndComments`, `TestRewriteManifestToRun_EmptyRunID`, `TestIsMediaURI`, `TestPublishManifest_RewritesToRunScopedURIs`, `TestPublishManifest_SlateNotRewritten`.
|
||||
- Docs updated: `STREAMING.md`, `operations/cloudflare-workers-setup.md`, `architecture/README.md` (incl. Level-2 diagram showing the R2-bound `linearvoid-media` Worker and run-scoped layout).
|
||||
- **Still pending (in TODO): live Validation** — deploy the updated playout to the test lab and confirm manifest/segment URIs resolve via `media.linearvoid.tv`.
|
||||
|
||||
## Test infrastructure & refactors (2026-08-07)
|
||||
|
||||
- Chat service `main.go` split into 8 packages (config, auth, db, handlers, ws, media, ratelimit, metrics); `db.RedisClient` + `db.PubSub` interfaces added for mocking.
|
||||
@@ -37,3 +44,35 @@
|
||||
- Test in Firefox: live playback, rebuffer/recovery, chat sync; record quirks in `docs/STREAMING.md`
|
||||
- Rebuild + run playout; confirm `playlist.m3u8` lists ~12 segments at 2s each
|
||||
- Manual: verify viewer latency and cache-hit behavior after the change
|
||||
|
||||
## Chat sidebar (2026-08-17)
|
||||
|
||||
- [x] remove search bar from chat sidebar (not needed for single-room)
|
||||
- [x] remove "user" toggle button from chat sidebar (hide for now, add back when user interface is stable)
|
||||
- [x] make login/registration form a modal and only toggle it when user is not logged in and clicks "Login" or "Register"
|
||||
- [x] when user is logged in, do not show login/registration form
|
||||
- [x] make sure each message is only rendered once in the chat log (no duplicates on reconnect or initial load)
|
||||
|
||||
### Plan
|
||||
|
||||
Files: `frontend/index.html`, `frontend/src/main.ts`, `frontend/src/style.css`.
|
||||
|
||||
1. **Remove search bar** (single-room)
|
||||
- `index.html`: delete the `<div id="dm-search-row">` block.
|
||||
- `main.ts`: remove `searchDM()` and the `dm-search-btn` listener; drop the `#dm-search-row` show/hide logic in `switchRoom()` (the `searchRow.hidden = !isDM` lines). Keep `currentDMTarget` only if the "New DM" flow stays.
|
||||
|
||||
2. **Remove "Users" toggle button**
|
||||
- `index.html`: delete the `<button id="users-toggle-btn">` button.
|
||||
- `main.ts`: remove `toggleUserList()` and the `users-toggle-btn` listener; drop the now-unused `usersVisible` variable.
|
||||
|
||||
3. **Login/registration form as a modal**
|
||||
- `index.html`: wrap the `#auth-panel` form in a modal structure (`.auth-modal` + backdrop) with a close button.
|
||||
- `style.css`: add `.auth-modal` / `.auth-modal-backdrop` styles.
|
||||
- `main.ts`: add `openAuthModal()` / `closeAuthModal()`; change the `auth-toggle-btn` handler to open the modal (only when not authenticated) instead of toggling `#auth-panel` inline; wire a close/backdrop-click to `closeAuthModal()`.
|
||||
|
||||
4. **Hide form when logged in**
|
||||
- `main.ts` `updateAuthUI()`: when `authenticated` → hide the "Sign in" button and call `closeAuthModal()`; show the user label + "Sign out" button. When not authenticated → show "Sign in" and hide "Sign out".
|
||||
|
||||
5. **No duplicate messages**
|
||||
- `main.ts`: add `clearMessages()` (empties `#messages`) and call it at the top of `connectWebSocket()` so each (re)connect renders history fresh instead of appending.
|
||||
- Fix the initial double-connect: make the initial `connectWebSocket("main")` wait for `refreshUserSession()` (or connect once with the resolved username) so history isn't pushed twice on load.
|
||||
@@ -2,37 +2,19 @@
|
||||
|
||||
> Completed tasks moved to [DONE](./DONE.md). Remaining tasks in [BACKLOG](./BACKLOG.md).
|
||||
|
||||
## Chat sidebar
|
||||
## Playout / R2 manifest–segment layout mismatch
|
||||
|
||||
- [x] remove search bar from chat sidebar (not needed for single-room)
|
||||
- [x] remove "user" toggle button from chat sidebar (hide for now, add back when user interface is stable)
|
||||
- [x] make login/registration form a modal and only toggle it when user is not logged in and clicks "Login" or "Register"
|
||||
- [x] when user is logged in, do not show login/registration form
|
||||
- [x] make sure each message is only rendered once in the chat log (no duplicates on reconnect or initial load)
|
||||
**Chosen approach: candidate (a)** — rewrite the live manifest so its media URIs point into the current run-scoped folder before the atomic R2 publish. Go change in `backend/playout/src/playlist.go`. The `linearvoid-media` Worker stays a faithful path→R2 mapper (no serve-time rewrite).
|
||||
|
||||
### Plan
|
||||
### Remaining — Validation
|
||||
|
||||
Files: `frontend/index.html`, `frontend/src/main.ts`, `frontend/src/style.css`.
|
||||
- [ ] Run playout (local or test lab), then:
|
||||
- `curl https://media.linearvoid.tv/hls/playlist.m3u8` → 200, short TTL.
|
||||
- Every URI in the manifest resolves `200` with `Cache-Control: public, max-age=31536000, immutable` (via the `linearvoid-media` Worker).
|
||||
- Confirm manifest refresh + `CF-Cache-Status` progression and no 404 on segment fetch across a run loop.
|
||||
- [ ] Verify slate fallback still serves correctly after the rewrite change.
|
||||
|
||||
1. **Remove search bar** (single-room)
|
||||
- `index.html`: delete the `<div id="dm-search-row">` block.
|
||||
- `main.ts`: remove `searchDM()` and the `dm-search-btn` listener; drop the `#dm-search-row` show/hide logic in `switchRoom()` (the `searchRow.hidden = !isDM` lines). Keep `currentDMTarget` only if the "New DM" flow stays.
|
||||
|
||||
2. **Remove "Users" toggle button**
|
||||
- `index.html`: delete the `<button id="users-toggle-btn">` button.
|
||||
- `main.ts`: remove `toggleUserList()` and the `users-toggle-btn` listener; drop the now-unused `usersVisible` variable.
|
||||
|
||||
3. **Login/registration form as a modal**
|
||||
- `index.html`: wrap the `#auth-panel` form in a modal structure (`.auth-modal` + backdrop) with a close button.
|
||||
- `style.css`: add `.auth-modal` / `.auth-modal-backdrop` styles.
|
||||
- `main.ts`: add `openAuthModal()` / `closeAuthModal()`; change the `auth-toggle-btn` handler to open the modal (only when not authenticated) instead of toggling `#auth-panel` inline; wire a close/backdrop-click to `closeAuthModal()`.
|
||||
|
||||
4. **Hide form when logged in**
|
||||
- `main.ts` `updateAuthUI()`: when `authenticated` → hide the "Sign in" button and call `closeAuthModal()`; show the user label + "Sign out" button. When not authenticated → show "Sign in" and hide "Sign out".
|
||||
|
||||
5. **No duplicate messages**
|
||||
- `main.ts`: add `clearMessages()` (empties `#messages`) and call it at the top of `connectWebSocket()` so each (re)connect renders history fresh instead of appending.
|
||||
- Fix the initial double-connect: make the initial `connectWebSocket("main")` wait for `refreshUserSession()` (or connect once with the resolved username) so history isn't pushed twice on load.
|
||||
> Implementation (candidate a), Go unit tests, and documentation are complete — see [DONE.md](./DONE.md).
|
||||
|
||||
## Frontend Tests
|
||||
|
||||
|
||||
+17
-25
@@ -2,43 +2,35 @@
|
||||
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
## [Unreleased]
|
||||
## [1.1.0] - 2026-09-03
|
||||
|
||||
### Added
|
||||
|
||||
- **Cloudflare media edge worker (`linearvoid-media`)**: New self-contained project under `infrastructure/cloudflare/` that owns `media.linearvoid.tv` and serves HLS objects from the `linearvoid-media` R2 bucket via an `r2_buckets` binding. Enforces immutable `Cache-Control` for segments (`max-age=31536000, immutable`) and a short TTL for live `.m3u8`, with HLS CORS, OPTIONS preflight, Range support, and a `/health` probe. Replaces the ad-hoc `worker.js`. Includes `wrangler.toml` (default preview + `production` env attaching the media custom domain), Vitest unit tests, Miniflare/workerd R2 integration tests, and automated production deploy via `.gitea/workflows/workers-media-deploy.yml`.
|
||||
- **Bash equivalents of Windows scripts**: Added `scripts/cache-hit-ratio.sh`, `scripts/check-env.sh`, `scripts/check-tokens.sh`, and `scripts/loadtest/run-loadtests.sh`, `scripts/loadtest/summarize-artillery.sh`, `scripts/loadtest/summarize-k6.sh` — Linux/macOS ports of the existing PowerShell scripts (the `.ps1` files are retained).
|
||||
- **Transient presence flash**: Join/leave announcements now appear as a 3-second flash at the top of the chat panel (`#chat-flash`) with a fade-out, instead of permanent chat rows. Presence indicators still update from the same events.
|
||||
- **Cloudflare media edge worker (`linearvoid-media`)**: new self-contained project under `infrastructure/cloudflare/` — an R2-bound Worker owning `media.linearvoid.tv` that enforces immutable segment caching (`public, max-age=31536000, immutable`) and short-TTL live `.m3u8`, with HLS CORS/OPTIONS, byte-Range, and a `/health` probe. Includes Vitest unit tests + Miniflare/workerd R2 integration tests and production deploy via `.gitea/workflows/workers-media-deploy.yml`. Replaces the ad-hoc `worker.js`.
|
||||
- **Bash script ports**: Linux/macOS equivalents of the Windows ops/load-test scripts (`cache-hit-ratio`, `check-env`, `check-tokens`, loadtest run/summarize); `.ps1` files retained.
|
||||
- **Transient presence flash**: join/leave events show as a 3-second `#chat-flash` instead of permanent chat rows.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Cloudflare worker docs updated**: README, SETUP, architecture, and Cloudflare ops guides now describe the `linearvoid-media` edge worker (R2-bound on `media.linearvoid.tv`) and the `linearvoid-frontend` SPA worker, replacing stale `worker.js` references and the R2 custom-domain wording for the media host.
|
||||
- **Coolify host-port de-collision**: Published playout on host port `8081` (`${PLAYOUT_PORT:-8081}:8081`) and moved the nginx host port from `8080`/`80` to a free `8082` (`${NGINX_PORT:-8082}:80`) in both `docker-compose.yml` and `docker-compose.prod.yml`, so no service binds ports reserved by the Coolify environment (Traefik proxy `80`/`443`, dashboard/Portainer `8000`, and the taken `8080`). `NGINX_PORT` default in `.env.example` updated from `80` to `8082`.
|
||||
|
||||
- **Docs condensed and de-Windows'd**: Removed Windows/PowerShell-specific instructions from the docs and condensed the longest documents into tighter, bash-first guides — `SETUP.md`, `DEPLOYMENT.md`, `API.md`, `MONITORING.md`, `STREAMING.md`, `COOLIFY.md`, `incident-runbook.md`, `operations/scaling-performance-playbook.md`, `architecture/README.md`, and the Prometheus metrics inventory. `scripts/loadtest/README.md` now documents the bash load-test helpers.
|
||||
- **System messages no longer persisted**: The chat backend no longer stores `system`-type messages; chat history and the admin chat-log table only contain real chat messages (moderation actions remain in the audit log).
|
||||
- **Reconnect grace period**: Disconnects schedule a delayed "left" announcement (5s); a reconnect within 5s cancels it and skips "joined", eliminating join/leave spam on brief reconnects.
|
||||
- **HLS segment duration 2s**: Default `HLS_SEGMENT_DURATION` changed from 6 to 2 and `HLS_WINDOW_SIZE` from 24 to 12 (2s × 12 = 24s live window), reducing end-to-end latency; the 2s GOP was already in place.
|
||||
- **Chat font sizing**: `.message-text` set to `0.95rem` (slightly smaller than the default, larger than the `0.83rem` system/flash text).
|
||||
- **R2 retention by run count + size**: Playout now keeps the newest `R2_KEEP_RUNS` run folders (default 2) and enforces an `R2_MAX_RETENTION_BYTES` cap (default 10 GiB), replacing the old 24h time-based `R2_SEGMENT_RETENTION_HOURS` cleanup. Cleanup runs after each new run plus an hourly backstop, and never deletes the live manifest. Adds `linearvoid_playout_r2_runs_retained` and `linearvoid_playout_r2_bytes_retained` gauges.
|
||||
- **Cloudflare Workers frontend hosting**: Replaced Cloudflare Pages with a Cloudflare Worker (`linearvoid-frontend`) that serves the static SPA via Workers Static Assets. Added `frontend/wrangler.toml` and renamed the deploy workflow to `.gitea/workflows/workers-deploy.yml` (builds the SPA and runs `npx wrangler deploy`). Architecture, setup/deployment, and ops documentation updated from Pages to Workers.
|
||||
- **Viewer session cookie cross-site**: The `linearvoid_user_session` cookie now uses `SameSite=None; Secure` over HTTPS (so it is sent cross-site from the Workers-served SPA) and `SameSite=Lax` over plain HTTP (local dev). The admin session cookie remains `SameSite=Strict`.
|
||||
- **Parallel playout startup validation**: `ValidatePlaylist` now deduplicates repeated paths and probes unique files concurrently through a bounded worker pool (`PLAYOUT_VALIDATE_WORKERS`, default `runtime.NumCPU()`); `ValidateVideo` runs its stream/format and keyframe `ffprobe` probes concurrently; and `main()` performs the strict validation once (removing the redundant `-validate` pre-step in `start-playout.sh`). This sharply cuts startup time on large or duplicated playlists.
|
||||
- **Chat sidebar cleanup**: Removed the DM search bar and the "Users" toggle button, and converted the login/register form into a modal that opens from "Sign in" only when signed out and is hidden once logged in.
|
||||
- **Cloudflare Workers hosting**: frontend moved from Cloudflare Pages to the `linearvoid-frontend` static-assets Worker; docs now describe `linearvoid-frontend` (SPA) and `linearvoid-media` (edge HLS), correcting stale `worker.js` and R2 custom-domain wording.
|
||||
- **Docs de-Windows'd & condensed**: bash-first guides for `SETUP`, `DEPLOYMENT`, `API`, `MONITORING`, `STREAMING`, `COOLIFY`, incident runbook, scaling playbook, architecture README, and Prometheus metrics inventory.
|
||||
- **Compose port de-collision**: playout on host port `8081`, nginx moved to `8082` (`NGINX_PORT` default updated) to avoid Coolify-reserved ports.
|
||||
- **Playout**: R2 retention now by run count + byte cap (`R2_KEEP_RUNS` default 2, `R2_MAX_RETENTION_BYTES` default 10 GiB, new gauges) replacing the 24h time-based cleanup; default `HLS_SEGMENT_DURATION` 2s / `HLS_WINDOW_SIZE` 12; deduplicated, concurrent startup validation.
|
||||
- **Chat/UX**: 5s reconnect grace window (no join/leave spam); `system` messages no longer persisted; smaller `.message-text`; sidebar cleanup (removed DM search + Users toggle, login/register now a modal).
|
||||
- **Auth**: viewer session cookie uses `SameSite=None; Secure` over HTTPS (cross-site SPA) and `Lax` over plain HTTP.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Login/register cross-origin API URLs**: `submitAuth` in `frontend/src/main.ts` now prefixes `/api/auth/login` and `/api/auth/register` with `VITE_PUBLIC_APP_URL` (falling back to same-origin), matching the other API calls so the Workers-served SPA reaches the app origin instead of a same-origin path.
|
||||
- **Duplicate chat history on reconnect/initial load**: `connectWebSocket` now clears `#messages` before each (re)connect, and the initial connect waits for session resolution, so room history renders once instead of being appended repeatedly.
|
||||
- **Run-scoped R2 manifest URIs (playout)**: `publishManifest` now rewrites bare relative media URIs to the current run folder (`r-{runID}/…`) via `rewriteManifestToRun` before the atomic `.tmp` → `CopyObject` publish, so the live `hls/playlist.m3u8` references resolve against the run-scoped segment keys on `media.linearvoid.tv`. Skipped in slate mode. (Live validation pending — see `TODO.md`.)
|
||||
- **Cross-origin auth**: login/register API calls now prefix `VITE_PUBLIC_APP_URL` so the Workers-served SPA reaches the app origin.
|
||||
- **Duplicate chat history**: `connectWebSocket` clears messages on each (re)connect and the initial connect waits for session resolution.
|
||||
|
||||
### Tests
|
||||
|
||||
- WebSocket: `TestHub_Broadcast_SystemNotPersisted`, `TestHub_ReconnectGracePeriod`.
|
||||
- Handlers: `TestHandleAdminChatLog_ExcludesSystem`.
|
||||
- Frontend: `presence flash` test.
|
||||
- Playout: `TestCleanupOldRuns`, `TestCleanupOldRuns_SizeCap`, `TestR2KeepRuns`, `TestR2MaxRetentionBytes`, `TestR2RunFromKey`.
|
||||
- Playout: `TestValidateWorkersConfig`, `TestValidatePlaylist_Dedup`.
|
||||
- Cloudflare media worker: Vitest unit tests (`cacheControlForPath`, `contentTypeForPath`, `parseByteRange`) and Miniflare R2 integration tests (immutable segment, short-TTL manifest, Range `206`, `404`, OPTIONS CORS, method guard).
|
||||
- WebSocket (`TestHub_Broadcast_SystemNotPersisted`, `TestHub_ReconnectGracePeriod`), handlers (`TestHandleAdminChatLog_ExcludesSystem`), frontend presence flash.
|
||||
- Playout: `TestCleanupOldRuns(_SizeCap)`, `TestR2KeepRuns`, `TestR2MaxRetentionBytes`, `TestR2RunFromKey`, `TestValidateWorkersConfig`, `TestValidatePlaylist_Dedup`.
|
||||
- Playout (run-scoped manifest rewrite): `TestRewriteManifestToRun`, `TestRewriteManifestToRun_AlreadyPrefixedUnchanged`, `TestRewriteManifestToRun_HandlesTSAndComments`, `TestRewriteManifestToRun_EmptyRunID`, `TestIsMediaURI`, `TestPublishManifest_RewritesToRunScopedURIs`, `TestPublishManifest_SlateNotRewritten`.
|
||||
- Cloudflare media worker: Vitest unit (`cacheControlForPath`, `contentTypeForPath`, `parseByteRange`) + Miniflare R2 integration (immutable segment, short-TTL manifest, Range `206`, `404`, OPTIONS, method guard).
|
||||
|
||||
## [1.0.13] - 2026-08-13
|
||||
|
||||
|
||||
@@ -7,7 +7,7 @@
|
||||
[](https://git.allucanget.biz/zwitschi/linearvoid/actions)
|
||||
[](https://git.allucanget.biz)
|
||||
[](LICENSE)
|
||||
[](https://git.allucanget.biz/zwitschi/linearvoid/releases)
|
||||
[](https://git.allucanget.biz/zwitschi/linearvoid/releases)
|
||||
[](CHANGELOG.md)
|
||||
|
||||
## Architecture
|
||||
|
||||
@@ -44,6 +44,9 @@ var (
|
||||
deleteObjects = func(client *s3.Client, input *s3.DeleteObjectsInput) (*s3.DeleteObjectsOutput, error) {
|
||||
return client.DeleteObjects(context.TODO(), input)
|
||||
}
|
||||
copyObject = func(client *s3.Client, input *s3.CopyObjectInput) (*s3.CopyObjectOutput, error) {
|
||||
return client.CopyObject(context.TODO(), input)
|
||||
}
|
||||
bucket = os.Getenv("R2_BUCKET_NAME")
|
||||
playoutMode = "starting" // "live", "slate", "degraded"
|
||||
restartCount = 0
|
||||
@@ -185,6 +188,8 @@ func watchAndPublishSegments(ctx context.Context, hlsOutputDir string) {
|
||||
}
|
||||
|
||||
// publishManifest uploads the manifest atomically: temp key first, then copy to final key (STREAMING.md contract).
|
||||
// Bare media URIs in the manifest are rewritten to the run-scoped R2 layout
|
||||
// (r-{runID}/…) so that every referenced segment/init resolves on the media host.
|
||||
func publishManifest(localManifestPath string) error {
|
||||
if r2Client == nil {
|
||||
if err := initR2Client(); err != nil {
|
||||
@@ -192,13 +197,33 @@ func publishManifest(localManifestPath string) error {
|
||||
}
|
||||
}
|
||||
|
||||
data, err := os.ReadFile(localManifestPath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("read manifest failed: %w", err)
|
||||
}
|
||||
|
||||
// Rewrite bare relative media URIs to the current run folder so the manifest
|
||||
// matches the run-scoped segment keys uploaded by watchAndPublishSegments /
|
||||
// syncLatestToR2 (candidate (a)). Slate mode is skipped: its manifest references
|
||||
// slate assets that are not published under the live run prefix.
|
||||
if playoutMode != "slate" {
|
||||
data = rewriteManifestToRun(runID, data)
|
||||
}
|
||||
|
||||
// Stage the (rewritten) manifest locally so the existing SyncToR2 path is reused.
|
||||
staging := localManifestPath + ".r2staging"
|
||||
if err := os.WriteFile(staging, data, 0o644); err != nil {
|
||||
return fmt.Errorf("write staging manifest failed: %w", err)
|
||||
}
|
||||
defer os.Remove(staging)
|
||||
|
||||
// 1. Upload to temp key
|
||||
if err := SyncToR2(localManifestPath, "hls/playlist.m3u8.tmp"); err != nil {
|
||||
if err := SyncToR2(staging, "hls/playlist.m3u8.tmp"); err != nil {
|
||||
return fmt.Errorf("temp manifest upload failed: %w", err)
|
||||
}
|
||||
|
||||
// 2. Atomic copy to final key
|
||||
_, err := r2Client.CopyObject(context.TODO(), &s3.CopyObjectInput{
|
||||
_, err = copyObject(r2Client, &s3.CopyObjectInput{
|
||||
Bucket: aws.String(bucket),
|
||||
CopySource: aws.String(bucket + "/hls/playlist.m3u8.tmp"),
|
||||
Key: aws.String("hls/playlist.m3u8"),
|
||||
@@ -210,6 +235,55 @@ func publishManifest(localManifestPath string) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// isMediaURI reports whether s names an HLS media file (a segment or init file).
|
||||
func isMediaURI(s string) bool {
|
||||
name := s
|
||||
if i := strings.IndexByte(name, '?'); i >= 0 {
|
||||
name = name[:i]
|
||||
}
|
||||
switch strings.ToLower(filepath.Ext(name)) {
|
||||
case ".m4s", ".ts", ".mp4", ".m4a":
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// rewriteManifestToRun prefixes bare relative media URIs in an HLS manifest with the
|
||||
// run folder, e.g. "seg-000001.m4s" -> "r-20260903T…/seg-000001.m4s" and
|
||||
// #EXT-X-MAP:URI="init.mp4" -> URI="r-20260903T…/init.mp4". This keeps the published
|
||||
// manifest consistent with the run-scoped R2 layout (STREAMING.md contract). Directives,
|
||||
// comments, blank lines, and URIs that already contain a path are left unchanged.
|
||||
// Returns data unchanged when runID is empty.
|
||||
func rewriteManifestToRun(runID string, data []byte) []byte {
|
||||
if runID == "" {
|
||||
return data
|
||||
}
|
||||
prefix := runID + "/"
|
||||
lines := strings.Split(string(data), "\n")
|
||||
out := make([]string, 0, len(lines))
|
||||
for _, line := range lines {
|
||||
trimmed := strings.TrimSpace(line)
|
||||
switch {
|
||||
case strings.HasPrefix(trimmed, "#EXT-X-MAP:URI="):
|
||||
if open := strings.Index(trimmed, `URI="`); open >= 0 {
|
||||
rest := trimmed[open+len(`URI="`):]
|
||||
if closeIdx := strings.IndexByte(rest, '"'); closeIdx > 0 {
|
||||
uri := rest[:closeIdx]
|
||||
if uri != "" && !strings.Contains(uri, "/") {
|
||||
line = strings.Replace(line, `URI="`+uri+`"`, `URI="`+prefix+uri+`"`, 1)
|
||||
}
|
||||
}
|
||||
}
|
||||
out = append(out, line)
|
||||
case trimmed != "" && !strings.HasPrefix(trimmed, "#") && !strings.Contains(trimmed, "/") && isMediaURI(trimmed):
|
||||
out = append(out, prefix+trimmed)
|
||||
default:
|
||||
out = append(out, line)
|
||||
}
|
||||
}
|
||||
return []byte(strings.Join(out, "\n"))
|
||||
}
|
||||
|
||||
// LoadPlaylist reads a playlist file and returns a list of VideoFile entries
|
||||
func LoadPlaylist(playlistPath string) ([]VideoFile, error) {
|
||||
file, err := os.Open(playlistPath)
|
||||
|
||||
@@ -0,0 +1,227 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/aws/aws-sdk-go-v2/aws"
|
||||
"github.com/aws/aws-sdk-go-v2/service/s3"
|
||||
)
|
||||
|
||||
const sampleManifest = `#EXTM3U
|
||||
#EXT-X-VERSION:7
|
||||
#EXT-X-TARGETDURATION:2
|
||||
#EXT-X-MEDIA-SEQUENCE:0
|
||||
#EXT-X-INDEPENDENT-SEGMENTS
|
||||
#EXT-X-MAP:URI="init.mp4"
|
||||
#EXTINF:2.000000,
|
||||
seg-000000.m4s
|
||||
#EXTINF:2.000000,
|
||||
seg-000001.m4s
|
||||
#EXT-X-ENDLIST
|
||||
`
|
||||
|
||||
func TestRewriteManifestToRun(t *testing.T) {
|
||||
runID := "r-20260903T120000Z"
|
||||
out := string(rewriteManifestToRun(runID, []byte(sampleManifest)))
|
||||
|
||||
wants := []string{
|
||||
`#EXT-X-MAP:URI="r-20260903T120000Z/init.mp4"`,
|
||||
"r-20260903T120000Z/seg-000000.m4s",
|
||||
"r-20260903T120000Z/seg-000001.m4s",
|
||||
"#EXTM3U",
|
||||
"#EXT-X-VERSION:7",
|
||||
"#EXT-X-TARGETDURATION:2",
|
||||
"#EXT-X-MEDIA-SEQUENCE:0",
|
||||
"#EXT-X-INDEPENDENT-SEGMENTS",
|
||||
"#EXT-X-ENDLIST",
|
||||
"#EXTINF:2.000000,",
|
||||
}
|
||||
for _, want := range wants {
|
||||
if !contains(out, want) {
|
||||
t.Errorf("output missing %q\noutput:\n%s", want, out)
|
||||
}
|
||||
}
|
||||
|
||||
// Media URIs must be prefixed (no bare segments left).
|
||||
for _, bare := range []string{"\nseg-000000.m4s\n", "\nseg-000001.m4s\n", `URI="init.mp4"`} {
|
||||
if contains(out, bare) {
|
||||
t.Errorf("output still contains bare %q\noutput:\n%s", bare, out)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRewriteManifestToRun_AlreadyPrefixedUnchanged(t *testing.T) {
|
||||
manifest := "#EXTM3U\n#EXT-X-MAP:URI=\"r-old/init.mp4\"\nseg-000000.m4s\nr-old/seg-000001.m4s\n#EXT-X-ENDLIST\n"
|
||||
out := string(rewriteManifestToRun("r-new", []byte(manifest)))
|
||||
|
||||
if !contains(out, `#EXT-X-MAP:URI="r-old/init.mp4"`) {
|
||||
t.Errorf("EXT-X-MAP already-prefixed URI was rewritten: %s", out)
|
||||
}
|
||||
// A URI line that already contains a path is left alone.
|
||||
if !contains(out, "r-old/seg-000001.m4s") {
|
||||
t.Errorf("already-prefixed segment URI was changed:\n%s", out)
|
||||
}
|
||||
// Bare URI still gets the run prefix.
|
||||
if !contains(out, "r-new/seg-000000.m4s") {
|
||||
t.Errorf("bare segment not prefixed:\n%s", out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRewriteManifestToRun_HandlesTSAndComments(t *testing.T) {
|
||||
manifest := "#EXTM3U\n# a comment\nvideo-1.ts\ninit.mp4\n#EXT-X-ENDLIST\n"
|
||||
out := string(rewriteManifestToRun("r-x", []byte(manifest)))
|
||||
|
||||
for _, want := range []string{"r-x/video-1.ts", "r-x/init.mp4", "# a comment", "#EXTM3U", "#EXT-X-ENDLIST"} {
|
||||
if !contains(out, want) {
|
||||
t.Errorf("output missing %q\noutput:\n%s", want, out)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRewriteManifestToRun_EmptyRunID(t *testing.T) {
|
||||
out := rewriteManifestToRun("", []byte(sampleManifest))
|
||||
if string(out) != sampleManifest {
|
||||
t.Errorf("empty runID should return input unchanged")
|
||||
}
|
||||
}
|
||||
|
||||
func TestIsMediaURI(t *testing.T) {
|
||||
for _, ok := range []string{"seg-000001.m4s", "init.mp4", "a.ts", "x.m4a"} {
|
||||
if !isMediaURI(ok) {
|
||||
t.Errorf("isMediaURI(%q) = false, want true", ok)
|
||||
}
|
||||
}
|
||||
for _, notOK := range []string{"playlist.m3u8", "readme.txt", "seg-000001.m4s?x=1"} {
|
||||
// Note: query strings are stripped; '?x=1' on m4s should still be media.
|
||||
if notOK == "seg-000001.m4s?x=1" {
|
||||
continue
|
||||
}
|
||||
if isMediaURI(notOK) {
|
||||
t.Errorf("isMediaURI(%q) = true, want false", notOK)
|
||||
}
|
||||
}
|
||||
if !isMediaURI("seg-000001.m4s?token=abc") {
|
||||
t.Errorf("isMediaURI with query should strip query and be true")
|
||||
}
|
||||
}
|
||||
|
||||
func TestPublishManifest_RewritesToRunScopedURIs(t *testing.T) {
|
||||
prevClient, prevBucket := r2Client, bucket
|
||||
prevRunID, prevMode := runID, playoutMode
|
||||
prevPut, prevCopy := putObject, copyObject
|
||||
defer func() {
|
||||
r2Client, bucket = prevClient, prevBucket
|
||||
runID, playoutMode = prevRunID, prevMode
|
||||
putObject, copyObject = prevPut, prevCopy
|
||||
}()
|
||||
|
||||
tmpDir := t.TempDir()
|
||||
mp := filepath.Join(tmpDir, "playlist.m3u8")
|
||||
if err := os.WriteFile(mp, []byte(sampleManifest), 0o644); err != nil {
|
||||
t.Fatalf("write manifest: %v", err)
|
||||
}
|
||||
|
||||
bucket = "linearvoid-media"
|
||||
runID = "r-20260903T120000Z"
|
||||
playoutMode = "live"
|
||||
r2Client = &s3.Client{}
|
||||
|
||||
var gotKey, gotCopySource, gotCopyKey string
|
||||
var gotBody []byte
|
||||
putObject = func(_ *s3.Client, in *s3.PutObjectInput) (*s3.PutObjectOutput, error) {
|
||||
gotKey = aws.ToString(in.Key)
|
||||
b, err := io.ReadAll(in.Body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
gotBody = b
|
||||
return &s3.PutObjectOutput{}, nil
|
||||
}
|
||||
copyObject = func(_ *s3.Client, in *s3.CopyObjectInput) (*s3.CopyObjectOutput, error) {
|
||||
gotCopySource = aws.ToString(in.CopySource)
|
||||
gotCopyKey = aws.ToString(in.Key)
|
||||
return &s3.CopyObjectOutput{}, nil
|
||||
}
|
||||
|
||||
if err := publishManifest(mp); err != nil {
|
||||
t.Fatalf("publishManifest error: %v", err)
|
||||
}
|
||||
|
||||
if gotKey != "hls/playlist.m3u8.tmp" {
|
||||
t.Errorf("temp upload key = %q, want hls/playlist.m3u8.tmp", gotKey)
|
||||
}
|
||||
if gotCopySource != "linearvoid-media/hls/playlist.m3u8.tmp" {
|
||||
t.Errorf("CopySource = %q, want linearvoid-media/hls/playlist.m3u8.tmp", gotCopySource)
|
||||
}
|
||||
if gotCopyKey != "hls/playlist.m3u8" {
|
||||
t.Errorf("final key = %q, want hls/playlist.m3u8", gotCopyKey)
|
||||
}
|
||||
|
||||
body := string(gotBody)
|
||||
for _, want := range []string{
|
||||
"r-20260903T120000Z/seg-000000.m4s",
|
||||
"r-20260903T120000Z/seg-000001.m4s",
|
||||
`#EXT-X-MAP:URI="r-20260903T120000Z/init.mp4"`,
|
||||
} {
|
||||
if !contains(body, want) {
|
||||
t.Errorf("published manifest missing %q:\n%s", want, body)
|
||||
}
|
||||
}
|
||||
for _, bare := range []string{"\nseg-000000.m4s\n", "\nseg-000001.m4s\n"} {
|
||||
if contains(body, bare) {
|
||||
t.Errorf("published manifest still contains bare %q:\n%s", bare, body)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestPublishManifest_SlateNotRewritten(t *testing.T) {
|
||||
prevClient, prevBucket := r2Client, bucket
|
||||
prevRunID, prevMode := runID, playoutMode
|
||||
prevPut, prevCopy := putObject, copyObject
|
||||
defer func() {
|
||||
r2Client, bucket = prevClient, prevBucket
|
||||
runID, playoutMode = prevRunID, prevMode
|
||||
putObject, copyObject = prevPut, prevCopy
|
||||
}()
|
||||
|
||||
tmpDir := t.TempDir()
|
||||
mp := filepath.Join(tmpDir, "playlist.m3u8")
|
||||
if err := os.WriteFile(mp, []byte(sampleManifest), 0o644); err != nil {
|
||||
t.Fatalf("write manifest: %v", err)
|
||||
}
|
||||
|
||||
bucket = "linearvoid-media"
|
||||
runID = "r-20260903T120000Z"
|
||||
playoutMode = "slate"
|
||||
r2Client = &s3.Client{}
|
||||
|
||||
var gotBody []byte
|
||||
putObject = func(_ *s3.Client, in *s3.PutObjectInput) (*s3.PutObjectOutput, error) {
|
||||
b, _ := io.ReadAll(in.Body)
|
||||
gotBody = b
|
||||
return &s3.PutObjectOutput{}, nil
|
||||
}
|
||||
copyObject = func(_ *s3.Client, _ *s3.CopyObjectInput) (*s3.CopyObjectOutput, error) {
|
||||
return &s3.CopyObjectOutput{}, nil
|
||||
}
|
||||
|
||||
if err := publishManifest(mp); err != nil {
|
||||
t.Fatalf("publishManifest error: %v", err)
|
||||
}
|
||||
|
||||
if string(gotBody) != sampleManifest {
|
||||
t.Errorf("slate manifest should be published unmodified; got:\n%s", string(gotBody))
|
||||
}
|
||||
}
|
||||
|
||||
func contains(s, sub string) bool {
|
||||
for i := 0; i+len(sub) <= len(s); i++ {
|
||||
if s[i:i+len(sub)] == sub {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
+35
-8
@@ -3,7 +3,7 @@
|
||||
## Environments
|
||||
|
||||
| Environment | Host | Docker Target | Purpose |
|
||||
|---|---|---|---|
|
||||
| -------------- | -------------------------- | -------------------------- | ---------------------------------------------- |
|
||||
| **Test Lab** | `192.168.88.40` (internal) | `tcp://192.168.88.40:2375` | Integration testing, staging, CI/CD validation |
|
||||
| **Production** | Hetzner VPS (TBD) | `tcp://<hetzner-ip>:2375` | Public linearvoid.tv |
|
||||
|
||||
@@ -14,12 +14,14 @@
|
||||
**Status: Not provisioned.** Plan documented and pipeline ready; only the VPS needs creation.
|
||||
|
||||
### Planned Infrastructure
|
||||
|
||||
- Provider: Hetzner Cloud, Region: Ashburn, VA (us-east)
|
||||
- Instance: CX22 (2 vCPU, 4 GB RAM, 40 GB SSD) or equivalent
|
||||
- OS: Ubuntu 24.04 LTS or Debian 12
|
||||
- Bandwidth: 20 TB included traffic/month; Estimated cost: ~€7–10/month base
|
||||
|
||||
### Cutover Plan
|
||||
|
||||
1. Provision Hetzner VPS
|
||||
2. Add production secrets to Gitea (`CI_PROD_*` prefix)
|
||||
3. Update `DOCKER_HOST` in CD workflow to Hetzner public IP
|
||||
@@ -29,6 +31,7 @@
|
||||
7. Decommission test lab as public target
|
||||
|
||||
### Production Preflight (minimum readiness gate)
|
||||
|
||||
1. Verify Hetzner host reachable over SSH and Docker remote API TLS works for the Gitea runner
|
||||
2. Confirm all production secrets exist under `CI_PROD_*` prefix
|
||||
3. Confirm `linearvoid.tv`, `www.linearvoid.tv`, `media.linearvoid.tv` DNS targets match intended phase
|
||||
@@ -37,8 +40,9 @@
|
||||
6. Verify rollback possible before final DNS switch
|
||||
|
||||
### Firewall / Security Group
|
||||
|
||||
| Port | Source | Purpose |
|
||||
|---|---|---|
|
||||
| ---- | --------------------- | ------------------------------ |
|
||||
| 22 | Admin IPs | SSH management |
|
||||
| 80 | 0.0.0.0/0 | HTTP (redirect to HTTPS) |
|
||||
| 443 | 0.0.0.0/0 | HTTPS (nginx) |
|
||||
@@ -47,8 +51,9 @@
|
||||
| 3001 | Admin IPs or internal | Grafana (or tunnel via SSH) |
|
||||
|
||||
### Pipeline Readiness
|
||||
|
||||
| Workflow | Trigger | Purpose |
|
||||
|---|---|---|
|
||||
| -------------------- | ----------------- | --------------------------------------------------------------------------------- |
|
||||
| `ci.yml` | Push to `main` | Lint, build frontend, vet Go, build & push images (`:latest`, `:main`) |
|
||||
| `cd.yml` | Push to `main` | Deploy to test lab (`192.168.88.40:2375`) — runs **in parallel** with CI |
|
||||
| `release.yml` | Tag push `*.*.*` | Build & push versioned images (`:X.Y.Z`, `:X.Y`, `:latest`), create Gitea Release |
|
||||
@@ -67,6 +72,7 @@ DOCKER_HOST=tcp://<hetzner-public-ip>:2375
|
||||
No pipeline code changes needed to switch targets.
|
||||
|
||||
## Target Topology
|
||||
|
||||
- Host: Linux server with Docker Engine
|
||||
- Reverse proxy: nginx container (linearvoid-nginx)
|
||||
- Core services: playout, chat, redis, postgres
|
||||
@@ -74,6 +80,7 @@ No pipeline code changes needed to switch targets.
|
||||
- CDN/edge: Cloudflare
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Docker Engine + Docker Compose plugin
|
||||
- Network access to git.allucanget.biz
|
||||
- Valid `.env` file with production values
|
||||
@@ -82,6 +89,7 @@ No pipeline code changes needed to switch targets.
|
||||
## Environment File
|
||||
|
||||
`.env` requires at minimum:
|
||||
|
||||
- `DATABASE_URL` — PostgreSQL connection string
|
||||
- `REDIS_URL` — Redis connection string
|
||||
- `ADMIN_USERNAME` / `ADMIN_PASSWORD` — admin dashboard credentials
|
||||
@@ -89,18 +97,21 @@ No pipeline code changes needed to switch targets.
|
||||
- `VITE_PUBLIC_APP_URL` / `VITE_PUBLIC_WS_URL` / `VITE_PUBLIC_HLS_URL` — frontend URLs
|
||||
|
||||
**Cloudflare R2 (required for HLS segment sync to edge):**
|
||||
|
||||
- `R2_ACCESS_KEY_ID` / `R2_SECRET_ACCESS_KEY`
|
||||
- `R2_BUCKET_NAME` (e.g. `linearvoid-media`)
|
||||
- `R2_ENDPOINT` — S3-compatible endpoint URL
|
||||
- `R2_REGION` — set to `auto`
|
||||
|
||||
**Cloudflare (optional, for Workers):**
|
||||
|
||||
- `CLOUDFLARE_API_TOKEN` / `CLOUDFLARE_ACCOUNT_ID`
|
||||
- `MEDIA_DOMAIN` — split media domain (e.g. `media.linearvoid.tv`)
|
||||
|
||||
See `.env.example` and `SETUP.md` for full variable reference.
|
||||
|
||||
### Gitea Secrets
|
||||
|
||||
`cd.yml` injects sensitive values from Gitea secrets into `.env` at deploy time. Configure in the Gitea repository:
|
||||
|
||||
`DATABASE_URL`, `ADMIN_USERNAME`, `ADMIN_PASSWORD`, `R2_ACCESS_KEY_ID`, `R2_SECRET_ACCESS_KEY`, `R2_BUCKET_NAME`, `R2_ENDPOINT`, `R2_ADMIN_ACCESS_KEY_ID` (optional), `R2_ADMIN_SECRET_ACCESS_KEY` (optional), `R2_ADMIN_ENDPOINT` (optional), `CLOUDFLARE_API_TOKEN` (optional), `CLOUDFLARE_ACCOUNT_ID` (optional), `CI_REGISTRY_USER`, `CI_REGISTRY_TOKEN`
|
||||
@@ -116,6 +127,7 @@ docker compose -f docker-compose.prod.yml up -d --remove-orphans
|
||||
```
|
||||
|
||||
## First-Time or Recovery Provisioning
|
||||
|
||||
```bash
|
||||
mkdir -p data/chat data/playlist data/media data/hls
|
||||
docker volume ls
|
||||
@@ -124,10 +136,13 @@ docker compose -f docker-compose.prod.yml up -d
|
||||
```
|
||||
|
||||
## Health Verification
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.prod.yml ps
|
||||
```
|
||||
|
||||
Endpoint checks:
|
||||
|
||||
- App: `http://{host}:8080/`
|
||||
- Nginx health: `http://{host}:8080/health`
|
||||
- Chat health: `http://{host}:3000/health`
|
||||
@@ -136,30 +151,38 @@ Endpoint checks:
|
||||
- Grafana: `http://{host}:3001`
|
||||
|
||||
Domain checks:
|
||||
|
||||
- `https://linearvoid.tv`
|
||||
- `https://linearvoid.tv/admin`
|
||||
- `https://linearvoid.tv/metrics`
|
||||
|
||||
## Zero/Low-Downtime Update Pattern
|
||||
|
||||
1. Pull newest images.
|
||||
2. Recreate changed services only:
|
||||
```bash
|
||||
docker compose -f docker-compose.prod.yml pull
|
||||
docker compose -f docker-compose.prod.yml up -d nginx chat playout
|
||||
```
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.prod.yml pull
|
||||
docker compose -f docker-compose.prod.yml up -d nginx chat playout
|
||||
```
|
||||
|
||||
3. Re-run health checks.
|
||||
|
||||
## Rollback
|
||||
|
||||
If new images are unhealthy:
|
||||
|
||||
1. Pin previous tags in `docker-compose.prod.yml`.
|
||||
2. Pull pinned tags.
|
||||
3. Restart impacted services:
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.prod.yml pull
|
||||
docker compose -f docker-compose.prod.yml up -d nginx chat playout
|
||||
```
|
||||
|
||||
## Operational Tasks
|
||||
|
||||
```bash
|
||||
# View logs
|
||||
docker compose -f docker-compose.prod.yml logs -f nginx chat playout
|
||||
@@ -175,23 +198,27 @@ docker compose -f docker-compose.prod.yml down -v
|
||||
```
|
||||
|
||||
## Nginx and Proxy Considerations
|
||||
|
||||
- If an external proxy (e.g. Nginx Proxy Manager) fronts the stack, align request body limits with internal nginx (see `infrastructure/nginx/nginx.conf`).
|
||||
- For resumable admin uploads, ensure `client_max_body_size` supports chunk requests in both proxy layers.
|
||||
|
||||
## Security Baseline
|
||||
|
||||
- Use a strong `ADMIN_PASSWORD` and rotate regularly.
|
||||
- Keep Docker host access restricted.
|
||||
- Prefer HTTPS/WSS through Cloudflare.
|
||||
- Restrict exposed ports if only proxy access is needed.
|
||||
|
||||
## Common Deployment Pitfalls
|
||||
|
||||
- Missing frontend build artifacts in local-only compose workflows.
|
||||
- Wrong registry credentials or missing login.
|
||||
- Misaligned domain env vars (app/ws/hls) causing mixed-origin errors.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Check | Fix |
|
||||
|---|---|---|
|
||||
| -------------------------- | --------------------------------------------- | ------------------------------------------------------------- |
|
||||
| Service not starting | `docker logs linearvoid-<svc> --tail 50` | Check `.env` vars, volume mounts, port conflicts |
|
||||
| Playout stall / slate mode | `curl http://<host>:8081/health` — mode=slate | Check playlist file, media paths, disk space |
|
||||
| Chat not connecting | `curl http://<host>:3000/health` | Check Redis: `docker exec linearvoid-redis redis-cli ping` |
|
||||
|
||||
+14
-4
@@ -27,7 +27,7 @@ Nginx `/hls/` is a local/test fallback, not the production origin. The SPA, admi
|
||||
Only validated files may enter the production playlist.
|
||||
|
||||
| Property | Required profile |
|
||||
| ---------- | ------------------------------------------------------------------------ |
|
||||
| ---------- | ----------------------------------------------------------------------- |
|
||||
| Container | MP4 |
|
||||
| Video | H.264/AVC, `yuv420p` |
|
||||
| Resolution | 1280x720 or 1920x1080; avoid resolution changes within a playlist block |
|
||||
@@ -63,7 +63,7 @@ ffmpeg -i input.mov \
|
||||
The playout service wraps FFmpeg, tracks the active item/revision, restarts failed processes, and exposes `/health` plus Prometheus metrics.
|
||||
|
||||
| Setting | Value |
|
||||
| -------------- | ---------------------------------------------------------------------------------- |
|
||||
| ---------------- | ------------------------------------------------------------------------ |
|
||||
| Protocol | Standard live HLS, not LL-HLS |
|
||||
| Segment format | fMP4 (`.m4s`) with init segment; `.ts` may be supported during migration |
|
||||
| Segment target | 2 seconds (`HLS_SEGMENT_DURATION=2`) |
|
||||
@@ -96,12 +96,22 @@ Order prevents a manifest referencing an unavailable segment:
|
||||
|
||||
Never upload an actively written segment or publish the manifest first. Use a single publisher (or fencing/leader election) so two workers can't update the live manifest concurrently.
|
||||
|
||||
**Run-scoped manifest URIs.** Segments and the init file are stored run-scoped under `hls/r-{runID}/…`, so the live `hls/playlist.m3u8` must reference them with that run prefix. Before the final commit step, the publisher rewrites each bare relative media URI (segments plus `#EXT-X-MAP:URI`) to `r-{runID}/<name>`. `publishManifest` in `backend/playout/src/playlist.go` does this via `rewriteManifestToRun`, using the same `runID` under which the segments were uploaded; the rewrite is skipped in slate mode (its manifest is served as-is). The published manifest therefore contains, e.g.:
|
||||
|
||||
```m3u8
|
||||
#EXT-X-MAP:URI="r-20260807T170412Z/init.mp4"
|
||||
#EXTINF:2.000000,
|
||||
r-20260807T170412Z/seg-000001234.m4s
|
||||
```
|
||||
|
||||
The `linearvoid-media` Worker on `media.linearvoid.tv` is a faithful path→R2 mapper and serves these run-scoped paths directly; it does **not** rewrite URIs at serve time.
|
||||
|
||||
`data/hls/` is regenerable staging. Source media, playlist revisions, and schedule state are authoritative; generated HLS is not backed up as application state.
|
||||
|
||||
## Cache contract
|
||||
|
||||
| Objects | Edge policy | Browser policy | Required headers/behavior |
|
||||
| -------------------------- | --------------------------------- | --------------------------------------- | ---------------------------------------------------------------------------- |
|
||||
| -------------------------- | --------------------------- | --------------------------------------- | ---------------------------------------------------------------------------- |
|
||||
| `.m4s`, `.ts`, init `.mp4` | Cache eligible; TTL 1 year | TTL 1 year | `Cache-Control: public, max-age=31536000, immutable`; full path in cache key |
|
||||
| Live `.m3u8` | Cache eligible; edge TTL 2s | Do not retain beyond freshness interval | `Cache-Control: public, max-age=1, s-maxage=2, must-revalidate` |
|
||||
| Versioned JS/CSS | Cache eligible; TTL 1 year | TTL 1 year | Content-hashed filenames and `immutable` |
|
||||
@@ -160,7 +170,7 @@ A release is streaming-ready only when: one full playlist boundary plays without
|
||||
## Required configuration
|
||||
|
||||
| Variable | Purpose |
|
||||
| ------------------------------------------ | -------------------------------------------------- |
|
||||
| ------------------------------------------ | ------------------------------------- |
|
||||
| `PLAYLIST_PATH` | Active playlist file |
|
||||
| `HLS_SEGMENT_DURATION` | Target segment duration; baseline `2` |
|
||||
| `HLS_WINDOW_SIZE` | Manifest window; baseline `12` |
|
||||
|
||||
@@ -156,11 +156,12 @@ graph TD
|
||||
graph TB
|
||||
subgraph "Cloudflare Edge"
|
||||
CF[CDN + WAF<br/>Cache Rules]
|
||||
R2[R2 Bucket<br/>media + HLS]
|
||||
WORKERS[Workers<br/>SPA hosting]
|
||||
R2["R2 Bucket<br/>media + HLS<br/>hls/r-{runID}/… run-scoped"]
|
||||
MEDIA[linearvoid-media Worker<br/>R2-bound · media.linearvoid.tv]
|
||||
WORKERS[linearvoid-frontend Worker<br/>SPA hosting]
|
||||
end
|
||||
|
||||
subgraph "Docker Host (Hetzner CX22)"
|
||||
subgraph "Docker Host"
|
||||
NGINX[nginx<br/>:80/:443<br/>proxy + auth]
|
||||
CHAT[chat service<br/>:3000<br/>Go + gorilla/websocket]
|
||||
PLAYOUT[playout service<br/>:8081<br/>Go + FFmpeg]
|
||||
@@ -173,13 +174,14 @@ graph TB
|
||||
ADMIN[Admin Browser<br/>SPA + uploads]
|
||||
end
|
||||
|
||||
VIEWER -->|HTTPS| CF
|
||||
VIEWER -->|WSS| NGINX
|
||||
VIEWER -->|HTTPS SPA| CF
|
||||
VIEWER -->|WSS Chat| NGINX
|
||||
VIEWER -->|HTTPS /hls| MEDIA
|
||||
ADMIN -->|HTTPS| CF
|
||||
CF --> NGINX
|
||||
MEDIA --> R2
|
||||
NGINX --> CHAT
|
||||
NGINX --> PLAYOUT
|
||||
NGINX -->|/hls/*| R2
|
||||
CHAT --> REDIS
|
||||
CHAT --> DB
|
||||
PLAYOUT --> R2
|
||||
@@ -189,7 +191,8 @@ graph TB
|
||||
**Playout Service** (`backend/playout/src/playlist.go`)
|
||||
|
||||
- Role: continuous HLS encoding loop via FFmpeg concat demuxer.
|
||||
- Input: `data/playlist/playlist.txt` (ordered MP4 paths); Output: HLS segments (`.m4s`, `.ts`) + live `.m3u8` to `data/hls/`, mirrored to R2.
|
||||
- Input: `data/playlist/playlist.txt` (ordered MP4 paths); Output: HLS segments (`.m4s`, `.ts`) + live `.m3u8` to `data/hls/`, mirrored to R2 run-scoped under `hls/r-{runID}/…`.
|
||||
- Run-scoped manifest: before publishing the live `hls/playlist.m3u8`, `publishManifest` rewrites bare media URIs to the current run folder (`rewriteManifestToRun`), so manifest → segment links resolve on `media.linearvoid.tv`. See [`STREAMING.md`](../STREAMING.md).
|
||||
- Resilience: restart counter (>5/60s → slate); Prometheus on 8081.
|
||||
- Validation: ffprobe gate (codec/fps/resolution/keyframe); `-validate` mode checks playlist without running.
|
||||
- Seamless looping: resets HLS only when playlist fingerprint (SHA-256) changes, crash/slate recovery, or first run; else continues via `-start_number`.
|
||||
@@ -267,7 +270,7 @@ graph TB
|
||||
|
||||
1. Admin adds normalized MP4 to R2 + updates playlist.
|
||||
2. Playout worker (FFmpeg concat loop) reads next file, packages HLS segments (2s).
|
||||
3. Segments uploaded to R2 with unique immutable names; live `.m3u8` updated.
|
||||
3. Segments uploaded to R2 with unique immutable, run-scoped names (`hls/r-{runID}/…`); the live `.m3u8` is published referencing those run-prefixed URIs (see `STREAMING.md`).
|
||||
4. Viewer loads SPA → requests manifest → hls.js follows current edge.
|
||||
5. Cloudflare serves `.m4s`/`.ts` from nearest edge cache (origin hit once per POP/region).
|
||||
6. Parallel WebSocket to chat service for synchronized interaction.
|
||||
@@ -400,7 +403,7 @@ _Cache rules baseline_: `.ts`/`.m4s` Cache Everything, long TTL, immutable; `.m3
|
||||
|
||||
_Edge security/controls_: enforce HTTPS/WSS + redirect HTTP; WAF baseline; rate limits on abusive patterns (auth/admin); minimize origin exposure.
|
||||
|
||||
_Worker & header policy_: the `linearvoid-media` Worker ([`infrastructure/cloudflare/`](../../infrastructure/cloudflare/), R2-bound) owns `media.linearvoid.tv` and enforces immutable caching for segments, short TTL for live manifests, plus HLS CORS/Range; the `linearvoid-frontend` Worker ([`frontend/wrangler.toml`](../../frontend/wrangler.toml)) serves the static SPA on `linearvoid.tv`. Automated deploys: `workers-deploy.yml` (SPA) and `workers-media-deploy.yml` (media).
|
||||
_Worker & header policy_: the `linearvoid-media` Worker ([`infrastructure/cloudflare/`](../../infrastructure/cloudflare/), R2-bound) owns `media.linearvoid.tv` and enforces immutable caching for segments, short TTL for live manifests, plus HLS CORS/Range; it is a faithful path→R2 mapper and does **not** rewrite manifest URIs at serve time (playout rewrites them to run-scoped paths at publish — see [`STREAMING.md`](../STREAMING.md)). The `linearvoid-frontend` Worker ([`frontend/wrangler.toml`](../../frontend/wrangler.toml)) serves the static SPA on `linearvoid.tv`. Automated deploys: `workers-deploy.yml` (SPA) and `workers-media-deploy.yml` (media). Ops detail: [`cloudflare-workers-setup.md`](../operations/cloudflare-workers-setup.md).
|
||||
|
||||
_Operations & validation_: verify cache status progression (MISS→HIT), manifest freshness under active playout; track hit ratio, request/error rates, ToS/cost posture.
|
||||
|
||||
|
||||
@@ -21,6 +21,8 @@ Both are managed under **Workers & Pages** in the Cloudflare dashboard.
|
||||
|
||||
The Worker in [`infrastructure/cloudflare/`](../../infrastructure/cloudflare/) runs on the `linearvoid-media` Workers service. It owns the `media.linearvoid.tv` host, serves HLS objects from the R2 bucket (bound as `MEDIA_BUCKET`), and stamps immutable `Cache-Control` on segments plus a short TTL on live manifests (CORS + Range supported). See its [README](../../infrastructure/cloudflare/README.md).
|
||||
|
||||
**Run-scoped layout.** R2 objects are stored run-scoped (`hls/r-{runID}/seg-*.m4s`, `…/init.mp4`), and the live `hls/playlist.m3u8` references them with the run prefix (the playout rewrites bare URIs before publishing — see [`docs/STREAMING.md`](../STREAMING.md)). The Worker is a faithful path→R2 mapper: it serves whatever key the request path resolves to (e.g. `/hls/r-…/seg-*.m4s`), adds the cache headers by file type, and does not rewrite manifest URIs at serve time.
|
||||
|
||||
Test / deploy:
|
||||
|
||||
```bash
|
||||
|
||||
Generated
+2
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "frontend",
|
||||
"version": "1.0.11",
|
||||
"version": "1.1.0",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "frontend",
|
||||
"version": "1.0.11",
|
||||
"version": "1.1.0",
|
||||
"dependencies": {
|
||||
"hls.js": "^1.6.16"
|
||||
},
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "frontend",
|
||||
"private": true,
|
||||
"version": "1.0.13",
|
||||
"version": "1.1.0",
|
||||
"type": "module",
|
||||
"engines": {
|
||||
"node": ">=22.0.0"
|
||||
|
||||
@@ -18,8 +18,14 @@ bucket_name = "linearvoid-media"
|
||||
# npx wrangler deploy
|
||||
#
|
||||
# Production environment: attach the media host as a custom domain on this worker.
|
||||
# Note: bindings are NOT inherited by environments, so r2_buckets is redeclared here.
|
||||
# npx wrangler deploy --env production
|
||||
[env.production]
|
||||
routes = [
|
||||
{ pattern = "media.linearvoid.tv", custom_domain = true }
|
||||
]
|
||||
|
||||
[[env.production.r2_buckets]]
|
||||
binding = "MEDIA_BUCKET"
|
||||
bucket_name = "linearvoid-media"
|
||||
|
||||
Reference in new issue
Block a user