docs: fix filename typo and add Cloudflare Pages deploy details across docs
This commit is contained in:
zwitschi committed
1 parent
7be66a703c
commit
1019972e25
13 files changed
+393
-403
No files matched your search
@@ -2,10 +2,6 @@
|
||||
|
||||
Agents instructions for the linearvoid project.
|
||||
|
||||
## Output
|
||||
|
||||
- use /caveman skill, latest version from <https://github.com/JuliusBrussee/caveman> is <https://github.com/JuliusBrussee/caveman/tree/main/skills/caveman>
|
||||
|
||||
## General Guidelines
|
||||
|
||||
- Use the architecture documentation (arc42 template and ADRs in `docs`) as the source of truth for design decisions.
|
||||
|
||||
+142
-31
@@ -1,62 +1,173 @@
|
||||
# BACKLOG
|
||||
|
||||
> Consolidated from `BACKLOG.md`, `DEVELOPMENT.md`, and `TODO.md` on 2026-08-11.
|
||||
|
||||
## Cloudflare Configuration
|
||||
|
||||
### DNS & Pages
|
||||
|
||||
- [x] Configure DNS for linearvoid.tv apex and www redirect (manual)
|
||||
- [x] Configure media.linearvoid.tv for split-domain HLS delivery (manual)
|
||||
- [x] Configure Cloudflare Pages deploy workflow (`.gitea/workflows/pages-deploy.yml`) and setup docs (`docs/operations/cloudflare-pages-setup.md`, `docs/operations/gitea-cloudflare-pages.md`)
|
||||
- [x] Create Cloudflare Pages project in dashboard (unavailable, created `linearvoid` via API)
|
||||
- [ ] Cloudflare Pages: link custom domain (manual)
|
||||
- [ ] Create Cloudflare Pages project and cut over frontend hosting (manual)
|
||||
|
||||
### Storage & Cache
|
||||
|
||||
- [x] Create R2 bucket (manual)
|
||||
- [x] Configure Cache Rules: .ts/.m4s + .m3u8 (manual)
|
||||
- [x] Configure Cloudflare Pages deploy workflow (`.gitea/workflows/pages-deploy.yml`) and setup docs (`docs/operations/cloudflare-pages-setup.md`, `docs/operations/gitea-cloudflare.pages.md`)
|
||||
- [ ] Create Cloudflare Pages project in dashboard and link custom domain (manual)
|
||||
- [ ] Enable Tiered Cache, WAF, rate limiting (manual)
|
||||
- [x] Implement Worker for immutable cache headers on segment paths
|
||||
- [ ] Enforce HTTPS/WSS, redirect HTTP → HTTPS (manual)
|
||||
- [ ] CORS headers on HLS responses from media.linearvoid.tv (MEDIUM) — needs Cloudflare R2 CORS config
|
||||
|
||||
### Security & TLS
|
||||
|
||||
- [ ] Enable Tiered Cache, WAF, rate limiting (manual)
|
||||
- [ ] Tiered Cache: enable Smart Tiered Cache, validate `Cf-Cache-Status: HIT` from upper tier
|
||||
- [ ] WAF: enable OWASP Managed Ruleset, set to Simulate → review 48h → Block, enable Bot Fight Mode
|
||||
- [ ] Rate limiting: `/api/admin/login` + `/api/admin/*` → 10 req/10s per IP, Challenge action
|
||||
- [ ] HTTPS/WSS: Always Use HTTPS, HSTS max-age=31536000 includeSubDomains preload, min TLS 1.2
|
||||
- [ ] HTTPS/WSS: enforce at edge (Cloudflare) and origin (nginx), redirect HTTP → HTTPS
|
||||
|
||||
## Monitoring & Observability
|
||||
|
||||
### Prometheus & Grafana
|
||||
|
||||
- [x] Set up Prometheus with chat/playout/nginx/redis/postgres/node scrape targets
|
||||
- [x] Add nginx-exporter, redis-exporter, postgres-exporter, node-exporter
|
||||
- [x] Import Grafana app + operational dashboards
|
||||
- [x] Define Prometheus alerting rules (8 rules: ChatDown, PlayoutDown, RedisDown, etc.)
|
||||
|
||||
### External Monitoring
|
||||
|
||||
- [ ] Set up UptimeRobot external monitoring
|
||||
- [ ] UptimeRobot: add HTTP monitor for `https://linearvoid.tv/health` (5min interval)
|
||||
- [ ] UptimeRobot: add keyword monitor for `https://linearvoid.tv/hls/playlist.m3u8` (keyword: `#EXTM3U`, 5min)
|
||||
- [ ] Alert on 2 consecutive failures, notify via email + Discord/Slack webhook
|
||||
|
||||
## Security
|
||||
|
||||
- [x] Implement admin cookie-session auth (HttpOnly, SameSite=Strict, Secure when TLS)
|
||||
- [x] Implement server-side session expiration
|
||||
- [ ] Enforce HTTPS/WSS at edge and origin
|
||||
- [x] Implement rate limiting for chat messages (Redis windowing)
|
||||
- [x] Implement WebSocket connection rate limiting (5/IP/60s)
|
||||
- [x] Implement admin login rate limiting (5/IP/5min)
|
||||
- [ ] Enforce HTTPS/WSS at edge and origin
|
||||
|
||||
## Scaling & Performance
|
||||
## Test Fixtures
|
||||
|
||||
- [x] Create scaling/performance playbook (methods, thresholds, execution order)
|
||||
- [x] Load-test scaffolding: ws-chat.yml, admin-api.js, run-loadtests.ps1, summarizers
|
||||
- [x] Cache-hit ratio exporter (scripts/cache-hit-ratio.js + .ps1)
|
||||
- [x] Saturation alerts: chat p95, Redis memory, cache-hit ratio
|
||||
- [ ] Create Cloudflare Pages project and cut over frontend hosting (manual)
|
||||
- [ ] Optimize segment immutability and manifest TTL for cache efficiency
|
||||
- [ ] Measure playback TTFB from multiple US regions
|
||||
- [ ] Execute WebSocket load tests with real infrastructure
|
||||
- [ ] Execute admin API benchmarks with real infrastructure
|
||||
- [ ] Track cache hit ratio and R2 ops in production
|
||||
- [ ] Plan Phase 2 service separation
|
||||
- [ ] Plan Phase 3 multi-origin resilience
|
||||
- [x] Create test media fixtures in `backend/playout/testdata/`: valid H.264/AAC, invalid codec, invalid framerate
|
||||
- [x] Create test playlist fixtures: `valid_playlist.txt`, `empty_playlist.txt`, `comments_only.txt`, `mixed_valid_invalid.txt`
|
||||
- [x] Document ffmpeg fixture-generation commands in test file headers
|
||||
|
||||
## Frontend Tests
|
||||
|
||||
### User Frontend
|
||||
|
||||
- [x] Video player behavior (play, pause, seek, buffering, error handling) — HLS player startup, native/fallback, URL resolution, error recovery with retries/backoff/reconnecting overlay
|
||||
- [x] Chat functionality (send/receive messages, presence, rate limiting, moderation) — send, receive, presence join/leave, DM flow, reconnect covered
|
||||
- [x] State management and data flow
|
||||
- [x] Responsive design and layout across different screen sizes — chat-collapsed toggle, CSS grid, now-playing aria
|
||||
- [x] Accessibility testing (keyboard navigation, screen reader support) — video controls, aria-labels, aria-expanded, input labels, reconnecting overlay aria-live/role
|
||||
- [x] Integration with backend APIs (mocking API responses for testing)
|
||||
- [ ] Test coverage for all critical frontend components and interactions
|
||||
- [ ] Add code coverage reporting and enforce minimum coverage threshold in CI/CD pipeline
|
||||
- [ ] Document test cases and expected outcomes
|
||||
|
||||
### Admin Frontend
|
||||
|
||||
- [x] Playlist management UI (add/remove videos, reorder, normalize) — save/error handling, sortable rendering
|
||||
- [ ] User moderation tools (mute/ban/report users, view moderation logs)
|
||||
- [x] Metrics and monitoring dashboards (Prometheus/Grafana integration) — overview + analytics rendering
|
||||
- [x] State management and data flow
|
||||
- [x] Responsive design and layout across different screen sizes — tab content visibility, essential tabs
|
||||
- [x] Accessibility testing (keyboard navigation, screen reader support) — nav aria-label, active tab aria-current, nav button focusability
|
||||
- [x] Integration with backend APIs (mocking API responses for testing)
|
||||
- [ ] Test coverage for all critical admin UI components and interactions
|
||||
- [ ] Add code coverage reporting and enforce minimum coverage threshold in CI/CD pipeline
|
||||
- [ ] Document test cases and expected outcomes
|
||||
|
||||
## Backend Integration Tests
|
||||
|
||||
### Cloudflare R2
|
||||
|
||||
- [x] Upload test HLS segments and manifests to R2 bucket (SyncToR2 with mock, verified bucket/key/body)
|
||||
- [x] Verify correct cache headers are applied (immutable segments via segmentCacheControl, short-lived manifests via manifestCacheControl)
|
||||
- [ ] Test retrieval of segments and manifests from R2 via Cloudflare edge
|
||||
- [ ] Validate cache hit/miss behavior and performance metrics
|
||||
- [ ] Validate edge cache hit/miss behavior with repeat requests — script `scripts/validate-edge-cache.sh` created
|
||||
- [x] Ensure integration test can run in CI/CD pipeline with proper mocking/stubbing of R2 API (putObject mock, no network calls)
|
||||
- [x] Document test cases and expected outcomes
|
||||
- [x] Document required stubs or mocks (putObject variable injection pattern)
|
||||
|
||||
## Streaming Pipeline
|
||||
|
||||
Gaps from 2026-08-10 audit → 2026-08-10 implementation (STREAMING.md contract vs playlist.go implementation):
|
||||
- [x] **Continuous R2 sync during playout (HIGH)**: fsnotify watcher uploads segments in real-time (~7s delay)
|
||||
- [x] **Globally unique segment naming (HIGH)**: `r-{ISOTIME}/seg-{seq}.m4s` via run ID + ffmpeg monotonic sequence
|
||||
- [x] **Atomic manifest publication (MEDIUM)**: Upload to `playlist.m3u8.tmp`, then `CopyObject` to final key
|
||||
- [x] **Manifest Cache-Control alignment (LOW)**: `public, max-age=1, s-maxage=2, must-revalidate`
|
||||
- [ ] **CORS headers on HLS responses (MEDIUM)**: Needs Cloudflare R2 bucket CORS config (infrastructure, not code)
|
||||
- [x] **Player error recovery (MEDIUM)**: `hls.on(Hls.Events.ERROR)` handler with bounded retries (5x), exponential backoff, reconnecting overlay
|
||||
- [x] **Health endpoint detail (LOW)**: Added active item, run ID, playlist revision, last segment seq, last R2 pub, consecutive failures
|
||||
- [x] Source media contract + validation (H.264/AAC, 30fps, keyframe check)
|
||||
- [x] fMP4 HLS packaging (6s segments, 24 window)
|
||||
- [x] Slate fallback + recovery (`shouldEscalateToSlate`)
|
||||
- [x] R2 cache headers (immutable segments, short-lived manifest)
|
||||
- [x] Cloudflare Worker immutable `.m4s` headers
|
||||
|
||||
- [x] **Continuous R2 sync during playout (HIGH)**: `syncLatestToR2()` now uses fsnotify watcher to upload segments in real-time (~7s delay).
|
||||
- [x] **Globally unique segment naming (HIGH)**: Implemented `r-{ISOTIME}/seg-{seq}.m4s` naming via run ID + ffmpeg monotonic sequence.
|
||||
- [x] **Atomic manifest publication (MEDIUM)**: Upload manifest to `playlist.m3u8.tmp`, then `CopyObject` to final key.
|
||||
- [x] **Manifest Cache-Control alignment (LOW)**: Updated to `public, max-age=1, s-maxage=2, must-revalidate`.
|
||||
- [ ] **CORS headers on HLS responses (MEDIUM)**: Needs Cloudflare R2 bucket CORS config (infrastructure, not code) — CORS already set in nginx for same-origin; media.linearvoid.tv needs Cloudflare setting.
|
||||
- [x] **Player error recovery (MEDIUM)**: Added `hls.on(Hls.Events.ERROR)` handler with bounded retries (5x), exponential backoff, reconnecting overlay.
|
||||
- [x] **Health endpoint detail (LOW)**: Added active item, run ID, playlist revision, last segment seq, last R2 pub, consecutive failures.
|
||||
- [x] Source media contract + validation (H.264/AAC, 30fps, keyframe check) — implemented
|
||||
- [x] fMP4 HLS packaging (6s segments, 24 window) — implemented
|
||||
- [x] Slate fallback + recovery (`shouldEscalateToSlate`) — implemented
|
||||
- [x] R2 cache headers (immutable segments, short-lived manifest) — implemented
|
||||
- [x] Cloudflare Worker immutable `.m4s` headers — implemented
|
||||
## External Services
|
||||
|
||||
> Evidence tracker: `docs/operations/cloudflare-uptimerobot-checklist.md`
|
||||
|
||||
- [ ] UptimeRobot: add HTTP monitor for `https://linearvoid.tv/health` (5min interval)
|
||||
- [ ] UptimeRobot: add keyword monitor for `https://linearvoid.tv/hls/playlist.m3u8` (keyword: `#EXTM3U`, 5min)
|
||||
- [ ] Alert on 2 consecutive failures, notify via email + Discord/Slack webhook
|
||||
|
||||
## Production Deployment
|
||||
|
||||
### Host Provisioning (Hetzner — not provisioned)
|
||||
|
||||
- [ ] Provision Hetzner CX22 (2 vCPU, 4 GB, 40 GB) in Ashburn VA, Ubuntu 24.04
|
||||
- [ ] Install Docker Engine + Compose plugin, open firewall: 80/443 (public), 2375 (runner IP only)
|
||||
- [ ] Configure Docker daemon for remote TLS access (`-H tcp://0.0.0.0:2376` with certs)
|
||||
- [ ] Create `/srv/linearvoid/data/{playlist,media,hls,chat,slate}` directories
|
||||
|
||||
### CI/CD & Secrets
|
||||
|
||||
- [ ] Add `CI_PROD_*` secrets to Gitea: DOCKER_HOST, R2 keys, admin password, DB password, Grafana password
|
||||
- [ ] Trigger CD workflow with `DOCKER_HOST` pointing to Hetzner IP
|
||||
|
||||
### Smoke Test & Cutover
|
||||
|
||||
- [ ] Verify all 11 containers start healthy, check `/health`, `/hls/playlist.m3u8`, WebSocket
|
||||
- [ ] Update Cloudflare A record `linearvoid.tv` → Hetzner IP, SSL Full (strict), test HTTPS + WSS
|
||||
- [ ] Smoke test: viewer SPA loads, HLS plays, chat connects, admin login → upload → playlist → playout
|
||||
|
||||
### Production Operations
|
||||
|
||||
- [ ] Set up Prometheus alerting notifications + UptimeRobot on production URL
|
||||
- [ ] Run `scripts/backup-db.sh` + `scripts/verify-backups.sh` on production host
|
||||
- [ ] Document production IP, SSH access, rollback procedure, scaling thresholds
|
||||
|
||||
## Scaling & Performance
|
||||
|
||||
### Load Testing
|
||||
|
||||
- [x] Load-test scaffolding: `ws-chat.yml`, `admin-api.js`, `run-loadtests.ps1`, summarizers, README
|
||||
- [x] Cache-hit ratio exporter (`scripts/cache-hit-ratio.js` + `.ps1`)
|
||||
- [x] Saturation alerts: chat p95 latency, Redis memory usage, cache-hit ratio
|
||||
- [ ] Execute WebSocket load tests with real infrastructure — ramp 10→100→500→1000 connections, measure p95 latency + errors
|
||||
- [ ] Execute admin API benchmarks with real infrastructure — `wrk`/`k6` against `/api/admin/*`, target p95 <1s
|
||||
|
||||
### Cache & TTFB Optimization
|
||||
|
||||
- [ ] Optimize segment immutability and manifest TTL for cache efficiency
|
||||
- [ ] Tune manifest TTL: measure `Cf-Cache-Status` distribution, adjust to 0.5× segment duration
|
||||
- [ ] Measure playback TTFB from multiple US regions — WebPageTest from 3+ regions, target <4s warm cache
|
||||
- [ ] Track cache hit ratio and R2 ops in production — Cloudflare GraphQL → Grafana panel, alert <90%
|
||||
|
||||
### Capacity & Future Planning
|
||||
|
||||
- [ ] Capacity triggers: Prometheus rules for cache-miss >10%/15min, CPU >80%/10min
|
||||
- [ ] Plan Phase 2 service separation — document boundaries (playout CPU, chat latency, DB state, monitoring), estimate per-service resources
|
||||
- [ ] Plan Phase 3 multi-origin resilience — research second Hetzner location, design warm standby + DNS failover + DB/Redis replication
|
||||
@@ -48,67 +48,173 @@ flowchart TD
|
||||
|
||||
## Project Structure
|
||||
|
||||
This structure directly reflects the building blocks, runtime flows, deployment view, and cross-cutting concepts defined in the arc42 documentation.
|
||||
|
||||
```text
|
||||
linearvoid/
|
||||
├── frontend/ # Static Single-Page Application (HTML/CSS/JS)
|
||||
│ ├── src/
|
||||
│ │ ├── components/ # Video player, chat sidebar, now-playing overlay
|
||||
│ │ ├── js/ # hls.js integration, WebSocket client, UI logic
|
||||
│ │ └── styles/ # Tailwind / custom CSS
|
||||
│ ├── index.html
|
||||
│ └── package.json # Build tools (Vite recommended)
|
||||
├── backend/
|
||||
│ ├── playout/ # Continuous HLS generation from playlist
|
||||
│ │ ├── src/ # FFmpeg orchestration, playlist manager
|
||||
│ │ ├── config/
|
||||
│ │ └── Dockerfile
|
||||
│ ├── chat/ # Real-time WebSocket service
|
||||
│ │ ├── src/ # Rooms, DMs, moderation, presence (Node/Go)
|
||||
│ │ ├── config/
|
||||
│ │ └── Dockerfile
|
||||
│ └── shared/ # Common utilities, types, normalization helpers
|
||||
├── infrastructure/ # Deployment, IaC, Cloudflare configs
|
||||
│ ├── docker-compose.yml # Local/dev orchestration
|
||||
│ ├── nginx/ # Proxy, static serving, HLS delivery
|
||||
│ ├── cloudflare/ # Cache rule examples, Workers
|
||||
│ └── terraform/ # (optional) provisioning
|
||||
├── scripts/ # Automation scripts
|
||||
│ ├── normalize-video.sh # Pre-process MP4s (codec, keyframes, CRF)
|
||||
│ ├── sync-to-r2.sh # Upload segments/manifests to R2
|
||||
│ └── backup.sh
|
||||
├── monitoring/ # Dashboards, alerts, logging
|
||||
│ ├── grafana/
|
||||
│ └── prometheus/
|
||||
├── docs/
|
||||
│ └── architecture/
|
||||
│ ├── README.md # arc42 template (fully populated)
|
||||
│ └── adr/
|
||||
│ ├── ADR-001.md # Consolidated decision record
|
||||
│ ├── ADR-002-module-structure.md
|
||||
│ ├── ADR-003-tech-stack.md
|
||||
│ ├── ADR-004-admin-video.md
|
||||
│ ├── ADR-005-admin-moderation-analytics.md
|
||||
│ ├── ADR-006-data-storage.md
|
||||
│ └── ADR-007-redis-usage.md
|
||||
├── .env.example
|
||||
├── .gitignore
|
||||
├── docker-compose.yml # Top-level for dev (playout + chat + nginx)
|
||||
├── README.md
|
||||
└── LICENSE
|
||||
├── docs/ # see Documentation section below
|
||||
├── backend/ # see Backend section below
|
||||
├── frontend/ # see Frontend section below
|
||||
├── infrastructure/ # see Infrastructure section below
|
||||
├── monitoring/ # see Monitoring section below
|
||||
├── scripts/ # see Scripts section below
|
||||
├── .editorconfig # EditorConfig for consistent coding styles
|
||||
├── .env.example # Example environment variables for local development
|
||||
├── .gitignore # Ignore node_modules, build artifacts, etc.
|
||||
├── .prettierignore # Ignore files for Prettier formatting
|
||||
├── CHANGELOG.md # Release notes and version history
|
||||
├── CONTRIBUTING.md # Contribution guidelines
|
||||
├── docker-compose.prod.yml # Production deployment (Hetzner + Cloudflare)
|
||||
├── docker-compose.yml # Top-level for dev (playout + chat + nginx)
|
||||
└── README.md # This file
|
||||
```
|
||||
|
||||
### Folder Explanations
|
||||
### Documentation
|
||||
|
||||
- **`frontend/`**: The viewer experience — minimal, modern SPA with Tailwind + Alpine.js (or vanilla). Uses `hls.js` for playback and a WebSocket client for chat.
|
||||
- **`backend/playout/`**: Core linear engine. FFmpeg consumes a normalized playlist and outputs continuous HLS. Includes normalization validation.
|
||||
- **`backend/chat/`**: Independent WebSocket service (Node.js/Socket.io or Go). Handles public rooms, private DMs, presence (Redis), persistence (PostgreSQL/SQLite), rate limiting, and moderation.
|
||||
- **`backend/shared/`**: Reusable code (types, utils, video validation).
|
||||
- **`infrastructure/`**: Everything needed to deploy and configure the stack (Docker, Nginx, Cloudflare rules, future Terraform).
|
||||
- **`scripts/`**: One-off and CI-friendly scripts for video preparation, R2 sync, backups, and playlist management.
|
||||
- **`monitoring/`**: Grafana dashboards, Prometheus rules, logging configuration.
|
||||
- **`docs/architecture/`**: Complete arc42 documentation + ADRs 001-007 (the single source of truth for all architectural decisions).
|
||||
- `docs/`: Folder containing all documentation for the project, including architecture, ADRs, operational procedures, and setup guides.
|
||||
- `docs/architecture/`: Architecture documentation, including high-level system architecture, component interactions, and decision records (ADRs).
|
||||
|
||||
This structure directly reflects the building blocks, runtime flows, deployment view, and cross-cutting concepts defined in the arc42 documentation.
|
||||
see [README.md](docs/README.md) for full architecture, ADRs, and operational documentation.
|
||||
|
||||
### Backend
|
||||
|
||||
- `backend/playout/`: Core linear engine. FFmpeg consumes a normalized playlist and outputs continuous HLS. Includes normalization validation.
|
||||
- `backend/chat/`: Independent WebSocket service (Node.js/Socket.io or Go). Handles public rooms, private DMs, presence (Redis), persistence (PostgreSQL/SQLite), rate limiting, and moderation.
|
||||
- `backend/shared/`: Reusable code (types, utils, video validation).
|
||||
|
||||
```text
|
||||
backend/
|
||||
├── chat/ # WebSocket chat service (Node.js or Go)
|
||||
│ ├── src/ # Chat service source code (rooms, DMs, moderation)
|
||||
| | ├── auth/ # Admin authentication and session management
|
||||
| | | ├── auth_test.go # Unit tests for authentication
|
||||
| | | └── auth.go # Authentication logic (login, session validation)
|
||||
| | ├── config/ # Configuration files for chat service
|
||||
| | | ├── config_test.go # Unit tests for configuration
|
||||
| | | └── config.go # Configuration loading and validation
|
||||
| | ├── db/ # Database access and models
|
||||
| | | ├── db_test.go # Unit tests for database access
|
||||
| | | ├── db.go # Database connection and query logic
|
||||
| | | ├── models.go # Database models (users, messages, rooms)
|
||||
| | | └── redis.go # Redis connection and presence management
|
||||
| | ├── handlers/ # WebSocket message handlers
|
||||
| | | ├── handlers_test.go # Unit tests for WebSocket message handlers
|
||||
| | | └── handlers.go # WebSocket message handling logic
|
||||
| | ├── media/ # Media-related utilities (e.g., video validation)
|
||||
| | | ├── media_test.go # Unit tests for media-related utilities
|
||||
| | | └── media.go # Media-related utility functions
|
||||
| | ├── metrics/ # Prometheus metrics endpoints
|
||||
| | | ├── metrics_test.go # Unit tests for Prometheus metrics
|
||||
| | | └── metrics.go # Prometheus metrics collection and exposition
|
||||
| | ├── ratelimit/ # Rate limiting logic (Redis-based)
|
||||
| | | ├── ratelimit_test.go # Unit tests for rate limiting
|
||||
| | | └── ratelimit.go # Rate limiting logic using Redis
|
||||
| | ├── ws/ # WebSocket handlers
|
||||
| | | ├── ws_test.go # Unit tests for WebSocket handlers
|
||||
| | | └── ws.go # WebSocket connection and message handling
|
||||
| | └── main.go # Entry point for chat service
|
||||
| ├── Dockerfile # Dockerfile for chat service
|
||||
| ├── go.mod # Go module file for chat service
|
||||
| └── go.sum # Go checksum file for chat service
|
||||
└── playout/ # Continuous HLS generation from playlist (Go)
|
||||
├── demodata/ # Sample media and playlist files for testing
|
||||
| ├── media/ # Sample media files for testing
|
||||
| └── playlist/ # Sample playlist files for testing
|
||||
├── src/ # Playout service source code (FFmpeg orchestration)
|
||||
| ├── go.mod # Go module file for playout service
|
||||
| ├── go.sum # Go checksum file for playout service
|
||||
| ├── playlist_test.go # Unit tests for playlist management
|
||||
| └── playlist.go # Playlist management logic
|
||||
└── testdata/ # Test data for playout service (sample playlists, media)
|
||||
| ├── comments_only.txt # Sample playlist with comments only
|
||||
| ├── empty_playlist.txt # Sample empty playlist
|
||||
| ├── invalid_24fps.mp4 # Sample invalid 24fps video for testing
|
||||
| ├── invalid_h265.mp4 # Sample invalid H.265 video for testing
|
||||
| ├── mixed_valid_invalid.txt # Sample playlist with mixed valid and invalid entries
|
||||
| ├── valid_720p30.mp4 # Sample valid 720p30 video for testing
|
||||
| └── valid_playlist.txt # Sample valid playlist for testing
|
||||
├── Dockerfile # Dockerfile for the playout service
|
||||
└── start-playout.sh # Script to start the playout service
|
||||
```
|
||||
|
||||
### Frontend
|
||||
|
||||
The viewer experience — minimal, modern SPA with custom CSS + vanilla JS. Uses `hls.js` for playback and a WebSocket client for chat.
|
||||
|
||||
```text
|
||||
frontend/
|
||||
├── admin/ # Admin SPA (playlist management, moderation, analytics)
|
||||
│ ├── login/ # Login page for admin authentication
|
||||
| | └── index.html # Admin login page
|
||||
| └── index.html # Admin SPA
|
||||
├── public/ # Static assets (favicon, images, etc.)
|
||||
│ ├── admin/ # Admin static assets
|
||||
| | ├── analytics.html # Analytics dashboard page
|
||||
| | ├── chat-log.html # Chat log viewer page
|
||||
| | ├── dashboard.html # Admin dashboard page
|
||||
| | ├── moderation.html # User moderation page
|
||||
| | ├── playlist.html # Playlist management page
|
||||
| | ├── schedule.html # Schedule overrides page
|
||||
| | ├── system.html # System configuration page
|
||||
| | ├── upload.html # Video upload page
|
||||
| | └── users.html # User management page
|
||||
│ ├── favicon.ico # Favicon for the SPA
|
||||
| └── icons.svg # SVG icons for the SPA
|
||||
├── src/ # Source code for the SPA
|
||||
│ ├── assets/ # Images, fonts, and other static assets
|
||||
| | ├── hero.png # Hero image for the SPA
|
||||
| | ├── typescript.svg # TypeScript logo for the SPA
|
||||
| | └── vite.svg # Vite logo for the SPA
|
||||
│ ├── admin.test.ts # Admin frontend tests (Vitest)
|
||||
│ ├── admin.ts # Admin SPA logic (playlist, moderation, analytics)
|
||||
│ ├── counter.ts # Counter component for testing
|
||||
│ ├── main.test.ts # Main frontend tests (Vitest)
|
||||
│ ├── main.ts # Main SPA logic (hls.js player, WebSocket chat)
|
||||
│ └── style.css # Custom CSS for the SPA
|
||||
├── .gitignore # Ignore node_modules, build artifacts, etc.
|
||||
├── index.html # Main SPA entry point
|
||||
├── package-lock.json # Lockfile for npm dependencies
|
||||
├── package.json # Build tools (Vite recommended)
|
||||
├── tsconfig.json # TypeScript configuration
|
||||
├── vite.config.ts # Vite configuration
|
||||
└── vitest.config.ts # Vitest configuration
|
||||
```
|
||||
|
||||
### Infrastructure
|
||||
|
||||
Everything needed to deploy and configure the stack (Docker, Nginx, Cloudflare rules, future Terraform).
|
||||
|
||||
- Cloudflare
|
||||
- [worker.js](infrastructure/cloudflare/worker.js) — adds immutable cache headers to HLS segments
|
||||
- Nginx
|
||||
- [Dockerfile](infrastructure/nginx/Dockerfile) — builds Nginx with custom config
|
||||
- [nginx.conf](infrastructure/nginx/nginx.conf) — reverse proxy, static SPA, HLS delivery
|
||||
|
||||
### Monitoring
|
||||
|
||||
Grafana dashboards, Prometheus rules, logging configuration.
|
||||
|
||||
- Grafana
|
||||
- dashboards
|
||||
- [dashboard.json](monitoring//grafana/dashboards/dashboard.json) — operational dashboards for playout, chat, and Redis
|
||||
- [dashboard.yml](monitoring//grafana/dashboards/dashboard.yml) — operational dashboards for playout, chat, and Redis
|
||||
- datasources
|
||||
- [datasource.yml](monitoring/grafana/datasources/datasource.yml) — Prometheus datasource configuration
|
||||
- [dashboard.json](monitoring/grafana/dashboard.json) — operational dashboards for playout, chat, and Redis
|
||||
- Prometheus
|
||||
- [alerts.yml](monitoring/prometheus/alerts.yml) — alerting rules for playout, chat, Redis, and system health
|
||||
- [prometheus.yml](monitoring/prometheus/prometheus.yml) — scrape targets for playout, chat, Redis, and system metrics
|
||||
|
||||
### Scripts
|
||||
|
||||
One-off and CI-friendly scripts for video preparation, R2 sync, backups, and playlist management.
|
||||
|
||||
- loadtest/
|
||||
- [admin-api.js](scripts/loadtest/admin-api.js) — admin API load test script
|
||||
- [run-loadtests.ps1](scripts/loadtest/run-loadtests.ps1) — PowerShell script to run load tests
|
||||
- [summarize-artillery.ps1](scripts/loadtest/summarize-artillery.ps1) — PowerShell script to summarize Artillery load test results
|
||||
- [summarize-k6.ps1](scripts/loadtest/summarize-k6.ps1) — PowerShell script to summarize k6 load test results
|
||||
- [ws-chat.yml](scripts/loadtest/ws-chat.yml) — WebSocket chat load test configuration
|
||||
|
||||
## Getting Started
|
||||
|
||||
|
||||
+4
-115
@@ -1,116 +1,5 @@
|
||||
# DEVELOPMENT.md — Linear Void Remaining Tasks
|
||||
# DEVELOPMENT.md
|
||||
|
||||
## Test Fixtures (COMPLETE)
|
||||
|
||||
- [x] Create test media fixtures in `backend/playout/testdata/`:
|
||||
- `valid_720p30.mp4` — small (1-2s) H.264/AAC compliant file
|
||||
- `invalid_h265.mp4` — wrong codec
|
||||
- `invalid_24fps.mp4` — wrong framerate
|
||||
- Generate via `ffmpeg` commands documented in test file headers
|
||||
- Windows dev path: `C:\Users\zwits\AppData\Local\Microsoft\WinGet\Packages\Gyan.FFmpeg_Microsoft.Winget.Source_8wekyb3d8bbwe\ffmpeg-9.0-full_build\bin\ffmpeg.exe`
|
||||
|
||||
- [x] Create test playlist fixtures in `backend/playout/testdata/`:
|
||||
- `valid_playlist.txt`, `empty_playlist.txt`, `comments_only.txt`, `mixed_valid_invalid.txt`
|
||||
|
||||
- [x] Document ffmpeg fixture-generation commands in `playlist_test.go` header
|
||||
|
||||
## User Frontend Tests (PARTIALLY COMPLETE)
|
||||
|
||||
- [x] Video player behavior (play, pause, seek, buffering, error handling) - HLS player startup, native/fallback, URL resolution, error recovery with retries/backoff/reconnecting overlay
|
||||
- [x] Chat functionality (send/receive messages, presence, rate limiting, moderation) - send, receive, presence join/leave, DM flow, reconnect covered
|
||||
- [x] State management and data flow
|
||||
- [x] Responsive design and layout across different screen sizes - chat-collapsed toggle, CSS grid, now-playing aria
|
||||
- [x] Accessibility testing (keyboard navigation, screen reader support) - video controls, aria-labels, aria-expanded, input labels, reconnecting overlay aria-live/role
|
||||
- [x] Integration with backend APIs (mocking API responses for testing)
|
||||
- [ ] Test coverage for all critical frontend components and interactions
|
||||
- [ ] Add code coverage reporting and enforce minimum coverage threshold in CI/CD pipeline
|
||||
- [ ] Document test cases and expected outcomes
|
||||
|
||||
## Admin Frontend Tests (PARTIALLY COMPLETE)
|
||||
|
||||
- [x] Playlist management UI (add/remove videos, reorder, normalize) - save/error handling, sortable rendering
|
||||
- [ ] User moderation tools (mute/ban/report users, view moderation logs)
|
||||
- [x] Metrics and monitoring dashboards (Prometheus/Grafana integration) - overview + analytics rendering
|
||||
- [x] State management and data flow
|
||||
- [x] Responsive design and layout across different screen sizes - tab content visibility, essential tabs
|
||||
- [x] Accessibility testing (keyboard navigation, screen reader support) - nav aria-label, active tab aria-current, nav button focusability
|
||||
- [x] Integration with backend APIs (mocking API responses for testing)
|
||||
- [ ] Test coverage for all critical admin UI components and interactions
|
||||
- [ ] Add code coverage reporting and enforce minimum coverage threshold in CI/CD pipeline
|
||||
- [ ] Document test cases and expected outcomes
|
||||
|
||||
## Cloudflare R2 Integration Test (PARTIALLY COMPLETE)
|
||||
|
||||
- [x] Upload test HLS segments and manifests to R2 bucket (SyncToR2 with mock, verified bucket/key/body)
|
||||
- [x] Verify correct cache headers are applied (immutable segments via segmentCacheControl, short-lived manifests via manifestCacheControl)
|
||||
- [ ] Test retrieval of segments and manifests from R2 via Cloudflare edge
|
||||
- [ ] Validate cache hit/miss behavior and performance metrics
|
||||
- [x] Ensure integration test can run in CI/CD pipeline with proper mocking/stubbing of R2 API (putObject mock, no network calls)
|
||||
- [x] Document test cases and expected outcomes (TestSyncToR2 + TestSyncToR2_ManifestCacheHeaders + TestSyncToR2_NoCacheHeadersForUnknownExt)
|
||||
- [x] Document required stubs or mocks (putObject variable injection pattern)
|
||||
|
||||
## Cloudflare Configuration (manual — Cloudflare dashboard)
|
||||
|
||||
> Evidence tracker: `docs/operations/cloudflare-uptimerobot-checklist.md`
|
||||
> Cloudflare Pages deploy workflow created (`.gitea/workflows/pages-deploy.yml`); setup docs written (`docs/operations/cloudflare-pages-setup.md`, `docs/operations/gitea-cloudflare.pages.md`).
|
||||
|
||||
- [ ] Cloudflare Pages: create project in dashboard, link custom domain, or deploy via Gitea Actions Direct Upload
|
||||
- [ ] Tiered Cache: enable Smart Tiered Cache, validate `Cf-Cache-Status: HIT` from upper tier
|
||||
- [ ] WAF: enable OWASP Managed Ruleset, set to Simulate → review 48h → Block, enable Bot Fight Mode
|
||||
- [ ] Rate limiting: `/api/admin/login` + `/api/admin/*` → 10 req/10s per IP, Challenge action
|
||||
- [ ] HTTPS/WSS: Always Use HTTPS, HSTS max-age=31536000 includeSubDomains preload, min TLS 1.2
|
||||
- [ ] HTTPS/WSS: enforce at edge (Cloudflare) and origin (nginx), redirect HTTP → HTTPS
|
||||
|
||||
## External Services
|
||||
|
||||
> Evidence tracker: `docs/operations/cloudflare-uptimerobot-checklist.md`
|
||||
|
||||
- [ ] UptimeRobot: add HTTP monitor for `https://linearvoid.tv/health` (5min interval)
|
||||
- [ ] UptimeRobot: add keyword monitor for `https://linearvoid.tv/hls/playlist.m3u8` (keyword: `#EXTM3U`, 5min)
|
||||
- [ ] Alert on 2 consecutive failures, notify via email + Discord/Slack webhook
|
||||
|
||||
## Production Deployment (Hetzner — not provisioned)
|
||||
|
||||
- [ ] Provision Hetzner CX22 (2 vCPU, 4 GB, 40 GB) in Ashburn VA, Ubuntu 24.04
|
||||
- [ ] Install Docker Engine + Compose plugin, open firewall: 80/443 (public), 2375 (runner IP only)
|
||||
- [ ] Configure Docker daemon for remote TLS access (`-H tcp://0.0.0.0:2376` with certs)
|
||||
- [ ] Create `/srv/linearvoid/data/{playlist,media,hls,chat,slate}` directories
|
||||
- [ ] Add `CI_PROD_*` secrets to Gitea: DOCKER_HOST, R2 keys, admin password, DB password, Grafana password
|
||||
- [ ] Trigger CD workflow with `DOCKER_HOST` pointing to Hetzner IP
|
||||
- [ ] Verify all 11 containers start healthy, check `/health`, `/hls/playlist.m3u8`, WebSocket
|
||||
- [ ] Update Cloudflare A record `linearvoid.tv` → Hetzner IP, SSL Full (strict), test HTTPS + WSS
|
||||
- [ ] Smoke test: viewer SPA loads, HLS plays, chat connects, admin login → upload → playlist → playout
|
||||
- [ ] Set up Prometheus alerting notifications + UptimeRobot on production URL
|
||||
- [ ] Run `scripts/backup-db.sh` + `scripts/verify-backups.sh` on production host
|
||||
- [ ] Document production IP, SSH access, rollback procedure, scaling thresholds
|
||||
|
||||
## Scaling & Performance (planning only)
|
||||
|
||||
- [ ] Tune manifest TTL: measure `Cf-Cache-Status` distribution, adjust to 0.5× segment duration
|
||||
- [ ] Measure TTFB: WebPageTest from 3+ US regions, target <4s warm cache, document baseline
|
||||
- [ ] Load-test WebSocket: `artillery` or Go script, ramp 10→100→500→1000 connections, measure p95 latency + errors (scaffolding complete; benchmark execution pending)
|
||||
- [ ] Benchmark admin API: `wrk`/`k6` against `/api/admin/*`, target p95 <1s, identify slow queries (scaffolding complete; benchmark execution pending)
|
||||
- [x] Load-test scaffolding: ws-chat.yml, admin-api.js, run-loadtests.ps1, summarizers, README
|
||||
- [x] Cache-hit ratio exporter (scripts/cache-hit-ratio.js + .ps1)
|
||||
- [x] Saturation alerts: chat p95 latency, Redis memory usage, cache-hit ratio
|
||||
- [ ] Track cache hit ratio: Cloudflare GraphQL → Grafana panel, alert <90%
|
||||
- [ ] Capacity triggers: Prometheus rules for cache-miss >10%/15min, CPU >80%/10min (chat p95, Redis memory, cache-hit already done)
|
||||
- [ ] Phase 2 plan: document boundaries (playout CPU, chat latency, DB state, monitoring), estimate per-service resources
|
||||
- [ ] Phase 3 plan: research second Hetzner location, design warm standby + DNS failover + DB/Redis replication
|
||||
|
||||
## Streaming Pipeline (2026-08-10 audit → 2026-08-10 implemented)
|
||||
|
||||
Gaps between STREAMING.md contract and playlist.go/main.ts implementation:
|
||||
|
||||
- [x] **Continuous R2 sync during playout (HIGH)**: fsnotify watcher uploads segments in real-time (~7s delay)
|
||||
- [x] **Globally unique segment naming (HIGH)**: `r-{ISOTIME}/seg-{seq}.m4s` via run ID + monotonic sequence
|
||||
- [x] **Atomic manifest publication (MEDIUM)**: Upload to `.tmp`, then `CopyObject` to final key
|
||||
- [x] **Manifest Cache-Control alignment (LOW)**: `public, max-age=1, s-maxage=2, must-revalidate`
|
||||
- [ ] **CORS headers on HLS responses (MEDIUM)**: Needs Cloudflare R2 CORS config (infrastructure, not code)
|
||||
- [x] **Player error recovery (MEDIUM)**: `hls.on(ERROR)` handler with 5x retries, exponential backoff (2s→32s), reconnecting overlay
|
||||
- [x] **Health endpoint detail (LOW)**: Added run_id, active_item, playlist_revision, last_segment_seq, last_r2_pub, consecutive_failures, unhealthy detection
|
||||
- [x] Source media contract + validation — implemented
|
||||
- [x] fMP4 HLS packaging (6s, 24 window) — implemented
|
||||
- [x] Slate fallback + recovery — implemented
|
||||
- [x] R2 cache headers (immutable segments, short-lived manifest) — implemented
|
||||
- [x] Cloudflare Worker immutable `.m4s` headers — implemented
|
||||
> **All task lists have been consolidated into [`.github/instructions/BACKLOG.md`](../.github/instructions/BACKLOG.md).**
|
||||
>
|
||||
> This file is kept as a reference for development setup instructions. See [`SETUP.md`](SETUP.md) for environment setup.
|
||||
@@ -87,12 +87,7 @@
|
||||
|
||||
## Companion Documentation
|
||||
|
||||
- [API.md](../API.md) - Implemented WebSocket and REST API reference.
|
||||
- [DEPLOYMENT.md](../DEPLOYMENT.md) - Production deployment and operational runbook.
|
||||
- [MONITORING.md](../MONITORING.md) - Metrics, dashboards, and monitoring troubleshooting.
|
||||
- [SECURITY.md](../SECURITY.md) - Security model, controls, and hardening baseline.
|
||||
- [SETUP.md](../SETUP.md) - Test lab setup, local development, and CI/CD workflow.
|
||||
- [STREAMING.md](../STREAMING.md) - Streaming architecture, playout, and delivery details.
|
||||
see [Documentation README](../README.md) for an overview of all documentation, including architecture, ADRs, operational procedures, and setup guides.
|
||||
|
||||
## Introduction and Goals
|
||||
|
||||
@@ -727,7 +722,7 @@ These ADRs together form the canonical decision log.
|
||||
- Self-managed US-East playout (Hetzner Ashburn) with FFmpeg + MediaMTX/SRS → HLS → Cloudflare R2 + edge caching.
|
||||
- Module structure that directly reflects the arc42 building blocks and context diagram.
|
||||
- Go preferred for performance-critical services; TypeScript for frontend; Docker for all deployment.
|
||||
- Admin interfaces for video management, moderation, analytics, and system configuration using HTMX + Tailwind.
|
||||
- Admin interfaces for video management, moderation, analytics, and system configuration using HTMX + custom CSS.
|
||||
- SQLite as primary database for v1 with PostgreSQL migration path if needed.
|
||||
- Redis for rate limiting, presence, pub/sub, session storage with defined TTL policies and fallback behavior.
|
||||
|
||||
|
||||
@@ -69,7 +69,7 @@ Adopt a **self-managed US-East playout + Cloudflare R2 + edge-cached HLS** archi
|
||||
|
||||
### 3. Frontend
|
||||
|
||||
- Static SPA (HTML/CSS/JS + Tailwind + Alpine.js or vanilla) hosted on Cloudflare Pages or same VPS/Nginx.
|
||||
- Static SPA (HTML + custom CSS + vanilla JS) hosted on Cloudflare Pages or same VPS/Nginx.
|
||||
- Video player: `hls.js` (Chromium/Firefox) or native HLS (Safari). Single live `master.m3u8`.
|
||||
- Embedded chat sidebar with room switcher, private DMs, now-playing overlay.
|
||||
|
||||
|
||||
@@ -25,7 +25,7 @@ We must explicitly document language/runtime choices per module and the assumpti
|
||||
|
||||
| Layer | Technology | Language | Rationale |
|
||||
| ------------------- | ------------------------------------------ | ----------------------- | ---------------------------------------------------------------------------------------------------------------------- |
|
||||
| **Frontend SPA** | HTML + Tailwind + Alpine.js + hls.js | TypeScript / JavaScript | Minimal bundle, excellent HLS support, fast iteration. Vite for build. |
|
||||
| **Frontend SPA** | HTML + custom CSS + vanilla JS + hls.js | TypeScript / JavaScript | Minimal bundle, excellent HLS support, fast iteration. Vite for build. |
|
||||
| **Playout Service** | FFmpeg + MediaMTX (or SRS) | Bash + Go (wrapper) | FFmpeg is the proven tool for continuous playlist HLS. Go wrapper for reliability and monitoring. |
|
||||
| **Chat Backend** | Go (preferred) or Node.js + Socket.io | Go | High-performance WebSockets, low memory, excellent Redis/PostgreSQL clients. Node.js acceptable for rapid prototyping. |
|
||||
| **Shared Code** | TypeScript definitions + Go structs | TS + Go | Clear contracts between frontend and backend. |
|
||||
|
||||
@@ -19,7 +19,7 @@ This interface must be lightweight, secure, and integrated with the playout serv
|
||||
|
||||
## Decision Drivers
|
||||
|
||||
1. **Lightweight & Maintainable** – Prefer server-rendered or minimal JS (HTMX + Tailwind) over heavy SPA frameworks to align with ADR-003's "lightweight" principle.
|
||||
1. **Lightweight & Maintainable** – Prefer server-rendered or minimal JS (HTMX + custom CSS) over heavy SPA frameworks to align with ADR-003's "lightweight" principle.
|
||||
2. **Security** – Strict authentication, file type/size validation, and R2 upload permissions.
|
||||
3. **Usability** – Drag-and-drop playlist editor, progress indicators for normalization and upload, real-time playout status.
|
||||
4. **Integration** – Must call playout service APIs for validation and playlist updates; upload directly to R2 with metadata.
|
||||
@@ -31,7 +31,7 @@ Create a dedicated **admin web interface** in a new `frontend/admin/` directory
|
||||
|
||||
**Tech Stack (per ADR-003)**:
|
||||
|
||||
- **Frontend**: HTMX + Tailwind CSS + Alpine.js (minimal JS, server-rendered HTML). Alternative: React if complex interactions are needed later.
|
||||
- **Frontend**: HTMX + custom CSS + minimal JS (server-rendered HTML). Alternative: React if complex interactions are needed later.
|
||||
- **Backend**: Go (or Node.js) API endpoints exposed by the playout service or a dedicated `backend/admin/` service.
|
||||
- **Authentication**: JWT or session-based (admin roles only). Use Cloudflare Access or simple OAuth for production.
|
||||
- **File Handling**: Upload → validate with ffprobe (codec, fps, keyframes, duration) → normalize if needed → store in R2 with metadata.
|
||||
@@ -48,7 +48,7 @@ Create a dedicated **admin web interface** in a new `frontend/admin/` directory
|
||||
|
||||
**Folder Placement** (per ADR-002):
|
||||
|
||||
- `frontend/admin/` – Admin-specific HTML/JS components (HTMX + Tailwind).
|
||||
- `frontend/admin/` – Admin-specific HTML/JS components (HTMX + custom CSS).
|
||||
- `backend/admin/` – Dedicated API handlers for validation, playlist updates, and configuration (or extend playout/chat services).
|
||||
- Protected route in nginx with authentication (JWT or Cloudflare Access).
|
||||
- Documentation and API specs added to `docs/architecture/`.
|
||||
@@ -95,7 +95,7 @@ Create a dedicated **admin web interface** in a new `frontend/admin/` directory
|
||||
|
||||
- ADR-001: Normalization requirements, HLS playout, R2 usage.
|
||||
- ADR-002: Folder structure (admin UI belongs under frontend or a protected route).
|
||||
- ADR-003: Lightweight frontend preference (HTMX/Tailwind over heavy frameworks).
|
||||
- ADR-003: Lightweight frontend preference (HTMX/custom CSS over heavy frameworks).
|
||||
- arc42 documentation: Building Block View (admin as part of the overall system).
|
||||
|
||||
This ADR ensures all admin interface decisions are documented before code is written.
|
||||
@@ -17,7 +17,7 @@ These interfaces must be secure, auditable, and integrated with the chat backend
|
||||
|
||||
## Decision Drivers
|
||||
|
||||
1. **Consistency** – Use the same lightweight tech stack decided in ADR-004 (HTMX + Tailwind + Alpine.js) to avoid introducing new frameworks.
|
||||
1. **Consistency** – Use the same lightweight tech stack decided in ADR-004 (HTMX + custom CSS + vanilla JS) to avoid introducing new frameworks.
|
||||
2. **Security & Auditability** – All actions must require admin authentication and be logged immutably.
|
||||
3. **Real-time Needs** – Moderation dashboard should show live chat activity (WebSocket integration).
|
||||
4. **Simplicity** – Analytics should use existing Prometheus/Grafana data where possible rather than building a duplicate system.
|
||||
@@ -29,7 +29,7 @@ Extend the admin interface started in ADR-004 with additional protected sections
|
||||
|
||||
**Tech Stack (consistent with ADR-003 and ADR-004)**:
|
||||
|
||||
- **Frontend**: HTMX + Tailwind + Alpine.js for all admin pages. Shared layout with sidebar navigation.
|
||||
- **Frontend**: HTMX + custom CSS + vanilla JS for all admin pages. Shared layout with sidebar navigation.
|
||||
- **Backend**: Go API endpoints (protected by JWT/session middleware) in `backend/admin/` or extended from the chat/playout services.
|
||||
- **Authentication**: Role-based (admin, moderator, viewer). Use Cloudflare Access or JWT with short expiry for production.
|
||||
- **Data Sources**:
|
||||
@@ -54,7 +54,7 @@ Extend the admin interface started in ADR-004 with additional protected sections
|
||||
|
||||
## Rationale
|
||||
|
||||
- Reusing HTMX/Tailwind from ADR-004 keeps the admin interface lightweight and consistent.
|
||||
- Reusing HTMX/custom CSS from ADR-004 keeps the admin interface lightweight and consistent.
|
||||
- Centralizing admin functionality under `/admin/` reduces maintenance overhead.
|
||||
- Leveraging Prometheus/Grafana for analytics avoids duplicating metrics collection.
|
||||
- Immutable audit logging supports future compliance and moderation review (key requirement for a public chat system).
|
||||
@@ -94,7 +94,7 @@ Extend the admin interface started in ADR-004 with additional protected sections
|
||||
|
||||
- ADR-001: Chat features, moderation requirements, monitoring needs.
|
||||
- ADR-002: Folder structure – admin UI belongs under `frontend/admin/`.
|
||||
- ADR-003: Lightweight frontend (HTMX/Tailwind) and Go backend preference.
|
||||
- ADR-003: Lightweight frontend (HTMX/custom CSS) and Go backend preference.
|
||||
- ADR-004: Video admin interface (shared layout and authentication approach).
|
||||
- arc42 documentation: Cross-cutting Concepts (security, monitoring) and Building Block View (admin as extension of the system).
|
||||
|
||||
|
||||
@@ -20,6 +20,5 @@ $env:CLOUDFLARE_ACCOUNT_ID = (Get-Content .env | Select-String 'CLOUDFLARE_ACCOU
|
||||
curl -X GET "https://api.cloudflare.com/client/v4/accounts/$env:CLOUDFLARE_ACCOUNT_ID/pages/projects" -H "Authorization: Bearer $env:CLOUDFLARE_API_TOKEN" -H "Content-Type: application/json"
|
||||
|
||||
# Create a new Pages project (if needed)
|
||||
curl -X POST "https://api.cloudflare.com/client/v4/accounts/$env:CLOUDFLARE_ACCOUNT_ID/pages/projects" -H "Authorization: Bearer $env:CLOUDFLARE_API_TOKEN" -H "Content-Type: application/json" -d '{"name":"linearvoid","production_branch":"main"}'
|
||||
|
||||
# curl -X POST "https://api.cloudflare.com/client/v4/accounts/$env:CLOUDFLARE_ACCOUNT_ID/pages/projects" -H "Authorization: Bearer $env:CLOUDFLARE_API_TOKEN" -H "Content-Type: application/json" -d '{"name":"linearvoid","production_branch":"main"}'
|
||||
```
|
||||
@@ -11,14 +11,14 @@ Both are managed under **Workers & Pages** in the Cloudflare dashboard.
|
||||
|
||||
## Dashboard URLs
|
||||
|
||||
| Resource | URL |
|
||||
| ------------------------ | ---------------------------------------------------------------------------------------------- |
|
||||
| Workers & Pages overview | `https://dash.cloudflare.com/{accountId}/workers-and-pages` |
|
||||
| Worker: linearvoid | `https://dash.cloudflare.com/{accountId}/workers/services/view/linearvoid/production/settings` |
|
||||
| Pages project | `https://dash.cloudflare.com/{accountId}/pages/view/linearvoid` |
|
||||
| API tokens | <https://dash.cloudflare.com/profile/api-tokens> |
|
||||
| Resource | URL |
|
||||
| ------------------------ | ------------------------------------------------------------------------------------------------------------------- |
|
||||
| Workers & Pages overview | `https://dash.cloudflare.com/deb11491c54ec53a24772e98ac6d8128/workers-and-pages` |
|
||||
| Worker: linearvoid | `https://dash.cloudflare.com/deb11491c54ec53a24772e98ac6d8128/workers/services/view/linearvoid/production/settings` |
|
||||
| Pages project | `https://dash.cloudflare.com/deb11491c54ec53a24772e98ac6d8128/pages/view/linearvoid` |
|
||||
| API tokens | <https://dash.cloudflare.com/profile/api-tokens> |
|
||||
|
||||
## 1) Worker: `linearvoid` (edge cache headers)
|
||||
## Worker: `linearvoid` (edge cache headers)
|
||||
|
||||
The Worker at `infrastructure/cloudflare/worker.js` runs on the `linearvoid` Workers service. It intercepts requests and adds immutable `Cache-Control` headers to `.m4s` segment files.
|
||||
|
||||
@@ -30,31 +30,34 @@ npx wrangler deploy infrastructure/cloudflare/worker.js
|
||||
|
||||
Or paste into the Workers dashboard at the URL above (Quick Edit).
|
||||
|
||||
## 2) Pages: Static frontend
|
||||
## Pages: Static frontend
|
||||
|
||||
Two approaches — pick one:
|
||||
|
||||
### Option A: Git integration (recommended)
|
||||
|
||||
Connect the Gitea repository directly in Cloudflare Pages:
|
||||
|
||||
1. Pages → Create a project → Connect to Git
|
||||
2. Link `git.allucanget.biz/zwitschi/linearvoid`
|
||||
3. Build command: `cd frontend && npm run build`
|
||||
4. Output directory: `frontend/dist`
|
||||
5. Add custom domain: `linearvoid.tv`
|
||||
|
||||
Cloudflare handles builds and deploys automatically on push.
|
||||
|
||||
> **Naming:** If `linearvoid` is already used by the Worker service, name the Pages project `linearvoid-site` (or similar). Workers and Pages share a namespace.
|
||||
|
||||
### Option B: Direct Upload via Gitea Actions
|
||||
### Direct Upload via Gitea Actions
|
||||
|
||||
Use `.gitea/workflows/pages-deploy.yml` to build in Gitea Actions and upload via the Cloudflare API.
|
||||
|
||||
See [gitea-cloudflare-pages.md](./gitea-cloudflare-pages.md) for the workflow.
|
||||
|
||||
## 3) Custom domains
|
||||
### Variables and secrets
|
||||
|
||||
On Cloudflare pages, set the following environment variables (needed to run the frontend)
|
||||
|
||||
| Type | Name | Value |
|
||||
| --------- | -------------------------------------- | ----------------------------------------------- |
|
||||
| Secret | ADMIN_PASSWORD | `not-a-real-admin-password` |
|
||||
| Plaintext | ADMIN_SESSION_TTL_HOURS | `12` |
|
||||
| Plaintext | ADMIN_USERNAME | `boogers` |
|
||||
| Plaintext | HLS_SEGMENT_DURATION | `6` |
|
||||
| Plaintext | HLS_WINDOW_SIZE | `24` |
|
||||
| Plaintext | PLAYLIST_PATH | `/app/playlist/playlist.txt` |
|
||||
| Plaintext | VITE_PUBLIC_ADMIN_DEFAULT_MODERATOR | `boogers` |
|
||||
| Plaintext | VITE_PUBLIC_ADMIN_DEFAULT_ROOM | `main` |
|
||||
| Plaintext | VITE_PUBLIC_ADMIN_MODERATION_LOG_LIMIT | `50` |
|
||||
| Plaintext | VITE_PUBLIC_APP_URL | `https://linearvoid.tv` |
|
||||
| Plaintext | VITE_PUBLIC_HLS_URL | `https://media.linearvoid.tv/hls/playlist.m3u8` |
|
||||
| Plaintext | VITE_PUBLIC_WS_URL | `wss://linearvoid.tv/ws` |
|
||||
|
||||
## Custom domains
|
||||
|
||||
- `linearvoid.tv` → Pages project
|
||||
- `www.linearvoid.tv` → redirect to `linearvoid.tv`
|
||||
|
||||
@@ -6,7 +6,12 @@ Deploy the Linear Void frontend (Vite + TypeScript SPA) from Gitea to Cloudflare
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. **Cloudflare Pages project** created (Dashboard → Workers & Pages → Create → Pages), see [Cloudflare Pages Setup](./cloudflare-pages-setup.md).
|
||||
> **Important:** The Pages project must exist before the API deploy call works. You'll get error 8000007 ("Project not found") if you skip this step.
|
||||
|
||||
1. **Cloudflare Pages project** created first:
|
||||
- Dashboard → Workers & Pages → Create → Pages → Direct Upload
|
||||
- Name it `linearvoid` (or a different name if `linearvoid` is already taken by a Worker — see note below)
|
||||
- See [Cloudflare Pages Setup](./cloudflare-pages-setup.md) for full details.
|
||||
2. **Project name**: note the exact name (e.g., `linearvoid`).
|
||||
3. **API token** with Pages permissions:
|
||||
- Create at <https://dash.cloudflare.com/profile/api-tokens>
|
||||
@@ -14,6 +19,8 @@ Deploy the Linear Void frontend (Vite + TypeScript SPA) from Gitea to Cloudflare
|
||||
- Store in Gitea → Settings → Secrets as `CLOUDFLARE_API_TOKEN`
|
||||
4. **Account ID** from Cloudflare dashboard overview → store as `CLOUDFLARE_ACCOUNT_ID`
|
||||
|
||||
> **Naming conflict:** If a Worker already exists with the name `linearvoid`, the Pages project must use a different name (e.g., `linearvoid-site`). Update the `projectName` in the workflow accordingly. Workers and Pages share a namespace under "Workers & Pages".
|
||||
|
||||
## Project-specific values
|
||||
|
||||
| Setting | Value |
|
||||
@@ -46,7 +53,7 @@ jobs:
|
||||
- name: Setup Node
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 20
|
||||
node-version: 22
|
||||
|
||||
- name: Install dependencies
|
||||
run: cd frontend && npm ci
|
||||
@@ -64,24 +71,50 @@ jobs:
|
||||
gitHubToken: ${{ secrets.GITEA_TOKEN }}
|
||||
```
|
||||
|
||||
If the `cloudflare/pages-action@v1` action is unavailable from Gitea, use the direct API approach:
|
||||
If the `cloudflare/pages-action@v1` action is unavailable from Gitea, use the direct Cloudflare Pages API (multipart/form-data with manifest):
|
||||
|
||||
Snippet to generate a manifest in format `-F manifest='{"index.html": "abc123", "style.css": "def456"}'`
|
||||
|
||||
```bash
|
||||
# Build manifest form argument from all files in frontend/dist
|
||||
form_args="-F manifest={"
|
||||
for file in $(find frontend/dist -type f | sed 's|^frontend/dist/||'); do
|
||||
hash=$(sha256sum "frontend/dist/$file" | awk '{print $1}')
|
||||
form_args="$form_args\"$file\":\"$hash\","
|
||||
done
|
||||
form_args="${form_args%,}}" # remove trailing comma and close JSON
|
||||
```
|
||||
|
||||
```yaml
|
||||
- name: Create deployment artifact
|
||||
run: tar -czf site.tgz -C frontend/dist .
|
||||
- name: Generate deployment manifest
|
||||
run: |
|
||||
cd frontend/dist
|
||||
form_args="-F manifest={"
|
||||
for file in $(find . -type f ! -name manifest.json | sed 's|^\./||'); do
|
||||
hash=$(sha256sum "$file" | awk '{print $1}')
|
||||
form_args="$form_args\"$file\":\"$hash\","
|
||||
done
|
||||
form_args="${form_args%,}}" # remove trailing comma and close JSON
|
||||
echo "Form arguments: $form_args"
|
||||
echo "FORM_ARGS=$form_args" >> $GITHUB_ENV
|
||||
|
||||
- name: Deploy to Cloudflare Pages
|
||||
env:
|
||||
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
||||
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
|
||||
run: |
|
||||
cd frontend/dist
|
||||
form_args="$FORM_ARGS" # use form arguments generated in previous step
|
||||
COMMIT_HASH=$(git rev-parse --short HEAD 2>/dev/null || echo "unknown")
|
||||
curl -sS -X POST \
|
||||
"https://api.cloudflare.com/client/v4/accounts/${CLOUDFLARE_ACCOUNT_ID}/pages/projects/linearvoid/deployments" \
|
||||
-H "Authorization: Bearer ${CLOUDFLARE_API_TOKEN}" \
|
||||
-H "Content-Type: application/octet-stream" \
|
||||
--data-binary @site.tgz
|
||||
-F "commit_hash=${COMMIT_HASH}" \
|
||||
$form_args
|
||||
```
|
||||
|
||||
This matches the Cloudflare Pages Direct Upload API: `Content-Type: multipart/form-data`, each file as a separate form field, and a `manifest` JSON mapping filenames to SHA-256 hashes.
|
||||
|
||||
## Secrets to add in Gitea
|
||||
|
||||
| Secret | Value |
|
||||
@@ -102,8 +135,10 @@ If the `cloudflare/pages-action@v1` action is unavailable from Gitea, use the di
|
||||
- Multi-page SPA (`vite.config.ts`) produces three entry points — Cloudflare Pages serves them correctly.
|
||||
- For SPA fallback routing, add a `frontend/public/_redirects` file with `/* /index.html 200` before building.
|
||||
- If deployments fail with 403, verify the API token has `Account → Cloudflare Pages → Edit` permission.
|
||||
- If you get error **8000007** ("Project not found"), the Pages project hasn't been created yet. Create it via the Cloudflare dashboard (Workers & Pages → Create → Pages → Direct Upload) first, then re-run.
|
||||
- If `linearvoid` is already taken by a Worker, use a different project name like `linearvoid-site` and update the `projectName` and `PROJECT_NAME` values in the workflow.
|
||||
|
||||
## Related resources
|
||||
|
||||
- Cloudflare Workers dashboard: `https://dash.cloudflare.com/{accountId}/workers/services/view/linearvoid/production/settings` — for managing the `infrastructure/cloudflare/worker.js` deployment
|
||||
- Cloudflare Pages project: `https://dash.cloudflare.com/{accountId}/pages/view/linearvoid` — deployment history and custom domain settings
|
||||
- Cloudflare Workers dashboard: `https://dash.cloudflare.com/deb11491c54ec53a24772e98ac6d8128/workers/services/view/linearvoid/production/settings` — for managing the `infrastructure/cloudflare/worker.js` deployment
|
||||
- Cloudflare Pages project: `https://dash.cloudflare.com/deb11491c54ec53a24772e98ac6d8128/pages/view/linearvoid` — deployment history and custom domain settings
|
||||
@@ -1,144 +0,0 @@
|
||||
# Gitea to Cloudflare Pages — Linear Void
|
||||
|
||||
## Goal
|
||||
|
||||
Deploy the Linear Void frontend (Vite + TypeScript SPA) from Gitea to Cloudflare Pages using Gitea Actions.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
> **Important:** The Pages project must exist before the API deploy call works. You'll get error 8000007 ("Project not found") if you skip this step.
|
||||
|
||||
1. **Cloudflare Pages project** created first:
|
||||
- Dashboard → Workers & Pages → Create → Pages → Direct Upload
|
||||
- Name it `linearvoid` (or a different name if `linearvoid` is already taken by a Worker — see note below)
|
||||
- See [Cloudflare Pages Setup](./cloudflare-pages-setup.md) for full details.
|
||||
2. **Project name**: note the exact name (e.g., `linearvoid`).
|
||||
3. **API token** with Pages permissions:
|
||||
- Create at <https://dash.cloudflare.com/profile/api-tokens>
|
||||
- Permission: `Account → Cloudflare Pages → Edit`
|
||||
- Store in Gitea → Settings → Secrets as `CLOUDFLARE_API_TOKEN`
|
||||
4. **Account ID** from Cloudflare dashboard overview → store as `CLOUDFLARE_ACCOUNT_ID`
|
||||
|
||||
> **Naming conflict:** If a Worker already exists with the name `linearvoid`, the Pages project must use a different name (e.g., `linearvoid-site`). Update the `projectName` in the workflow accordingly. Workers and Pages share a namespace under "Workers & Pages".
|
||||
|
||||
## Project-specific values
|
||||
|
||||
| Setting | Value |
|
||||
| ------------------ | ---------------------------------------------------------- |
|
||||
| Framework | Vite + TypeScript |
|
||||
| Build command | `cd frontend && npm install && npm run build` |
|
||||
| Output directory | `frontend/dist` |
|
||||
| Node version | 20 |
|
||||
| Multi-page entries | `index.html`, `admin/index.html`, `admin/login/index.html` |
|
||||
|
||||
## Gitea Actions workflow
|
||||
|
||||
Create `.gitea/workflows/pages-deploy.yml`:
|
||||
|
||||
```yaml
|
||||
name: Deploy to Cloudflare Pages
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: ["main"]
|
||||
workflow_dispatch: {}
|
||||
|
||||
jobs:
|
||||
deploy:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 22
|
||||
|
||||
- name: Install dependencies
|
||||
run: cd frontend && npm ci
|
||||
|
||||
- name: Build
|
||||
run: cd frontend && npm run build
|
||||
|
||||
- name: Deploy to Cloudflare Pages
|
||||
uses: cloudflare/pages-action@v1
|
||||
with:
|
||||
apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
||||
accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
|
||||
projectName: linearvoid
|
||||
directory: frontend/dist
|
||||
gitHubToken: ${{ secrets.GITEA_TOKEN }}
|
||||
```
|
||||
|
||||
If the `cloudflare/pages-action@v1` action is unavailable from Gitea, use the direct Cloudflare Pages API (multipart/form-data with manifest):
|
||||
|
||||
Snippet to generate a manifest in format `-F manifest='{"index.html": "abc123", "style.css": "def456"}'`
|
||||
|
||||
```bash
|
||||
# Build manifest form argument from all files in frontend/dist
|
||||
form_args="-F manifest={"
|
||||
for file in $(find frontend/dist -type f | sed 's|^frontend/dist/||'); do
|
||||
hash=$(sha256sum "frontend/dist/$file" | awk '{print $1}')
|
||||
form_args="$form_args\"$file\":\"$hash\","
|
||||
done
|
||||
form_args="${form_args%,}}" # remove trailing comma and close JSON
|
||||
```
|
||||
|
||||
```yaml
|
||||
- name: Generate deployment manifest
|
||||
run: |
|
||||
cd frontend/dist
|
||||
form_args="-F manifest={"
|
||||
for file in $(find . -type f ! -name manifest.json | sed 's|^\./||'); do
|
||||
hash=$(sha256sum "$file" | awk '{print $1}')
|
||||
form_args="$form_args\"$file\":\"$hash\","
|
||||
done
|
||||
form_args="${form_args%,}}" # remove trailing comma and close JSON
|
||||
echo "Form arguments: $form_args"
|
||||
echo "FORM_ARGS=$form_args" >> $GITHUB_ENV
|
||||
|
||||
- name: Deploy to Cloudflare Pages
|
||||
env:
|
||||
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
||||
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
|
||||
run: |
|
||||
cd frontend/dist
|
||||
form_args="$FORM_ARGS" # use form arguments generated in previous step
|
||||
COMMIT_HASH=$(git rev-parse --short HEAD 2>/dev/null || echo "unknown")
|
||||
curl -sS -X POST \
|
||||
"https://api.cloudflare.com/client/v4/accounts/${CLOUDFLARE_ACCOUNT_ID}/pages/projects/linearvoid/deployments" \
|
||||
-H "Authorization: Bearer ${CLOUDFLARE_API_TOKEN}" \
|
||||
-F "commit_hash=${COMMIT_HASH}" \
|
||||
$form_args
|
||||
```
|
||||
|
||||
This matches the Cloudflare Pages Direct Upload API: `Content-Type: multipart/form-data`, each file as a separate form field, and a `manifest` JSON mapping filenames to SHA-256 hashes.
|
||||
|
||||
## Secrets to add in Gitea
|
||||
|
||||
| Secret | Value |
|
||||
| ----------------------- | ------------------------------------ |
|
||||
| `CLOUDFLARE_API_TOKEN` | API token with Pages:Edit permission |
|
||||
| `CLOUDFLARE_ACCOUNT_ID` | Cloudflare account ID from dashboard |
|
||||
|
||||
## Verify deployment
|
||||
|
||||
1. Push to `main` or trigger manually from Gitea Actions.
|
||||
2. Check **Gitea Actions** → job logs for the deploy step.
|
||||
3. Check **Cloudflare Pages** → Deployments for the new deployment.
|
||||
4. Visit the `.pages.dev` preview URL, then promote to `linearvoid.tv` when ready.
|
||||
|
||||
## Notes
|
||||
|
||||
- The existing `ci.yml` already builds the frontend; the Pages deploy workflow is additive and independent.
|
||||
- Multi-page SPA (`vite.config.ts`) produces three entry points — Cloudflare Pages serves them correctly.
|
||||
- For SPA fallback routing, add a `frontend/public/_redirects` file with `/* /index.html 200` before building.
|
||||
- If deployments fail with 403, verify the API token has `Account → Cloudflare Pages → Edit` permission.
|
||||
- If you get error **8000007** ("Project not found"), the Pages project hasn't been created yet. Create it via the Cloudflare dashboard (Workers & Pages → Create → Pages → Direct Upload) first, then re-run.
|
||||
- If `linearvoid` is already taken by a Worker, use a different project name like `linearvoid-site` and update the `projectName` and `PROJECT_NAME` values in the workflow.
|
||||
|
||||
## Related resources
|
||||
|
||||
- Cloudflare Workers dashboard: `https://dash.cloudflare.com/{accountId}/workers/services/view/linearvoid/production/settings` — for managing the `infrastructure/cloudflare/worker.js` deployment
|
||||
- Cloudflare Pages project: `https://dash.cloudflare.com/{accountId}/pages/view/linearvoid` — deployment history and custom domain settings
|
||||
Reference in new issue
Block a user