docs: adding more details for monitoring and operations
This commit is contained in:
zwitschi committed
1 parent
8b8f2335e1
commit
98fafcd93c
6 files changed
+295
-1
No files matched your search
@@ -52,6 +52,13 @@ Config: `monitoring/prometheus/prometheus.yml` (15s scrape interval)
|
||||
- **Linear Void app**: `monitoring/grafana/dashboard.json` — clients, messages, moderation, Redis, sessions, scrape health
|
||||
- **Prometheus ops**: `monitoring/grafana/dashboards/dashboard.json` — TSDB, CPU/memory, WAL, compactions, query latency
|
||||
|
||||
## Metrics checking
|
||||
|
||||
See detailed scripts for querying Prometheus scrape targets:
|
||||
|
||||
- [Linux/macOS](./infrastructure/monitoring/prometheus_curl.md)
|
||||
- [Windows](./infrastructure/monitoring/prometheus_curl_win.md)
|
||||
|
||||
## Key Metrics
|
||||
|
||||
- `linearvoid_chat_connected_clients` — active WebSocket connections
|
||||
|
||||
+1
-1
@@ -5,7 +5,7 @@ This document is the operational contract for Linear Void video ingest, playout,
|
||||
## Production flow
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
flowchart TD
|
||||
A["Normalized MP4 assets"] --> B["Playlist and schedule"]
|
||||
B --> C["Playout service and FFmpeg"]
|
||||
C --> D["Local HLS staging"]
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
# Prometheus scrape target metric names
|
||||
|
||||
Here are ready‑to‑run **cURL** commands you can use to query each Prometheus scrape target and list the metric names they expose.
|
||||
The Prometheus HTTP API endpoint `http://localhost:9090/api/v1/label/__name__/values` returns all metric names currently known for the target’s job/instance.
|
||||
|
||||
---
|
||||
|
||||
## 1. Chat (`http://chat:3000/metrics`)
|
||||
|
||||
```bash
|
||||
curl -s 'http://localhost:9090/api/v1/label/__name__/values?match[]={job="chat",instance="chat:3000"}' | jq .
|
||||
```
|
||||
|
||||
## 2. NGINX exporter (`http://nginx-exporter:9113/metrics`)
|
||||
|
||||
```bash
|
||||
curl -s 'http://localhost:9090/api/v1/label/__name__/values?match[]={job="nginx",instance="nginx-exporter:9113"}' | jq .
|
||||
```
|
||||
|
||||
## 3. Node exporter (`http://node-exporter:9100/metrics`)
|
||||
|
||||
```bash
|
||||
curl -s 'http://localhost:9090/api/v1/label/__name__/values?match[]={job="node",instance="node-exporter:9100"}' | jq .
|
||||
```
|
||||
|
||||
## 4. Playout (`http://playout:8081/metrics`)
|
||||
|
||||
```bash
|
||||
curl -s 'http://localhost:9090/api/v1/label/__name__/values?match[]={job="playout",instance="playout:8081"}' | jq .
|
||||
```
|
||||
|
||||
## 5. Postgres exporter (`http://postgres-exporter:9187/metrics`)
|
||||
|
||||
```bash
|
||||
curl -s 'http://localhost:9090/api/v1/label/__name__/values?match[]={job="postgres",instance="postgres-exporter:9187"}' | jq .
|
||||
```
|
||||
|
||||
## 6. Prometheus itself (`http://prometheus:9090/metrics`)
|
||||
|
||||
```bash
|
||||
curl -s 'http://localhost:9090/api/v1/label/__name__/values?match[]={job="prometheus",instance="prometheus:9090"}' | jq .
|
||||
```
|
||||
|
||||
## 7. Redis exporter (`http://redis-exporter:9121/metrics`)
|
||||
|
||||
```bash
|
||||
curl -s 'http://localhost:9090/api/v1/label/__name__/values?match[]={job="redis",instance="redis-exporter:9121"}' | jq .
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### How it works
|
||||
|
||||
- `http://localhost:9090` – address of your Prometheus server.
|
||||
- `/api/v1/label/__name__/values` – API call that lists _all_ metric names (`__name__` label) known to Prometheus.
|
||||
- `match[]` – a selector that restricts the result to the specific `job` and `instance` you care about.
|
||||
- `-s` – silent mode (no progress meter).
|
||||
- `| jq .` – pretty‑prints the JSON response (optional but handy).
|
||||
|
||||
Running any of these commands will output a JSON array like:
|
||||
|
||||
```json
|
||||
[
|
||||
"up",
|
||||
"process_cpu_seconds_total",
|
||||
"http_requests_total",
|
||||
...
|
||||
]
|
||||
```
|
||||
|
||||
which is the full set of metrics currently being scraped from that target. Adjust the Prometheus host/port if it differs from `http://localhost:9090`.
|
||||
@@ -0,0 +1,96 @@
|
||||
# Prometheus scrape target metric names (Windows)
|
||||
|
||||
Below are **cURL** one‑liners you can run from a Windows command prompt (or PowerShell) that query the Prometheus HTTP API and print the raw JSON list of metric names for each target.
|
||||
Since **jq** isn’t installed, the commands simply echo the JSON response; you can pipe it to `findstr` or PowerShell’s `ConvertFrom‑Json` if you want a prettier view.
|
||||
|
||||
---
|
||||
|
||||
## 1. Chat (`http://chat:3000/metrics`)
|
||||
|
||||
```cmd
|
||||
curl "http://localhost:9090/api/v1/label/__name__/values?match[]={job=\"chat\",instance=\"chat:3000\"}"
|
||||
```
|
||||
|
||||
## 2. NGINX exporter (`http://nginx-exporter:9113/metrics`)
|
||||
|
||||
```cmd
|
||||
curl "http://localhost:9090/api/v1/label/__name__/values?match[]={job=\"nginx\",instance=\"nginx-exporter:9113\"}"
|
||||
```
|
||||
|
||||
## 3. Node exporter (`http://node-exporter:9100/metrics`)
|
||||
|
||||
```cmd
|
||||
curl "http://localhost:9090/api/v1/label/__name__/values?match[]={job=\"node\",instance=\"node-exporter:9100\"}"
|
||||
```
|
||||
|
||||
## 4. Playout (`http://playout:8081/metrics`)
|
||||
|
||||
```cmd
|
||||
curl "http://localhost:9090/api/v1/label/__name__/values?match[]={job=\"playout\",instance=\"playout:8081\"}"
|
||||
```
|
||||
|
||||
## 5. Postgres exporter (`http://postgres-exporter:9187/metrics`)
|
||||
|
||||
```cmd
|
||||
curl "http://localhost:9090/api/v1/label/__name__/values?match[]={job=\"postgres\",instance=\"postgres-exporter:9187\"}"
|
||||
```
|
||||
|
||||
## 6. Prometheus itself (`http://prometheus:9090/metrics`)
|
||||
|
||||
```cmd
|
||||
curl "http://localhost:9090/api/v1/label/__name__/values?match[]={job=\"prometheus\",instance=\"prometheus:9090\"}"
|
||||
```
|
||||
|
||||
## 7. Redis exporter (`http://redis-exporter:9121/metrics`)
|
||||
|
||||
```cmd
|
||||
curl "http://localhost:9090/api/v1/label/__name__/values?match[]={job=\"redis\",instance=\"redis-exporter:9121\"}"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Making the output a bit more readable (optional)
|
||||
|
||||
If you have **PowerShell** (available on Windows 10/11), you can format the JSON nicely:
|
||||
|
||||
```powershell
|
||||
curl "http://localhost:9090/api/v1/label/__name__/values?match[]={job=`"chat`",instance=`"chat:3000`"}" |
|
||||
ConvertFrom-Json |
|
||||
Select-Object -ExpandProperty data |
|
||||
Sort-Object
|
||||
```
|
||||
|
||||
Replace the URL part with the one for the target you’re interested in. The pipeline:
|
||||
|
||||
1. Calls `curl` (the built‑in alias for `Invoke-WebRequest` in PowerShell).
|
||||
2. `ConvertFrom-Json` parses the JSON.
|
||||
3. `Select-Object -ExpandProperty data` extracts the array of metric names.
|
||||
4. `Sort-Object` alphabetizes them.
|
||||
|
||||
You can similarly pipe the output to `findstr /i "http_requests_total"` to search for a particular metric name without any extra tools.
|
||||
|
||||
---
|
||||
|
||||
### Quick checklist
|
||||
|
||||
| Item | What to adjust |
|
||||
| ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **Prometheus address** | If your Prometheus container isn’t bound to `localhost:9090`, replace `localhost:9090` with the appropriate host/port (e.g., `127.0.0.1:9090`). |
|
||||
| **Target job/instance** | The `job` and `instance` values must match the labels shown in your UI (they’re case‑sensitive). |
|
||||
| **PowerShell vs CMD** | The raw `curl` commands work in both CMD and PowerShell; the PowerShell‑only pretty‑print uses `ConvertFrom‑Json`. |
|
||||
|
||||
Run the command for any target and you’ll receive a JSON payload like:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "success",
|
||||
"data": [
|
||||
"up",
|
||||
"process_cpu_seconds_total",
|
||||
"http_requests_total",
|
||||
"... more metrics ..."
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
That array (`data`) is the complete list of metric names currently scraped from the specified endpoint.
|
||||
@@ -46,6 +46,8 @@ Connect the Gitea repository directly in Cloudflare Pages:
|
||||
|
||||
Cloudflare handles builds and deploys automatically on push.
|
||||
|
||||
> **Naming:** If `linearvoid` is already used by the Worker service, name the Pages project `linearvoid-site` (or similar). Workers and Pages share a namespace.
|
||||
|
||||
### Option B: Direct Upload via Gitea Actions
|
||||
|
||||
Use `.gitea/workflows/pages-deploy.yml` to build in Gitea Actions and upload via the Cloudflare API.
|
||||
|
||||
@@ -0,0 +1,118 @@
|
||||
# Gitea to Cloudflare Pages — Linear Void
|
||||
|
||||
## Goal
|
||||
|
||||
Deploy the Linear Void frontend (Vite + TypeScript SPA) from Gitea to Cloudflare Pages using Gitea Actions.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
> **Important:** The Pages project must exist before the API deploy call works. You'll get error 8000007 ("Project not found") if you skip this step.
|
||||
|
||||
1. **Cloudflare Pages project** created first:
|
||||
- Dashboard → Workers & Pages → Create → Pages → Direct Upload
|
||||
- Name it `linearvoid` (or a different name if `linearvoid` is already taken by a Worker — see note below)
|
||||
- See [Cloudflare Pages Setup](./cloudflare-pages-setup.md) for full details.
|
||||
2. **Project name**: note the exact name (e.g., `linearvoid`).
|
||||
3. **API token** with Pages permissions:
|
||||
- Create at <https://dash.cloudflare.com/profile/api-tokens>
|
||||
- Permission: `Account → Cloudflare Pages → Edit`
|
||||
- Store in Gitea → Settings → Secrets as `CLOUDFLARE_API_TOKEN`
|
||||
4. **Account ID** from Cloudflare dashboard overview → store as `CLOUDFLARE_ACCOUNT_ID`
|
||||
|
||||
> **Naming conflict:** If a Worker already exists with the name `linearvoid`, the Pages project must use a different name (e.g., `linearvoid-site`). Update the `projectName` in the workflow accordingly. Workers and Pages share a namespace under "Workers & Pages".
|
||||
|
||||
## Project-specific values
|
||||
|
||||
| Setting | Value |
|
||||
| ------------------ | ---------------------------------------------------------- |
|
||||
| Framework | Vite + TypeScript |
|
||||
| Build command | `cd frontend && npm install && npm run build` |
|
||||
| Output directory | `frontend/dist` |
|
||||
| Node version | 20 |
|
||||
| Multi-page entries | `index.html`, `admin/index.html`, `admin/login/index.html` |
|
||||
|
||||
## Gitea Actions workflow
|
||||
|
||||
Create `.gitea/workflows/pages-deploy.yml`:
|
||||
|
||||
```yaml
|
||||
name: Deploy to Cloudflare Pages
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: ["main"]
|
||||
workflow_dispatch: {}
|
||||
|
||||
jobs:
|
||||
deploy:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 20
|
||||
|
||||
- name: Install dependencies
|
||||
run: cd frontend && npm ci
|
||||
|
||||
- name: Build
|
||||
run: cd frontend && npm run build
|
||||
|
||||
- name: Deploy to Cloudflare Pages
|
||||
uses: cloudflare/pages-action@v1
|
||||
with:
|
||||
apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
||||
accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
|
||||
projectName: linearvoid
|
||||
directory: frontend/dist
|
||||
gitHubToken: ${{ secrets.GITEA_TOKEN }}
|
||||
```
|
||||
|
||||
If the `cloudflare/pages-action@v1` action is unavailable from Gitea, use the direct API approach:
|
||||
|
||||
```yaml
|
||||
- name: Create deployment artifact
|
||||
run: tar -czf site.tgz -C frontend/dist .
|
||||
|
||||
- name: Deploy to Cloudflare Pages
|
||||
env:
|
||||
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
||||
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
|
||||
run: |
|
||||
curl -sS -X POST \
|
||||
"https://api.cloudflare.com/client/v4/accounts/${CLOUDFLARE_ACCOUNT_ID}/pages/projects/linearvoid/deployments" \
|
||||
-H "Authorization: Bearer ${CLOUDFLARE_API_TOKEN}" \
|
||||
-H "Content-Type: application/octet-stream" \
|
||||
--data-binary @site.tgz
|
||||
```
|
||||
|
||||
## Secrets to add in Gitea
|
||||
|
||||
| Secret | Value |
|
||||
| ----------------------- | ------------------------------------ |
|
||||
| `CLOUDFLARE_API_TOKEN` | API token with Pages:Edit permission |
|
||||
| `CLOUDFLARE_ACCOUNT_ID` | Cloudflare account ID from dashboard |
|
||||
|
||||
## Verify deployment
|
||||
|
||||
1. Push to `main` or trigger manually from Gitea Actions.
|
||||
2. Check **Gitea Actions** → job logs for the deploy step.
|
||||
3. Check **Cloudflare Pages** → Deployments for the new deployment.
|
||||
4. Visit the `.pages.dev` preview URL, then promote to `linearvoid.tv` when ready.
|
||||
|
||||
## Notes
|
||||
|
||||
- The existing `ci.yml` already builds the frontend; the Pages deploy workflow is additive and independent.
|
||||
- Multi-page SPA (`vite.config.ts`) produces three entry points — Cloudflare Pages serves them correctly.
|
||||
- For SPA fallback routing, add a `frontend/public/_redirects` file with `/* /index.html 200` before building.
|
||||
- If deployments fail with 403, verify the API token has `Account → Cloudflare Pages → Edit` permission.
|
||||
- If you get error **8000007** ("Project not found"), the Pages project hasn't been created yet. Create it via the Cloudflare dashboard (Workers & Pages → Create → Pages → Direct Upload) first, then re-run.
|
||||
- If `linearvoid` is already taken by a Worker, use a different project name like `linearvoid-site` and update the `projectName` and `PROJECT_NAME` values in the workflow.
|
||||
|
||||
## Related resources
|
||||
|
||||
- Cloudflare Workers dashboard: `https://dash.cloudflare.com/{accountId}/workers/services/view/linearvoid/production/settings` — for managing the `infrastructure/cloudflare/worker.js` deployment
|
||||
- Cloudflare Pages project: `https://dash.cloudflare.com/{accountId}/pages/view/linearvoid` — deployment history and custom domain settings
|
||||
Reference in new issue
Block a user