docs: align CI/CD and release policy wording
This commit is contained in:
1 parent
a8df164cc2
commit
2ed89760d6
2 files changed
+31
-29
No files matched your search
+30
-28
@@ -394,7 +394,7 @@ Notable aspects: Manifest freshness ensures all viewers stay on same timeline; c
|
||||
|
||||
### CI/CD Pipeline Architecture
|
||||
|
||||
The deployment follows a two-stage pipeline: **CI builds and pushes images**, then **CD pulls and deploys** them. Currently targeting the **test lab** (192.168.88.40). Production Hetzner VPS is planned but not yet provisioned — the same pipeline will deploy there by changing `DOCKER_HOST`.
|
||||
The deployment uses parallel workflows on each push to `main`: **CI builds and pushes images** while **CD deploys latest images** to the test lab. A dedicated release workflow handles semantic version tags. Current runtime target is the **test lab** (192.168.88.40). Production Hetzner VPS is planned but not yet provisioned — the same pipeline will deploy there by changing `DOCKER_HOST`.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
@@ -431,18 +431,22 @@ flowchart LR
|
||||
**Pipeline Sequence:**
|
||||
|
||||
1. **CI Workflow** (`.gitea/workflows/ci.yml`):
|
||||
- Triggers on `push` to `main` or `pull_request`
|
||||
- Runs `test` job: frontend lint/build, Go validation, `docker compose build`
|
||||
- Triggers on `push` to `main` and `pull_request`
|
||||
- Runs `test` job: frontend lint/build and Go validation
|
||||
- Runs `docker` job (after `test` passes, only on `main` push):
|
||||
- Builds `playout`, `chat`, `nginx` images with Buildx
|
||||
- Pushes to Gitea Container Registry with `main` and `latest` tags
|
||||
2. **CD Workflow** (`.gitea/workflows/cd.yml`):
|
||||
- Triggers on `workflow_run` when CI completes successfully, or manually via `workflow_dispatch`
|
||||
- Pulls latest images from registry
|
||||
- Triggers on `push` to `main` (parallel to CI), or manually via `workflow_dispatch`
|
||||
- Logs in remote Docker host to private registry
|
||||
- Generates `.env` from secrets
|
||||
- Seeds Prometheus config into named volume
|
||||
- Deploys via `docker compose -f docker-compose.prod.yml up -d` to remote host (`192.168.88.40:2375` — test lab; switch to Hetzner IP for production)
|
||||
- Seeds Prometheus and Grafana provisioning into named volumes
|
||||
- Deploys via `docker compose -f docker-compose.prod.yml up -d --remove-orphans --force-recreate` to remote host (`192.168.88.40:2375` — test lab; switch to Hetzner IP for production)
|
||||
- Runs health checks with retry logic for playout initialization
|
||||
3. **Release Workflow** (`.gitea/workflows/release.yml`):
|
||||
- Triggers on semantic tags in `*.*.*` format (for example `1.0.9`)
|
||||
- Builds and pushes versioned image tags (`X.Y.Z`, `X.Y`, `latest`)
|
||||
- Creates release entry in Gitea
|
||||
|
||||
### Infrastructure Level 1
|
||||
|
||||
@@ -524,14 +528,14 @@ Linear Void uses a two-layer monitoring model:
|
||||
|
||||
- Config file: `monitoring/prometheus/prometheus.yml`
|
||||
- Default scrape interval: `15s`
|
||||
- Current scrape jobs: `chat` -> `chat:3000`, `prometheus` -> `prometheus:9090`.
|
||||
|
||||
Recommended next-step scrape targets (if/when exporters are introduced):
|
||||
|
||||
- `nginx` (via nginx-exporter)
|
||||
- `redis` (via redis-exporter)
|
||||
- `postgres` (via postgres-exporter)
|
||||
- `node` (via node-exporter)
|
||||
- Current scrape jobs (7 total):
|
||||
- `chat` -> `chat:3000`
|
||||
- `playout` -> `playout:8081`
|
||||
- `nginx` -> `nginx-exporter:9113`
|
||||
- `redis` -> `redis-exporter:9121`
|
||||
- `postgres` -> `postgres-exporter:9187`
|
||||
- `node` -> `node-exporter:9100`
|
||||
- `prometheus` -> `localhost:9090`
|
||||
|
||||
#### Grafana Dashboards
|
||||
|
||||
@@ -542,22 +546,20 @@ Use the app dashboard for service-level behavior and the Prometheus overview das
|
||||
|
||||
#### Alerting Rules (Prometheus Rule Groups)
|
||||
|
||||
Define alert rules as grouped policies (for example in `monitoring/prometheus/alerts.yml`) and include them from `prometheus.yml`.
|
||||
Rules are implemented in `monitoring/prometheus/alerts.yml` and loaded via `rule_files` from `prometheus.yml`.
|
||||
|
||||
Minimum recommended alert set:
|
||||
Current rule set (8):
|
||||
|
||||
- **Availability**
|
||||
`up{job="chat"} == 0` for `2m`; `up{job="prometheus"} == 0` for `2m`.
|
||||
- **Chat service health**
|
||||
`linearvoid_chat_redis_up == 0` for `2m`; sudden disconnect spike / connected clients collapse.
|
||||
- **Playout continuity**
|
||||
No HLS segment progression for `30-60s` (from playout-specific metric when available).
|
||||
- **Prometheus capacity and ingestion health**
|
||||
Sustained growth in `prometheus_tsdb_wal_writes_failed_total` or `prometheus_tsdb_compactions_failed_total`, abnormal jump in `prometheus_tsdb_out_of_order_samples_total`, and high `prometheus_engine_queries` near concurrent limit.
|
||||
- **Latency and user-impact**
|
||||
Elevated p99 handler latency from `prometheus_http_request_duration_seconds_bucket` and elevated p99 query execution latency from `prometheus_engine_query_duration_histogram_seconds_bucket`.
|
||||
- `ChatDown`
|
||||
- `PlayoutDown`
|
||||
- `RedisDown`
|
||||
- `PlayoutSlateActive`
|
||||
- `HighChatLatency`
|
||||
- `PrometheusWALFailing`
|
||||
- `HighCPU`
|
||||
- `LowDiskSpace`
|
||||
|
||||
Route warning/critical severities to the team channel and keep runbook links attached to each alert definition.
|
||||
Route warning/critical severities to the operations channel and keep runbook links attached to each alert definition.
|
||||
|
||||
Operational resilience baseline:
|
||||
|
||||
|
||||
Reference in new issue
Block a user