From f18987ec8b83ce0d2182427c2e36efbb299e1ad6 Mon Sep 17 00:00:00 2001 From: zwitschi Date: Sat, 15 Nov 2025 13:21:59 +0100 Subject: [PATCH] docs: add Coolify deployment reference and configuration details --- CONTRIBUTING.md | 42 ++++++++++++++++++++-------- admin/runbooks/coolify_deployment.md | 41 +++++++++++++++++++++++++++ 2 files changed, 71 insertions(+), 12 deletions(-) create mode 100644 admin/runbooks/coolify_deployment.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 8e7d08f..28fbc49 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -42,22 +42,40 @@ Once you are satisfied with your changes, submit a pull request to the main repo ## Continuous Integration -Calminer uses Gitea Actions for automated testing, linting, and deployment. The CI pipeline is defined in `.gitea/workflows/cicache.yml` and runs on pushes and pull requests to the `main` and `develop` branches. +Calminer uses Gitea Actions for automated testing, linting, and deployment. The pipeline is orchestrated by `.gitea/workflows/ci.yml`, which delegates to reusable stage workflows that run entirely on the self-hosted Gitea runner: -### Pipeline Stages - -1. **Lint**: Checks code style with Ruff and Black. -2. **Test**: Runs pytest with coverage enforcement (80% threshold), using a PostgreSQL service. Uploads coverage.xml and pytest-report.xml artifacts. -3. **Build**: Builds Docker image and pushes to registry only on `main` branch pushes (not PRs) if registry secrets are configured. +- `.gitea/workflows/ci-lint.yml` checks out the repository, installs dependencies, and runs Ruff, Black, and Bandit. +- `.gitea/workflows/ci-test.yml` provisions PostgreSQL 17, installs dependencies, executes pytest with an 80% coverage threshold, and uploads `coverage.xml` plus `pytest-report.xml` artifacts. +- `.gitea/workflows/ci-build.yml` builds the Docker image, pushing tags only for `main` branch pushes (never for pull requests) when registry credentials are configured. It retains the optional deployment job gated by commit messages (`[deploy staging]`, `[deploy production]`). The workflow validates that the configured registry host matches the current Gitea instance to avoid publishing to the wrong registry. +- `.gitea/workflows/deploy-coolify.yml` runs on `push` events to `main` (and via manual dispatch) once the repository is checked out. It bundles `docker-compose.prod.yml`, requires Coolify secrets (`COOLIFY_BASE_URL`, `COOLIFY_API_TOKEN`, `COOLIFY_APPLICATION_ID`, optional `COOLIFY_DEPLOY_ENV`), and calls the Coolify `/api/v1/deploy` endpoint with the application UUID. ### Workflow Behavior -- Triggers on push/PR to `main` or `develop`. -- Linting must pass before tests run. -- Tests must pass before build runs. -- Coverage below 80% fails the test stage. -- Artifacts are available for PR inspection. -- Docker push occurs only for main branch commits with valid registry credentials. +- `CI - Lint` triggers on push/PR to `main`, `develop`, or `v2` and must succeed before downstream workflows start. +- `CI - Test` and `CI - Build` run as dependent jobs within the orchestrating workflow, so artifacts and images reflect the linted commit. +- Coverage below 80% fails the test workflow and stops the build sequence. +- Artifacts remain available for PR inspection via the test workflow. +- Docker pushes occur only for main-branch pushes (PRs will never publish images). Deployments run exclusively on `main` pushes and require explicit `[deploy staging]` / `[deploy production]` markers. +- Coolify automation also runs only for `main` pushes and relies on the configured application UUID to target the correct deployment. + +### Gitea Runner and Registry Configuration + +The runner and registry are self-hosted within Gitea. Configure the following secrets in the repository settings before enabling the workflows: + +- `REGISTRY_URL`: Base URL of the Gitea container registry (for example `https://git.example.com`). +- `REGISTRY_USERNAME` / `REGISTRY_PASSWORD`: Credentials with permission to push images to the project namespace. +- `COOLIFY_BASE_URL`: Base URL to the Coolify instance (no trailing slash). +- `COOLIFY_API_TOKEN`: Personal API token from Coolify with deploy permission. +- `COOLIFY_APPLICATION_ID`: Coolify application UUID (see **Applications → Settings → UUID** in the Coolify UI). +- `COOLIFY_DEPLOY_ENV` (optional): Multiline environment file content; omit if `deploy/.env` is stored in the repo. +- `KUBE_CONFIG`, `STAGING_KUBE_CONFIG`, `PROD_KUBE_CONFIG` (optional): Base64 encoded kubeconfig for the optional k8s deployment job. +- `K8S_DEPLOY_ENABLED` (optional): Set to `true` to allow the CI deploy job to run Kubernetes steps; leave unset/any other value to skip k8s updates (Coolify deploy still runs). + +Tips: + +- Ensure DNS records resolve the registry host for the runner network. Runner logs such as `logs/ci-lint-399.log` will show failures if the host cannot be resolved. +- The workflows prefer `GITEA_*` environment variables when available (for example `GITEA_REF`, `GITEA_SHA`) and fall back to GitHub-compatible names, guaranteeing operation on a native Gitea runner. +- If you change the registry endpoint, update `REGISTRY_URL` and verify the workflow warning emitted by the “Validate registry configuration” step to avoid pushing images to the wrong registry. ### Local Testing diff --git a/admin/runbooks/coolify_deployment.md b/admin/runbooks/coolify_deployment.md new file mode 100644 index 0000000..712ff75 --- /dev/null +++ b/admin/runbooks/coolify_deployment.md @@ -0,0 +1,41 @@ +# Coolify Deployment Reference + +This note captures the current production deployment inputs so the automated Coolify workflow can be wired up consistently. + +## Compose bundle + +- Use `docker-compose.prod.yml` as the base service definition. +- The compose file expects the following variables supplied by the deployment runner (either through an `.env` file or Coolify secret variables): + - `DATABASE_HOST` + - `DATABASE_PORT` (defaults to `5432` if omitted) + - `DATABASE_USER` + - `DATABASE_PASSWORD` + - `DATABASE_NAME` + - Optional `APT_CACHE_URL` build arg when a proxy is required. +- Expose port `8003` for the FastAPI service and `5432` for PostgreSQL. + +## Runtime expectations + +- Application container runs with `ENVIRONMENT=production`, `LOG_LEVEL=WARNING`, and enables large import/export limits via: + - `CALMINER_EXPORT_MAX_ROWS` + - `CALMINER_IMPORT_MAX_ROWS` + - `CALMINER_EXPORT_METADATA` + - `CALMINER_IMPORT_STAGING_TTL` +- PostgreSQL container requires persistent storage (volume `postgres_data`). +- The app health check hits `http://localhost:8003/health`. + +## Coolify integration inputs (to provision secrets later) + +- Coolify instance URL (e.g. `https://coolify.example.com`) and target project/app identifier. +- API token or CLI credentials with permission to trigger deployments. +- SSH key or repository token already configured in Coolify to pull this repository. +- Optional: webhook/event endpoint if we want to observe deployment status from CI. + +## Manual deployment checklist (current state) + +1. Ensure the compose environment variables above are defined in Coolify under the application settings. +2. Rebuild the application image with the latest commit (using `docker-compose.prod.yml`). +3. Trigger a deployment in Coolify so the platform pulls the new image and restarts the service. +4. Confirm the health check passes and review logs in Coolify. + +These details will feed into the new `deploy-coolify` workflow so it can authenticate, trigger the deployment, and surface logs automatically.