Compare commits
1 Commits
4dea0a9ae1
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| f18987ec8b |
@@ -42,22 +42,40 @@ Once you are satisfied with your changes, submit a pull request to the main repo
|
||||
|
||||
## Continuous Integration
|
||||
|
||||
Calminer uses Gitea Actions for automated testing, linting, and deployment. The CI pipeline is defined in `.gitea/workflows/cicache.yml` and runs on pushes and pull requests to the `main` and `develop` branches.
|
||||
Calminer uses Gitea Actions for automated testing, linting, and deployment. The pipeline is orchestrated by `.gitea/workflows/ci.yml`, which delegates to reusable stage workflows that run entirely on the self-hosted Gitea runner:
|
||||
|
||||
### Pipeline Stages
|
||||
|
||||
1. **Lint**: Checks code style with Ruff and Black.
|
||||
2. **Test**: Runs pytest with coverage enforcement (80% threshold), using a PostgreSQL service. Uploads coverage.xml and pytest-report.xml artifacts.
|
||||
3. **Build**: Builds Docker image and pushes to registry only on `main` branch pushes (not PRs) if registry secrets are configured.
|
||||
- `.gitea/workflows/ci-lint.yml` checks out the repository, installs dependencies, and runs Ruff, Black, and Bandit.
|
||||
- `.gitea/workflows/ci-test.yml` provisions PostgreSQL 17, installs dependencies, executes pytest with an 80% coverage threshold, and uploads `coverage.xml` plus `pytest-report.xml` artifacts.
|
||||
- `.gitea/workflows/ci-build.yml` builds the Docker image, pushing tags only for `main` branch pushes (never for pull requests) when registry credentials are configured. It retains the optional deployment job gated by commit messages (`[deploy staging]`, `[deploy production]`). The workflow validates that the configured registry host matches the current Gitea instance to avoid publishing to the wrong registry.
|
||||
- `.gitea/workflows/deploy-coolify.yml` runs on `push` events to `main` (and via manual dispatch) once the repository is checked out. It bundles `docker-compose.prod.yml`, requires Coolify secrets (`COOLIFY_BASE_URL`, `COOLIFY_API_TOKEN`, `COOLIFY_APPLICATION_ID`, optional `COOLIFY_DEPLOY_ENV`), and calls the Coolify `/api/v1/deploy` endpoint with the application UUID.
|
||||
|
||||
### Workflow Behavior
|
||||
|
||||
- Triggers on push/PR to `main` or `develop`.
|
||||
- Linting must pass before tests run.
|
||||
- Tests must pass before build runs.
|
||||
- Coverage below 80% fails the test stage.
|
||||
- Artifacts are available for PR inspection.
|
||||
- Docker push occurs only for main branch commits with valid registry credentials.
|
||||
- `CI - Lint` triggers on push/PR to `main`, `develop`, or `v2` and must succeed before downstream workflows start.
|
||||
- `CI - Test` and `CI - Build` run as dependent jobs within the orchestrating workflow, so artifacts and images reflect the linted commit.
|
||||
- Coverage below 80% fails the test workflow and stops the build sequence.
|
||||
- Artifacts remain available for PR inspection via the test workflow.
|
||||
- Docker pushes occur only for main-branch pushes (PRs will never publish images). Deployments run exclusively on `main` pushes and require explicit `[deploy staging]` / `[deploy production]` markers.
|
||||
- Coolify automation also runs only for `main` pushes and relies on the configured application UUID to target the correct deployment.
|
||||
|
||||
### Gitea Runner and Registry Configuration
|
||||
|
||||
The runner and registry are self-hosted within Gitea. Configure the following secrets in the repository settings before enabling the workflows:
|
||||
|
||||
- `REGISTRY_URL`: Base URL of the Gitea container registry (for example `https://git.example.com`).
|
||||
- `REGISTRY_USERNAME` / `REGISTRY_PASSWORD`: Credentials with permission to push images to the project namespace.
|
||||
- `COOLIFY_BASE_URL`: Base URL to the Coolify instance (no trailing slash).
|
||||
- `COOLIFY_API_TOKEN`: Personal API token from Coolify with deploy permission.
|
||||
- `COOLIFY_APPLICATION_ID`: Coolify application UUID (see **Applications → Settings → UUID** in the Coolify UI).
|
||||
- `COOLIFY_DEPLOY_ENV` (optional): Multiline environment file content; omit if `deploy/.env` is stored in the repo.
|
||||
- `KUBE_CONFIG`, `STAGING_KUBE_CONFIG`, `PROD_KUBE_CONFIG` (optional): Base64 encoded kubeconfig for the optional k8s deployment job.
|
||||
- `K8S_DEPLOY_ENABLED` (optional): Set to `true` to allow the CI deploy job to run Kubernetes steps; leave unset/any other value to skip k8s updates (Coolify deploy still runs).
|
||||
|
||||
Tips:
|
||||
|
||||
- Ensure DNS records resolve the registry host for the runner network. Runner logs such as `logs/ci-lint-399.log` will show failures if the host cannot be resolved.
|
||||
- The workflows prefer `GITEA_*` environment variables when available (for example `GITEA_REF`, `GITEA_SHA`) and fall back to GitHub-compatible names, guaranteeing operation on a native Gitea runner.
|
||||
- If you change the registry endpoint, update `REGISTRY_URL` and verify the workflow warning emitted by the “Validate registry configuration” step to avoid pushing images to the wrong registry.
|
||||
|
||||
### Local Testing
|
||||
|
||||
|
||||
41
admin/runbooks/coolify_deployment.md
Normal file
41
admin/runbooks/coolify_deployment.md
Normal file
@@ -0,0 +1,41 @@
|
||||
# Coolify Deployment Reference
|
||||
|
||||
This note captures the current production deployment inputs so the automated Coolify workflow can be wired up consistently.
|
||||
|
||||
## Compose bundle
|
||||
|
||||
- Use `docker-compose.prod.yml` as the base service definition.
|
||||
- The compose file expects the following variables supplied by the deployment runner (either through an `.env` file or Coolify secret variables):
|
||||
- `DATABASE_HOST`
|
||||
- `DATABASE_PORT` (defaults to `5432` if omitted)
|
||||
- `DATABASE_USER`
|
||||
- `DATABASE_PASSWORD`
|
||||
- `DATABASE_NAME`
|
||||
- Optional `APT_CACHE_URL` build arg when a proxy is required.
|
||||
- Expose port `8003` for the FastAPI service and `5432` for PostgreSQL.
|
||||
|
||||
## Runtime expectations
|
||||
|
||||
- Application container runs with `ENVIRONMENT=production`, `LOG_LEVEL=WARNING`, and enables large import/export limits via:
|
||||
- `CALMINER_EXPORT_MAX_ROWS`
|
||||
- `CALMINER_IMPORT_MAX_ROWS`
|
||||
- `CALMINER_EXPORT_METADATA`
|
||||
- `CALMINER_IMPORT_STAGING_TTL`
|
||||
- PostgreSQL container requires persistent storage (volume `postgres_data`).
|
||||
- The app health check hits `http://localhost:8003/health`.
|
||||
|
||||
## Coolify integration inputs (to provision secrets later)
|
||||
|
||||
- Coolify instance URL (e.g. `https://coolify.example.com`) and target project/app identifier.
|
||||
- API token or CLI credentials with permission to trigger deployments.
|
||||
- SSH key or repository token already configured in Coolify to pull this repository.
|
||||
- Optional: webhook/event endpoint if we want to observe deployment status from CI.
|
||||
|
||||
## Manual deployment checklist (current state)
|
||||
|
||||
1. Ensure the compose environment variables above are defined in Coolify under the application settings.
|
||||
2. Rebuild the application image with the latest commit (using `docker-compose.prod.yml`).
|
||||
3. Trigger a deployment in Coolify so the platform pulls the new image and restarts the service.
|
||||
4. Confirm the health check passes and review logs in Coolify.
|
||||
|
||||
These details will feed into the new `deploy-coolify` workflow so it can authenticate, trigger the deployment, and surface logs automatically.
|
||||
Reference in New Issue
Block a user