allucanget/calminer
2
0
Fork 0
Code Pull Requests Actions Packages Releases Activity

refactor: Update CI configuration and local setup documentation; remove obsolete currency migration scripts

Browse Source
This commit is contained in:
zwitschi
2025-10-25 16:59:35 +02:00
parent bff75a722e
commit 4e1658a638
5 changed files with 61 additions and 100 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
docs/architecture/14_testing_ci.md
View File

@@ -21,7 +21,7 @@ CalMiner uses a combination of unit, integration, and end-to-end tests to ensure
### CI/CD
- Use Gitea Actions for CI/CD; workflows live under `.gitea/workflows/`.
- `test.yml` runs on every push with cached Python dependencies via `actions/cache@v3`.
- `test.yml` runs on every push, provisions a temporary Postgres 16 service, waits for readiness, executes the setup script in dry-run and live modes, installs Playwright browsers, and finally runs the full pytest suite.
- `build-and-push.yml` builds the Docker image with `docker/build-push-action@v2`, reusing GitHub Actions cache-backed layers, and pushes to the Gitea registry.
- `deploy.yml` connects to the target host (via `appleboy/ssh-action`) to pull the freshly pushed image and restart the container.
- Mandatory secrets: `REGISTRY_USERNAME`, `REGISTRY_PASSWORD`, `REGISTRY_URL`, `SSH_HOST`, `SSH_USERNAME`, `SSH_PRIVATE_KEY`.
@@ -99,10 +99,11 @@ pytest tests/e2e/ --headed
`test.yml` encapsulates the steps below:
- Check out the repository and set up Python 3.10.
- Restore the pip cache (keyed by `requirements.txt`).
- Install project dependencies and Playwright browsers (if needed for E2E).
- Configure the runner's apt proxy (if available), install project dependencies (requirements + test extras), and download Playwright browsers.
- Run `pytest` (extend with `--cov` flags when enforcing coverage).
> The pip cache step is temporarily disabled in `test.yml` until the self-hosted cache service is exposed (see `docs/ci-cache-troubleshooting.md`).
`build-and-push.yml` adds:
- Registry login using repository secrets.

36
docs/quickstart.md
View File

@@ -120,6 +120,38 @@ Typical log output confirms:
After a successful run the target database contains all application tables plus `schema_migrations`, and that table records each applied migration file. New installations only record `000_base.sql`; upgraded environments retain historical entries alongside the baseline.
### Local Postgres via Docker Compose
For local validation without installing Postgres directly, use the provided compose file:
```powershell
docker compose -f docker-compose.postgres.yml up -d
```
The service exposes Postgres 16 on `localhost:5433` with database `calminer_local` and role `calminer`/`secret`. When the container is running, set the granular environment variables before invoking the setup script:
```powershell
$env:DATABASE_DRIVER = 'postgresql'
$env:DATABASE_HOST = '127.0.0.1'
$env:DATABASE_PORT = '5433'
$env:DATABASE_USER = 'calminer'
$env:DATABASE_PASSWORD = 'secret'
$env:DATABASE_NAME = 'calminer_local'
$env:DATABASE_SCHEMA = 'public'
python scripts/setup_database.py --ensure-database --ensure-role --ensure-schema --initialize-schema --run-migrations --seed-data --dry-run -v
python scripts/setup_database.py --ensure-database --ensure-role --ensure-schema --initialize-schema --run-migrations --seed-data -v
```
When testing is complete, shut down the container (and optional persistent volume) with:
```powershell
docker compose -f docker-compose.postgres.yml down
docker volume rm calminer_postgres_local_postgres_data # optional cleanup
```
Document successful runs (or issues encountered) in `.github/instructions/DONE.TODO.md` for future reference.
### Seeding reference data
`scripts/seed_data.py` provides targeted control over the baseline datasets when the full setup script is not required:
@@ -154,7 +186,7 @@ The `.gitea/workflows/test.yml` job spins up a temporary PostgreSQL 16 container
| Variable | Value | Purpose |
| --- | --- | --- |
| `DATABASE_DRIVER` | `postgresql` | Signals the driver to the setup script |
| `DATABASE_HOST` | `127.0.0.1` | Points to the linked job service |
| `DATABASE_HOST` | `postgres` | Hostname of the Postgres job service container |
| `DATABASE_PORT` | `5432` | Default service port |
| `DATABASE_NAME` | `calminer_ci` | Target database created by the workflow |
| `DATABASE_USER` | `calminer` | Application role used during tests |
@@ -166,7 +198,7 @@ The `.gitea/workflows/test.yml` job spins up a temporary PostgreSQL 16 container
The workflow also updates `DATABASE_URL` for pytest to point at the CI Postgres instance. Existing tests continue to work unchanged, since SQLAlchemy reads the URL exactly as it does locally.
Because the workflow provisions everything inline, no repository or organization secrets need to be configured for basic CI runs. If you later move the setup step to staging or production pipelines, replace these inline values with secrets managed by the CI platform.
Because the workflow provisions everything inline, no repository or organization secrets need to be configured for basic CI runs. If you later move the setup step to staging or production pipelines, replace these inline values with secrets managed by the CI platform. When running on self-hosted runners behind an HTTP proxy or apt cache, ensure Playwright dependencies and OS packages inherit the same proxy settings that the workflow configures prior to installing browsers.
## Database Objects