feat: Add currency management feature with CRUD operations
Some checks failed
Run Tests / test (push) Failing after 5m2s
Some checks failed
Run Tests / test (push) Failing after 5m2s
- Introduced a new template for currency overview and management (`currencies.html`). - Updated footer to include attribution to AllYouCanGET. - Added "Currencies" link to the main navigation header. - Implemented end-to-end tests for currency creation, update, and activation toggling. - Created unit tests for currency API endpoints, including creation, updating, and activation toggling. - Added a fixture to seed default currencies for testing. - Enhanced database setup tests to ensure proper seeding and migration handling.
This commit is contained in:
@@ -68,13 +68,11 @@ pytest
|
||||
|
||||
E2E tests use Playwright and a session-scoped `live_server` fixture that starts the app at `http://localhost:8001` for browser-driven tests.
|
||||
|
||||
## Migrations & Currency Backfill
|
||||
## Migrations & Baseline
|
||||
|
||||
The project includes a referential `currency` table and migration/backfill tooling to normalize legacy currency fields.
|
||||
A consolidated baseline migration (`scripts/migrations/000_base.sql`) captures all schema changes required for a fresh installation. The script is idempotent: it creates the `currency` and `measurement_unit` reference tables, ensures consumption and production records expose unit metadata, and enforces the foreign keys used by CAPEX and OPEX.
|
||||
|
||||
### Run migrations and backfill (development)
|
||||
|
||||
Configure the granular database settings in your PowerShell session before running migrations.
|
||||
Configure granular database settings in your PowerShell session before running migrations:
|
||||
|
||||
```powershell
|
||||
$env:DATABASE_DRIVER = 'postgresql'
|
||||
@@ -84,14 +82,91 @@ $env:DATABASE_USER = 'calminer'
|
||||
$env:DATABASE_PASSWORD = 's3cret'
|
||||
$env:DATABASE_NAME = 'calminer'
|
||||
$env:DATABASE_SCHEMA = 'public'
|
||||
python scripts/run_migrations.py
|
||||
python scripts/backfill_currency.py --dry-run
|
||||
python scripts/backfill_currency.py --create-missing
|
||||
python scripts/setup_database.py --run-migrations --seed-data --dry-run
|
||||
python scripts/setup_database.py --run-migrations --seed-data
|
||||
```
|
||||
|
||||
The dry-run invocation reports which steps would execute without making changes. The live run applies the baseline (if not already recorded in `schema_migrations`) and seeds the reference data relied upon by the UI and API.
|
||||
|
||||
> ℹ️ The application still accepts `DATABASE_URL` as a fallback if the granular variables are not set.
|
||||
|
||||
Use `--dry-run` first to verify what will change.
|
||||
## Database bootstrap workflow
|
||||
|
||||
Provision or refresh a database instance with `scripts/setup_database.py`. Populate the required environment variables (an example lives at `config/setup_test.env.example`) and run:
|
||||
|
||||
```powershell
|
||||
# Load test credentials (PowerShell)
|
||||
Get-Content .\config\setup_test.env.example |
|
||||
ForEach-Object {
|
||||
if ($_ -and -not $_.StartsWith('#')) {
|
||||
$name, $value = $_ -split '=', 2
|
||||
Set-Item -Path Env:$name -Value $value
|
||||
}
|
||||
}
|
||||
|
||||
# Dry-run to inspect the planned actions
|
||||
python scripts/setup_database.py --ensure-database --ensure-role --ensure-schema --initialize-schema --run-migrations --seed-data --dry-run -v
|
||||
|
||||
# Execute the full workflow
|
||||
python scripts/setup_database.py --ensure-database --ensure-role --ensure-schema --initialize-schema --run-migrations --seed-data -v
|
||||
```
|
||||
|
||||
Typical log output confirms:
|
||||
|
||||
- Admin and application connections succeed for the supplied credentials.
|
||||
- Database and role creation are idempotent (`already present` when rerun).
|
||||
- SQLAlchemy metadata either reports missing tables or `All tables already exist`.
|
||||
- Migrations list pending files and finish with `Applied N migrations` (a new database reports `Applied 1 migrations` for `000_base.sql`).
|
||||
|
||||
After a successful run the target database contains all application tables plus `schema_migrations`, and that table records each applied migration file. New installations only record `000_base.sql`; upgraded environments retain historical entries alongside the baseline.
|
||||
|
||||
### Seeding reference data
|
||||
|
||||
`scripts/seed_data.py` provides targeted control over the baseline datasets when the full setup script is not required:
|
||||
|
||||
```powershell
|
||||
python scripts/seed_data.py --currencies --units --dry-run
|
||||
python scripts/seed_data.py --currencies --units
|
||||
```
|
||||
|
||||
The seeder upserts the canonical currency catalog (`USD`, `EUR`, `CLP`, `RMB`, `GBP`, `CAD`, `AUD`) using ASCII-safe symbols (`USD$`, `EUR`, etc.) and the measurement units referenced by the UI (`tonnes`, `kilograms`, `pounds`, `liters`, `cubic_meters`, `kilowatt_hours`). The setup script invokes the same seeder when `--seed-data` is provided and verifies the expected rows afterward, warning if any are missing or inactive.
|
||||
|
||||
### Rollback guidance
|
||||
|
||||
`scripts/setup_database.py` now tracks compensating actions when it creates the database or application role. If a later step fails, the script replays those rollback actions (dropping the newly created database or role and revoking grants) before exiting. Dry runs never register rollback steps and remain read-only.
|
||||
|
||||
If the script reports that some rollback steps could not complete—for example because a connection cannot be established—rerun the script with `--dry-run` to confirm the desired end state and then apply the outstanding cleanup manually:
|
||||
|
||||
```powershell
|
||||
python scripts/setup_database.py --ensure-database --ensure-role --dry-run -v
|
||||
|
||||
# Manual cleanup examples when automation cannot connect
|
||||
psql -d postgres -c "DROP DATABASE IF EXISTS calminer"
|
||||
psql -d postgres -c "DROP ROLE IF EXISTS calminer"
|
||||
```
|
||||
|
||||
After a failure and rollback, rerun the full setup once the environment issues are resolved.
|
||||
|
||||
### CI pipeline environment
|
||||
|
||||
The `.gitea/workflows/test.yml` job spins up a temporary PostgreSQL 16 container and runs the setup script twice: once with `--dry-run` to validate the plan and again without it to apply migrations and seeds. No external secrets are required; the workflow sets the following environment variables for both invocations and for pytest:
|
||||
|
||||
| Variable | Value | Purpose |
|
||||
| --- | --- | --- |
|
||||
| `DATABASE_DRIVER` | `postgresql` | Signals the driver to the setup script |
|
||||
| `DATABASE_HOST` | `127.0.0.1` | Points to the linked job service |
|
||||
| `DATABASE_PORT` | `5432` | Default service port |
|
||||
| `DATABASE_NAME` | `calminer_ci` | Target database created by the workflow |
|
||||
| `DATABASE_USER` | `calminer` | Application role used during tests |
|
||||
| `DATABASE_PASSWORD` | `secret` | Password for both admin and app role |
|
||||
| `DATABASE_SCHEMA` | `public` | Default schema for the tests |
|
||||
| `DATABASE_SUPERUSER` | `calminer` | Setup script uses the same role for admin actions |
|
||||
| `DATABASE_SUPERUSER_PASSWORD` | `secret` | Matches the Postgres service password |
|
||||
| `DATABASE_SUPERUSER_DB` | `calminer_ci` | Database to connect to for admin operations |
|
||||
|
||||
The workflow also updates `DATABASE_URL` for pytest to point at the CI Postgres instance. Existing tests continue to work unchanged, since SQLAlchemy reads the URL exactly as it does locally.
|
||||
|
||||
Because the workflow provisions everything inline, no repository or organization secrets need to be configured for basic CI runs. If you later move the setup step to staging or production pipelines, replace these inline values with secrets managed by the CI platform.
|
||||
|
||||
## Database Objects
|
||||
|
||||
|
||||
Reference in New Issue
Block a user