calminer/docs/architecture/07_deployment/07_02_database.md

Database Deployment

Migrations & Baseline

A consolidated baseline migration (scripts/migrations/000_base.sql) captures all schema changes required for a fresh installation. The script is idempotent: it creates the currency and measurement_unit reference tables, provisions the application_setting store for configurable UI/system options, ensures consumption and production records expose unit metadata, and enforces the foreign keys used by CAPEX and OPEX.

Configure granular database settings in your PowerShell session before running migrations:

$env:DATABASE_DRIVER = 'postgresql'
$env:DATABASE_HOST = 'localhost'
$env:DATABASE_PORT = '5432'
$env:DATABASE_USER = 'calminer'
$env:DATABASE_PASSWORD = 's3cret'
$env:DATABASE_NAME = 'calminer'
$env:DATABASE_SCHEMA = 'public'
python scripts/setup_database.py --run-migrations --seed-data --dry-run
python scripts/setup_database.py --run-migrations --seed-data

The dry-run invocation reports which steps would execute without making changes. The live run applies the baseline (if not already recorded in schema_migrations) and seeds the reference data relied upon by the UI and API.

ℹ️ When --seed-data is supplied without --run-migrations, the bootstrap script automatically applies any pending SQL migrations first so the application_setting table (and future settings-backed features) are present before seeding.

ℹ️ The application still accepts DATABASE_URL as a fallback if the granular variables are not set.

Database bootstrap workflow

Provision or refresh a database instance with scripts/setup_database.py. Populate the required environment variables (an example lives at config/setup_test.env.example) and run:

# Load test credentials (PowerShell)
Get-Content .\config\setup_test.env.example |
  ForEach-Object {
    if ($_ -and -not $_.StartsWith('#')) {
      $name, $value = $_ -split '=', 2
      Set-Item -Path Env:$name -Value $value
    }
  }

# Dry-run to inspect the planned actions
python scripts/setup_database.py --ensure-database --ensure-role --ensure-schema --initialize-schema --run-migrations --seed-data --dry-run -v

# Execute the full workflow
python scripts/setup_database.py --ensure-database --ensure-role --ensure-schema --initialize-schema --run-migrations --seed-data -v

Typical log output confirms:

• Admin and application connections succeed for the supplied credentials.
• Database and role creation are idempotent (already present when rerun).
• SQLAlchemy metadata either reports missing tables or All tables already exist.
• Migrations list pending files and finish with Applied N migrations (a new database reports Applied 1 migrations for 000_base.sql).

After a successful run the target database contains all application tables plus schema_migrations, and that table records each applied migration file. New installations only record 000_base.sql; upgraded environments retain historical entries alongside the baseline.

Seeding reference data

scripts/seed_data.py provides targeted control over the baseline datasets when the full setup script is not required:

python scripts/seed_data.py --currencies --units --dry-run
python scripts/seed_data.py --currencies --units

The seeder upserts the canonical currency catalog (USD, EUR, CLP, RMB, GBP, CAD, AUD) using ASCII-safe symbols (USD$, EUR, etc.) and the measurement units referenced by the UI (tonnes, kilograms, pounds, liters, cubic_meters, kilowatt_hours). The setup script invokes the same seeder when --seed-data is provided and verifies the expected rows afterward, warning if any are missing or inactive.

Rollback guidance

scripts/setup_database.py now tracks compensating actions when it creates the database or application role. If a later step fails, the script replays those rollback actions (dropping the newly created database or role and revoking grants) before exiting. Dry runs never register rollback steps and remain read-only.

If the script reports that some rollback steps could not complete—for example because a connection cannot be established—rerun the script with --dry-run to confirm the desired end state and then apply the outstanding cleanup manually:

python scripts/setup_database.py --ensure-database --ensure-role --dry-run -v

# Manual cleanup examples when automation cannot connect
psql -d postgres -c "DROP DATABASE IF EXISTS calminer"
psql -d postgres -c "DROP ROLE IF EXISTS calminer"

After a failure and rollback, rerun the full setup once the environment issues are resolved.