calminer/docs/migrations/consolidated_baseline_plan.md

Consolidated Migration Baseline Plan

This note outlines the content and structure of the planned baseline migration (scripts/migrations/000_base.sql). The objective is to capture the currently required schema changes in a single idempotent script so that fresh environments only need to apply one SQL file before proceeding with incremental migrations.

Guiding Principles

1. Idempotent DDL: Every CREATE or ALTER statement must tolerate repeated execution. Use IF NOT EXISTS guards or existence checks (information_schema) where necessary.
2. Order of Operations: Create reference tables first, then update dependent tables, finally enforce foreign keys and constraints.
3. Data Safety: Default data seeded by migrations should be minimal and in ASCII-only form to avoid encoding issues in various shells and CI logs.
4. Compatibility: The baseline must reflect the schema shape expected by the current SQLAlchemy models, API routes, and seeding scripts.

Schema Elements to Include

1. currency Table

• Columns: id SERIAL PRIMARY KEY, code VARCHAR(3) UNIQUE NOT NULL, name VARCHAR(128) NOT NULL, symbol VARCHAR(8), is_active BOOLEAN NOT NULL DEFAULT TRUE.
• Index: implicit via unique constraint on code.
• Seed rows matching scripts.seed_data.CURRENCY_SEEDS (ASCII-only symbols such as USD$, CAD$).
• Upsert logic using ON CONFLICT (code) DO UPDATE to keep names/symbols in sync when rerun.

2. Currency Integration for CAPEX/OPEX

• Add currency_id INTEGER columns with IF NOT EXISTS guards.
• Populate currency_id from legacy currency_code if the column exists.
• Default null currency_id values to the USD row, then ALTER to SET NOT NULL.
• Create fk_capex_currency and fk_opex_currency constraints with ON DELETE RESTRICT semantics.
• Drop legacy currency_code column if it exists (safe because new column holds data).

3. Measurement Metadata on Consumption/Production

• Ensure consumption and production_output tables have unit_name VARCHAR(64) and unit_symbol VARCHAR(16) columns with IF NOT EXISTS guards.

4. measurement_unit Reference Table

• Columns: id SERIAL PRIMARY KEY, code VARCHAR(64) UNIQUE NOT NULL, name VARCHAR(128) NOT NULL, symbol VARCHAR(16), unit_type VARCHAR(32) NOT NULL, is_active BOOLEAN NOT NULL DEFAULT TRUE, created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(), updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW().
• Assume a simple trigger to maintain updated_at is deferred: automate via application layer later; for now, omit trigger.
• Seed rows matching MEASUREMENT_UNIT_SEEDS (ASCII names/symbols). Use ON CONFLICT (code) DO UPDATE to keep descriptive fields aligned.

5. Transaction Handling

• Wrap the main operations in a single BEGIN; ... COMMIT; block.
• Use subtransactions (DO $$ ... $$;) only where conditional logic is required (e.g., checking column existence before backfill).

Migration Tracking Alignment

• Baseline file will be named 000_base.sql. After execution, insert a row into schema_migrations with filename 000_base.sql to keep the tracking table aligned.
• Existing migrations (20251021_add_currency_and_unit_fields.sql, 20251022_create_currency_table_and_fks.sql) remain for historical reference but will no longer be applied to new environments once the baseline is present.

Next Steps

1. Draft 000_base.sql reflecting the steps above.
2. Update run_migrations to recognise the baseline file and mark older migrations as applied when the baseline exists.
3. Provide documentation in docs/quickstart.md explaining how to reset an environment using the baseline plus seeds.