docs: add Coolify deployment reference and configuration details

Add detailed SQLAlchemy models, navigation metadata, enumerations, Pydantic schemas, monitoring, and auditing documentation
- Introduced SQLAlchemy models for user management, project management, financial inputs, and pricing configuration. - Created navigation metadata tables for sidebar and top-level menus. - Cataloged enumerations used across ORM models and Pydantic schemas. - Documented Pydantic schemas for API request/response validation, including authentication, project, scenario, import, and export schemas. - Added monitoring and auditing tables for performance metrics and import/export logs. - Updated security documentation to reflect changes in data model references.
2025-11-15 13:21:59 +01:00 · 2025-11-13 20:23:09 +01:00 · 2025-11-13 14:10:47 +01:00 · 2025-11-13 09:20:10 +01:00 · 2025-11-12 18:17:39 +01:00 · 2025-11-12 18:17:20 +01:00
34 changed files with 3046 additions and 565 deletions
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,3 +1,8 @@
 # Change Log
-<!-- TODO: Add description for CHANGELOG.md -->
+## 2025-11-10
 - Completed export workflow implementation (query builders, CSV/XLSX serializers, streaming API endpoints, UI modals, automated tests).
 - Added export modal UI and client script to trigger downloads directly from dashboard.
 - Documented import/export field mapping and usage guidelines in FR-008.
 - Updated installation guide with export environment variables, dependencies, and CLI/CI usage instructions.
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -2,14 +2,98 @@
 We welcome contributions to Calminer! If you would like to contribute, please follow these guidelines:
-1. **Fork the Repository**: Create a personal copy of the repository on your GitHub account.
+## Fork the Repository
-2. **Create a Branch**: Before making changes, create a new branch for your feature or bug fix.
+Create a personal copy of the repository on your GitHub account.
-3. **Make Your Changes**: Implement your changes in the new branch.
+## Create a Branch
-4. **Write Tests**: Ensure that your changes are covered by tests.
+Before making changes, create a new branch for your feature or bug fix.
-5. **Submit a Pull Request**: Once you are satisfied with your changes, submit a pull request to the main repository.
+## Make Your Changes
 Implement your changes in the new branch.
 ## Write Tests
 Ensure that your changes are covered by tests.
 ## Run Test Suite With Coverage
 Execute the default pytest run to enforce the 80% project-wide coverage threshold and review missing lines in the terminal report.
 ```bash
 pytest
 ```
 ## Run Export Test Suite
 Before opening a pull request, run the export-focused pytest module to verify CSV/XLSX streaming endpoints.
 ```bash
 pytest tests/test_export_routes.py
 ```
 This ensures the API headers, download content, and modal routes remain functional.
 ## Submit a Pull Request
 Once you are satisfied with your changes, submit a pull request to the main repository.
 ## Continuous Integration
 Calminer uses Gitea Actions for automated testing, linting, and deployment. The pipeline is orchestrated by `.gitea/workflows/ci.yml`, which delegates to reusable stage workflows that run entirely on the self-hosted Gitea runner:
 - `.gitea/workflows/ci-lint.yml` checks out the repository, installs dependencies, and runs Ruff, Black, and Bandit.
 - `.gitea/workflows/ci-test.yml` provisions PostgreSQL 17, installs dependencies, executes pytest with an 80% coverage threshold, and uploads `coverage.xml` plus `pytest-report.xml` artifacts.
 - `.gitea/workflows/ci-build.yml` builds the Docker image, pushing tags only for `main` branch pushes (never for pull requests) when registry credentials are configured. It retains the optional deployment job gated by commit messages (`[deploy staging]`, `[deploy production]`). The workflow validates that the configured registry host matches the current Gitea instance to avoid publishing to the wrong registry.
 - `.gitea/workflows/deploy-coolify.yml` runs on `push` events to `main` (and via manual dispatch) once the repository is checked out. It bundles `docker-compose.prod.yml`, requires Coolify secrets (`COOLIFY_BASE_URL`, `COOLIFY_API_TOKEN`, `COOLIFY_APPLICATION_ID`, optional `COOLIFY_DEPLOY_ENV`), and calls the Coolify `/api/v1/deploy` endpoint with the application UUID.
 ### Workflow Behavior
 - `CI - Lint` triggers on push/PR to `main`, `develop`, or `v2` and must succeed before downstream workflows start.
 - `CI - Test` and `CI - Build` run as dependent jobs within the orchestrating workflow, so artifacts and images reflect the linted commit.
 - Coverage below 80% fails the test workflow and stops the build sequence.
 - Artifacts remain available for PR inspection via the test workflow.
 - Docker pushes occur only for main-branch pushes (PRs will never publish images). Deployments run exclusively on `main` pushes and require explicit `[deploy staging]` / `[deploy production]` markers.
 - Coolify automation also runs only for `main` pushes and relies on the configured application UUID to target the correct deployment.
 ### Gitea Runner and Registry Configuration
 The runner and registry are self-hosted within Gitea. Configure the following secrets in the repository settings before enabling the workflows:
 - `REGISTRY_URL`: Base URL of the Gitea container registry (for example `https://git.example.com`).
 - `REGISTRY_USERNAME` / `REGISTRY_PASSWORD`: Credentials with permission to push images to the project namespace.
 - `COOLIFY_BASE_URL`: Base URL to the Coolify instance (no trailing slash).
 - `COOLIFY_API_TOKEN`: Personal API token from Coolify with deploy permission.
 - `COOLIFY_APPLICATION_ID`: Coolify application UUID (see **Applications → Settings → UUID** in the Coolify UI).
 - `COOLIFY_DEPLOY_ENV` (optional): Multiline environment file content; omit if `deploy/.env` is stored in the repo.
 - `KUBE_CONFIG`, `STAGING_KUBE_CONFIG`, `PROD_KUBE_CONFIG` (optional): Base64 encoded kubeconfig for the optional k8s deployment job.
 - `K8S_DEPLOY_ENABLED` (optional): Set to `true` to allow the CI deploy job to run Kubernetes steps; leave unset/any other value to skip k8s updates (Coolify deploy still runs).
 Tips:
 - Ensure DNS records resolve the registry host for the runner network. Runner logs such as `logs/ci-lint-399.log` will show failures if the host cannot be resolved.
 - The workflows prefer `GITEA_*` environment variables when available (for example `GITEA_REF`, `GITEA_SHA`) and fall back to GitHub-compatible names, guaranteeing operation on a native Gitea runner.
 - If you change the registry endpoint, update `REGISTRY_URL` and verify the workflow warning emitted by the “Validate registry configuration” step to avoid pushing images to the wrong registry.
 ### Local Testing
 To replicate CI locally:
 ```bash
 # Install test deps
 pip install -r requirements-test.txt
 # Run linting
 ruff check .
 black --check .
 # Run tests with coverage
 pytest --cov=. --cov-report=term-missing --cov-fail-under=80
 # Build image
 docker build -t calminer .
 ```
 Thank you for your interest in contributing to Calminer!
--- a/admin/installation.md
+++ b/admin/installation.md
@@ -27,34 +27,93 @@ Before you begin, ensure that you have the following prerequisites installed on
   cd calminer
   ```
-2. **Build and Start the Docker Containers**
+2. **Environment Configuration**
   Copy the appropriate environment file for your deployment:
   ```bash
   # For development
   cp .env.development .env
   # For staging
   cp .env.staging .env
   # For production
   cp .env.production .env
   # Then edit .env with your actual database credentials
   ```
 3. **Build and Start the Docker Containers**
   Run the following command to build and start the Docker containers:
   ```bash
   # For development (includes live reload and source mounting)
   docker compose up --build
   # For staging
   docker compose -f docker-compose.yml -f docker-compose.staging.yml up --build
   # For production
   docker compose -f docker-compose.yml -f docker-compose.prod.yml up --build
   ```
   This command will build the Docker images and start the containers as defined in the `docker-compose.yml` file.
-3. **Access the Application**
+4. **Access the Application**
-   Once the containers are up and running, you can access the Calminer application by navigating to `http://localhost:3000` in your web browser.
+   Once the containers are up and running, you can access the Calminer application by navigating to `http://localhost:8003` in your web browser.
   If you are running the application on a remote server, replace `localhost` with the server's IP address or domain name.
-4. **Database Initialization**
+5. **Database Initialization & Seed Data**
-   The first time you run the application, the database will be initialized automatically. Ensure that the database container is running and accessible.
+   On container startup the FastAPI application executes `scripts.init_db` during the first `startup` event. This idempotent initializer performs the following steps:
-5. **Stopping the Application**
+   - ensures PostgreSQL enum types exist (no duplicates are created),
   - creates the core tables required for authentication, pricing, projects, scenarios, financial inputs, and simulation parameters,
   - seeds default roles, admin user, pricing settings, and demo projects/scenarios/financial inputs using `INSERT ... ON CONFLICT DO NOTHING` semantics.
-   To stop the application, run the following command in the terminal:
+   No additional action is required when using Docker Compose—the `uvicorn` process runs the initializer automatically before the bootstrap routines.
   For local development without Docker, run the initializer manually after exporting your environment variables:
   ```bash
-   docker compose down
+   # activate your virtualenv first
   python -m scripts.init_db
   ```
-   This command will stop and remove the containers, networks, and volumes created by Docker Compose.
+   Running the script multiple times is safe; it will reconcile records without duplicating data.
 6. **Resetting the Schema (optional)**
   To rebuild the schema from a clean slate in development, use the reset utility which drops the managed tables and enum types before rerunning `init_db`:
   ```bash
   # activate your virtualenv first
   CALMINER_ENV=development python -m scripts.reset_db
   python -m scripts.init_db
   ```
   The reset utility refuses to run when `CALMINER_ENV` indicates a production or staging environment, providing an extra safeguard against accidental destructive operations.
 ### Export Dependencies
 Export and monitoring workflows require the following Python packages in addition to the core dependencies:
 - `pandas`
 - `openpyxl`
 - `prometheus-client`
 These libraries are already listed in `requirements.txt`. Ensure they are installed in your virtual environment if you are not using Docker.
 ### Environment Variables for Export Features
 While exports reuse the existing database configuration, you may optionally set the following variables to adjust behavior:
 - `CALMINER_EXPORT_MAX_ROWS` — override default pagination when generating exports (optional).
 - `CALMINER_EXPORT_METADATA` — enable (`true`) or disable (`false`) the metadata sheet in Excel exports by default (UI form still allows per-request overrides).
 Set these variables in your `.env` file or compose environment section before launching the stack.
 ## Docker Configuration
@@ -62,7 +121,26 @@ The `docker-compose.yml` file contains the configuration for the Calminer applic
 ### Environment Variables
-The application uses environment variables to configure various settings. You can set these variables in a `.env` file in the root directory of the project. Refer to the `docker-compose.yml` file for a list of available environment variables and their default values.
+The application uses environment variables to configure various settings. You can set these variables in a `.env` file in the root directory of the project. Refer to the provided `.env.*` files for examples and default values.
 Key variables relevant to import/export workflows:
 | Variable                      | Development | Staging | Production | Description                                                                     |
 | ----------------------------- | ----------- | ------- | ---------- | ------------------------------------------------------------------------------- |
 | `CALMINER_EXPORT_MAX_ROWS`    | 1000        | 50000   | 100000     | Optional safety guard to limit the number of rows exported in a single request. |
 | `CALMINER_EXPORT_METADATA`    | `true`      | `true`  | `true`     | Controls whether metadata sheets are generated by default during Excel exports. |
 | `CALMINER_IMPORT_STAGING_TTL` | `300`       | `600`   | `3600`     | Controls how long staged import tokens remain valid before expiration.          |
 | `CALMINER_IMPORT_MAX_ROWS`    | `10000`     | `50000` | `100000`   | Optional guard to prevent excessively large import files.                       |
 ### Docker Environment Parity
 The Docker Compose configurations ensure environment parity across development, staging, and production:
 - **Development**: Uses `docker-compose.override.yml` with live code reloading, debug logging, and relaxed resource limits.
 - **Staging**: Uses `docker-compose.staging.yml` with health checks, moderate resource limits, and staging-specific configurations.
 - **Production**: Uses `docker-compose.prod.yml` with strict resource limits, production logging, and required external database configuration.
 All environments use the same base `docker-compose.yml` and share common environment variables for consistency.
 ### Volumes
@@ -73,6 +151,106 @@ The application uses Docker volumes to persist data. The following volumes are d
 Ensure that these volumes are properly configured to avoid data loss during container restarts or removals.
 ## Kubernetes Deployment
 For production deployments, Calminer can be deployed on a Kubernetes cluster using the provided manifests in the `k8s/` directory.
 ### K8s Prerequisites
 - Kubernetes cluster (e.g., minikube for local testing, or cloud provider like GKE, EKS)
 - kubectl configured to access the cluster
 - Helm (optional, for advanced deployments)
 ### K8s Deployment Steps
 1. **Clone the Repository and Build Image**
   ```bash
   git clone https://git.allucanget.biz/allucanget/calminer.git
   cd calminer
   docker build -t registry.example.com/calminer:latest .
   docker push registry.example.com/calminer:latest
   ```
 2. **Update Manifests**
   Edit the manifests in `k8s/` to match your environment:
   - Update image registry in `deployment.yaml`
   - Update host in `ingress.yaml`
   - Update secrets in `secret.yaml` with base64 encoded values
 3. **Deploy to Kubernetes**
   ```bash
   kubectl apply -f k8s/
   ```
 4. **Verify Deployment**
   ```bash
   kubectl get pods
   kubectl get services
   kubectl get ingress
   ```
 5. **Access the Application**
   The application will be available at the ingress host (e.g., `https://calminer.example.com`).
 ### Environment Parity
 The Kubernetes deployment uses the same environment variables as Docker Compose, ensuring consistency across environments. Secrets are managed via Kubernetes Secrets, and configurations via ConfigMaps.
 ### Scaling
 The deployment is configured with 3 replicas for high availability. You can scale as needed:
 ```bash
 kubectl scale deployment calminer-app --replicas=5
 ```
 ### Monitoring
 Ensure your monitoring stack (e.g., Prometheus) scrapes the `/metrics` endpoint from the service.
 ## CI/CD Pipeline
 Calminer uses Gitea Actions for continuous integration and deployment. The CI/CD pipeline is defined in `.gitea/workflows/cicache.yml` and includes the following stages:
 ### CI Stages
 1. **Lint**: Runs Ruff for Python linting, Black for code formatting, and Bandit for security scanning.
 2. **Test**: Executes the full pytest suite with coverage reporting (80% minimum), using a PostgreSQL service container.
 3. **Build**: Builds Docker images using Buildx and pushes to the Gitea registry on the main branch.
 ### CD Stages
 1. **Deploy**: Deploys to staging or production Kubernetes clusters based on commit messages containing `[deploy staging]` or `[deploy production]`.
 ### Required Secrets
 The following secrets must be configured in your Gitea repository:
 - `REGISTRY_URL`: Gitea registry URL
 - `REGISTRY_USERNAME`: Registry username
 - `REGISTRY_PASSWORD`: Registry password
 - `STAGING_KUBE_CONFIG`: Base64-encoded kubeconfig for staging cluster
 - `PROD_KUBE_CONFIG`: Base64-encoded kubeconfig for production cluster
 ### Deployment Triggers
 - **Automatic**: Images are built and pushed on every push to `main` or `develop` branches.
 - **Staging Deployment**: Include `[deploy staging]` in your commit message to trigger staging deployment.
 - **Production Deployment**: Include `[deploy production]` in your commit message to trigger production deployment.
 ### Monitoring CI/CD
 - View pipeline status in the Gitea Actions tab.
 - Test artifacts (coverage, pytest reports) are uploaded for each run.
 - Docker build logs are available for troubleshooting build failures.
 - Deployment runs publish Kubernetes rollout diagnostics under the `deployment-logs` artifact (`/logs/deployment/`), which includes pod listings, deployment manifests, and recent container logs.
 ## Troubleshooting
 If you encounter any issues during the installation or deployment process, refer to the following troubleshooting tips:
--- a/admin/runbooks/coolify_deployment.md
+++ b/admin/runbooks/coolify_deployment.md
@@ -0,0 +1,41 @@
 # Coolify Deployment Reference
 This note captures the current production deployment inputs so the automated Coolify workflow can be wired up consistently.
 ## Compose bundle
 - Use `docker-compose.prod.yml` as the base service definition.
 - The compose file expects the following variables supplied by the deployment runner (either through an `.env` file or Coolify secret variables):
  - `DATABASE_HOST`
  - `DATABASE_PORT` (defaults to `5432` if omitted)
  - `DATABASE_USER`
  - `DATABASE_PASSWORD`
  - `DATABASE_NAME`
  - Optional `APT_CACHE_URL` build arg when a proxy is required.
 - Expose port `8003` for the FastAPI service and `5432` for PostgreSQL.
 ## Runtime expectations
 - Application container runs with `ENVIRONMENT=production`, `LOG_LEVEL=WARNING`, and enables large import/export limits via:
  - `CALMINER_EXPORT_MAX_ROWS`
  - `CALMINER_IMPORT_MAX_ROWS`
  - `CALMINER_EXPORT_METADATA`
  - `CALMINER_IMPORT_STAGING_TTL`
 - PostgreSQL container requires persistent storage (volume `postgres_data`).
 - The app health check hits `http://localhost:8003/health`.
 ## Coolify integration inputs (to provision secrets later)
 - Coolify instance URL (e.g. `https://coolify.example.com`) and target project/app identifier.
 - API token or CLI credentials with permission to trigger deployments.
 - SSH key or repository token already configured in Coolify to pull this repository.
 - Optional: webhook/event endpoint if we want to observe deployment status from CI.
 ## Manual deployment checklist (current state)
 1. Ensure the compose environment variables above are defined in Coolify under the application settings.
 2. Rebuild the application image with the latest commit (using `docker-compose.prod.yml`).
 3. Trigger a deployment in Coolify so the platform pulls the new image and restarts the service.
 4. Confirm the health check passes and review logs in Coolify.
 These details will feed into the new `deploy-coolify` workflow so it can authenticate, trigger the deployment, and surface logs automatically.
--- a/admin/runbooks/export_operations.md
+++ b/admin/runbooks/export_operations.md
@@ -0,0 +1,81 @@
 # Export Operations Runbook
 ## Purpose
 This runbook provides step-by-step guidance for operators to execute project and scenario exports, monitor their status, and troubleshoot common issues.
 ## Prerequisites
 - Access to the CalMiner web UI with role `analyst`, `project_manager`, or `admin`.
 - Direct API access (curl or HTTP client) if performing scripted exports.
 - Environment variables configured per [Installation Guide](installation.md), especially:
  - `CALMINER_EXPORT_MAX_ROWS`
  - `CALMINER_EXPORT_METADATA`
 ## Success Path
 ### Export via Web UI
 1. Sign in to CalMiner.
 2. Navigate to the dashboard and click **Export** next to either _Recent Projects_ or _Scenario Alerts_.
 3. In the modal dialog:
   - Choose **CSV** or **Excel (.xlsx)**.
   - Toggle **Include metadata sheet** (Excel only) as needed.
   - Click **Download**.
 4. Confirm that the browser downloads a file named `projects-YYYYMMDD-HHMMSS.csv` (or `.xlsx`).
 ### Export via API (curl)
 ```bash
 # CSV export of projects
 curl -X POST https://<host>/exports/projects \
  -H "Content-Type: application/json" \
  -d '{"format": "csv"}' \
  --output projects.csv
 # Excel export of scenarios
 curl -X POST https://<host>/exports/scenarios \
  -H "Content-Type: application/json" \
  -d '{"format": "xlsx"}' \
  --output scenarios.xlsx
 ```
 Expected response headers:
 - `Content-Type: text/csv` or `application/vnd.openxmlformats-officedocument.spreadsheetml.sheet`
 - `Content-Disposition: attachment; filename=...`
 ## Troubleshooting
 | Symptom                                  | Likely Cause                                   | Resolution                                                                  |
 | ---------------------------------------- | ---------------------------------------------- | --------------------------------------------------------------------------- |
 | `403 Forbidden`                          | User lacks analyst/project_manager/admin role. | Assign appropriate role or escalate to administrator.                       |
 | `400 Bad Request` with validation errors | Unsupported format or malformed filters.       | Verify payload matches schema (`format` = `csv` or `xlsx`); review filters. |
 | Empty dataset                            | No matching records for filters.               | Validate data exists; adjust filters or check project/scenario status.      |
 | Large exports time out                   | Dataset exceeds `CALMINER_EXPORT_MAX_ROWS`.    | Increase limit (with caution) or export narrower dataset.                   |
 ## Monitoring & Logging
 - Success and error events are logged via structured events (`import.preview`, `import.commit`, `export`). Ensure your log sink (e.g., ELK) captures JSON payloads and index fields such as `dataset`, `status`, `row_count`, and `token` for filtering.
 - Prometheus endpoint: `GET /metrics`
  - Sample scrape config:
    ```yaml
    scrape_configs:
      - job_name: calminer
        static_configs:
          - targets: ["calminer.local:8000"]
    ```
  - Key metrics:
    - `calminer_import_total` / `calminer_export_total` — counters labelled by `dataset`, `action`, and `status` (imports) or `dataset`, `status`, and `format` (exports).
    - `calminer_import_duration_seconds` / `calminer_export_duration_seconds` — histograms for measuring operation duration.
  - Alerting suggestions:
    - Trigger when `calminer_export_total{status="failure"}` increases for the last 5 minutes.
    - Trigger when 95th percentile of `calminer_export_duration_seconds` exceeds your SLA threshold.
 - Dashboard recommendations:
  - Plot export/import throughput split by dataset and format.
  - Surface recent failures with `detail` and `error` metadata pulled from the `import_export_logs` table.
  - Combine logs, metrics, and DB audit records to trace user actions end-to-end.
--- a/api/README.md
+++ b/api/README.md
@@ -1,3 +1,22 @@
 # API Documentation
-<!-- TODO: Add API documentation -->
+## Project & Scenario Endpoints
 | Method | Path | Roles | Success | Common Errors | Notes |
 | --- | --- | --- | --- | --- | --- |
 | `GET` | `/projects` | viewer, analyst, project_manager, admin | 200 + `ProjectRead[]` | 401 unauthenticated, 403 insufficient role | Lists all projects visible to the caller. |
 | `POST` | `/projects` | project_manager, admin | 201 + `ProjectRead` | 401, 403, 409 name conflict, 422 validation | Creates a project and seeds default pricing settings. |
 | `GET` | `/projects/{project_id}` | viewer, analyst, project_manager, admin | 200 + `ProjectRead` | 401, 403, 404 missing project | Returns a single project by id. |
 | `PUT` | `/projects/{project_id}` | project_manager, admin | 200 + `ProjectRead` | 401, 403, 404, 422 | Updates mutable fields (name, location, operation_type, description). |
 | `DELETE` | `/projects/{project_id}` | project_manager, admin | 204 | 401, 403, 404 | Removes the project and cascading scenarios. |
 | `GET` | `/projects/{project_id}/scenarios` | viewer, analyst, project_manager, admin | 200 + `ScenarioRead[]` | 401, 403, 404 | Lists scenarios that belong to the project. |
 | `POST` | `/projects/{project_id}/scenarios` | project_manager, admin | 201 + `ScenarioRead` | 401, 403, 404 missing project, 409 duplicate name, 422 validation | Creates a scenario under the project. Currency defaults to pricing metadata when omitted. |
 | `POST` | `/projects/{project_id}/scenarios/compare` | viewer, analyst, project_manager, admin | 200 + `ScenarioComparisonResponse` | 401, 403, 404, 422 mismatch/validation | Validates a list of scenario ids before comparison. |
 | `GET` | `/scenarios/{scenario_id}` | viewer, analyst, project_manager, admin | 200 + `ScenarioRead` | 401, 403, 404 | Retrieves a single scenario. |
 | `PUT` | `/scenarios/{scenario_id}` | project_manager, admin | 200 + `ScenarioRead` | 401, 403, 404, 422 | Updates scenario metadata (status, dates, currency, resource). |
 | `DELETE` | `/scenarios/{scenario_id}` | project_manager, admin | 204 | 401, 403, 404 | Removes the scenario and dependent data. |
 ## OpenAPI Reference
 - Interactive docs: `GET /docs`
 - Raw schema (JSON): `GET /openapi.json`
--- a/architecture/08_concepts.md
+++ b/architecture/08_concepts.md
@@ -48,10 +48,6 @@ CalMiner aims to provide a comprehensive platform for mining project scenario an
    - [Data Synchronization](#data-synchronization)
    - [Error Handling](#error-handling)
  - [Reporting and Analytics](#reporting-and-analytics)
    - [Data Visualization](#data-visualization)
    - [Custom Reports](#custom-reports)
    - [Real-time Analytics](#real-time-analytics)
    - [Historical Analysis](#historical-analysis)
 ## Multitenancy
@@ -71,9 +67,27 @@ Each tenant can have customized settings and preferences, managed through the Ma
 ## Data Model
-The system employs a relational data model to manage and store information efficiently. The primary entities include Users, Projects, Datasets, and Results. Each entity is represented as a table in the database, with relationships defined through foreign keys.
+The system employs a relational data model to manage and store information efficiently. Core entities include Users, Projects, Scenarios, Financial Inputs, Simulation Parameters, Pricing Settings, and supporting metadata tables. Relationships are enforced using foreign keys with cascading deletes where appropriate (e.g., scenarios cascade from projects, financial inputs cascade from scenarios).
-A detailed [Data Model](08_concepts/02_data_model.md) documentation is available.
+Domain-specific enumerations are mapped to PostgreSQL enum types and mirrored in the ORM layer via `models.enums`:
 - `MiningOperationType` — categorises projects (open pit, underground, in-situ leach, etc.).
 - `ScenarioStatus` — tracks lifecycle state (draft, active, archived).
 - `FinancialCategory` and `CostBucket` — classify financial inputs for reporting granularity.
 - `DistributionType` and `StochasticVariable` — describe probabilistic simulation parameters.
 - `ResourceType` — annotates primary resource consumption for scenarios and stochastic parameters.
 These enums back the relevant model columns through named SQLAlchemy `Enum` definitions referencing the same PostgreSQL type names, ensuring parity between application logic and database constraints.
 Schema creation and baseline seed data are handled by the idempotent initializer (`scripts/init_db.py`). On every application startup the initializer guarantees that:
 1. Enum types exist exactly once.
 2. Tables for authentication, pricing, and mining domain entities are present.
 3. Default roles, administrator account, pricing settings, and representative demo projects/scenarios/financial inputs are available for immediate use.
 Developers can reset to a clean slate with `scripts/reset_db.py`, which safely drops managed tables and enum types in non-production environments before rerunning the initializer.
 A detailed [Data Model](08_concepts/02_data_model.md) documentation is available. Discounted cash-flow metrics (NPV, IRR, Payback) referenced by the economic portion of the data model are described in the [Financial Metrics Specification](../specifications/financial_metrics.md), while stochastic extensions leverage the Monte Carlo engine documented in the [Monte Carlo Simulation Specification](../specifications/monte_carlo_simulation.md).
 All data interactions are handled through the [Data Access Layer](05_building_block_view.md#data-access-layer), ensuring consistency and integrity across operations.
@@ -227,20 +241,4 @@ Robust error handling mechanisms are implemented to manage integration failures
 ## Reporting and Analytics
-The Calminer system includes comprehensive reporting and analytics capabilities to support data-driven decision-making.
+The Calminer system includes comprehensive reporting and analytics capabilities to support data-driven decision-making. Detailed [Reporting and Analytics](08_concepts/11_reporting_analytics.md) documentation is available.
 ### Data Visualization
 The system provides tools for visualizing data through charts, graphs, and dashboards, making it easier to identify trends and insights.
 ### Custom Reports
 Users can create custom reports based on specific criteria, allowing for tailored analysis of project performance and resource utilization.
 ### Real-time Analytics
 The system supports real-time data processing and analytics, enabling users to access up-to-date information and respond quickly to changing conditions.
 ### Historical Analysis
 The system maintains a history of key metrics and events, allowing for retrospective analysis and identification of long-term trends.
--- a/architecture/08_concepts/02_data_model.md
+++ b/architecture/08_concepts/02_data_model.md
@@ -1,529 +1,23 @@
-# Data Model
+# Data Model Overview
-The data model for the system is designed to capture the essential entities and their relationships involved in mining projects. Each entity is represented as a table in the relational database, with attributes defining their properties and foreign keys establishing relationships between them.
+Calminer’s data model spans several distinct layers: persisted ORM entities, Pydantic schemas used by the API, navigation metadata, shared enumerations, and operational telemetry tables. To make the material easier to scan, the original monolithic document has been split into focused reference pages.
-## Table of Contents
+## Reference Structure
- [Data Model](#data-model)
+- [SQLAlchemy Models](./02_data_model/01_sqlalchemy_models.md) — Domain entities that persist projects, scenarios, pricing configuration, snapshots, and supporting records.
-  - [Table of Contents](#table-of-contents)
+- [Navigation Metadata](./02_data_model/02_navigation.md) — Sidebar and menu configuration tables plus seeding/runtime notes.
-  - [Relationships Diagram](#relationships-diagram)
+- [Enumerations](./02_data_model/03_enumerations.md) — Shared enum definitions used across ORM models and schemas.
-  - [User](#user)
+- [Pydantic Schemas](./02_data_model/04_pydantic.md) — Request/response models, import/export payloads, and validation nuances.
-    - [User Diagram](#user-diagram)
+- [Monitoring and Auditing](./02_data_model/05_monitoring.md) — Telemetry and audit tables supporting observability.
    - [User Roles](#user-roles)
      - [System Administrator](#system-administrator)
      - [Mandator Administrator](#mandator-administrator)
      - [Project Manager](#project-manager)
      - [Standard User](#standard-user)
    - [Permissions](#permissions)
  - [Mandator](#mandator)
  - [Project](#project)
  - [Location](#location)
  - [Currency](#currency)
  - [Unit](#unit)
  - [Mining Technology](#mining-technology)
  - [Product Model](#product-model)
    - [Quality Metrics](#quality-metrics)
  - [Financial Model](#financial-model)
    - [Cost Model](#cost-model)
      - [CAPEX](#capex)
      - [OPEX](#opex)
    - [Revenue Model](#revenue-model)
      - [Revenue Streams](#revenue-streams)
      - [Product Sales](#product-sales)
    - [Investment Model](#investment-model)
    - [Economic Model](#economic-model)
    - [Risk Model](#risk-model)
  - [Parameter](#parameter)
  - [Scenario](#scenario)
-## Relationships Diagram
+Each detailed page retains the original headings and tables, so existing anchors and references can migrate with minimal disruption.
-```mermaid
+## How to Use This Overview
 erDiagram
    User ||--o{ UserRole : has
    UserRole ||--o{ RolePermission : has
    Permission ||--o{ RolePermission : assigned_to
-    User ||--o{ Mandator : belongs_to
+- Start with the SQLAlchemy reference when you need to understand persistence concerns or relationships between core domain objects.
-    Mandator ||--o{ Project : has_many
+- Jump to the Pydantic schemas document when adjusting API payloads or validation logic.
-    Project ||--|| Location : located_at
+- Consult the enumerations list before introducing new enum values to keep backend and frontend usage aligned.
-    Project ||--o{ MiningTechnology : uses
+- Review the navigation metadata page when seeding or modifying the application sidebar.
-    Project ||--o{ FinancialModel : has_many
+- Use the monitoring and auditing section to track telemetry fields that drive dashboards and compliance reporting.
    Project ||--o{ Parameter : has_many
    Project ||--o{ Scenario : has_many
-    ProductModel ||--o{ QualityMetric : has
+Cross-links between these documents mirror the previous inline references. Update any external links to point at the new files during your next documentation touchpoint.
    FinancialModel ||--o{ CostModel : includes
    FinancialModel ||--o{ RevenueModel : includes
    FinancialModel ||--o{ InvestmentModel : includes
    FinancialModel ||--o{ EconomicModel : includes
    FinancialModel ||--o{ RiskModel : includes
    MiningTechnology ||--o{ Parameter : has_many
    MiningTechnology ||--o{ QualityMetric : has_many
    CostModel ||--|| CAPEX : has
    CostModel ||--|| OPEX : has
    RevenueModel ||--o{ RevenueStream : has
    RevenueModel ||--o{ ProductSale : has
    Currency ||--o{ CAPEX : used_in
    Currency ||--o{ OPEX : used_in
    Currency ||--o{ RevenueStream : used_in
    Currency ||--o{ ProductSale : used_in
    Unit ||--o{ ProductModel : used_in
    Unit ||--o{ QualityMetric : used_in
    Unit ||--o{ Parameter : used_in
 ```
 ## User
 Users are individuals who have access to the Calminer system.
 Each User is associated with a single Mandator and can be assigned to multiple Projects under that Mandator. Users have specific roles and permissions that determine their level of access within the system.
 ### User Diagram
 ```mermaid
 erDiagram
    User {
        string UserID PK
        string Name
        string Email
        string PasswordHash
        datetime CreatedAt
        datetime UpdatedAt
        datetime LastLogin
        boolean IsActive
    }
 ```
 ### User Roles
 Users can have different roles within the Calminer system, which define their permissions and access levels.
 ```mermaid
 erDiagram
    User ||--o{ UserRole : has
    UserRole {
        string RoleID PK
        string RoleName
        string Description
    }
 ```
 #### System Administrator
 The System Administrator role is a special user role that has elevated privileges across the entire Calminer system. This role is responsible for managing system settings, configurations, managing Mandators, user accounts, and overall system maintenance.
 #### Mandator Administrator
 The Mandator Administrator role is a special user role assigned to individuals who manage a specific Mandator. This role has the authority to oversee Users and Projects within their Mandator, including user assignments and project configurations.
 #### Project Manager
 The Project Manager role is responsible for overseeing specific Projects within a Mandator. This role has permissions to manage project settings, assign Users to the Project, and monitor project progress.
 #### Standard User
 The Standard User role can participate in Projects assigned to them but has limited access to administrative functions.
 ### Permissions
 Permissions are defined based on user roles, determining what actions a user can perform within the system. Permissions include:
 - View Projects
 - Edit Projects
 - Manage Users
 - Configure System Settings
 - Access Reports
 - Run Simulations
 - Manage Financial Models
 - Export Data
 - Import Data
 - View Logs
 - Manage Notifications
 - Access API
 Permissions are assigned to roles, and users inherit permissions based on their assigned roles. For this purpose, a helper entity `RolePermission` is defined to map roles to their respective permissions.
 The Permissions entity (and RolePermission entity) is defined as follows:
 ```mermaid
 erDiagram
    RolePermission {
        string RoleID FK
        string PermissionID FK
    }
    Permission {
        string PermissionID PK
        string PermissionName
        string Description
    }
    UserRole ||--o{ RolePermission : has
    Permission ||--o{ RolePermission : assigned_to
 ```
 ## Mandator
 The Mandator entity represents organizational units or clients using the system. A Mandator can have multiple Users and Projects associated with it.
 ```mermaid
 erDiagram
    Mandator {
        string MandatorID PK
        string Name
        string ContactInfo
        datetime CreatedAt
        datetime UpdatedAt
        boolean IsActive
    }
 ```
 ## Project
 The Project entity encapsulates the details of mining projects. Attributes include ProjectID, ProjectName, Description, LocationID (linking to the Location entity), CreationDate, and OwnerID (linking to the User entity). Projects can have multiple Datasets associated with them.
 ```mermaid
 erDiagram
    Project {
        string ProjectID PK
        string ProjectName
        string Description
        string LocationID FK
        datetime CreationDate
        string OwnerID FK
        string Status
        datetime StartDate
    }
 ```
 ## Location
 The Location entity represents the geographical context of the data being mined. Locations can be linked to multiple Projects.
 ```mermaid
 erDiagram
    Location {
        string LocationID PK
        string Name
        string Description
        string Coordinates
        string Region
        string Country
    }
 ```
 ## Currency
 The Currency entity defines the monetary units used in the system. Currency is used when dealing with any monetary values in Projects or Results.
 ```mermaid
 erDiagram
    Currency {
        string CurrencyID PK
        string Code
        string Name
        string Symbol
        datetime CreatedAt
        datetime UpdatedAt
        boolean IsActive
    }
 ```
 ## Unit
 The Unit entity defines measurement units used in the system. Units are essential for standardizing data representation across mining projects, financial models, and results.
 ```mermaid
 erDiagram
    Unit {
        string UnitID PK
        string Name
        string Symbol
        string Description
    }
 ```
 ## Mining Technology
 The Mining Technology entity encompasses the various tools and techniques used in data mining projects. A Mining Project typically utilizes one specific Mining Technology.
 ```mermaid
 erDiagram
    MiningTechnology {
        string TechnologyID PK
        string Name
        string Description
    }
 ```
 ## Product Model
 The Product Model entity captures the details of products extracted or processed in mining projects. It includes attributes such as ProductID, Name, Type and UnitID (linking to the Unit entity). A Product Model can have multiple Quality Metrics associated with it.
 ```mermaid
 erDiagram
    ProductModel {
        string ProductID PK
        string Name
        string Type
        string UnitID FK
    }
    ProductModel ||--o{ QualityMetric : has
    Unit ||--o{ ProductModel : used_in
    Unit ||--o{ QualityMetric : used_in
 ```
 ### Quality Metrics
 Quality Metrics define the standards and specifications for products derived from mining projects. These metrics ensure that products meet industry and regulatory requirements.
 For example, Quality Metrics for a mineral product may include:
 - Purity Level
 - Moisture Content
 - Impurity Levels
 - Grain Size Distribution
 - Chemical Composition
 - Physical Properties
 These metrics are essential for assessing the value and usability of the products in various applications and markets.
 ```mermaid
 erDiagram
    QualityMetric {
        string MetricID PK
        string Name
        string Description
        float MinValue
        float MaxValue
        string UnitID FK
        string AssociatedProductID FK
    }
    ProductModel ||--o{ QualityMetric : has
    Unit ||--o{ QualityMetric : used_in
 ```
 ## Financial Model
 The Financial Model entity captures the economic aspects of mining projects. The Financial Model includes various sub-models and components that detail costs, revenues, investments, and risks associated with mining projects. Financial Models are the basis for Scenario analyses within Projects.
 ```mermaid
 erDiagram
    FinancialModel {
        string ModelID PK
        string Name
        string Description
        string AssociatedProjectID FK
    }
 ```
 ### Cost Model
 The Cost Model details the expenses associated with mining projects, including capital expenditures (CAPEX) and operational expenditures (OPEX).
 - CAPEX: Initial costs for equipment, infrastructure, and setup.
 - OPEX: Ongoing costs for labor, maintenance, and operations.
 ```mermaid
 erDiagram
    CostModel {
        string ModelID PK
        string Name
        string Description
        string AssociatedProjectID FK
    }
    CAPEX {
        string CAPEXID PK
        float EquipmentCost
        float InfrastructureCost
        float SetupCost
        float ContingencyCost
        float TotalCAPEX
        string CurrencyID FK
    }
    OPEX {
        string OPEXID PK
        float LaborCost
        float MaterialsCost
        float EquipmentRentalCost
        float UtilitiesCost
        float MaintenanceCost
        float TotalOPEX
        string CurrencyID FK
    }
    FinancialModel ||--o{ CostModel : includes
    CostModel ||--|| CAPEX : has
    CostModel ||--|| OPEX : has
    Currency ||--o{ CAPEX : used_in
    Currency ||--o{ OPEX : used_in
 ```
 #### CAPEX
 For a calculation of Capital Expenditures (CAPEX), the following attributes are included:
 - EquipmentCost
 - InfrastructureCost
 - SetupCost
 - ContingencyCost
 - TotalCAPEX
 - CurrencyID (Foreign Key to Currency)
 #### OPEX
 For a calculation of Operational Expenditures (OPEX), the following attributes are included:
 - LaborCost
 - MaterialsCost
 - EquipmentRentalCost
 - UtilitiesCost
 - MaintenanceCost
 - TotalOPEX
 - CurrencyID (Foreign Key to Currency)
 ### Revenue Model
 The Revenue Model outlines the income generated from mining projects. It includes various revenue streams and product sales.
 ```mermaid
 erDiagram
    RevenueModel {
        string ModelID PK
        string Name
        string Description
        string AssociatedProjectID FK
    }
    FinancialModel ||--o{ RevenueModel : includes
    RevenueStream {
        string RevenueStreamID PK
        string Name
        string Description
        float Amount
        string CurrencyID FK
        string AssociatedRevenueModelID FK
        string Frequency
        datetime StartDate
        datetime EndDate
        boolean IsRecurring
    }
    FinancialModel ||--o{ RevenueModel : includes
    RevenueModel ||--o{ RevenueStream : has
    Currency ||--o{ RevenueStream : used_in
 ```
 #### Revenue Streams
 Revenue streams can include product sales, service fees, and other income sources related to the mining project.
 ```mermaid
 erDiagram
    RevenueStream {
        string RevenueStreamID PK
        string Name
        string Description
        float Amount
        string CurrencyID FK
        string AssociatedRevenueModelID FK
        string Frequency
        datetime StartDate
        datetime EndDate
        boolean IsRecurring
    }
 ```
 #### Product Sales
 Mining product sales represent the primary revenue stream for most mining projects. Product sales revenue is calculated based on the quantity of product sold, price per unit, and any applicable adjustments for quality or other factors like penalties for impurities. Also see [Price Calculation](../../specifications/price_calculation.md) for more details.
 ```mermaid
 erDiagram
    ProductSale {
        string SaleID PK
        string ProductID FK
        float QuantitySold
        float PricePerUnit
        float TotalRevenue
        string CurrencyID FK
        datetime SaleDate
    }
    ProductModel ||--o{ ProductSale : has
    Currency ||--o{ ProductSale : used_in
 ```
 ### Investment Model
 The Investment Model focuses on the funding and financial backing of mining projects. It includes details about investors, investment amounts, and dates.
 ```mermaid
 erDiagram
    InvestmentModel {
        string InvestmentID PK
        string InvestorID FK
        string ProjectID FK
        float Amount
        datetime InvestmentDate
    }
    FinancialModel ||--o{ InvestmentModel : includes
 ```
 ### Economic Model
 The Economic Model assesses the overall economic viability and impact of mining projects. Economic indicators such as Net Present Value (NPV), Internal Rate of Return (IRR), and Payback Period are calculated within this model.
 ```mermaid
 erDiagram
    EconomicModel {
        string ModelID PK
        string Name
        string Description
        string AssociatedProjectID FK
    }
    FinancialModel ||--o{ EconomicModel : includes
 ```
 ### Risk Model
 The Risk Model identifies and evaluates potential risks associated with mining projects. It includes risk factors, their probabilities, and potential impacts on project outcomes.
 ```mermaid
 erDiagram
    RiskModel {
        string ModelID PK
        string Name
        string Description
        string AssociatedProjectID FK
    }
    FinancialModel ||--o{ RiskModel : includes
 ```
 ## Parameter
 Parameters are configurable inputs used within Financial Models to simulate different economic conditions and scenarios for mining projects. Each Parameter has a name, type, default value, and is linked to a specific Technology.
 ```mermaid
 erDiagram
    Parameter {
        string ParameterID PK
        string Name
        string Type
        string DefaultValue
        string Description
        string AssociatedTechnologyID FK
    }
 ```
 ## Scenario
 A Scenario represents a specific configuration or analysis case within a Project. Scenarios utilize various Financial Models and Parameters to simulate different outcomes for mining projects.
 ```mermaid
 erDiagram
    Scenario {
        string ScenarioID PK
        string Name
        string Description
        string AssociatedProjectID FK
    }
 ```
--- a/architecture/08_concepts/02_data_model/01_sqlalchemy_models.md
+++ b/architecture/08_concepts/02_data_model/01_sqlalchemy_models.md
@@ -0,0 +1,369 @@
 # SQLAlchemy Models
 This reference describes the primary SQLAlchemy ORM models that underpin Calminer. It mirrors the original hierarchy from `02_data_model.md`, extracted so each domain area can evolve independently. See the [data model overview](../02_data_model.md) for context and navigation.
 ## User Management
 ### User
 Represents authenticated platform users with optional elevated privileges.
 **Table:** `users`
 | Attribute     | Type         | Description               |
 | ------------- | ------------ | ------------------------- |
 | id            | Integer (PK) | Primary key               |
 | email         | String(255)  | Unique email address      |
 | username      | String(128)  | Unique username           |
 | password_hash | String(255)  | Argon2 hashed password    |
 | is_active     | Boolean      | Account activation status |
 | is_superuser  | Boolean      | Superuser privileges      |
 | last_login_at | DateTime     | Last login timestamp      |
 | created_at    | DateTime     | Creation timestamp        |
 | updated_at    | DateTime     | Last update timestamp     |
 **Relationships:**
 - `role_assignments`: Many-to-many with Role via UserRole
 ### Role
 Defines user roles for role-based access control (RBAC).
 **Table:** `roles`
 | Attribute    | Type         | Description           |
 | ------------ | ------------ | --------------------- |
 | id           | Integer (PK) | Primary key           |
 | name         | String(64)   | Unique role name      |
 | display_name | String(128)  | Human-readable name   |
 | description  | Text         | Role description      |
 | created_at   | DateTime     | Creation timestamp    |
 | updated_at   | DateTime     | Last update timestamp |
 **Relationships:**
 - `assignments`: One-to-many with UserRole
 - `users`: Many-to-many with User (viewonly)
 ### UserRole
 Association between users and roles with assignment metadata.
 **Table:** `user_roles`
 | Attribute  | Type                    | Description          |
 | ---------- | ----------------------- | -------------------- |
 | user_id    | Integer (FK → users.id) | User foreign key     |
 | role_id    | Integer (FK → roles.id) | Role foreign key     |
 | granted_at | DateTime                | Assignment timestamp |
 | granted_by | Integer (FK → users.id) | Granting user        |
 **Relationships:**
 - `user`: Many-to-one with User
 - `role`: Many-to-one with Role
 - `granted_by_user`: Many-to-one with User
 ## Project Management
 ### Project
 Top-level mining project grouping multiple scenarios.
 **Table:** `projects`
 | Attribute           | Type                               | Description           |
 | ------------------- | ---------------------------------- | --------------------- |
 | id                  | Integer (PK)                       | Primary key           |
 | name                | String(255)                        | Unique project name   |
 | location            | String(255)                        | Project location      |
 | operation_type      | MiningOperationType                | Mining operation type |
 | description         | Text                               | Project description   |
 | pricing_settings_id | Integer (FK → pricing_settings.id) | Pricing settings      |
 | created_at          | DateTime                           | Creation timestamp    |
 | updated_at          | DateTime                           | Last update timestamp |
 **Relationships:**
 - `scenarios`: One-to-many with Scenario
 - `pricing_settings`: Many-to-one with PricingSettings
 ### Scenario
 A specific configuration of assumptions for a project.
 **Table:** `scenarios`
 | Attribute        | Type                       | Description               |
 | ---------------- | -------------------------- | ------------------------- |
 | id               | Integer (PK)               | Primary key               |
 | project_id       | Integer (FK → projects.id) | Project foreign key       |
 | name             | String(255)                | Scenario name             |
 | description      | Text                       | Scenario description      |
 | status           | ScenarioStatus             | Scenario lifecycle status |
 | start_date       | Date                       | Scenario start date       |
 | end_date         | Date                       | Scenario end date         |
 | discount_rate    | Numeric(5,2)               | Discount rate percentage  |
 | currency         | String(3)                  | ISO currency code         |
 | primary_resource | ResourceType               | Primary resource type     |
 | created_at       | DateTime                   | Creation timestamp        |
 | updated_at       | DateTime                   | Last update timestamp     |
 **Relationships:**
 - `project`: Many-to-one with Project
 - `financial_inputs`: One-to-many with FinancialInput
 - `simulation_parameters`: One-to-many with SimulationParameter
 ### Projects → Scenarios → Profitability Calculations
 Calminer organises feasibility data in a nested hierarchy. A project defines the overarching mining context and exposes a one-to-many `scenarios` collection. Each scenario captures a self-contained assumption set and anchors derived artefacts such as financial inputs, simulation parameters, and profitability snapshots. Profitability calculations execute at the scenario scope; when triggered, the workflow in `services/calculations.py` persists a `ScenarioProfitability` record and can optionally roll results up to project level by creating a `ProjectProfitability` snapshot. Consumers typically surface the most recent metrics via the `latest_profitability` helpers on both ORM models.
 | Layer                      | ORM models                                                                                                  | Pydantic schema(s)                                                                                            | Key relationships                                                                                                 |
 | -------------------------- | ----------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- |
 | Project                    | `models.project.Project`                                                                                    | `schemas.project.ProjectRead`                                                                                 | `Project.scenarios`, `Project.profitability_snapshots`, `Project.latest_profitability`                            |
 | Scenario                   | `models.scenario.Scenario`                                                                                  | `schemas.scenario.ScenarioRead`                                                                               | `Scenario.project`, `Scenario.profitability_snapshots`, `Scenario.latest_profitability`                           |
 | Profitability calculations | `models.profitability_snapshot.ProjectProfitability`, `models.profitability_snapshot.ScenarioProfitability` | `schemas.calculations.ProfitabilityCalculationRequest`, `schemas.calculations.ProfitabilityCalculationResult` | Persisted via `services.calculations.calculate_profitability`; aggregates scenario metrics into project snapshots |
 Detailed CRUD endpoint behaviour for projects and scenarios is documented in `calminer-docs/api/README.md`.
 ### FinancialInput
 Line-item financial assumption attached to a scenario.
 **Table:** `financial_inputs`
 | Attribute      | Type                        | Description                |
 | -------------- | --------------------------- | -------------------------- |
 | id             | Integer (PK)                | Primary key                |
 | scenario_id    | Integer (FK → scenarios.id) | Scenario foreign key       |
 | name           | String(255)                 | Input name                 |
 | category       | FinancialCategory           | Financial category         |
 | cost_bucket    | CostBucket                  | Cost bucket classification |
 | amount         | Numeric(18,2)               | Monetary amount            |
 | currency       | String(3)                   | ISO currency code          |
 | effective_date | Date                        | Effective date             |
 | notes          | Text                        | Additional notes           |
 | created_at     | DateTime                    | Creation timestamp         |
 | updated_at     | DateTime                    | Last update timestamp      |
 **Relationships:**
 - `scenario`: Many-to-one with Scenario
 ## Project and Scenario Snapshots
 ### ProjectCapexSnapshot
 Project-level snapshot capturing aggregated initial capital expenditure metrics.
 **Table:** `project_capex_snapshots`
 | Attribute              | Type                              | Description                                             |
 | ---------------------- | --------------------------------- | ------------------------------------------------------- |
 | id                     | Integer (PK)                      | Primary key                                             |
 | project_id             | Integer (FK → projects.id)        | Associated project                                      |
 | created_by_id          | Integer (FK → users.id, nullable) | User that triggered persistence                         |
 | calculation_source     | String(64), nullable              | Originating workflow identifier (UI, API, etc.)         |
 | calculated_at          | DateTime                          | Timestamp the calculation completed                     |
 | currency_code          | String(3), nullable               | Currency for totals                                     |
 | total_capex            | Numeric(18,2), nullable           | Aggregated capex before contingency                     |
 | contingency_pct        | Numeric(12,6), nullable           | Applied contingency percentage                          |
 | contingency_amount     | Numeric(18,2), nullable           | Monetary contingency amount                             |
 | total_with_contingency | Numeric(18,2), nullable           | Capex total after contingency                           |
 | component_count        | Integer, nullable                 | Number of normalized components captured                |
 | payload                | JSON, nullable                    | Serialized component breakdown and calculation metadata |
 | created_at             | DateTime                          | Record creation timestamp                               |
 | updated_at             | DateTime                          | Last update timestamp                                   |
 **Relationships:**
 - `project`: Many-to-one with Project
 - `created_by`: Many-to-one with User (nullable)
 ### ScenarioCapexSnapshot
 Scenario-level snapshot storing detailed capex results.
 **Table:** `scenario_capex_snapshots`
 | Attribute              | Type                              | Description                                             |
 | ---------------------- | --------------------------------- | ------------------------------------------------------- |
 | id                     | Integer (PK)                      | Primary key                                             |
 | scenario_id            | Integer (FK → scenarios.id)       | Associated scenario                                     |
 | created_by_id          | Integer (FK → users.id, nullable) | User that triggered persistence                         |
 | calculation_source     | String(64), nullable              | Originating workflow identifier                         |
 | calculated_at          | DateTime                          | Timestamp the calculation completed                     |
 | currency_code          | String(3), nullable               | Currency for totals                                     |
 | total_capex            | Numeric(18,2), nullable           | Aggregated capex before contingency                     |
 | contingency_pct        | Numeric(12,6), nullable           | Applied contingency percentage                          |
 | contingency_amount     | Numeric(18,2), nullable           | Monetary contingency amount                             |
 | total_with_contingency | Numeric(18,2), nullable           | Capex total after contingency                           |
 | component_count        | Integer, nullable                 | Number of normalized components captured                |
 | payload                | JSON, nullable                    | Serialized component breakdown and calculation metadata |
 | created_at             | DateTime                          | Record creation timestamp                               |
 | updated_at             | DateTime                          | Last update timestamp                                   |
 **Relationships:**
 - `scenario`: Many-to-one with Scenario
 - `created_by`: Many-to-one with User (nullable)
 ### ProjectOpexSnapshot
 Project-level snapshot persisting recurring opex metrics.
 **Table:** `project_opex_snapshots`
 | Attribute                | Type                              | Description                                             |
 | ------------------------ | --------------------------------- | ------------------------------------------------------- |
 | id                       | Integer (PK)                      | Primary key                                             |
 | project_id               | Integer (FK → projects.id)        | Associated project                                      |
 | created_by_id            | Integer (FK → users.id, nullable) | User that triggered persistence                         |
 | calculation_source       | String(64), nullable              | Originating workflow identifier                         |
 | calculated_at            | DateTime                          | Timestamp the calculation completed                     |
 | currency_code            | String(3), nullable               | Currency for totals                                     |
 | overall_annual           | Numeric(18,2), nullable           | Total annual opex                                       |
 | escalated_total          | Numeric(18,2), nullable           | Escalated cost across the evaluation horizon            |
 | annual_average           | Numeric(18,2), nullable           | Average annual cost over the horizon                    |
 | evaluation_horizon_years | Integer, nullable                 | Number of years included in the timeline                |
 | escalation_pct           | Numeric(12,6), nullable           | Escalation percentage applied                           |
 | apply_escalation         | Boolean                           | Flag indicating whether escalation was applied          |
 | component_count          | Integer, nullable                 | Number of normalized components captured                |
 | payload                  | JSON, nullable                    | Serialized component breakdown and calculation metadata |
 | created_at               | DateTime                          | Record creation timestamp                               |
 | updated_at               | DateTime                          | Last update timestamp                                   |
 **Relationships:**
 - `project`: Many-to-one with Project
 - `created_by`: Many-to-one with User (nullable)
 ### ScenarioOpexSnapshot
 Scenario-level snapshot persisting recurring opex metrics.
 **Table:** `scenario_opex_snapshots`
 | Attribute                | Type                              | Description                                             |
 | ------------------------ | --------------------------------- | ------------------------------------------------------- |
 | id                       | Integer (PK)                      | Primary key                                             |
 | scenario_id              | Integer (FK → scenarios.id)       | Associated scenario                                     |
 | created_by_id            | Integer (FK → users.id, nullable) | User that triggered persistence                         |
 | calculation_source       | String(64), nullable              | Originating workflow identifier                         |
 | calculated_at            | DateTime                          | Timestamp the calculation completed                     |
 | currency_code            | String(3), nullable               | Currency for totals                                     |
 | overall_annual           | Numeric(18,2), nullable           | Total annual opex                                       |
 | escalated_total          | Numeric(18,2), nullable           | Escalated cost across the evaluation horizon            |
 | annual_average           | Numeric(18,2), nullable           | Average annual cost over the horizon                    |
 | evaluation_horizon_years | Integer, nullable                 | Number of years included in the timeline                |
 | escalation_pct           | Numeric(12,6), nullable           | Escalation percentage applied                           |
 | apply_escalation         | Boolean                           | Flag indicating whether escalation was applied          |
 | component_count          | Integer, nullable                 | Number of normalized components captured                |
 | payload                  | JSON, nullable                    | Serialized component breakdown and calculation metadata |
 | created_at               | DateTime                          | Record creation timestamp                               |
 | updated_at               | DateTime                          | Last update timestamp                                   |
 **Relationships:**
 - `scenario`: Many-to-one with Scenario
 - `created_by`: Many-to-one with User (nullable)
 ### SimulationParameter
 Probability distribution settings for scenario simulations.
 **Table:** `simulation_parameters`
 | Attribute          | Type                        | Description              |
 | ------------------ | --------------------------- | ------------------------ |
 | id                 | Integer (PK)                | Primary key              |
 | scenario_id        | Integer (FK → scenarios.id) | Scenario foreign key     |
 | name               | String(255)                 | Parameter name           |
 | distribution       | DistributionType            | Distribution type        |
 | variable           | StochasticVariable          | Stochastic variable type |
 | resource_type      | ResourceType                | Resource type            |
 | mean_value         | Numeric(18,4)               | Mean value               |
 | standard_deviation | Numeric(18,4)               | Standard deviation       |
 | minimum_value      | Numeric(18,4)               | Minimum value            |
 | maximum_value      | Numeric(18,4)               | Maximum value            |
 | unit               | String(32)                  | Unit of measurement      |
 | configuration      | JSON                        | Additional configuration |
 | created_at         | DateTime                    | Creation timestamp       |
 | updated_at         | DateTime                    | Last update timestamp    |
 **Relationships:**
 - `scenario`: Many-to-one with Scenario
 ## Pricing Configuration
 ### PricingSettings
 Persisted pricing defaults applied to scenario evaluations.
 **Table:** `pricing_settings`
 | Attribute                | Type          | Description                   |
 | ------------------------ | ------------- | ----------------------------- |
 | id                       | Integer (PK)  | Primary key                   |
 | name                     | String(128)   | Unique settings name          |
 | slug                     | String(64)    | Unique slug identifier        |
 | description              | Text          | Settings description          |
 | default_currency         | String(3)     | Default ISO currency code     |
 | default_payable_pct      | Numeric(5,2)  | Default payable percentage    |
 | moisture_threshold_pct   | Numeric(5,2)  | Moisture threshold percentage |
 | moisture_penalty_per_pct | Numeric(14,4) | Moisture penalty per percent  |
 | metadata_payload         | JSON          | Additional metadata           |
 | created_at               | DateTime      | Creation timestamp            |
 | updated_at               | DateTime      | Last update timestamp         |
 **Relationships:**
 - `metal_overrides`: One-to-many with PricingMetalSettings
 - `impurity_overrides`: One-to-many with PricingImpuritySettings
 - `projects`: One-to-many with Project
 ### PricingMetalSettings
 Contract-specific overrides for a particular metal.
 **Table:** `pricing_metal_settings`
 | Attribute                | Type                               | Description                  |
 | ------------------------ | ---------------------------------- | ---------------------------- |
 | id                       | Integer (PK)                       | Primary key                  |
 | pricing_settings_id      | Integer (FK → pricing_settings.id) | Pricing settings foreign key |
 | metal_code               | String(32)                         | Metal code                   |
 | payable_pct              | Numeric(5,2)                       | Payable percentage           |
 | moisture_threshold_pct   | Numeric(5,2)                       | Moisture threshold           |
 | moisture_penalty_per_pct | Numeric(14,4)                      | Moisture penalty             |
 | data                     | JSON                               | Additional data              |
 | created_at               | DateTime                           | Creation timestamp           |
 | updated_at               | DateTime                           | Last update timestamp        |
 **Relationships:**
 - `pricing_settings`: Many-to-one with PricingSettings
 ### PricingImpuritySettings
 Impurity penalty thresholds associated with pricing settings.
 **Table:** `pricing_impurity_settings`
 | Attribute           | Type                               | Description                  |
 | ------------------- | ---------------------------------- | ---------------------------- |
 | id                  | Integer (PK)                       | Primary key                  |
 | pricing_settings_id | Integer (FK → pricing_settings.id) | Pricing settings foreign key |
 | impurity_code       | String(32)                         | Impurity code                |
 | threshold_ppm       | Numeric(14,4)                      | Threshold in ppm             |
 | penalty_per_ppm     | Numeric(14,4)                      | Penalty per ppm              |
 | notes               | Text                               | Additional notes             |
 | created_at          | DateTime                           | Creation timestamp           |
 | updated_at          | DateTime                           | Last update timestamp        |
 **Relationships:**
 - `pricing_settings`: Many-to-one with PricingSettings
--- a/architecture/08_concepts/02_data_model/02_navigation.md
+++ b/architecture/08_concepts/02_data_model/02_navigation.md
@@ -0,0 +1,65 @@
 # Navigation Metadata
 This page details the navigation metadata tables that drive the Calminer sidebar and top-level menus. It was split from `02_data_model.md` to isolate runtime navigation behaviour from the broader ORM catalogue.
 ## NavigationGroup
 Represents a logical grouping of navigation links shown in the UI sidebar or header.
 **Table:** `navigation_groups`
 | Attribute    | Type         | Description                                     |
 | ------------ | ------------ | ----------------------------------------------- |
 | id           | Integer (PK) | Primary key                                     |
 | slug         | String(64)   | Unique slug identifier                          |
 | title        | String(128)  | Display title                                   |
 | description  | Text         | Optional descriptive text                       |
 | match_prefix | String(255)  | URL prefix used to auto-expand in UI            |
 | weighting    | Integer      | Ordering weight                                 |
 | icon_name    | String(64)   | Optional icon identifier                        |
 | feature_flag | String(64)   | Optional feature flag for conditional rendering |
 | created_at   | DateTime     | Creation timestamp                              |
 | updated_at   | DateTime     | Last update timestamp                           |
 **Relationships:**
 - `links`: One-to-many with NavigationLink
 ## NavigationLink
 Individual navigation entry that may have an associated feature flag or role restriction.
 **Table:** `navigation_links`
 | Attribute     | Type                                | Description                                          |
 | ------------- | ----------------------------------- | ---------------------------------------------------- |
 | id            | Integer (PK)                        | Primary key                                          |
 | group_id      | Integer (FK → navigation_groups.id) | Parent navigation group                              |
 | parent_id     | Integer (FK → navigation_links.id)  | Optional parent link (for nested menus)              |
 | slug          | String(64)                          | Unique slug identifier                               |
 | title         | String(128)                         | Display title                                        |
 | description   | Text                                | Optional descriptive text                            |
 | href          | String(255)                         | Static fallback URL                                  |
 | match_prefix  | String(255)                         | URL prefix for expansion                             |
 | feature_flag  | String(64)                          | Optional feature flag for conditional rendering      |
 | required_role | String(64)                          | Optional role required to view the link              |
 | weighting     | Integer                             | Ordering weight within the parent scope              |
 | icon_name     | String(64)                          | Optional icon identifier                             |
 | metadata      | JSON                                | Additional configuration (e.g. legacy route aliases) |
 | created_at    | DateTime                            | Creation timestamp                                   |
 | updated_at    | DateTime                            | Last update timestamp                                |
 **Relationships:**
 - `group`: Many-to-one with NavigationGroup
 - `parent`: Many-to-one self-reference
 - `children`: One-to-many self-reference
 ## Seeding and Runtime Consumption
 - **Seed script:** `scripts/init_db.py` provisions baseline groups and links, including nested scenario calculators and role-gated admin entries.
 - **Service layer:** `services/navigation.py` maps ORM entities into response DTOs, resolves contextual URLs for projects and scenarios, and applies feature flag as well as role filters.
 - **API exposure:** `routes/navigation.py` serves the `/navigation/sidebar` endpoint, combining seeded data with runtime context to produce navigation trees consumed by the frontend.
 - **Testing:** `tests/services/test_navigation_service.py` covers URL resolution and child filtering logic, while `tests/integration/test_navigation_sidebar_calculations.py` verifies scenario calculator entries in the API payload.
 Refer to the [navigation service documentation](../../../../api/README.md#navigation) for endpoint-level behaviour.
--- a/architecture/08_concepts/02_data_model/03_enumerations.md
+++ b/architecture/08_concepts/02_data_model/03_enumerations.md
@@ -0,0 +1,83 @@
 # Enumerations
 This page catalogs the enumerations shared by Calminer ORM models and Pydantic schemas. The entries mirror the original content from `02_data_model.md` so enum updates can be maintained independently of the broader data model reference.
 ## MiningOperationType
 Supported mining operation categories.
 - `OPEN_PIT`: Open pit mining
 - `UNDERGROUND`: Underground mining
 - `IN_SITU_LEACH`: In-situ leaching
 - `PLACER`: Placer mining
 - `QUARRY`: Quarry operations
 - `MOUNTAINTOP_REMOVAL`: Mountaintop removal
 - `OTHER`: Other operations
 ## ScenarioStatus
 Lifecycle states for project scenarios.
 - `DRAFT`: Draft status
 - `ACTIVE`: Active status
 - `ARCHIVED`: Archived status
 ## FinancialCategory
 Enumeration of cost and revenue classifications.
 - `CAPITAL_EXPENDITURE`: Capital expenditures
 - `OPERATING_EXPENDITURE`: Operating expenditures
 - `REVENUE`: Revenue
 - `CONTINGENCY`: Contingency
 - `OTHER`: Other
 ## DistributionType
 Supported stochastic distribution families for simulations.
 - `NORMAL`: Normal distribution
 - `TRIANGULAR`: Triangular distribution
 - `UNIFORM`: Uniform distribution
 - `LOGNORMAL`: Lognormal distribution
 - `CUSTOM`: Custom distribution
 ## ResourceType
 Primary consumables and resources used in mining operations.
 - `DIESEL`: Diesel fuel
 - `ELECTRICITY`: Electrical power
 - `WATER`: Process water
 - `EXPLOSIVES`: Blasting agents
 - `REAGENTS`: Processing reagents
 - `LABOR`: Direct labor
 - `EQUIPMENT_HOURS`: Equipment operating hours
 - `TAILINGS_CAPACITY`: Tailings storage
 ## CostBucket
 Granular cost buckets aligned with project accounting.
 - `CAPITAL_INITIAL`: Initial capital
 - `CAPITAL_SUSTAINING`: Sustaining capital
 - `OPERATING_FIXED`: Fixed operating costs
 - `OPERATING_VARIABLE`: Variable operating costs
 - `MAINTENANCE`: Maintenance costs
 - `RECLAMATION`: Reclamation costs
 - `ROYALTIES`: Royalties
 - `GENERAL_ADMIN`: General and administrative
 ## StochasticVariable
 Domain variables that typically require probabilistic modelling.
 - `ORE_GRADE`: Ore grade variability
 - `RECOVERY_RATE`: Metallurgical recovery
 - `METAL_PRICE`: Commodity price
 - `OPERATING_COST`: Operating cost per tonne
 - `CAPITAL_COST`: Capital cost
 - `DISCOUNT_RATE`: Discount rate
 - `THROUGHPUT`: Plant throughput
 Refer back to the [data model overview](../02_data_model.md) when choosing the appropriate enum for new features.
--- a/architecture/08_concepts/02_data_model/04_pydantic.md
+++ b/architecture/08_concepts/02_data_model/04_pydantic.md
@@ -0,0 +1,267 @@
 # Pydantic Schemas
 This page documents the Pydantic models used for request/response validation across the Calminer API surface. It was extracted from `02_data_model.md` so API-facing contracts can evolve separately from the ORM layer.
 ## Authentication Schemas (`schemas/auth.py`)
 ### RegistrationForm
 Form model for user registration.
 | Field            | Type | Validation                      |
 | ---------------- | ---- | ------------------------------- |
 | username         | str  | 3-128 characters                |
 | email            | str  | Valid email format, 5-255 chars |
 | password         | str  | 8-256 characters                |
 | confirm_password | str  | Must match password             |
 ### LoginForm
 Form model for user login.
 | Field    | Type | Validation       |
 | -------- | ---- | ---------------- |
 | username | str  | 1-255 characters |
 | password | str  | 1-256 characters |
 ### PasswordResetRequestForm
 Form model for password reset request.
 | Field | Type | Validation                      |
 | ----- | ---- | ------------------------------- |
 | email | str  | Valid email format, 5-255 chars |
 ### PasswordResetForm
 Form model for password reset.
 | Field            | Type | Validation          |
 | ---------------- | ---- | ------------------- |
 | token            | str  | Required            |
 | password         | str  | 8-256 characters    |
 | confirm_password | str  | Must match password |
 ## Project Schemas (`schemas/project.py`)
 ### ProjectBase
 Base schema for project data.
 | Field          | Type                | Description           |
 | -------------- | ------------------- | --------------------- |
 | name           | str                 | Project name          |
 | location       | str \| None         | Project location      |
 | operation_type | MiningOperationType | Mining operation type |
 | description    | str \| None         | Project description   |
 ### ProjectCreate
 Schema for creating projects (inherits ProjectBase).
 ### ProjectUpdate
 Schema for updating projects.
 | Field          | Type                        | Description           |
 | -------------- | --------------------------- | --------------------- |
 | name           | str \| None                 | Project name          |
 | location       | str \| None                 | Project location      |
 | operation_type | MiningOperationType \| None | Mining operation type |
 | description    | str \| None                 | Project description   |
 ### ProjectRead
 Schema for reading projects (inherits ProjectBase).
 | Field      | Type     | Description        |
 | ---------- | -------- | ------------------ |
 | id         | int      | Project ID         |
 | created_at | datetime | Creation timestamp |
 | updated_at | datetime | Update timestamp   |
 ## Scenario Schemas (`schemas/scenario.py`)
 ### ScenarioBase
 Base schema for scenario data.
 | Field            | Type                 | Description                      |
 | ---------------- | -------------------- | -------------------------------- |
 | name             | str                  | Scenario name                    |
 | description      | str \| None          | Scenario description             |
 | status           | ScenarioStatus       | Scenario status (default: DRAFT) |
 | start_date       | date \| None         | Start date                       |
 | end_date         | date \| None         | End date                         |
 | discount_rate    | float \| None        | Discount rate                    |
 | currency         | str \| None          | ISO currency code                |
 | primary_resource | ResourceType \| None | Primary resource                 |
 ### ScenarioCreate
 Schema for creating scenarios (inherits ScenarioBase).
 ### ScenarioUpdate
 Schema for updating scenarios.
 | Field            | Type                   | Description          |
 | ---------------- | ---------------------- | -------------------- |
 | name             | str \| None            | Scenario name        |
 | description      | str \| None            | Scenario description |
 | status           | ScenarioStatus \| None | Scenario status      |
 | start_date       | date \| None           | Start date           |
 | end_date         | date \| None           | End date             |
 | discount_rate    | float \| None          | Discount rate        |
 | currency         | str \| None            | ISO currency code    |
 | primary_resource | ResourceType \| None   | Primary resource     |
 ### ScenarioRead
 Schema for reading scenarios (inherits ScenarioBase).
 | Field      | Type     | Description        |
 | ---------- | -------- | ------------------ |
 | id         | int      | Scenario ID        |
 | project_id | int      | Project ID         |
 | created_at | datetime | Creation timestamp |
 | updated_at | datetime | Update timestamp   |
 ### ScenarioComparisonRequest
 Schema for scenario comparison requests.
 | Field        | Type      | Description                             |
 | ------------ | --------- | --------------------------------------- |
 | scenario_ids | list[int] | List of scenario IDs (minimum 2 unique) |
 ### ScenarioComparisonResponse
 Schema for scenario comparison responses.
 | Field      | Type               | Description           |
 | ---------- | ------------------ | --------------------- |
 | project_id | int                | Project ID            |
 | scenarios  | list[ScenarioRead] | List of scenario data |
 ## Import Schemas (`schemas/imports.py`)
 ### ProjectImportRow
 Schema for importing project data.
 | Field          | Type                | Description             |
 | -------------- | ------------------- | ----------------------- |
 | name           | str                 | Project name (required) |
 | location       | str \| None         | Project location        |
 | operation_type | MiningOperationType | Mining operation type   |
 | description    | str \| None         | Project description     |
 | created_at     | datetime \| None    | Creation timestamp      |
 | updated_at     | datetime \| None    | Update timestamp        |
 ### ScenarioImportRow
 Schema for importing scenario data.
 | Field            | Type                 | Description                      |
 | ---------------- | -------------------- | -------------------------------- |
 | project_name     | str                  | Project name (required)          |
 | name             | str                  | Scenario name (required)         |
 | status           | ScenarioStatus       | Scenario status (default: DRAFT) |
 | start_date       | date \| None         | Start date                       |
 | end_date         | date \| None         | End date                         |
 | discount_rate    | float \| None        | Discount rate                    |
 | currency         | str \| None          | ISO currency code                |
 | primary_resource | ResourceType \| None | Primary resource                 |
 | description      | str \| None          | Scenario description             |
 | created_at       | datetime \| None     | Creation timestamp               |
 | updated_at       | datetime \| None     | Update timestamp                 |
 ## Export Schemas (`schemas/exports.py`)
 ### ExportFormat
 Enumeration for export formats.
 - `CSV`: CSV format
 - `XLSX`: Excel format
 ### BaseExportRequest
 Base schema for export requests.
 | Field            | Type         | Description                       |
 | ---------------- | ------------ | --------------------------------- |
 | format           | ExportFormat | Export format (default: CSV)      |
 | include_metadata | bool         | Include metadata (default: False) |
 ### ProjectExportRequest
 Schema for project export requests (inherits BaseExportRequest).
 | Field   | Type                         | Description    |
 | ------- | ---------------------------- | -------------- |
 | filters | ProjectExportFilters \| None | Export filters |
 ### ScenarioExportRequest
 Schema for scenario export requests (inherits BaseExportRequest).
 | Field   | Type                          | Description    |
 | ------- | ----------------------------- | -------------- |
 | filters | ScenarioExportFilters \| None | Export filters |
 ### ExportTicket
 Schema for export tickets.
 | Field    | Type                             | Description   |
 | -------- | -------------------------------- | ------------- |
 | token    | str                              | Export token  |
 | format   | ExportFormat                     | Export format |
 | resource | Literal["projects", "scenarios"] | Resource type |
 ### ExportResponse
 Schema for export responses.
 | Field  | Type         | Description   |
 | ------ | ------------ | ------------- |
 | ticket | ExportTicket | Export ticket |
 ## Discrepancies Between Data Models and Pydantic Schemas
 ### Missing Pydantic Schemas
 The following SQLAlchemy models do not have corresponding Pydantic schemas:
 - `User` - Authentication handled through forms, not API schemas
 - `Role` - Role management not exposed via API
 - `UserRole` - User-role associations not exposed via API
 - `FinancialInput` - Financial inputs managed through scenario context
 - `SimulationParameter` - Simulation parameters managed through scenario context
 - `PricingSettings` - Pricing settings managed through project context
 - `PricingMetalSettings` - Pricing overrides managed through settings context
 - `PricingImpuritySettings` - Pricing overrides managed through settings context
 - `PerformanceMetric` - Metrics not exposed via API
 - `ImportExportLog` - Audit logs not exposed via API
 ### Schema Differences
 - **Project schemas** include `operation_type` as required enum, while the model allows null but defaults to `OTHER`
 - **Scenario schemas** include currency normalization validation not present in the model validator
 - **Import schemas** include extensive validation and coercion logic for CSV/Excel data parsing
 - **Export schemas** reference filter classes (`ProjectExportFilters`, `ScenarioExportFilters`) not defined in this document
 ### Additional Validation
 Pydantic schemas include additional validation beyond SQLAlchemy model constraints:
 - Email format validation in auth forms
 - Password confirmation matching
 - Currency code normalization and validation
 - Date range validation (end_date >= start_date)
 - Required field validation for imports
 - Enum value coercion with aliases for imports
 Refer back to the [data model overview](../02_data_model.md) when aligning ORM entities with these schemas.
--- a/architecture/08_concepts/02_data_model/05_monitoring.md
+++ b/architecture/08_concepts/02_data_model/05_monitoring.md
@@ -0,0 +1,41 @@
 # Monitoring and Auditing
 This page documents the monitoring- and auditing-related tables that were previously embedded in `02_data_model.md`. Separating them keeps operational telemetry isolated from the core project/scenario data model reference.
 ## PerformanceMetric
 Tracks API performance metrics used by the monitoring pipeline.
 **Table:** `performance_metrics`
 | Attribute        | Type         | Description           |
 | ---------------- | ------------ | --------------------- |
 | id               | Integer (PK) | Primary key           |
 | timestamp        | DateTime     | Metric timestamp      |
 | metric_name      | String       | Metric name           |
 | value            | Float        | Metric value          |
 | labels           | String       | JSON string of labels |
 | endpoint         | String       | API endpoint          |
 | method           | String       | HTTP method           |
 | status_code      | Integer      | HTTP status code      |
 | duration_seconds | Float        | Request duration      |
 ## ImportExportLog
 Audit log for import and export operations initiated by users.
 **Table:** `import_export_logs`
 | Attribute  | Type                    | Description                           |
 | ---------- | ----------------------- | ------------------------------------- |
 | id         | Integer (PK)            | Primary key                           |
 | action     | String(32)              | Action type (preview, commit, export) |
 | dataset    | String(32)              | Dataset type (projects, scenarios)    |
 | status     | String(16)              | Operation status                      |
 | filename   | String(255)             | File name                             |
 | row_count  | Integer                 | Number of rows                        |
 | detail     | Text                    | Additional details                    |
 | user_id    | Integer (FK → users.id) | User foreign key                      |
 | created_at | DateTime                | Creation timestamp                    |
 These tables power telemetry dashboards and audit trails but are not exposed via public API endpoints.
--- a/architecture/08_concepts/03_security.md
+++ b/architecture/08_concepts/03_security.md
@@ -10,7 +10,21 @@ All sensitive data is encrypted at rest and in transit to prevent unauthorized a
 Role-based access controls (RBAC) are implemented to restrict data access based on user roles and responsibilities.
-Also see [Authentication and Authorization](../08_concepts.md#authentication-and-authorization) and the [Data Model](../08_concepts/02_data_model.md#user-roles) sections.
+Also see [Authentication and Authorization](../08_concepts.md#authentication-and-authorization) and the [Data Model](../08_concepts/02_data_model/01_sqlalchemy_models.md#userrole) sections.
 - Default administrative credentials are provided at deployment time through environment variables (`CALMINER_SEED_ADMIN_EMAIL`, `CALMINER_SEED_ADMIN_USERNAME`, `CALMINER_SEED_ADMIN_PASSWORD`, `CALMINER_SEED_ADMIN_ROLES`). These values are consumed by a shared bootstrap helper on application startup, ensuring mandatory roles and the administrator account exist before any user interaction.
 - Operators can request a managed credential reset by setting `CALMINER_SEED_FORCE=true`. On the next startup the helper rotates the admin password and reapplies role assignments, so downstream environments must update stored secrets immediately after the reset.
 - The bootstrap helper is idempotent; when no changes are required, startup completes without mutating the database, preserving audit trails while still verifying the presence of required roles.
 ### Route Guard Dependencies
 - `require_project_resource` and `require_scenario_resource` build on service-level authorization helpers to enforce role checks while resolving requested entities.
 - `require_project_scenario_resource` ensures the scenario referenced by a request belongs to the provided project identifier before continuing processing.
 - These dependencies surface 401/403/404 responses consistently across API and UI handlers and can be composed with additional ownership checks when project member metadata is introduced.
 ## Session Management
 Authentication relies on a pair of signing tokens issued as `calminer_access_token` and `calminer_refresh_token` HttpOnly cookies. An `AuthSessionMiddleware` component validates incoming access tokens, refreshes them when still covered by a valid refresh token, and attaches the resolved user context to `request.state.auth_session`. Logout clears both cookies and redirects users back to the login form. This approach keeps credentials out of JavaScript, supports transparent rotation of short-lived access tokens, and ensures templates can adapt their navigation to the current session state.
 ## Audit Logging
--- a/architecture/08_concepts/11_reporting_analytics.md
+++ b/architecture/08_concepts/11_reporting_analytics.md
@@ -0,0 +1,39 @@
 # Reporting and Analytics
 The Calminer application provides robust reporting and analytics features designed to help users visualize and interpret mining project data effectively.
 ## Dashboard Generation
 Users can create and customize dashboards that display key metrics and performance indicators related to mining projects. Dashboards support various visualization types, including charts, graphs, and tables, and allow users to drill down into specific data points for deeper analysis.
 ## Report Creation
 The system enables users to create detailed reports that summarize project scenarios, financial analyses, and simulation results. Reports can be exported in multiple formats, including PDF and Excel, for easy sharing and presentation.
 ## Data Visualization Tools
 The system provides tools for visualizing data through charts, graphs, and dashboards, making it easier to identify trends and insights. Users can interact with visualizations to explore data in more detail.
 ## Analytics Engine
 The backend analytics engine processes large datasets and performs complex calculations, including Monte Carlo simulations, to provide users with probabilistic assessments of project outcomes.
 ### Real-time Analytics
 The system supports real-time data processing and analytics, enabling users to access up-to-date information and respond quickly to changing conditions.
 ### Historical Analysis
 The system maintains a history of key metrics and events, allowing for retrospective analysis and identification of long-term trends.
 ## Custom Reports
 Users can create custom reports based on specific criteria, allowing for tailored analysis of project performance and resource utilization.
 ## User-Friendly Interface
 The reporting and analytics features are accessible through an intuitive web interface, allowing users to navigate reports and dashboards seamlessly.
 ## Integration with Scenario Management
 Reporting tools are tightly integrated with the scenario management system, enabling users to generate reports directly from specific project scenarios and compare results across different configurations.
--- a/requirements/FR-001.md
+++ b/requirements/FR-001.md
@@ -24,3 +24,8 @@ The ability to create and manage scenarios is crucial for effective decision-mak
 - User authentication and authorization mechanisms to ensure secure access to project data.
 - Data visualization libraries to support graphical representation of scenario analysis results.
 - Integration with other functional requirements, such as data import/export and reporting features.
 ## References
 - [Data Model Documentation](../architecture/08_concepts/02_data_model.md)
 - [Authentication and Authorization](../architecture/08_concepts.md#authentication-and-authorization)
--- a/requirements/FR-008.md
+++ b/requirements/FR-008.md
@@ -32,3 +32,44 @@ Exporting analysis results in multiple formats is essential for users who need t
 - The system should provide a user-friendly interface for configuring export options.
 - The export functionality should be accessible from relevant areas of the application (e.g., project dashboards, analysis results pages).
 - The system should log export activities for auditing and monitoring purposes.
 - Import and export flows must share a consistent schema contract so that data exported from the platform can be re-imported without loss.
 - Export endpoints must respect role-based access rules (analyst, project_manager, admin) and return streaming responses with proper content-disposition headers.
 - UI tooling must support triggered downloads via modal forms with format and metadata controls.
 ## Import/Export Field Mapping
 The following table inventories the core project and scenario attributes that must participate in bulk import/export workflows. It also documents whether a field is required during import, how validation should behave, and any special handling notes for generated values.
 | Dataset  | Field              | Data Type         | Required on Import | Validation & Normalization                                                 | Export Behaviour                             |
 | -------- | ------------------ | ----------------- | ------------------ | -------------------------------------------------------------------------- | -------------------------------------------- |
 | Project  | `name`             | string (≤255)     | Yes                | Must be unique (case-insensitive); trim whitespace.                        | Always include; acts as stable identifier.   |
 | Project  | `location`         | string (≤255)     | No                 | Accept blank values; normalize to title case where practical.              | Include when present; blank otherwise.       |
 | Project  | `operation_type`   | enum              | Yes                | Must map to `MiningOperationType`; allow human-friendly labels on import.  | Export canonical enum value.                 |
 | Project  | `description`      | text              | No                 | Permit multiline text; strip dangerous markup.                             | Export full text.                            |
 | Project  | `created_at`       | datetime (UTC)    | No                 | Ignored on import; system assigns timestamp.                               | Export for audit purposes (read-only).       |
 | Project  | `updated_at`       | datetime (UTC)    | No                 | Ignored on import; system manages.                                         | Export for audit purposes (read-only).       |
 | Scenario | `project_name`     | string (≤255)     | Yes                | Must resolve to an existing project `name`; case-insensitive lookup.       | Export to maintain relationship context.     |
 | Scenario | `name`             | string (≤255)     | Yes                | Unique per project; trim whitespace.                                       | Always include.                              |
 | Scenario | `status`           | enum              | Yes                | Map to `ScenarioStatus`; accept friendly labels (`Draft`, `Active`, etc.). | Export canonical enum value.                 |
 | Scenario | `start_date`       | date (ISO 8601)   | No                 | Validate ordering with `end_date` when provided.                           | Export ISO 8601 string or blank.             |
 | Scenario | `end_date`         | date (ISO 8601)   | No                 | Must be ≥ `start_date` when both present.                                  | Export ISO 8601 string or blank.             |
 | Scenario | `discount_rate`    | decimal (5,2)     | No                 | Accept percentage string or decimal; normalize to decimal (0-100).         | Export numeric percentage with two decimals. |
 | Scenario | `currency`         | string (ISO 4217) | No                 | Uppercase 3-letter ISO code; validate against supported list.              | Export uppercase code.                       |
 | Scenario | `primary_resource` | enum              | No                 | Map to `ResourceType`; accept human-friendly names on import.              | Export canonical enum value.                 |
 | Scenario | `description`      | text              | No                 | Allow multiline text; strip dangerous markup.                              | Export full text.                            |
 | Scenario | `created_at`       | datetime (UTC)    | No                 | Ignored on import; system assigns timestamp.                               | Export for audit purposes (read-only).       |
 | Scenario | `updated_at`       | datetime (UTC)    | No                 | Ignored on import; system manages.                                         | Export for audit purposes (read-only).       |
 Additional domain entities (financial inputs, simulation parameters, etc.) will be inventoried in subsequent iterations once their import/export scope is defined. This initial mapping focuses on the mandatory Project and Scenario records required to satisfy FR-008 and related reporting flows.
 ## Import/Export File Format Guidelines
 - **Supported formats:** CSV (`.csv`) and Excel (`.xlsx`).
 - **Encoding:** All text-based files must use UTF-8 encoding without byte-order marks to ensure consistent parsing across environments.
 - **Header row:** The first row must contain column headers matching the field names defined in the mapping table (case-insensitive). Additional columns are ignored during import but preserved during export when possible.
 - **CSV specifics:** Values are comma-delimited, double-quoted when containing commas or line breaks, and line endings follow `\n`. Empty strings are treated as null when the field is optional.
 - **Excel specifics:** Importers read from the first worksheet. Each sheet must use the same header row convention. Cells containing formulas are evaluated before ingestion (using stored values, not formulas).
 - **Date/time:** Dates and timestamps must be supplied in ISO 8601 format (`YYYY-MM-DD` for dates, `YYYY-MM-DDTHH:MM:SSZ` for timestamps). Exporters always output UTC timestamps.
 - **Numeric fields:** Decimal values use `.` as the separator. Discount rates accept either raw decimals (`7.5`) or percentages (`7.5%`).
 - **Enumerations:** Enum columns accept canonical values or the friendly labels documented in the alias table. Exporters always emit canonical enum codes.
 - **Validation feedback:** Import workflows return structured error payloads indicating row number, column, and validation failure to guide remediation.
--- a/requirements/FR-011.md
+++ b/requirements/FR-011.md
@@ -0,0 +1,53 @@
 # Functional Requirement FR-011: Basic Profitability Calculation
 ## Description
 The system shall provide functionality to perform basic profitability calculations for ore mining products based on user-defined input parameters.
 A detailed specification is available in [specifications/price_calculation.md](specifications/price_calculation.md)
 ## Rationale
 Profitability calculations are essential for mining companies to assess the financial viability of their operations. By enabling users to input key parameters and receive profitability metrics, the system will support informed decision-making and strategic planning.
 ## Acceptance Criteria
 1. **Input Parameters**: The system shall accept the following input parameters:
   - Metal type (e.g., copper, gold, lithium)
   - Ore tonnage processed
   - Head grade (%)
   - Recovery rate (%)
   - Treatment charge (currency/tonne)
   - Smelting charge (currency/tonne)
   - Moisture content (%)
   - Impurity content (ppm for relevant impurities)
   - Moisture penalty factor (currency/%)
   - Impurity penalty factor (currency/ppm)
   - Premiums/credits (currency)
   - FX rate (currency conversion rate)
 2. **Calculation Logic**: The system shall implement the profitability calculation logic as specified in the detailed specification document, including:
   - Calculation of metal content based on ore tonnage, head grade, and recovery rate.
   - Computation of gross revenue using metal content and reference prices.
   - Deduction of treatment and smelting charges.
   - Application of moisture and impurity penalties.
   - Adjustment for premiums/credits.
   - Conversion of final revenue to scenario currency using the FX rate.
 3. **Output Metrics**: The system shall provide the following output metrics:
   - Gross revenue (in scenario currency)
   - Total charges (treatment and smelting)
   - Total penalties (moisture and impurities)
   - Net revenue before premiums
   - Final adjusted revenue (after premiums/credits)
 ## Dependencies
 - Accurate and up-to-date reference prices for supported metals.
 - User interface components for inputting parameters and displaying results.
 - Data validation mechanisms to ensure input parameters are within acceptable ranges.
 - Integration with existing scenario management and reporting modules for seamless user experience.
 ## Notes
 - The system should allow for easy updates to reference prices and other key parameters as market conditions change.
 - User training and documentation will be essential to ensure effective use of the profitability calculation features.
 - Future enhancements may include sensitivity analysis and scenario comparisons based on profitability metrics.
--- a/requirements/FR-012.md
+++ b/requirements/FR-012.md
@@ -0,0 +1,50 @@
 # Functional Requirement FR-012: Monte Carlo simulation engine
 ## Description
 The system shall provide a Monte Carlo simulation engine to model uncertainties in mining project parameters. This engine will allow users to define probability distributions for key input variables and run simulations to assess the impact of these uncertainties on project outcomes.
 A detailed specification is available in [specifications/monte_carlo_simulation.md](specifications/monte_carlo_simulation.md)
 ## Rationale
 Monte Carlo simulations are a powerful tool for risk analysis and decision-making in mining projects. By modeling the variability of input parameters, users can better understand potential outcomes, identify risks, and make informed decisions based on probabilistic results.
 ## Acceptance Criteria
 1. **Input Parameter Definitions**: The system shall allow users to define probability distributions for key input parameters, including but not limited to:
   - Ore grade
   - Recovery rate
   - Operating costs
   - Metal prices
 2. **Simulation Configuration**: The system shall provide options for users to configure the simulation, including:
   - Number of simulation iterations
   - Random seed for reproducibility
   - Output metrics to be recorded
 3. **Result Analysis**: The system shall offer tools for analyzing simulation results, including:
   - Summary statistics (mean, median, percentiles)
   - Visualization of result distributions (histograms, box plots)
   - Comparison of different scenarios or input configurations
 4. **Integration with Existing Workflows**: The Monte Carlo simulation engine shall be integrated with existing scenario management and reporting tools within the system, allowing users to easily incorporate simulation results into their decision-making processes.
 5. **Documentation**: The system shall include comprehensive documentation on how to use the Monte Carlo simulation engine, including examples and best practices.
 ## Dependencies
 - Access to historical data for key input parameters to inform probability distribution definitions.
 - Computational resources to support the execution of large-scale simulations.
 - Integration with data management systems to facilitate the use of external data sources.
 - User interface components for defining input parameters, configuring simulations, and visualizing results.
 ## Notes
 - The Monte Carlo simulation engine should be designed for scalability to accommodate large datasets and complex models.
 - Users should be able to easily update probability distributions and other simulation parameters as new information becomes available.
 - Collaboration features may be beneficial to allow multiple users to work on simulation scenarios simultaneously.
 - Future enhancements may include sensitivity analysis and scenario optimization based on simulation results.
--- a/requirements/FR-013.md
+++ b/requirements/FR-013.md
@@ -0,0 +1,40 @@
 # Functional Requirement FR-013: Capital Expenditure (Capex) management and calculation tools for mining projects
 ## Description
 The system shall provide tools to calculate and manage Capital Expenditures (Capex) associated with mining projects. This includes the ability to input, store, and analyze various capital cost components such as equipment purchases, infrastructure development, land acquisition, and other one-time expenses.
 ## Rationale
 Accurate Capex calculations are essential for budgeting, financial planning, and profitability analysis in mining projects. By providing robust tools to manage these costs, the system will enable users to make informed decisions regarding project feasibility and capital allocation.
 ## Acceptance Criteria
 1. **Capex Component Definition**: The system shall allow users to define and categorize various Capex components, including but not limited to:
   - Equipment purchases
   - Infrastructure development
   - Land acquisition
   - Other one-time expenses
 2. **Input and Storage**: The system shall provide interfaces for users to input Capex data and store it in a structured format for easy retrieval and analysis.
 3. **Capex Calculation**: The system shall include tools to calculate total Capex based on the defined components and their respective values. This includes the ability to apply formulas and algorithms to derive insights from the data.
 4. **Reporting and Visualization**: The system shall offer reporting and visualization features to help users understand Capex trends, identify cost drivers, and make data-driven decisions.
 5. **Integration**: The system shall support integration with other financial and operational systems to ensure seamless data flow and consistency across platforms.
 ## Dependencies
 - Integration with existing financial modules for comprehensive cost analysis.
 - User interface components for inputting and managing Capex data.
 - Data validation mechanisms to ensure accuracy and consistency of Capex inputs.
 - Reporting and visualization libraries to support analysis and presentation of Capex data.
 ## Notes
 - The Capex management tools should be designed for scalability to accommodate large datasets and complex cost structures.
 - Users should be able to easily update Capex components and their values as project conditions change.
 - Collaboration features may be beneficial to allow multiple users to work on Capex scenarios simultaneously.
 - Future enhancements may include predictive analytics and machine learning capabilities to improve Capex forecasting and optimization.
--- a/requirements/FR-014.md
+++ b/requirements/FR-014.md
@@ -0,0 +1,41 @@
 # Functional Requirement FR-014: Operational Expenditure (Opex) management and calculation tools for mining projects
 ## Description
 The system shall provide tools to calculate and manage Operational Expenditures (Opex) associated with mining projects. This includes the ability to input, store, and analyze various operational cost components such as labor, materials, energy consumption, maintenance, and other recurring expenses.
 ## Rationale
 Accurate Opex calculations are essential for budgeting, financial planning, and profitability analysis in mining projects. By providing robust tools to manage these costs, the system will enable users to make informed decisions regarding project feasibility and operational efficiency.
 ## Acceptance Criteria
 1. **Opex Component Definition**: The system shall allow users to define and categorize various Opex components, including but not limited to:
   - Labor costs
   - Material costs
   - Energy costs
   - Maintenance costs
   - Other recurring expenses
 2. **Input and Storage**: The system shall provide interfaces for users to input Opex data and store it in a structured format for easy retrieval and analysis.
 3. **Opex Calculation**: The system shall include tools to calculate total Opex based on the defined components and their respective values. This includes the ability to apply formulas and algorithms to derive insights from the data.
 4. **Reporting and Visualization**: The system shall offer reporting and visualization features to help users understand Opex trends, identify cost drivers, and make data-driven decisions.
 5. **Integration**: The system shall support integration with other financial and operational systems to ensure seamless data flow and consistency across platforms.
 ## Dependencies
 - Integration with existing financial modules for comprehensive cost analysis.
 - User interface components for inputting and managing Opex data.
 - Data validation mechanisms to ensure accuracy and consistency of Opex inputs.
 - Reporting and visualization libraries to support analysis and presentation of Opex data.
 ## Notes
 - The Opex management tools should be designed for scalability to accommodate large datasets and complex cost structures.
 - Users should be able to easily update Opex components and their values as project conditions change.
 - Collaboration features may be beneficial to allow multiple users to work on Opex scenarios simultaneously.
 - Future enhancements may include predictive analytics and machine learning capabilities to improve Opex forecasting and optimization.
--- a/requirements/README.md
+++ b/requirements/README.md
@@ -22,5 +22,9 @@ This document outlines the key functional requirements for the CalMiner project.
 | FR-008         | Facilitate export of analysis results in various formats (PDF, Excel, etc.).          | Medium   | [FR-008](../requirements/FR-008.md) |
 | FR-009         | Provide a user-friendly interface accessible via web browsers.                        | High     | [FR-009](../requirements/FR-009.md) |
 | FR-010         | Enable collaboration features for multiple users to work on scenarios simultaneously. | Medium   | [FR-010](../requirements/FR-010.md) |
 | FR-011         | Basic Profitability calculation for ore mining products based on input parameters.    | High     | [FR-011](../requirements/FR-011.md) |
 | FR-012         | Monte Carlo simulation engine to model uncertainties in mining project parameters.    | High     | [FR-012](../requirements/FR-012.md) |
 | FR-013         | Capital Expenditure (Capex) management and calculation tools for mining projects.     | High     | [FR-013](../requirements/FR-013.md) |
 | FR-014         | Operational Expenditure (Opex) calculation tools for processing costs in mining.      | High     | [FR-014](../requirements/FR-014.md) |
 Each functional requirement is detailed in its respective document, providing a comprehensive description, rationale, acceptance criteria, and dependencies. Please refer to the linked documents for more information on each requirement.
--- a/specifications/dashboard_layout.md
+++ b/specifications/dashboard_layout.md
@@ -0,0 +1,45 @@
 # CalMiner Dashboard Layout Plan
 Last updated: 2025-11-09
 This document describes the intended layout structure for the CalMiner homepage/dashboard template.
 ## Page Goals
 - Provide a concise overview of key project and scenario metrics immediately after sign-in.
 - Offer quick actions for creating or navigating to core workflows (projects, scenarios, data uploads).
 - Surface recent activity in a scan-friendly, responsive layout that works on desktop and tablet.
 ## Layout Structure
 1. **Hero Header**
   - Title block with optional welcome message and current environment badge.
   - Primary CTA button group (e.g., "New Project", "Import Data").
   - Optional quick filter chips for project categories.
 2. **Summary Metrics Row**
   - Four metric cards showing: total projects, active scenarios, pending simulations, last import timestamp.
   - Cards adapt to a two-column grid on screens < 960px.
 3. **Recent Activity Panels**
   - Left column (2/3 width) displays a table of recent projects with status, updated timestamp.
   - Right column (1/3 width) hosts scenario alerts (validation conflicts, approaching deadlines) in a stacked list.
 4. **Secondary Widgets**
   - Simulation pipeline status timeline (if data present) or placeholder guidance.
   - Documentation and support links card.
 ## Responsiveness
 - Desktop: two-column layout for activity panels and secondary widgets.
 - Tablet: stacks columns vertically while keeping metric cards in two columns.
 - Mobile: single-column flow; CTA buttons collapse into a vertical stack.
 ## Implementation Notes
 - Reuse existing `.card`, `.page-header`, and `.container` classes from `static/css/main.css`.
 - Introduce specialized dashboard classes (e.g., `.dashboard-metrics`, `.dashboard-grid`) scoped within a new `static/css/dashboard.css` file if needed.
 - Provide placeholder content for areas where backend data integrations are pending (e.g., empty state messages).
--- a/specifications/financial_metrics.md
+++ b/specifications/financial_metrics.md
@@ -0,0 +1,184 @@
 # Financial Metrics Specification
 ## 1. Purpose
 Define the standard methodology CalMiner uses to compute discounted-cash-flow
 profitability metrics, including Net Present Value (NPV), Internal Rate of Return
 (IRR), and Payback Period. These calculations underpin scenario evaluation,
 reporting, and investment decision support within the platform.
 ## 2. Scope
 - Applies to scenario-level profitability analysis using cash flows stored in
  the application database.
 - Covers deterministic cash-flow evaluation; stochastic extensions (e.g., Monte
  Carlo) may overlay these metrics but should reference this specification.
 - Documents the assumptions implemented in `services/financial.py` and the
  related pytest coverage in `tests/test_financial.py`.
 ## 3. Inputs and Definitions
 | Symbol   | Name                    | Description                                 | Units / Domain    | Notes                                                     |
 | -------- | ----------------------- | ------------------------------------------- | ----------------- | --------------------------------------------------------- |
 | $CF_t$   | Cash flow at period $t$ | Currency amount (positive or negative)      | Scenario currency | Negative values typically represent investments/outflows. |
 | $t$      | Period index            | Fractional period (0 = anchor)              | Real number       | Derived from explicit index or calendar date.             |
 | $r$      | Discount rate           | Decimal representation of the annual rate   | $r > -1$          | Scenario configuration provides default rate.             |
 | $m$      | Compounds per year      | Compounding frequency                       | Positive integer  | Defaults to 1 (annual).                                   |
 | $RV$     | Residual value          | Terminal value realised after final period  | Scenario currency | Optional.                                                 |
 | $t_{RV}$ | Residual periods        | Timing of residual value relative to anchor | Real number       | Defaults to last period + 1.                              |
 ### Period Anchoring and Timing
 Cash flows can be supplied with either:
 1. `period_index` — explicit integers/floats overriding all other timing.
 2. `date` — calendar dates. The earliest dated flow anchors the timeline and
   subsequent dates convert the day difference into fractional periods using a
   365-day year divided by `compounds_per_year`.
 3. Neither — flows default to sequential periods based on input order.
 This aligns with `normalize_cash_flows` in `services/financial.py`, ensuring all
 calculations receive `(amount, periods)` tuples.
 ## 4. Net Present Value (NPV)
 ### Formula
 For a set of cash flows $CF_t$ with discount rate $r$ and compounding frequency
 $m$:
 $$
 \text{NPV} = \sum_{t=0}^{n} \frac{CF_t}{\left(1 + \frac{r}{m}\right)^{t}} +
 \begin{cases}
 \dfrac{RV}{\left(1 + \frac{r}{m}\right)^{t_{RV}}} & \text{if residual value present} \\
 0 & \text{otherwise}
 \end{cases}
 $$
 ### Implementation Notes
 - `discount_factor` computes $(1 + r/m)^{-t}`; NPV iterates over the normalised
 flows and sums `amount \* factor`.
 - Residual values default to one period after the final cash flow when
  `residual_periods` is omitted.
 - Empty cash-flow sequences return 0 unless a residual value is supplied.
 ## 5. Internal Rate of Return (IRR)
 ### Definition
 IRR is the discount rate $r$ for which NPV equals zero:
 $$
 0 = \sum_{t=0}^{n} \frac{CF_t}{\left(1 + \frac{r}{m}\right)^{t}}
 $$
 ### Solver Behaviour
 - Newton–Raphson iteration starts from `guess` (default 10%).
 - Derivative instability or non-finite values trigger a fallback to a bracketed
  bisection search between:
  - Lower bound: $-0.99 \times m$
  - Upper bound: $10.0$ (doubles until the root is bracketed or attempts exceed 12)
 - Raises `ConvergenceError` when no sign change is found or the bisection fails
  within the iteration budget (`max_iterations` default 100; bisection uses
  double this limit).
 - Validates that the cash-flow series includes at least one negative and one
  positive value; otherwise IRR is undefined and a `ValueError` is raised.
 ### Caveats
 - Multiple sign changes may yield multiple IRRs. The solver returns the root it
  finds within the configured bounds; scenarios must interpret the result in
  context.
 - Rates less than `-1 * m` imply nonphysical periodic rates and are excluded.
 ## 6. Payback Period
 ### Definition
 The payback period is the earliest period $t$ where cumulative cash flows become
 non-negative. With fractional interpolation (default behaviour), the period is
 calculated as:
 $$
 \text{Payback} = t_{prev} + \left(\frac{-\text{Cumulative}_{prev}}{CF_t}\right)
 \times (t - t_{prev})
 $$
 where $t_{prev}$ is the previous period with negative cumulative cash flow.
 ### Implementation Notes
 - Cash flows are sorted by period to ensure chronological accumulation.
 - When `allow_fractional` is `False`, the function returns the first period with
  non-negative cumulative total without interpolation.
 - `PaybackNotReachedError` is raised if the cumulative total never becomes
  non-negative.
 ## 7. Examples
 ### Example 1: Baseline Project
 - Initial investment: $-1,000,000$ at period 0.
 - Annual inflows: 300k, 320k, 340k, 360k, 450k (periods 1-5).
 - Discount rate: 8% annual, `compounds_per_year = 1`.
 | Period | Cash Flow (currency) |
 | ------ | -------------------- |
 | 0      | -1,000,000           |
 | 1      | 300,000              |
 | 2      | 320,000              |
 | 3      | 340,000              |
 | 4      | 360,000              |
 | 5      | 450,000              |
 - `net_present_value` ≈ 205,759
 - `internal_rate_of_return` ≈ 0.158
 - `payback_period` ≈ 4.13 periods
 ### Example 2: Residual Value with Irregular Timing
 - Investment: -500,000 on 2024-01-01
 - Cash inflows on irregular dates (2024-07-01: 180k, 2025-01-01: 200k,
  2025-11-01: 260k)
 - Residual value 150k realised two years after final inflow
 - Discount rate: 10%, `compounds_per_year = 4`
 NPV discounts each cash flow by converting day deltas to quarterly periods. The
 residual is discounted at `t_{RV} = last_period + 2` (because the override is
 supplied).
 ## 8. Testing Strategy
 `tests/test_financial.py` exercises:
 - `normalize_cash_flows` with date-based, index-based, and sequential cash-flow
  inputs.
 - NPV calculations with and without residual values, including discount-rate
  sensitivity checks.
 - IRR convergence success cases, invalid inputs, and non-converging scenarios.
 - Payback period exact, fractional, and never-payback cases.
 Developers extending the financial metrics should add regression tests covering
 new assumptions or solver behaviour.
 ## 9. Integration Notes
 - Scenario evaluation services should pass cash flows as `CashFlow` instances to
  reuse the shared normalisation logic.
 - UI and reporting layers should display rates as percentages but supply them as
  decimals to the service layer.
 - Future Monte Carlo or sensitivity analyses can reuse the same helpers to
  evaluate each simulated cash-flow path.
 ## 10. References
 - Internal implementation: `calminer/services/financial.py`
 - Tests: `calminer/tests/test_financial.py`
 - Related specification: `calminer-docs/specifications/price_calculation.md`
 - Architecture context: `calminer-docs/architecture/08_concepts/02_data_model.md`
 ```}
 ```
--- a/specifications/kpis.md
+++ b/specifications/kpis.md
@@ -0,0 +1,143 @@
 # Key Performance Indicators (KPIs) for Calminer
 ## Overview
 This document defines the key performance indicators (KPIs) that Calminer tracks to ensure system scalability, reliability, and optimal user experience as specified in FR-006.
 ## KPI Categories
 ### Application Performance Metrics
 #### Response Time
 - **Metric**: HTTP request duration (95th percentile)
 - **Target**: < 500ms for API endpoints, < 2s for UI pages
 - **Collection**: Automatic via MetricsMiddleware
 - **Alert Threshold**: > 1s (API), > 5s (UI)
 #### Error Rate
 - **Metric**: HTTP error responses (4xx/5xx) as percentage of total requests
 - **Target**: < 1% overall, < 0.1% for 5xx errors
 - **Collection**: Automatic via MetricsMiddleware
 - **Alert Threshold**: > 5% (4xx), > 0.5% (5xx)
 #### Throughput
 - **Metric**: Requests per second (RPS)
 - **Target**: > 100 RPS sustained
 - **Collection**: Automatic via MetricsMiddleware
 - **Alert Threshold**: < 10 RPS sustained
 ### Data Processing Metrics
 #### Import/Export Duration
 - **Metric**: Time to complete import/export operations
 - **Target**: < 30s for small datasets (< 10k rows), < 5min for large datasets
 - **Collection**: Via monitoring.metrics.observe_import/observe_export
 - **Alert Threshold**: > 10min for any operation
 #### Data Volume
 - **Metric**: Rows processed per operation
 - **Target**: Support up to 100k rows per import/export
 - **Collection**: Via import/export service instrumentation
 - **Alert Threshold**: Operations failing on > 10k rows
 ### System Resource Metrics
 #### Database Connections
 - **Metric**: Active database connections
 - **Target**: < 80% of max connections
 - **Collection**: Prometheus gauge (DB_CONNECTIONS)
 - **Alert Threshold**: > 90% of max connections
 #### Memory Usage
 - **Metric**: Application memory consumption
 - **Target**: < 512MB per worker
 - **Collection**: Container metrics (Kubernetes/Docker)
 - **Alert Threshold**: > 1GB per worker
 #### CPU Usage
 - **Metric**: Application CPU utilization
 - **Target**: < 70% sustained
 - **Collection**: Container metrics (Kubernetes/Docker)
 - **Alert Threshold**: > 85% sustained
 ### User Experience Metrics
 #### Concurrent Users
 - **Metric**: Active user sessions
 - **Target**: Support 100+ concurrent users
 - **Collection**: Session tracking via AuthSessionMiddleware
 - **Alert Threshold**: > 200 concurrent users (capacity planning)
 #### Session Duration
 - **Metric**: Average user session length
 - **Target**: 10-30 minutes typical
 - **Collection**: Session tracking
 - **Alert Threshold**: < 1 minute average (usability issue)
 ### Business Metrics
 #### Project/Scenario Operations
 - **Metric**: Projects/scenarios created per hour
 - **Target**: 50+ operations per hour
 - **Collection**: Repository operation logging
 - **Alert Threshold**: < 5 operations per hour (adoption issue)
 #### Simulation Performance
 - **Metric**: Monte Carlo simulation completion time
 - **Target**: < 10s for typical scenarios
 - **Collection**: Simulation service instrumentation
 - **Alert Threshold**: > 60s for any simulation
 ## Monitoring Implementation
 ### Data Collection
 - **HTTP Metrics**: Automatic collection via MetricsMiddleware
 - **Business Metrics**: Service-level instrumentation
 - **System Metrics**: Container orchestration (Kubernetes)
 - **Storage**: performance_metrics table + Prometheus
 ### Alerting
 - **Response Time**: P95 > 1s for 5 minutes
 - **Error Rate**: > 5% for 10 minutes
 - **Resource Usage**: > 90% for 15 minutes
 - **Data Processing**: Failures > 3 in 1 hour
 ### Dashboards
 - **Real-time**: Current performance via /metrics endpoint
 - **Historical**: Aggregated metrics via /performance endpoint
 - **Health**: Detailed health checks via /health endpoint
 ## Scaling Guidelines
 ### Horizontal Scaling Triggers
 - CPU > 70% sustained
 - Memory > 80% sustained
 - RPS > 80% of target
 ### Vertical Scaling Triggers
 - Memory > 90% sustained
 - Database connections > 80%
 ### Auto-scaling Configuration
 - Min replicas: 2
 - Max replicas: 10
 - Scale up: CPU > 70% for 5 minutes
 - Scale down: CPU < 30% for 10 minutes
--- a/specifications/monte_carlo_simulation.md
+++ b/specifications/monte_carlo_simulation.md
@@ -0,0 +1,141 @@
 # Monte Carlo Simulation Specification
 ## 1. Purpose
 Define the configuration, inputs, and outputs for CalMiner's Monte Carlo
 simulation engine used to evaluate project scenarios with stochastic cash-flow
 assumptions. The engine augments deterministic profitability metrics by
 sampling cash-flow distributions and aggregating resulting Net Present Value
 (NPV), Internal Rate of Return (IRR), and Payback Period statistics.
 ## 2. Scope
 - Applies to scenario-level profitability analysis executed via
  `services/simulation.py`.
 - Covers configuration dataclasses (`SimulationConfig`, `CashFlowSpec`,
  `DistributionSpec`) and supported distribution families.
 - Outlines expectations for downstream reporting and visualization modules that
  consume simulation results.
 ## 3. Inputs
 ### 3.1 Cash Flow Specifications
 Each Monte Carlo run receives an ordered collection of `CashFlowSpec` entries.
 Each spec pairs a deterministic `CashFlow` (amount, period index/date) with an
 optional `DistributionSpec`. When no distribution is provided the deterministic
 value is used for every iteration.
 ### 3.2 Simulation Configuration
 `SimulationConfig` controls execution:
 | Field                                 | Description                                                       |
 | ------------------------------------- | ----------------------------------------------------------------- |
 | `iterations`                          | Number of Monte Carlo iterations (must be > 0).                   |
 | `discount_rate`                       | Annual discount rate (decimal) passed to NPV helper.              |
 | `seed`                                | Optional RNG seed to ensure reproducible sampling.                |
 | `metrics`                             | Tuple of requested metrics (`npv`, `irr`, `payback`).             |
 | `percentiles`                         | Percentile cutoffs (0–100) computed for each metric.              |
 | `compounds_per_year`                  | Compounding frequency reused by financial helpers.                |
 | `return_samples`                      | When `True`, raw metric samples are returned alongside summaries. |
 | `residual_value` / `residual_periods` | Optional residual cash flow inputs reused by NPV.                 |
 ### 3.3 Context Metadata
 Optional dictionaries provide dynamic parameters when sourcing distribution
 means or other values:
 - `scenario_context`: scenario-specific values (e.g., salvage mean, cost
  overrides).
 - `metadata`: shared configuration (e.g., global commodity price expectations).
 ## 4. Distributions
 `DistributionSpec` defines stochastic behaviour:
 | Property     | Description                                                                     |
 | ------------ | ------------------------------------------------------------------------------- |
 | `type`       | `normal`, `lognormal`, `triangular`, or `discrete`.                             |
 | `parameters` | Mapping of required parameters per distribution family.                         |
 | `source`     | How base parameters are sourced: `static`, `scenario_field`, or `metadata_key`. |
 | `source_key` | Identifier used for non-static sources.                                         |
 ### 4.1 Parameter Validation
 - `normal`: requires non-negative `std_dev`; defaults `mean` to baseline cash
  flow amount when omitted.
 - `lognormal`: requires `mean` (mu in log space) and non-negative `sigma`.
 - `triangular`: requires `min`, `mode`, `max` with constraint `min <= mode <= max`.
 - `discrete`: requires paired `values`/`probabilities` sequences; probabilities
  must be non-negative and sum to 1.0.
 Invalid definitions raise `DistributionConfigError` before sampling.
 ## 5. Algorithm Overview
 1. Seed a NumPy `Generator` (`default_rng(seed)`) unless a generator instance is
   supplied.
 2. For each iteration:
   - Realise cash flows by sampling distributions or using deterministic
     values.
   - Compute requested metrics using shared helpers from
     `services/financial.py`:
     - NPV via `net_present_value` (respecting `residual_value` inputs).
     - IRR via `internal_rate_of_return`; non-converging or invalid trajectories
       return `NaN` and increment `failed_runs`.
     - Payback via `payback_period`; scenarios failing to hit non-negative
       cumulative cash flow record `NaN`.
 3. Aggregate results into per-metric arrays; calculate summary statistics:
   mean, sample standard deviation, min/max, and configured percentiles using
   `numpy.percentile`.
 4. Assemble `SimulationResult` containing summary descriptors and optional raw
   samples when `return_samples` is enabled.
 ## 6. Outputs
 `SimulationResult` includes:
 - `iterations`: total iteration count executed.
 - `summaries`: mapping of `SimulationMetric` to `MetricSummary` objects with:
  - `mean`, `std_dev`, `minimum`, `maximum`.
  - `percentiles`: mapping of configured percentile cutoffs to values.
  - `sample_size`: number of successful (non-NaN) samples.
  - `failed_runs`: count of iterations producing `NaN` for the metric.
 - `samples`: optional mapping of metric to raw `numpy.ndarray` of samples when
  detailed analysis is required downstream.
 ## 7. Error Handling
 - Invalid configuration or missing context raises `DistributionConfigError`.
 - Zero iterations or invalid percentile ranges raise `ValueError`.
 - Financial helper exceptions (`ConvergenceError`, `PaybackNotReachedError`)
  are captured per iteration and converted to `NaN` samples to preserve
  aggregate results while flagging failure counts.
 ## 8. Usage Guidance
 - Scenario services should construct `CashFlowSpec` instances from persisted
  financial inputs and optional uncertainty definitions stored alongside the
  scenario.
 - Reporting routes can request raw samples when producing histogram or violin
  plots; otherwise rely on `MetricSummary` statistics for tabular output.
 - Visualizations implementing FR-005 should leverage percentile outputs to
  render fan charts or confidence intervals.
 - When integrating with scheduling workflows, persist the deterministic seed to
  ensure repeated runs remain comparable.
 ## 9. Testing
 `tests/test_simulation.py` covers deterministic parity with financial helpers,
 seed reproducibility, context parameter sourcing, failure accounting for metrics
 that cannot be computed, error handling for misconfigured distributions, and
 sample-return functionality. Additional regression cases should accompany new
 metrics or distribution families.
 ## 10. References
 - Implementation: `calminer/services/simulation.py`
 - Financial helpers: `calminer/services/financial.py`
 - Tests: `calminer/tests/test_simulation.py`
 - Related specification: `calminer-docs/specifications/financial_metrics.md`
--- a/specifications/price_calculation.md
+++ b/specifications/price_calculation.md
@@ -1,4 +1,141 @@
-# Variables for Price Calculation
+# Product Price Calculation Specification
 ## 1. Purpose
 Provide a detailed reference for calculating product sale prices across supported metals, including adjustments for ore grade, recovery rate, moisture, and impurities. This document extends the initial variable list with explicit formula definitions, validation rules, and example workflows.
 ## 2. Scope
 - Applies to primary commodity outputs (e.g., copper, gold, lithium) cited in scenario models.
 - Supports integration into scenario profitability pipelines and exportable reporting.
 - Covers penalties/credits based on quality metrics (water content, impurity assays) and market adjustments.
 ## 3. Inputs & Parameters
 | Symbol        | Name                    | Description                                          | Units / Domain               | Validation              |
 | ------------- | ----------------------- | ---------------------------------------------------- | ---------------------------- | ----------------------- |
 | $M$           | Metal                   | Commodity identifier (e.g., `copper`, `gold`)        | Enum `MetalType`             | Required                |
 | $Q_{ore}$     | Ore tonnage processed   | Total ore mass entering processing                   | metric tonnes                | $Q_{ore} > 0$           |
 | $G$           | Head grade              | Percentage of target metal in ore (mass fraction)    | % (0–100)                    | $0 < G \leq 100$        |
 | $R$           | Recovery rate           | Plant recovery efficiency                            | % (0–100)                    | $0 < R \leq 100$        |
 | $T$           | Treatment charge        | Base processing fee negotiated with smelter/refiner  | currency / tonne concentrate | $T \geq 0$              |
 | $S$           | Smelting charge         | Additional fee tied to concentrate handling          | currency / tonne concentrate | $S \geq 0$              |
 | $M_{moist}$   | Moisture content        | Percentage water in concentrate                      | % (0–100)                    | $0 \leq M_{moist} < 40$ |
 | $M_{imp}^{i}$ | Impurity content        | For impurity _i_ (e.g., As, Pb, Zn) measure mass ppm | ppm                          | $0 \leq M_{imp}^{i}$    |
 | $F_{moist}$   | Moisture penalty factor | Currency impact per excess moisture percentage       | currency / %                 | Optional                |
 | $F_{imp}^{i}$ | Impurity penalty factor | Currency impact per ppm over threshold               | currency / ppm               | Optional                |
 | $Adj_{prem}$  | Premiums/credits        | Adders (e.g., gold credits in copper concentrate)    | currency                     | Optional                |
 | $FX$          | FX rate                 | Convert pricing currency to scenario currency        | currency conversion rate     | $FX > 0$                |
 ## 4. Derived Values
 1. **Metal content** (payable basis):
   $$ Q*{metal} = Q*{ore} \times \frac{G}{100} \times \frac{R}{100} $$
 2. **Payable mass after deductions** (if contractual payable is <100%): Introduce payable percentage $K_M$ (default 100%).
   $$ Q*{pay} = Q*{metal} \times \frac{K_M}{100} $$
 3. **Gross revenue** in reference currency:
   $$ Rev*{gross}^{ref} = Q*{pay} \times P\_{ref} $$
 4. **Treatment and smelting charges**:
   $$ Charges = T + S $$
 5. **Moisture penalty** (if moisture exceeds threshold $M_{moist}^{thr}$):
   $$ Pen*{moist} = \max(0, M*{moist} - M*{moist}^{thr}) \times F*{moist} $$
 6. **Impurity penalty** (sum over impurities with thresholds $M_{imp}^{i,thr}$):
   $$ Pen*{imp} = \sum_i \max(0, M*{imp}^{i} - M*{imp}^{i,thr}) \times F*{imp}^{i} $$
 7. **Net revenue before premiums**:
   $$ Rev*{net}^{ref} = Rev*{gross}^{ref} - Charges - Pen*{moist} - Pen*{imp} $$
 8. **Apply premiums/credits**:
   $$ Rev*{adj}^{ref} = Rev*{net}^{ref} + Adj\_{prem} $$
 9. **Convert to scenario currency**:
   $$ Rev*{adj} = Rev*{adj}^{ref} \times FX $$
 ## 5. Workflow Examples
 ### 5.1 Copper Concentrate
 - Ore feed $Q_{ore} = 100,000$ t, head grade $G = 1.2\%$, recovery $R = 90\%$.
 - Reference price $P_{ref} = \$8,500$/t, payable percentage $K_M = 96\%$.
 - Treatment and smelting charges $T+S = \$100/t$ concentrate equivalent.
 - Moisture threshold 8%, actual 10%, penalty factor $F_{moist} = \$3,000$ per %.
 - Arsenic impurity: threshold 0 ppm (premium for As-free), actual 100 ppm, penalty factor $F_{imp}^{As} = \$2$ per ppm.
 Calculations:
 1. $Q_{metal} = 100,000 \times 0.012 \times 0.9 = 1,080$ t.
 2. $Q_{pay} = 1,080 \times 0.96 = 1,036.8$ t.
 3. $Rev_{gross}^{ref} = 1,036.8 \times 8,500 = \$8,812,800$.
 4. $Pen_{moist} = (10 - 8) \times 3,000 = \$6,000$.
 5. $Pen_{imp} = (100 - 0) \times 2 = \$200$.
 6. $Rev_{net}^{ref} = 8,812,800 - 100,000 - 6,000 - 200 = \$8,706,600$.
 7. Assume premiums $Adj_{prem} = \$50,000$, FX = 1.0 → final $Rev_{adj} = \$8,756,600$.
 ### 5.2 Gold Doré
 - Ore feed 50,000 t, head grade 2.5 g/t (convert to % mass: 0.00025%), recovery 92%.
 - Reference price $P_{ref} = \$1,900$/oz, convert to per tonne of metal ($1\ \text{oz} = 0.0311035 \text{kg}$).
 - Treat $Q_{metal}$ in kilograms or troy ounces as implementation convenience.
 - No moisture/impurity penalties; add refining charge 1.5% of revenue.
 Implementation note: Provide conversion helpers for precious metals (grams per tonne to ounces, etc.).
 ## 6. Validation & Error Handling
 - Ensure required inputs are provided; raise descriptive validation errors per scenario when data missing.
 - Bound checks for percentage inputs; clamp or reject values outside (0,100].
 - Penalty factors default to zero if not configured; log warnings if impurities exceed supported list.
 - FX rate defaults to 1.0 when scenario currency matches reference currency; enforce positive values.
 ## 7. Data Sources & Configuration
 - Reference prices sourced from market data integration (e.g., LME, LBMA). Provide placeholder configuration until connectors exist.
 - Penalty thresholds and factors configurable per smelter contract; persisted in the `pricing_settings` tables (`pricing_settings`, `pricing_metal_settings`, `pricing_impurity_settings`).
 - The default configuration lives in the `pricing_settings` row with slug `default`. `services.bootstrap.bootstrap_pricing_settings` runs during FastAPI startup (and via `scripts/initial_data.py`) to ensure this row exists and mirrors the configured baseline metadata.
 - Runtime consumers obtain metadata through the FastAPI dependency `dependencies.get_pricing_metadata`, which loads the persisted defaults with impurity overrides. If the requested slug is missing, the dependency reseeds the table from `Settings.pricing_metadata()` before returning a fresh `PricingMetadata` instance.
 - Deployment environments can still influence the initial bootstrap by setting the following environment variables **before the database is seeded**. They are only read when creating or reseeding the `default` record:
 | Environment Variable                        | Default | Bootstrap Usage                                                                 |
 | ------------------------------------------- | ------- | ------------------------------------------------------------------------------- |
 | `CALMINER_PRICING_DEFAULT_PAYABLE_PCT`      | `100.0` | Initial payable percentage stored in the default pricing settings row.          |
 | `CALMINER_PRICING_DEFAULT_CURRENCY`         | `USD`   | Initial currency recorded in the persisted metadata (can be set to `null`).     |
 | `CALMINER_PRICING_MOISTURE_THRESHOLD_PCT`   | `8.0`   | Initial moisture threshold applied when seeding the database.                   |
 | `CALMINER_PRICING_MOISTURE_PENALTY_PER_PCT` | `0.0`   | Initial moisture penalty factor captured in the persisted configuration record. |
 Operators should update the database record (or project-specific overrides) after bootstrap to align with active smelter contracts. Subsequent application restarts reuse the stored values without re-reading environment variables.
 ## 8. Output Schema
 Define structured result for integration:
 ```json
 {
  "metal": "copper",
  "ore_tonnage": 100000,
  "head_grade_pct": 1.2,
  "recovery_pct": 90,
  "payable_metal_tonnes": 1036.8,
  "reference_price": 8500,
  "gross_revenue": 8812800,
  "moisture_penalty": 6000,
  "impurity_penalty": 200,
  "treatment_smelt_charges": 100000,
  "premiums": 50000,
  "net_revenue": 8756600,
  "currency": "USD"
 }
 ```
 ## 9. Dependencies & Next Steps
 - Align with FR-006 (performance monitoring) to collect calculation metrics.
 - Coordinate with reporting requirements to expose inputs/outputs in exports.
 - Next implementation steps: build pricing service module, integrate with scenario evaluation, and author unit tests using example data above.# Variables for Price Calculation
 ## Variables
--- a/specifications/pricing_settings_data_model.md
+++ b/specifications/pricing_settings_data_model.md
@@ -0,0 +1,77 @@
 # Pricing Settings Data Model
 ## Objective
 Persist pricing configuration values that currently live in environment variables so projects can own default payable percentages, currency, and penalty factors without redeploying the application.
 ## Core Entity: `pricing_settings`
 Holds the defaults that are injected into `PricingMetadata` when evaluating scenarios.
 | Column                     | Type                    | Nullable | Default  | Notes                                                                      |
 | -------------------------- | ----------------------- | -------- | -------- | -------------------------------------------------------------------------- |
 | `id`                       | Integer                 | No       | Identity | Primary key                                                                |
 | `name`                     | String(128)             | No       | —        | Human readable label displayed in the UI                                   |
 | `slug`                     | String(64)              | No       | —        | Unique code used for programmatic lookup (e.g. `default`, `contract-a`)    |
 | `description`              | Text                    | Yes      | `NULL`   | Optional descriptive text                                                  |
 | `default_currency`         | String(3)               | Yes      | `NULL`   | Normalised ISO-4217 code; fallback when scenario currency is absent        |
 | `default_payable_pct`      | Numeric(5,2)            | No       | `100.00` | Default payable percentage applied when not supplied with an input         |
 | `moisture_threshold_pct`   | Numeric(5,2)            | No       | `8.00`   | Percentage moisture threshold before penalties apply                       |
 | `moisture_penalty_per_pct` | Numeric(14,4)           | No       | `0.0000` | Currency amount deducted per percentage point above the moisture threshold |
 | `metadata`                 | JSON                    | Yes      | `NULL`   | Future extension bucket (e.g. FX assumptions)                              |
 | `created_at`               | DateTime(timezone=True) | No       | `now()`  | Creation timestamp                                                         |
 | `updated_at`               | DateTime(timezone=True) | No       | `now()`  | Auto updated timestamp                                                     |
 ## Child Entity: `pricing_metal_settings`
 Stores overrides that apply to specific commodities (payable percentage or alternate thresholds).
 | Column                     | Type                    | Nullable | Default  | Notes                                                                          |
 | -------------------------- | ----------------------- | -------- | -------- | ------------------------------------------------------------------------------ |
 | `id`                       | Integer                 | No       | Identity | Primary key                                                                    |
 | `pricing_settings_id`      | Integer (FK)            | No       | —        | References `pricing_settings.id` with cascade delete                           |
 | `metal_code`               | String(32)              | No       | —        | Normalised commodity identifier (e.g. `copper`, `gold`)                        |
 | `payable_pct`              | Numeric(5,2)            | Yes      | `NULL`   | Contractual payable percentage for this metal; overrides parent value when set |
 | `moisture_threshold_pct`   | Numeric(5,2)            | Yes      | `NULL`   | Optional metal specific moisture threshold                                     |
 | `moisture_penalty_per_pct` | Numeric(14,4)           | Yes      | `NULL`   | Optional metal specific penalty factor                                         |
 | `data`                     | JSON                    | Yes      | `NULL`   | Additional metal settings (credits, payable deductions)                        |
 | `created_at`               | DateTime(timezone=True) | No       | `now()`  | Creation timestamp                                                             |
 | `updated_at`               | DateTime(timezone=True) | No       | `now()`  | Auto updated timestamp                                                         |
 `metal_code` should have a unique constraint together with `pricing_settings_id` to prevent duplication.
 ## Child Entity: `pricing_impurity_settings`
 Represents impurity penalty factors and thresholds that are injected into `PricingMetadata.impurity_thresholds` and `PricingMetadata.impurity_penalty_per_ppm`.
 | Column                | Type                    | Nullable | Default  | Notes                                                |
 | --------------------- | ----------------------- | -------- | -------- | ---------------------------------------------------- |
 | `id`                  | Integer                 | No       | Identity | Primary key                                          |
 | `pricing_settings_id` | Integer (FK)            | No       | —        | References `pricing_settings.id` with cascade delete |
 | `impurity_code`       | String(32)              | No       | —        | Identifier such as `As`, `Pb`, `Zn`                  |
 | `threshold_ppm`       | Numeric(14,4)           | No       | `0.0000` | Contractual impurity allowance                       |
 | `penalty_per_ppm`     | Numeric(14,4)           | No       | `0.0000` | Currency penalty applied per ppm above the threshold |
 | `notes`               | Text                    | Yes      | `NULL`   | Optional narrative about the contract rule           |
 | `created_at`          | DateTime(timezone=True) | No       | `now()`  | Creation timestamp                                   |
 | `updated_at`          | DateTime(timezone=True) | No       | `now()`  | Auto updated timestamp                               |
 Add a unique constraint on `(pricing_settings_id, impurity_code)`.
 ## Mapping to `PricingMetadata`
 - `default_currency`, `default_payable_pct`, `moisture_threshold_pct`, and `moisture_penalty_per_pct` map directly to the dataclass fields.
 - `pricing_metal_settings` rows provide per-metal overrides. During load, prefer metal-specific values when present, falling back to the parent record. These values hydrate a composed `PricingMetadata` or supplementary structure passed to the evaluator.
 - `pricing_impurity_settings` rows populate `PricingMetadata.impurity_thresholds` and `PricingMetadata.impurity_penalty_per_ppm` dictionaries.
 ## Usage Notes
 - Global defaults can be represented by a `pricing_settings` row referenced by new projects. Future migrations will add `projects.pricing_settings_id` to point at the desired configuration.
 - The `metadata` JSON column gives room to store additional contract attributes (e.g. minimum lot penalties, premium formulas) without immediate schema churn.
 - Numeric precision follows existing financial models (two decimal places for percentages, four for monetary/ppm penalties) to align with current tests.
 ## Bootstrap & Runtime Loading
 - `services.bootstrap.bootstrap_pricing_settings` ensures a baseline record (slug `default`) exists during FastAPI startup and when running `scripts/initial_data.py`. Initial values come from `config.settings.Settings.pricing_metadata()`, allowing operators to shape the first record via environment variables.
 - When projects lack an explicit configuration, the bootstrap associates them with the default record through `UnitOfWork.set_project_pricing_settings`, guaranteeing every project has pricing metadata.
 - At request time, `dependencies.get_pricing_metadata` loads the persisted defaults using `UnitOfWork.get_pricing_metadata(include_children=True)`. If the slug is missing, it reseeds the default record from the bootstrap metadata before returning a `PricingMetadata` instance.
 - Callers therefore observe a consistent fallback chain: project-specific settings → default database record → freshly seeded defaults derived from environment values.
--- a/specifications/scenario_validation.md
+++ b/specifications/scenario_validation.md
@@ -0,0 +1,44 @@
 # Scenario Validation Rules
 Last updated: 2025-11-09
 This document captures the validation rules that must run before comparing two or more scenarios within the CalMiner application. The rules are enforced inside the scenario service layer and surfaced through the API and HTML workflows.
 ## Comparison Preconditions
 All scenarios included in a comparison request **must** satisfy the following preconditions:
 | Rule                       | Description                                                                                                                                                              | Error Code                   | Client Message                                                         |
 | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------- | ---------------------------------------------------------------------- |
 | Same project               | Every scenario must belong to the same project identifier.                                                                                                               | `SCENARIO_PROJECT_MISMATCH`  | "Selected scenarios do not belong to the same project."                |
 | Active status              | Only scenarios in `draft` or `active` status may be compared. Archived scenarios are rejected.                                                                           | `SCENARIO_STATUS_INVALID`    | "Archived scenarios cannot be compared."                               |
 | Shared currency            | Scenarios must share the same `currency` value when provided.                                                                                                            | `SCENARIO_CURRENCY_MISMATCH` | "Scenarios use different currencies and cannot be compared."           |
 | Timeline coherence         | When both scenarios provide date ranges, they must overlap or touch (inclusive). Non-overlapping ranges raise a conflict.                                                | `SCENARIO_TIMELINE_DISJOINT` | "Scenario timelines do not overlap; adjust the comparison window."     |
 | Primary resource alignment | Scenarios must declare the same `primary_resource` (if set). When one is unset, the non-null value is applied to both. If both are set but differ, the comparison fails. | `SCENARIO_RESOURCE_MISMATCH` | "Scenarios target different primary resources and cannot be compared." |
 ## Error Response Contract
 When a validation rule fails, the service layer raises a `ScenarioValidationError` (to be implemented) containing:
 - a machine-readable `code` from the table above,
 - a human-readable `message` designed for UI display,
 - a list of related `scenario_ids` that triggered the rule (if available).
 The API adapts the error into an HTTP `422 Unprocessable Entity` payload:
 ```jsonc
 {
  "detail": {
    "code": "SCENARIO_PROJECT_MISMATCH",
    "message": "Selected scenarios do not belong to the same project.",
    "scenario_ids": [12, 19]
  }
 }
 ```
 HTML form flows display the `message` as a banner and set the HTTP status to `400`.
 ## Future Considerations
 - Expand rules to cover incompatible simulation parameter configurations once parameter taxonomy stabilises.
 - Provide localized error message variants when the internationalization framework is introduced.
--- a/userguide/README.md
+++ b/userguide/README.md
@@ -1,3 +1,8 @@
 # User Guide
-<!-- TODO: Create a user guide for Calminer. This should include instructions on how to use the various features of the application, along with screenshots and examples where applicable. -->
+CalMiner user-facing documentation is organized by feature area. Start with the guides below and expand the repository as new workflows ship.
 ## Available Guides
 - [Capex Planner](initial_capex_planner.md) — capture upfront capital components, run calculations, and persist snapshots for projects and scenarios.
 - [Opex Planner](opex_planner.md) — manage recurring operational costs, apply escalation/discount assumptions, and store calculation snapshots.
--- a/userguide/data_import_export.md
+++ b/userguide/data_import_export.md
@@ -0,0 +1,99 @@
 # Data Import & Export Templates
 This guide explains how to capture profitability, capex, and opex data for bulk upload or download using the standardized CSV and Excel templates. It builds on the field inventory documented in [data_import_export_field_inventory.md](data_import_export_field_inventory.md) and should be shared with anyone preparing data for the calculation engine.
 ## Template Package
 The import/export toolkit contains two artifacts:
 | File                       | Purpose                                                                                              |
 | -------------------------- | ---------------------------------------------------------------------------------------------------- |
 | `calminer_financials.csv`  | Single-sheet CSV template for quick edits or integrations that generate structured rows dynamically. |
 | `calminer_financials.xlsx` | Excel workbook with dedicated sheets, lookup lists, and validation rules for guided data capture.    |
 Always distribute the templates as a matched set so contributors can choose the format that best suits their workflow.
 ## CSV Workflow
 ### Structure Overview
 The CSV template uses a `record_type` column to multiplex multiple logical tables inside one file. The exact column requirements, validation ranges, and record types are defined in the [Unified CSV Template Specification](data_import_export_field_inventory.md#unified-csv-template-specification). Key expectations:
 - All rows include the scenario code (`scenario_code`), and optionally the project code, to keep capex and opex data aligned with profitability assumptions.
 - Numeric cells must use `.` as the decimal separator and omit thousands separators, currency symbols, or formatting.
 - Boolean values accept `true`/`false`, `yes`/`no`, or `1`/`0`; exporters should emit `true` or `false`.
 ### Authoring Steps
 1. Populate at least one `profitability_input` row per scenario to establish the calculation context.
 2. Add optional `profitability_impurity` rows where penalty modelling is required.
 3. Capture capital costs via `capex_component` rows and configure options with a `capex_parameters` record.
 4. Capture recurring costs via `processing_opex_component` rows. Add a `processing_opex_parameters` row to define escalation settings.
 5. Validate the file against the rules listed in the specification prior to upload. Recommended tooling includes frictionless data checks or schema-enforced pipelines.
 ### Naming & Delivery
 - Use the pattern `calminer_financials_<project-or-scenario-code>_<YYYYMMDD>.csv` when transferring files between teams.
 - Provide a brief change log or summary per file so reviewers understand the scope of updates.
 ## Excel Workflow
 ### Workbook Layout
 The Excel workbook contains structured sheets described in the [Excel Workbook Layout & Validation Rules](data_import_export_field_inventory.md#excel-workbook-layout--validation-rules) section. Highlights:
 - The `Summary` sheet captures metadata (scenario, preparer, notes) that propagates into the downstream sheets via Excel Table references.
 - Each data sheet (`Profitability_Input`, `Capex_Components`, `Processing_Opex`, etc.) is formatted as an Excel Table with frozen headers and data validation to enforce numeric ranges, lookup values, and boolean lists.
 - The `Lookups` sheet holds named ranges for currencies, categories, and frequency values. Keep this sheet hidden to reduce accidental edits.
 ### Data Entry Checklist
 - Confirm the `Summary` sheet lists every scenario included in the workbook before editing dependent sheets.
 - Use the provided dropdowns for category, currency, frequency, and boolean fields to avoid invalid values.
 - When copying data from other workbooks, paste values only to preserve validation rules.
 - Turn on the totals row for component tables to quickly verify aggregate capex and opex amounts.
 ### Exporting From Excel
 When exporting to CSV or ingesting into the API:
 - Ensure the workbook is saved in `.xlsx` format; do not rename the sheets.
 - If a CSV export is required, export each sheet separately and merge using the column headers defined in the CSV specification.
 - Retain the original workbook as your source of truth; conversions to CSV should be treated as transient files for automation pipelines.
 ## Import Pipeline Expectations
 The planned FastAPI import endpoints will perform the following checks:
 - Validate record types, required columns, and numeric bounds according to the schemas described in the field inventory.
 - Verify that referenced project/scenario codes exist and that foreign key relationships (components, parameters) target a single scenario within the file.
 - Normalize casing for category slugs and currency codes to match the calculation services.
 - Reject files that mix multiple projects unless intentionally enabled by configuration.
 Until the endpoints are available, internal teams can use the specification as the contract for pre-validation scripts or interim ETL routines.
 ## Exporting Data
 Export functionality will mirror the same structures:
 - CSV exports will group rows by `record_type`, enabling direct round-tripping with minimal transformation.
 - Excel exports will populate the same workbook sheets, preserving lookup lists and validation to support offline edits.
 Document any local transformations or additional columns introduced during export so that downstream consumers can reconcile the data with the import templates.
 ## Change Management
 - Increment the `import_version` value on the `Summary` sheet (and document the change here) whenever template-breaking updates occur.
 - Update `data_import_export_field_inventory.md` first when adding or removing columns, then revise this guide to explain the user-facing workflow impact.
 - Communicate updates to stakeholders and archive superseded templates to avoid drift.
 ## Stakeholder Review Checklist
 Before finalising the templates for production use, circulate the following package to domain stakeholders (finance, engineering, data operations) and collect their feedback:
 - `calminer_financials.csv` sample populated with representative profitability, capex, and opex data.
 - `calminer_financials.xlsx` workbook showcasing all sheets, dropdowns, and validation rules.
 - Links to documentation: this guide and the [field inventory](data_import_export_field_inventory.md).
 - Summary of outstanding questions or assumptions (e.g., default currencies, additional categories).
 Record the review session outcomes in the project tracker or meeting notes. Once sign-off is received, mark the stakeholder feedback step as complete in `.github/instructions/TODO.md` and proceed with implementation of import/export endpoints.
--- a/userguide/data_import_export_field_inventory.md
+++ b/userguide/data_import_export_field_inventory.md
@@ -0,0 +1,421 @@
 # Data Import/Export Field Inventory
 This inventory captures the current data fields involved in profitability, capex, and opex workflows. It consolidates inputs accepted by the calculation services, derived outputs that should be available for export, and persisted snapshot columns. The goal is to ground the upcoming CSV/Excel template design in authoritative field definitions.
 ## Profitability
 ### Calculation Inputs (`schemas/calculations.py::ProfitabilityCalculationRequest`)
 | Field                      | Type                  | Constraints & Notes                                        |
 | -------------------------- | --------------------- | ---------------------------------------------------------- |
 | `metal`                    | `str`                 | Required; trimmed lowercase; ore/metal identifier.         |
 | `ore_tonnage`              | `PositiveFloat`       | Required; tonnes processed (> 0).                          |
 | `head_grade_pct`           | `float`               | Required; 0 < value ≤ 100.                                 |
 | `recovery_pct`             | `float`               | Required; 0 < value ≤ 100.                                 |
 | `payable_pct`              | `float \| None`       | Optional; 0 < value ≤ 100; overrides metadata default.     |
 | `reference_price`          | `PositiveFloat`       | Required; price per unit in base currency.                 |
 | `treatment_charge`         | `float`               | ≥ 0.                                                       |
 | `smelting_charge`          | `float`               | ≥ 0.                                                       |
 | `moisture_pct`             | `float`               | ≥ 0 and ≤ 100.                                             |
 | `moisture_threshold_pct`   | `float \| None`       | Optional; ≥ 0 and ≤ 100.                                   |
 | `moisture_penalty_per_pct` | `float \| None`       | Optional; penalty per excess moisture %.                   |
 | `premiums`                 | `float`               | Monetary premium adjustments (can be negative).            |
 | `fx_rate`                  | `PositiveFloat`       | Multiplier to convert to scenario currency; defaults to 1. |
 | `currency_code`            | `str \| None`         | Optional ISO 4217 code; uppercased.                        |
 | `opex`                     | `float`               | ≥ 0; feeds cost aggregation.                               |
 | `sustaining_capex`         | `float`               | ≥ 0.                                                       |
 | `capex`                    | `float`               | ≥ 0.                                                       |
 | `discount_rate`            | `float \| None`       | Optional; 0 ≤ value ≤ 100.                                 |
 | `periods`                  | `int`                 | Evaluation periods; 1 ≤ value ≤ 120.                       |
 | `impurities`               | `List[ImpurityInput]` | Optional; see table below.                                 |
 ### Impurity Rows (`schemas/calculations.py::ImpurityInput`)
 | Field       | Type            | Constraints & Notes                                |
 | ----------- | --------------- | -------------------------------------------------- |
 | `name`      | `str`           | Required; trimmed.                                 |
 | `value`     | `float \| None` | Optional; ≥ 0; measured in ppm.                    |
 | `threshold` | `float \| None` | Optional; ≥ 0.                                     |
 | `penalty`   | `float \| None` | Optional; penalty factor applied beyond threshold. |
 ### Derived Outputs (selected)
 | Structure                               | Fields                                                                                                                                                                                                                          |
 | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | `PricingResult` (`services/pricing.py`) | `metal`, `ore_tonnage`, `head_grade_pct`, `recovery_pct`, `payable_metal_tonnes`, `reference_price`, `gross_revenue`, `moisture_penalty`, `impurity_penalty`, `treatment_smelt_charges`, `premiums`, `net_revenue`, `currency`. |
 | `ProfitabilityCosts`                    | `opex_total`, `sustaining_capex_total`, `capex`.                                                                                                                                                                                |
 | `ProfitabilityMetrics`                  | `npv`, `irr`, `payback_period`, `margin`.                                                                                                                                                                                       |
 | `CashFlowEntry`                         | `period`, `revenue`, `opex`, `sustaining_capex`, `net`.                                                                                                                                                                         |
 ### Snapshot Columns (`models/profitability_snapshot.py`)
 | Field                        | Type             | Notes                                             |
 | ---------------------------- | ---------------- | ------------------------------------------------- |
 | `project_id` / `scenario_id` | `int`            | Foreign key; context.                             |
 | `created_by_id`              | `int \| None`    | Author reference.                                 |
 | `calculation_source`         | `str \| None`    | Identifier for calculation run.                   |
 | `calculated_at`              | `datetime`       | Server timestamp.                                 |
 | `currency_code`              | `str \| None`    | ISO code.                                         |
 | `npv`                        | `Numeric(18, 2)` | Net present value.                                |
 | `irr_pct`                    | `Numeric(12, 6)` | Internal rate of return (%).                      |
 | `payback_period_years`       | `Numeric(12, 4)` | Years to payback.                                 |
 | `margin_pct`                 | `Numeric(12, 6)` | Profit margin %.                                  |
 | `revenue_total`              | `Numeric(18, 2)` | Total revenue.                                    |
 | `opex_total`                 | `Numeric(18, 2)` | Linked opex cost.                                 |
 | `sustaining_capex_total`     | `Numeric(18, 2)` | Sustaining capital spend.                         |
 | `capex`                      | `Numeric(18, 2)` | Capex input.                                      |
 | `net_cash_flow_total`        | `Numeric(18, 2)` | Sum of discounted cash flows.                     |
 | `payload`                    | `JSON`           | Serialized detail (cash flow arrays, parameters). |
 | `created_at`, `updated_at`   | `datetime`       | Audit timestamps.                                 |
 ## Capex
 ### Component Inputs (`schemas/calculations.py::CapexComponentInput`)
 | Field        | Type          | Constraints & Notes                             |
 | ------------ | ------------- | ----------------------------------------------- |
 | `id`         | `int \| None` | Optional persistent identifier; ≥ 1.            |
 | `name`       | `str`         | Required; trimmed.                              |
 | `category`   | `str`         | Required; trimmed lowercase; category slug.     |
 | `amount`     | `float`       | Required; ≥ 0; nominal value.                   |
 | `currency`   | `str \| None` | Optional ISO 4217 code; uppercased.             |
 | `spend_year` | `int \| None` | Optional; 0 ≤ value ≤ 120; relative spend year. |
 | `notes`      | `str \| None` | Optional; max length 500.                       |
 ### Capex Parameters & Options
 | Structure                 | Field                      | Type            | Notes                          |
 | ------------------------- | -------------------------- | --------------- | ------------------------------ |
 | `CapexParameters`         | `currency_code`            | `str \| None`   | Optional ISO code; uppercased. |
 |                           | `contingency_pct`          | `float \| None` | 0 ≤ value ≤ 100.               |
 |                           | `discount_rate_pct`        | `float \| None` | 0 ≤ value ≤ 100.               |
 |                           | `evaluation_horizon_years` | `int \| None`   | 1 ≤ value ≤ 100.               |
 | `CapexCalculationOptions` | `persist`                  | `bool`          | Persist snapshot when true.    |
 ### Derived Outputs (`CapexCalculationResult`)
 | Segment                         | Fields                                                                                   |
 | ------------------------------- | ---------------------------------------------------------------------------------------- |
 | Totals (`CapexTotals`)          | `overall`, `contingency_pct`, `contingency_amount`, `with_contingency`, `by_category[]`. |
 | Timeline (`CapexTimelineEntry`) | `year`, `spend`, `cumulative`.                                                           |
 | Echoed Inputs                   | Normalized `components`, resolved `parameters`, `options`, `currency`.                   |
 ### Snapshot Columns (`models/capex_snapshot.py`)
 | Field                        | Type             | Notes                                        |
 | ---------------------------- | ---------------- | -------------------------------------------- |
 | `project_id` / `scenario_id` | `int`            | Foreign key; context.                        |
 | `created_by_id`              | `int \| None`    | Author reference.                            |
 | `calculation_source`         | `str \| None`    | Origin tag.                                  |
 | `calculated_at`              | `datetime`       | Server timestamp.                            |
 | `currency_code`              | `str \| None`    | ISO code.                                    |
 | `total_capex`                | `Numeric(18, 2)` | Sum of component spend.                      |
 | `contingency_pct`            | `Numeric(12, 6)` | Applied contingency %.                       |
 | `contingency_amount`         | `Numeric(18, 2)` | Monetary contingency.                        |
 | `total_with_contingency`     | `Numeric(18, 2)` | All-in capex.                                |
 | `component_count`            | `int \| None`    | Count of components.                         |
 | `payload`                    | `JSON`           | Serialized breakdown (components, timeline). |
 | `created_at`, `updated_at`   | `datetime`       | Audit timestamps.                            |
 ## Opex
 ### Component Inputs (`schemas/calculations.py::OpexComponentInput`)
 | Field          | Type          | Constraints & Notes                                |
 | -------------- | ------------- | -------------------------------------------------- |
 | `id`           | `int \| None` | Optional persistent identifier; ≥ 1.               |
 | `name`         | `str`         | Required; trimmed.                                 |
 | `category`     | `str`         | Required; trimmed lowercase; category slug.        |
 | `unit_cost`    | `float`       | Required; ≥ 0; base currency.                      |
 | `quantity`     | `float`       | Required; ≥ 0.                                     |
 | `frequency`    | `str`         | Required; trimmed lowercase; e.g. annual, monthly. |
 | `currency`     | `str \| None` | Optional ISO 4217 code; uppercased.                |
 | `period_start` | `int \| None` | Optional; ≥ 0; evaluation period index.            |
 | `period_end`   | `int \| None` | Optional; ≥ 0; must be ≥ `period_start`.           |
 | `notes`        | `str \| None` | Optional; max length 500.                          |
 ### Opex Parameters & Options
 | Structure        | Field                      | Type            | Notes                          |
 | ---------------- | -------------------------- | --------------- | ------------------------------ |
 | `OpexParameters` | `currency_code`            | `str \| None`   | Optional ISO code; uppercased. |
 |                  | `escalation_pct`           | `float \| None` | 0 ≤ value ≤ 100.               |
 |                  | `discount_rate_pct`        | `float \| None` | 0 ≤ value ≤ 100.               |
 |                  | `evaluation_horizon_years` | `int \| None`   | 1 ≤ value ≤ 100.               |
 |                  | `apply_escalation`         | `bool`          | Toggle escalation usage.       |
 | `OpexOptions`    | `persist`                  | `bool`          | Persist snapshot when true.    |
 |                  | `snapshot_notes`           | `str \| None`   | Optional; max length 500.      |
 ### Derived Outputs (`OpexCalculationResult`)
 | Segment                        | Fields                                                                  |
 | ------------------------------ | ----------------------------------------------------------------------- |
 | Totals (`OpexTotals`)          | `overall_annual`, `escalated_total`, `escalation_pct`, `by_category[]`. |
 | Timeline (`OpexTimelineEntry`) | `period`, `base_cost`, `escalated_cost`.                                |
 | Metrics (`OpexMetrics`)        | `annual_average`, `cost_per_ton`.                                       |
 | Echoed Inputs                  | Normalized `components`, `parameters`, `options`, resolved `currency`.  |
 ### Snapshot Columns (`models/opex_snapshot.py`)
 | Field                        | Type              | Notes                                                 |
 | ---------------------------- | ----------------- | ----------------------------------------------------- |
 | `project_id` / `scenario_id` | `int`             | Foreign key; context.                                 |
 | `created_by_id`              | `int \| None`     | Author reference.                                     |
 | `calculation_source`         | `str \| None`     | Origin tag.                                           |
 | `calculated_at`              | `datetime`        | Server timestamp.                                     |
 | `currency_code`              | `str \| None`     | ISO code.                                             |
 | `overall_annual`             | `Numeric(18, 2)`  | Annual recurring cost.                                |
 | `escalated_total`            | `Numeric(18, 2)`  | Total cost over horizon with escalation.              |
 | `annual_average`             | `Numeric(18, 2)`  | Average annual spend.                                 |
 | `evaluation_horizon_years`   | `Integer`         | Horizon length.                                       |
 | `escalation_pct`             | `Numeric(12, 6)`  | Escalation %.                                         |
 | `apply_escalation`           | `Boolean`         | Flag used for calculation.                            |
 | `component_count`            | `Integer \| None` | Component count.                                      |
 | `payload`                    | `JSON`            | Serialized breakdown (components, timeline, metrics). |
 | `created_at`, `updated_at`   | `datetime`        | Audit timestamps.                                     |
 ## Unified CSV Template Specification
 ### Overview
 The CSV template consolidates profitability, capex, and opex data into a single file. Each row declares its purpose via a `record_type` flag so that parsers can route data to the appropriate service. The format is UTF-8 with a header row and uses commas as delimiters. Empty string cells are interpreted as `null`. Monetary values must not include currency symbols; decimals use a period.
 ### Shared Columns
 | Column          | Required | Applies To                   | Validation & Notes                                                                |
 | --------------- | -------- | ---------------------------- | --------------------------------------------------------------------------------- |
 | `record_type`   | Yes      | All rows                     | Lowercase slug describing the row (see record types below).                       |
 | `project_code`  | No       | All rows                     | Optional external project identifier; trimmed; max length 64.                     |
 | `scenario_code` | No       | All rows                     | Optional external scenario identifier; trimmed; max length 64.                    |
 | `sequence`      | No       | Component and impurity rows  | Optional positive integer governing ordering in UI exports.                       |
 | `notes`         | No       | Component and parameter rows | Free-form text ≤ 500 chars; trimmed; mirrors request `notes` fields when present. |
 ### Record Types
 #### `profitability_input`
 Single row per scenario capturing the fields required by `ProfitabilityCalculationRequest`.
 | Column                     | Required | Validation Notes                       |
 | -------------------------- | -------- | -------------------------------------- |
 | `metal`                    | Yes      | Lowercase slug; 1–32 chars.            |
 | `ore_tonnage`              | Yes      | Decimal > 0.                           |
 | `head_grade_pct`           | Yes      | Decimal > 0 and ≤ 100.                 |
 | `recovery_pct`             | Yes      | Decimal > 0 and ≤ 100.                 |
 | `payable_pct`              | No       | Decimal > 0 and ≤ 100.                 |
 | `reference_price`          | Yes      | Decimal > 0.                           |
 | `treatment_charge`         | No       | Decimal ≥ 0.                           |
 | `smelting_charge`          | No       | Decimal ≥ 0.                           |
 | `moisture_pct`             | No       | Decimal ≥ 0 and ≤ 100.                 |
 | `moisture_threshold_pct`   | No       | Decimal ≥ 0 and ≤ 100.                 |
 | `moisture_penalty_per_pct` | No       | Decimal; allow negative for credits.   |
 | `premiums`                 | No       | Decimal; allow negative.               |
 | `fx_rate`                  | No       | Decimal > 0; defaults to 1 when blank. |
 | `currency_code`            | No       | ISO 4217 code; uppercase; length 3.    |
 | `opex`                     | No       | Decimal ≥ 0.                           |
 | `sustaining_capex`         | No       | Decimal ≥ 0.                           |
 | `capex`                    | No       | Decimal ≥ 0.                           |
 | `discount_rate`            | No       | Decimal ≥ 0 and ≤ 100.                 |
 | `periods`                  | No       | Integer 1–120.                         |
 #### `profitability_impurity`
 Multiple rows permitted per scenario; maps to `ImpurityInput`.
 | Column      | Required | Validation Notes                      |
 | ----------- | -------- | ------------------------------------- |
 | `name`      | Yes      | Trimmed string; 1–64 chars.           |
 | `value`     | No       | Decimal ≥ 0.                          |
 | `threshold` | No       | Decimal ≥ 0.                          |
 | `penalty`   | No       | Decimal; can be negative for credits. |
 #### `capex_component`
 One row per component feeding `CapexComponentInput`.
 | Column         | Required | Validation Notes                                     |
 | -------------- | -------- | ---------------------------------------------------- |
 | `component_id` | No       | Integer ≥ 1; references existing record for updates. |
 | `name`         | Yes      | Trimmed string; 1–128 chars.                         |
 | `category`     | Yes      | Lowercase slug; matches allowed UI categories.       |
 | `amount`       | Yes      | Decimal ≥ 0.                                         |
 | `currency`     | No       | ISO 4217 code; uppercase; length 3.                  |
 | `spend_year`   | No       | Integer 0–120.                                       |
 #### `capex_parameters`
 At most one row per scenario; populates `CapexParameters` and `CapexCalculationOptions`.
 | Column                     | Required | Validation Notes                                 |
 | -------------------------- | -------- | ------------------------------------------------ |
 | `currency_code`            | No       | ISO 4217 code; uppercase.                        |
 | `contingency_pct`          | No       | Decimal ≥ 0 and ≤ 100.                           |
 | `discount_rate_pct`        | No       | Decimal ≥ 0 and ≤ 100.                           |
 | `evaluation_horizon_years` | No       | Integer 1–100.                                   |
 | `persist_snapshot`         | No       | `true`/`false` case-insensitive; defaults false. |
 #### `opex_component`
 Maps to `OpexComponentInput`; multiple rows allowed.
 | Column         | Required | Validation Notes                                                          |
 | -------------- | -------- | ------------------------------------------------------------------------- |
 | `component_id` | No       | Integer ≥ 1; references existing record for updates.                      |
 | `name`         | Yes      | Trimmed string; 1–128 chars.                                              |
 | `category`     | Yes      | Lowercase slug; matches enum used in UI (e.g. energy, labor).             |
 | `unit_cost`    | Yes      | Decimal ≥ 0.                                                              |
 | `quantity`     | Yes      | Decimal ≥ 0.                                                              |
 | `frequency`    | Yes      | Lowercase slug; allowed values: `annual`, `monthly`, `quarterly`, `once`. |
 | `currency`     | No       | ISO 4217 code; uppercase; length 3.                                       |
 | `period_start` | No       | Integer ≥ 0; must be ≤ `period_end` when provided.                        |
 | `period_end`   | No       | Integer ≥ 0; defaults to `period_start` when blank.                       |
 #### `opex_parameters`
 At most one row per scenario; maps to `OpexParameters` and options.
 | Column                     | Required | Validation Notes                |
 | -------------------------- | -------- | ------------------------------- |
 | `currency_code`            | No       | ISO 4217 code; uppercase.       |
 | `escalation_pct`           | No       | Decimal ≥ 0 and ≤ 100.          |
 | `discount_rate_pct`        | No       | Decimal ≥ 0 and ≤ 100.          |
 | `evaluation_horizon_years` | No       | Integer 1–100.                  |
 | `apply_escalation`         | No       | `true`/`false`; defaults true.  |
 | `persist_snapshot`         | No       | `true`/`false`; defaults false. |
 | `snapshot_notes`           | No       | Free-form text ≤ 500 chars.     |
 ### Validation Rules Summary
 - Parsers must group rows by `project_code` + `scenario_code`; missing codes fall back to payload metadata supplied during import.
 - `record_type` values outside the table must raise a validation error.
 - Component identifiers (`component_id`) are optional for inserts but required to overwrite existing records.
 - Decimal columns should accept up to two fractional places for currency-aligned fields (`amount`, `overall`, etc.) and up to six for percentage columns.
 - Boolean columns accept `true`, `false`, `1`, `0`, `yes`, `no` (case-insensitive); exporters should emit `true`/`false`.
 ## Excel Workbook Layout & Validation Rules
 ### Workbook Structure
 | Sheet Name                 | Purpose                                                                               |
 | -------------------------- | ------------------------------------------------------------------------------------- |
 | `Summary`                  | Capture project/scenario metadata and import scope details.                           |
 | `Profitability_Input`      | Tabular input for `ProfitabilityCalculationRequest`.                                  |
 | `Profitability_Impurities` | Optional impurity rows linked to a scenario.                                          |
 | `Capex_Components`         | Component-level spend records for the capex planner.                                  |
 | `Capex_Parameters`         | Global capex parameters/options for a scenario.                                       |
 | `Opex`                     | Component-level recurring cost records for opex.                                      |
 | `Opex_Parameters`          | Global opex parameters/options for a scenario.                                        |
 | `Lookups`                  | Controlled vocabulary lists consumed by data validation (categories, booleans, etc.). |
 All sheets except `Lookups` start with a frozen header row and are formatted as Excel Tables (e.g., `tbl_profitability`). Tables enforce consistent column names and simplify import parsing.
 ### `Summary` Sheet
 | Column           | Required | Validation & Notes                                                                |
 | ---------------- | -------- | --------------------------------------------------------------------------------- |
 | `import_version` | Yes      | Static text `v1`; used to detect template drift.                                  |
 | `project_code`   | No       | Matches shared `project_code`; max 64 chars; trimmed.                             |
 | `scenario_code`  | Yes      | Identifier tying all sheets together; max 64 chars; duplicates allowed for batch. |
 | `prepared_by`    | No       | Free-form text ≤ 128 chars.                                                       |
 | `prepared_on`    | No       | Excel date; data validation restricts to dates ≥ `TODAY()-365`.                   |
 | `notes`          | No       | Free-form text ≤ 500 chars; carry-over to import metadata.                        |
 ### `Profitability_Input`
 Columns mirror the CSV specification; data validation rules apply per cell:
 - Numeric fields (`ore_tonnage`, `reference_price`, etc.) use decimal validation with explicit min/max aligned to service constraints.
 - Percentage fields (`head_grade_pct`, `discount_rate`) use decimals with bounds (e.g., 0–100). Apply `Data Validation → Decimal` settings.
 - `currency_code` validation references `Lookups!$A:$A` (ISO codes list).
 - Table default rows include scenario code reference via structured formula: `=[@Scenario_Code]` autocompletes when set.
 ### `Profitability_Impurities`
 | Column          | Required | Validation & Notes                                    |
 | --------------- | -------- | ----------------------------------------------------- |
 | `scenario_code` | Yes      | Drop-down referencing `Summary!C:C`; ensures linkage. |
 | `name`          | Yes      | Text ≤ 64 chars; duplicates allowed.                  |
 | `value`         | No       | Decimal ≥ 0.                                          |
 | `threshold`     | No       | Decimal ≥ 0.                                          |
 | `penalty`       | No       | Decimal; allow negatives.                             |
 ### `Capex_Components`
 | Column          | Required | Validation & Notes                                     |
 | --------------- | -------- | ------------------------------------------------------ |
 | `scenario_code` | Yes      | Drop-down referencing `Summary!C:C`.                   |
 | `component_id`  | No       | Whole number ≥ 1; optional when inserting new records. |
 | `name`          | Yes      | Text ≤ 128 chars.                                      |
 | `category`      | Yes      | Drop-down referencing `Lookups!category_values`.       |
 | `amount`        | Yes      | Decimal ≥ 0; formatted as currency (no symbol).        |
 | `currency`      | No       | Drop-down referencing `Lookups!currency_codes`.        |
 | `spend_year`    | No       | Whole number 0–120.                                    |
 | `notes`         | No       | Text ≤ 500 chars.                                      |
 ### `Capex_Parameters`
 Single-row table per scenario with structured references:
 | Column                     | Required | Validation & Notes                                           |
 | -------------------------- | -------- | ------------------------------------------------------------ |
 | `scenario_code`            | Yes      | Drop-down referencing `Summary!C:C`.                         |
 | `currency_code`            | No       | Drop-down referencing `Lookups!currency_codes`.              |
 | `contingency_pct`          | No       | Decimal 0–100 with two decimal places.                       |
 | `discount_rate_pct`        | No       | Decimal 0–100.                                               |
 | `evaluation_horizon_years` | No       | Whole number 1–100.                                          |
 | `persist_snapshot`         | No       | Drop-down referencing `Lookups!boolean_values` (True/False). |
 | `notes`                    | No       | Text ≤ 500 chars; maps to request options metadata.          |
 ### `Opex`
 | Column          | Required | Validation & Notes                                                                                                         |
 | --------------- | -------- | -------------------------------------------------------------------------------------------------------------------------- |
 | `scenario_code` | Yes      | Drop-down referencing `Summary!C:C`.                                                                                       |
 | `component_id`  | No       | Whole number ≥ 1; optional for inserts.                                                                                    |
 | `name`          | Yes      | Text ≤ 128 chars.                                                                                                          |
 | `category`      | Yes      | Drop-down referencing `Lookups!opex_categories`.                                                                           |
 | `unit_cost`     | Yes      | Decimal ≥ 0.                                                                                                               |
 | `quantity`      | Yes      | Decimal ≥ 0.                                                                                                               |
 | `frequency`     | Yes      | Drop-down referencing `Lookups!frequency_values` (annual, monthly, quarterly, once).                                       |
 | `currency`      | No       | Drop-down referencing `Lookups!currency_codes`.                                                                            |
 | `period_start`  | No       | Whole number ≥ 0; additional rule ensures `period_end` ≥ `period_start` via custom formula `=IF(ISBLANK(H2),TRUE,H2<=I2)`. |
 | `period_end`    | No       | Whole number ≥ 0.                                                                                                          |
 | `notes`         | No       | Text ≤ 500 chars.                                                                                                          |
 ### `Opex_Parameters`
 | Column                     | Required | Validation & Notes                              |
 | -------------------------- | -------- | ----------------------------------------------- |
 | `scenario_code`            | Yes      | Drop-down referencing `Summary!C:C`.            |
 | `currency_code`            | No       | Drop-down referencing `Lookups!currency_codes`. |
 | `escalation_pct`           | No       | Decimal 0–100 with up to two decimals.          |
 | `discount_rate_pct`        | No       | Decimal 0–100.                                  |
 | `evaluation_horizon_years` | No       | Whole number 1–100.                             |
 | `apply_escalation`         | No       | Drop-down referencing `Lookups!boolean_values`. |
 | `persist_snapshot`         | No       | Drop-down referencing `Lookups!boolean_values`. |
 | `snapshot_notes`           | No       | Text ≤ 500 chars.                               |
 ### `Lookups` Sheet
 Contains named ranges used by validation rules:
 | Named Range        | Column Contents                                |
 | ------------------ | ---------------------------------------------- |
 | `currency_codes`   | ISO 4217 codes supported by the platform.      |
 | `category_values`  | Allowed capex categories (e.g., engineering).  |
 | `opex_categories`  | Allowed opex categories (e.g., energy, labor). |
 | `frequency_values` | `annual`, `monthly`, `quarterly`, `once`.      |
 | `boolean_values`   | `TRUE`, `FALSE`.                               |
 The sheet is hidden by default to avoid accidental edits. Import logic should bundle the lookup dictionary alongside the workbook to verify user-supplied values.
 ### Additional Validation Guidance
 - Protect header rows to prevent renaming; enable `Allow Users to Edit Ranges` for data sections only.
 - Apply conditional formatting to highlight missing required fields (`ISBLANK`) and out-of-range values.
 - Provide data validation error messages explaining expected ranges to reduce back-and-forth with users.
 - Recommend enabling the Excel Table totals row for quick sanity checks (sum of amounts, counts of components).
 ## Next Use
 The consolidated tables above provide the authoritative field inventory required to draft CSV column layouts and Excel worksheet structures. They also surface validation ranges and metadata that must be preserved during import/export.
--- a/userguide/initial_capex_planner.md
+++ b/userguide/initial_capex_planner.md
@@ -0,0 +1,59 @@
 # Capex Planner
 The Capex Planner helps project teams capture upfront capital requirements, categorize spend, and produce a shareable breakdown that feeds downstream profitability analysis. The feature implements the acceptance criteria described in [FR-013](../requirements/FR-013.md) and persists calculation snapshots for both projects and scenarios when context is provided.
 ## Access Paths
 - **Workspace sidebar**: Navigate to _Workspace → Capex Planner_.
 - **Scenario detail page**: Use the _Capex Planner_ button in the scenario header to open the planner pre-loaded with the selected project and scenario.
 ## Prerequisites
 - Authenticated CalMiner account with at least _viewer_ permissions on the target project/scenario.
 - Baseline pricing settings seeded through the database initializer (the GET route resolves pricing metadata for defaults).
 ## Planner Workflow
 1. **Open the planner** via the sidebar or scenario action button. Query parameters (`project_id`, `scenario_id`) preload metadata for the selected context.
 2. **Review defaults** in the form header: currency code, contingency, discount rate, and evaluation horizon pull from the scenario, project, or system defaults.
 3. **Capture capex components** by adding rows to the components table. Each row records:
   - Category (equipment, infrastructure, land, miscellaneous, etc.)
   - Component name and optional internal identifier
   - Amount and currency (defaults to scenario or project currency)
   - Planned spend year and notes
 4. **Adjust global parameters** in the sidebar panel for contingency percentage, discount rate, and evaluation horizon years.
 5. **Run the calculation** with _Save & Calculate_. The POST request validates the payload, aggregates totals, applies contingency, and produces categorized and time-phased results.
 6. **Review results** in the Capex Summary cards, category table, and timeline table. Optional chart containers are rendered for future visualization enhancements.
 ## Persistence Behaviour
 - When a `project_id` or `scenario_id` is supplied, the POST handler stores a snapshot via `ProjectCapexSnapshot` and/or `ScenarioCapexSnapshot` before rendering results.
 - Snapshots include aggregate totals, contingency metrics, component counts, and the normalized payload, enabling auditing and downstream financial modelling.
 - JSON clients can set `options[persist]=0` to skip persistence in future iterations (HTML form defaults to persist enabled).
 ## API Reference
 | Route                                               | Method | Description                                                                                                                                    |
 | --------------------------------------------------- | ------ | ---------------------------------------------------------------------------------------------------------------------------------------------- |
 | `/calculations/capex` (`calculations.capex_form`)   | GET    | Renders the planner template with defaults for the provided project/scenario context.                                                          |
 | `/calculations/capex` (`calculations.capex_submit`) | POST   | Accepts form or JSON payload matching `CapexCalculationRequest`, returns HTML or JSON results, and persists snapshots when context is present. |
 Key schemas and services:
 - `schemas.calculations.CapexCalculationRequest`, `CapexComponentInput`, `CapexParameters`
 - `services.calculations.calculate_initial_capex`
 - `_persist_capex_snapshots` helper in `routes/calculations.py`
 Refer to `tests/integration/test_capex_calculations.py` for example payloads and persistence assertions.
 ## Troubleshooting
 - **Validation errors**: Field-level messages appear above the components table for row-specific issues and in the global error alert for general problems.
 - **Currency mismatch**: The service enforces a single currency across all components; ensure component rows use the same ISO-4217 code or adjust the default currency.
 - **Timeline gaps**: Rows without a `spend_year` are excluded from the timeline but still contribute to totals.
 ## Related Resources
 - [Capex Planner template](../../calminer/templates/scenarios/capex.html)
 - [Capex snapshot models](../../calminer/models/capex_snapshot.py)
 - [FR-013 Requirement](../requirements/FR-013.md)
--- a/userguide/processing_opex_planner.md
+++ b/userguide/processing_opex_planner.md
@@ -0,0 +1,114 @@
 # Opex Planner
 The Opex Planner captures recurring operational costs, applies escalation/discount assumptions, and persists calculation snapshots for projects and scenarios. It satisfies the Operational Expenditure tooling defined in [FR-014](../requirements/FR-014.md).
 ## Access Paths
 - **Workspace sidebar**: Navigate to _Workspace → Opex Planner_.
 - **Scenario detail header**: Use the _Processing Opex Planner_ button to open the planner pre-loaded with the active project/scenario context.
 ## Prerequisites
 - Authenticated CalMiner account with viewer or higher access to the target project or scenario.
 - Baseline scenario/project metadata (currency, discount rate, primary resource) to seed form defaults when IDs are provided.
 ## Planner Workflow
 1. **Open the planner** via the sidebar or scenario action. Query parameters (`project_id`, `scenario_id`) load project and scenario metadata plus the latest persisted snapshot, when available.
 2. **Review defaults** in the global parameters panel. Currency, discount rate, evaluation horizon, and persist toggle derive from the scenario, project, or system defaults surfaced by `_prepare_opex_context` in `routes/calculations.py`.
 3. **Capture opex components** in the components table. Each row records the details listed in the _Component Fields_ table below. Rows support dynamic add/remove interactions and inline validation messages for missing data.
 4. **Adjust global parameters** such as escalation percentage, discount rate, evaluation horizon, and whether escalation applies to the timeline. Parameter defaults derive from the active scenario, project, or environment configuration.
 5. **Add optional snapshot notes** that persist alongside stored results and appear in the snapshot history UI (planned) and API responses.
 6. **Run the calculation** with _Save & Calculate_. The POST handler validates input via `OpexCalculationRequest`, calls `services.calculations.calculate_opex`, and repopulates the form with normalized data. When validation fails, the page returns a 422 status with component-level alerts.
 7. **Review results** in the summary cards and detailed breakdowns: total annual opex, per-ton cost, category totals, escalated timeline table, and the normalized component listing. Alerts surface validation issues or component-level notices above the table.
 ### Component Fields
 | Field          | Description                                                                 | Notes                                                  |
 | -------------- | --------------------------------------------------------------------------- | ------------------------------------------------------ |
 | `category`     | Cost grouping (`labor`, `materials`, `energy`, `maintenance`, `other`).     | Drives category totals and stacked charts.             |
 | `name`         | Descriptive label for the component.                                        | Displayed in breakdown tables and persisted snapshots. |
 | `unit_cost`    | Cost per unit in the selected currency.                                     | Decimal, >= 0.                                         |
 | `quantity`     | Units consumed per frequency period.                                        | Decimal, >= 0.                                         |
 | `frequency`    | Recurrence cadence (`daily`, `weekly`, `monthly`, `quarterly`, `annually`). | Controls timeline expansion scaling.                   |
 | `currency`     | ISO-4217 currency code.                                                     | Must match parameters currency when persisting.        |
 | `period_start` | First evaluation period (1-indexed) to include the component.               | Optional; defaults to 1.                               |
 | `period_end`   | Final evaluation period to include the component.                           | Optional; defaults to evaluation horizon.              |
 | `notes`        | Free-form text stored with the component.                                   | Optional; hidden by default in HTML form.              |
 ### Global Parameters
 | Parameter                  | Purpose                                                                           |
 | -------------------------- | --------------------------------------------------------------------------------- |
 | `currency_code`            | Normalized currency for totals and timeline.                                      |
 | `escalation_pct`           | Annual escalation applied to eligible components when `apply_escalation` is true. |
 | `discount_rate_pct`        | Discount rate surfaced for downstream profitability workflows.                    |
 | `evaluation_horizon_years` | Number of years to expand the timeline.                                           |
 | `apply_escalation`         | Boolean flag enabling escalation across the timeline.                             |
 | `persist` (options)        | Persists the calculation when project/scenario context is present.                |
 | `snapshot_notes` (options) | Optional metadata attached to stored snapshots.                                   |
 ## Persistence Behaviour
 - When `project_id` or `scenario_id` is supplied and `options[persist]` evaluates true (default for HTML form), snapshots are stored via `ProjectOpexSnapshot` and `ScenarioOpexSnapshot` repositories before rendering the response.
 - Snapshot payloads capture normalized component entries, parameters, escalation settings, calculated totals, and optional notes, enabling historical comparison and downstream profitability inputs.
 - JSON clients can disable persistence by sending `"options": {"persist": false}` or omit identifiers for ad hoc calculations.
 ## API Reference
 | Route                                             | Method | Description                                                                                                                                   |
 | ------------------------------------------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------------- |
 | `/calculations/opex` (`calculations.opex_form`)   | GET    | Renders the planner template with defaults and any latest snapshot context for the supplied project/scenario IDs.                             |
 | `/calculations/opex` (`calculations.opex_submit`) | POST   | Accepts form or JSON payload matching `OpexCalculationRequest`, returns HTML or JSON results, and persists snapshots when context is present. |
 Key schemas and services:
 - `schemas.calculations.OpexComponentInput`, `OpexParameters`, `OpexCalculationRequest`, `OpexCalculationResult`
 - `services.calculations.calculate_opex`
 - `_prepare_opex_context`, `_persist_opex_snapshots`, and related helpers in `routes/calculations.py`
 ### Example JSON Request
 ```json
 {
  "components": [
    {
      "category": "energy",
      "name": "Processing Plant Power",
      "unit_cost": "480.00",
      "quantity": "1.0",
      "frequency": "monthly",
      "currency": "USD",
      "period_start": 1,
      "period_end": 12
    }
  ],
  "parameters": {
    "currency_code": "USD",
    "escalation_pct": "3.0",
    "discount_rate_pct": "8.0",
    "evaluation_horizon_years": 10,
    "apply_escalation": true
  },
  "options": {
    "persist": true,
    "snapshot_notes": "Baseline processing OPEX"
  }
 }
 ```
 The response payload mirrors `OpexCalculationResult`, returning normalized components, aggregated totals, timeline series, and snapshot metadata when persistence is enabled.
 ## Troubleshooting
 - **Validation errors**: The planner surfaces field-level issues above the component table. JSON responses include `errors` and `message` keys mirroring Pydantic validation output.
 - **Currency mismatch**: All component rows must share the same currency. Adjust row currencies or the default currency in the parameters panel to resolve mismatches enforced by the service layer.
 - **Timeline coverage**: Ensure `period_start` and `period_end` fall within the evaluation horizon. Rows outside the horizon are ignored in the timeline though they still influence totals.
 ## Related Resources
 - [Opex planner template](../../calminer/templates/scenarios/opex.html)
 - [Calculations route handlers](../../calminer/routes/calculations.py)
 - [Opex schemas and results](../../calminer/schemas/calculations.py)
 - [Service implementation and tests](../../calminer/services/calculations.py), [tests/services/test_calculations_opex.py](../../calminer/tests/services/test_calculations_opex.py)
 - [Integration coverage](../../calminer/tests/integration/test_opex_calculations.py)
Author	SHA1	Message	Date
zwitschi	f18987ec8b	docs: add Coolify deployment reference and configuration details	2025-11-15 13:21:59 +01:00
zwitschi	4dea0a9ae1	Add detailed SQLAlchemy models, navigation metadata, enumerations, Pydantic schemas, monitoring, and auditing documentation - Introduced SQLAlchemy models for user management, project management, financial inputs, and pricing configuration. - Created navigation metadata tables for sidebar and top-level menus. - Cataloged enumerations used across ORM models and Pydantic schemas. - Documented Pydantic schemas for API request/response validation, including authentication, project, scenario, import, and export schemas. - Added monitoring and auditing tables for performance metrics and import/export logs. - Updated security documentation to reflect changes in data model references.	2025-11-13 20:23:09 +01:00
zwitschi	07e68a553d	Enhance documentation for data model and import/export processes - Updated data model documentation to clarify relationships between projects, scenarios, and profitability calculations. - Introduced a new guide for data import/export templates, detailing CSV and Excel workflows for profitability, capex, and opex data. - Created a comprehensive field inventory for data import/export, outlining input fields, derived outputs, and snapshot columns. - Renamed "Initial Capex Planner" to "Capex Planner" and "Processing Opex Planner" to "Opex Planner" for consistency across user guides. - Adjusted access paths and related resources in user guides to reflect the new naming conventions. - Improved clarity and consistency in descriptions and instructions throughout the user documentation.	2025-11-13 14:10:47 +01:00
zwitschi	fb6be6d84f	feat: add functional requirements for profitability, Monte Carlo simulation, and Capex/Opex management; enhance user guide with planners	2025-11-13 09:20:10 +01:00
zwitschi	d3597bc8c9	feat: enhance data model documentation with detailed entity descriptions and relationships	2025-11-12 18:17:39 +01:00
zwitschi	747f430562	fix: update database initialization and seeding instructions in installation guide	2025-11-12 18:17:20 +01:00
zwitschi	c9ac7195c7	feat: add continuous integration and deployment documentation, including CI stages, local testing, and Kubernetes deployment guidelines	2025-11-12 12:04:25 +01:00
zwitschi	29f16139a3	feat: documentation update - Completed export workflow implementation (query builders, CSV/XLSX serializers, streaming API endpoints, UI modals, automated tests). - Added export modal UI and client script to trigger downloads directly from dashboard. - Documented import/export field mapping and usage guidelines in FR-008. - Updated installation guide with export environment variables, dependencies, and CLI/CI usage instructions.	2025-11-11 18:34:02 +01:00
zwitschi	02906bc960	Add dashboard layout plan and scenario validation rules documentation	2025-11-10 08:27:49 +01:00
zwitschi	f20ac85310	Add import/export file format guidelines to FR-008	2025-11-10 08:27:26 +01:00
zwitschi	e573f7dc7b	Add import/export field mapping for Project and Scenario attributes in FR-008	2025-11-10 08:25:56 +01:00
zwitschi	3d35e9ae60	Enhance security documentation with details on administrative credentials, credential reset process, route guard dependencies, and session management.	2025-11-10 08:25:38 +01:00
zwitschi	5536f2325a	Add instructions for seeding default accounts and roles in installation guide	2025-11-10 08:24:19 +01:00
zwitschi	1337bde936	Add references section to FR-001 for related documentation	2025-11-09 17:25:47 +01:00