allucanget/calminer-docs
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add detailed SQLAlchemy models, navigation metadata, enumerations, Pydantic schemas, monitoring, and auditing documentation

Browse Source 
- Introduced SQLAlchemy models for user management, project management, financial inputs, and pricing configuration.
- Created navigation metadata tables for sidebar and top-level menus.
- Cataloged enumerations used across ORM models and Pydantic schemas.
- Documented Pydantic schemas for API request/response validation, including authentication, project, scenario, import, and export schemas.
- Added monitoring and auditing tables for performance metrics and import/export logs.
- Updated security documentation to reflect changes in data model references.
This commit is contained in:
zwitschi
2025-11-13 20:23:09 +01:00
parent 07e68a553d
commit 4dea0a9ae1
7 changed files with 842 additions and 749 deletions
Download Patch File Download Diff File Expand all files Collapse all files

369
architecture/08_concepts/02_data_model/01_sqlalchemy_models.md Normal file
View File

@@ -0,0 +1,369 @@
# SQLAlchemy Models
This reference describes the primary SQLAlchemy ORM models that underpin Calminer. It mirrors the original hierarchy from `02_data_model.md`, extracted so each domain area can evolve independently. See the [data model overview](../02_data_model.md) for context and navigation.
## User Management
### User
Represents authenticated platform users with optional elevated privileges.
**Table:** `users`
| Attribute | Type | Description |
| ------------- | ------------ | ------------------------- |
| id | Integer (PK) | Primary key |
| email | String(255) | Unique email address |
| username | String(128) | Unique username |
| password_hash | String(255) | Argon2 hashed password |
| is_active | Boolean | Account activation status |
| is_superuser | Boolean | Superuser privileges |
| last_login_at | DateTime | Last login timestamp |
| created_at | DateTime | Creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `role_assignments`: Many-to-many with Role via UserRole
### Role
Defines user roles for role-based access control (RBAC).
**Table:** `roles`
| Attribute | Type | Description |
| ------------ | ------------ | --------------------- |
| id | Integer (PK) | Primary key |
| name | String(64) | Unique role name |
| display_name | String(128) | Human-readable name |
| description | Text | Role description |
| created_at | DateTime | Creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `assignments`: One-to-many with UserRole
- `users`: Many-to-many with User (viewonly)
### UserRole
Association between users and roles with assignment metadata.
**Table:** `user_roles`
| Attribute | Type | Description |
| ---------- | ----------------------- | -------------------- |
| user_id | Integer (FK → users.id) | User foreign key |
| role_id | Integer (FK → roles.id) | Role foreign key |
| granted_at | DateTime | Assignment timestamp |
| granted_by | Integer (FK → users.id) | Granting user |
**Relationships:**
- `user`: Many-to-one with User
- `role`: Many-to-one with Role
- `granted_by_user`: Many-to-one with User
## Project Management
### Project
Top-level mining project grouping multiple scenarios.
**Table:** `projects`
| Attribute | Type | Description |
| ------------------- | ---------------------------------- | --------------------- |
| id | Integer (PK) | Primary key |
| name | String(255) | Unique project name |
| location | String(255) | Project location |
| operation_type | MiningOperationType | Mining operation type |
| description | Text | Project description |
| pricing_settings_id | Integer (FK → pricing_settings.id) | Pricing settings |
| created_at | DateTime | Creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `scenarios`: One-to-many with Scenario
- `pricing_settings`: Many-to-one with PricingSettings
### Scenario
A specific configuration of assumptions for a project.
**Table:** `scenarios`
| Attribute | Type | Description |
| ---------------- | -------------------------- | ------------------------- |
| id | Integer (PK) | Primary key |
| project_id | Integer (FK → projects.id) | Project foreign key |
| name | String(255) | Scenario name |
| description | Text | Scenario description |
| status | ScenarioStatus | Scenario lifecycle status |
| start_date | Date | Scenario start date |
| end_date | Date | Scenario end date |
| discount_rate | Numeric(5,2) | Discount rate percentage |
| currency | String(3) | ISO currency code |
| primary_resource | ResourceType | Primary resource type |
| created_at | DateTime | Creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `project`: Many-to-one with Project
- `financial_inputs`: One-to-many with FinancialInput
- `simulation_parameters`: One-to-many with SimulationParameter
### Projects → Scenarios → Profitability Calculations
Calminer organises feasibility data in a nested hierarchy. A project defines the overarching mining context and exposes a one-to-many `scenarios` collection. Each scenario captures a self-contained assumption set and anchors derived artefacts such as financial inputs, simulation parameters, and profitability snapshots. Profitability calculations execute at the scenario scope; when triggered, the workflow in `services/calculations.py` persists a `ScenarioProfitability` record and can optionally roll results up to project level by creating a `ProjectProfitability` snapshot. Consumers typically surface the most recent metrics via the `latest_profitability` helpers on both ORM models.
| Layer | ORM models | Pydantic schema(s) | Key relationships |
| -------------------------- | ----------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- |
| Project | `models.project.Project` | `schemas.project.ProjectRead` | `Project.scenarios`, `Project.profitability_snapshots`, `Project.latest_profitability` |
| Scenario | `models.scenario.Scenario` | `schemas.scenario.ScenarioRead` | `Scenario.project`, `Scenario.profitability_snapshots`, `Scenario.latest_profitability` |
| Profitability calculations | `models.profitability_snapshot.ProjectProfitability`, `models.profitability_snapshot.ScenarioProfitability` | `schemas.calculations.ProfitabilityCalculationRequest`, `schemas.calculations.ProfitabilityCalculationResult` | Persisted via `services.calculations.calculate_profitability`; aggregates scenario metrics into project snapshots |
Detailed CRUD endpoint behaviour for projects and scenarios is documented in `calminer-docs/api/README.md`.
### FinancialInput
Line-item financial assumption attached to a scenario.
**Table:** `financial_inputs`
| Attribute | Type | Description |
| -------------- | --------------------------- | -------------------------- |
| id | Integer (PK) | Primary key |
| scenario_id | Integer (FK → scenarios.id) | Scenario foreign key |
| name | String(255) | Input name |
| category | FinancialCategory | Financial category |
| cost_bucket | CostBucket | Cost bucket classification |
| amount | Numeric(18,2) | Monetary amount |
| currency | String(3) | ISO currency code |
| effective_date | Date | Effective date |
| notes | Text | Additional notes |
| created_at | DateTime | Creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `scenario`: Many-to-one with Scenario
## Project and Scenario Snapshots
### ProjectCapexSnapshot
Project-level snapshot capturing aggregated initial capital expenditure metrics.
**Table:** `project_capex_snapshots`
| Attribute | Type | Description |
| ---------------------- | --------------------------------- | ------------------------------------------------------- |
| id | Integer (PK) | Primary key |
| project_id | Integer (FK → projects.id) | Associated project |
| created_by_id | Integer (FK → users.id, nullable) | User that triggered persistence |
| calculation_source | String(64), nullable | Originating workflow identifier (UI, API, etc.) |
| calculated_at | DateTime | Timestamp the calculation completed |
| currency_code | String(3), nullable | Currency for totals |
| total_capex | Numeric(18,2), nullable | Aggregated capex before contingency |
| contingency_pct | Numeric(12,6), nullable | Applied contingency percentage |
| contingency_amount | Numeric(18,2), nullable | Monetary contingency amount |
| total_with_contingency | Numeric(18,2), nullable | Capex total after contingency |
| component_count | Integer, nullable | Number of normalized components captured |
| payload | JSON, nullable | Serialized component breakdown and calculation metadata |
| created_at | DateTime | Record creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `project`: Many-to-one with Project
- `created_by`: Many-to-one with User (nullable)
### ScenarioCapexSnapshot
Scenario-level snapshot storing detailed capex results.
**Table:** `scenario_capex_snapshots`
| Attribute | Type | Description |
| ---------------------- | --------------------------------- | ------------------------------------------------------- |
| id | Integer (PK) | Primary key |
| scenario_id | Integer (FK → scenarios.id) | Associated scenario |
| created_by_id | Integer (FK → users.id, nullable) | User that triggered persistence |
| calculation_source | String(64), nullable | Originating workflow identifier |
| calculated_at | DateTime | Timestamp the calculation completed |
| currency_code | String(3), nullable | Currency for totals |
| total_capex | Numeric(18,2), nullable | Aggregated capex before contingency |
| contingency_pct | Numeric(12,6), nullable | Applied contingency percentage |
| contingency_amount | Numeric(18,2), nullable | Monetary contingency amount |
| total_with_contingency | Numeric(18,2), nullable | Capex total after contingency |
| component_count | Integer, nullable | Number of normalized components captured |
| payload | JSON, nullable | Serialized component breakdown and calculation metadata |
| created_at | DateTime | Record creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `scenario`: Many-to-one with Scenario
- `created_by`: Many-to-one with User (nullable)
### ProjectOpexSnapshot
Project-level snapshot persisting recurring opex metrics.
**Table:** `project_opex_snapshots`
| Attribute | Type | Description |
| ------------------------ | --------------------------------- | ------------------------------------------------------- |
| id | Integer (PK) | Primary key |
| project_id | Integer (FK → projects.id) | Associated project |
| created_by_id | Integer (FK → users.id, nullable) | User that triggered persistence |
| calculation_source | String(64), nullable | Originating workflow identifier |
| calculated_at | DateTime | Timestamp the calculation completed |
| currency_code | String(3), nullable | Currency for totals |
| overall_annual | Numeric(18,2), nullable | Total annual opex |
| escalated_total | Numeric(18,2), nullable | Escalated cost across the evaluation horizon |
| annual_average | Numeric(18,2), nullable | Average annual cost over the horizon |
| evaluation_horizon_years | Integer, nullable | Number of years included in the timeline |
| escalation_pct | Numeric(12,6), nullable | Escalation percentage applied |
| apply_escalation | Boolean | Flag indicating whether escalation was applied |
| component_count | Integer, nullable | Number of normalized components captured |
| payload | JSON, nullable | Serialized component breakdown and calculation metadata |
| created_at | DateTime | Record creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `project`: Many-to-one with Project
- `created_by`: Many-to-one with User (nullable)
### ScenarioOpexSnapshot
Scenario-level snapshot persisting recurring opex metrics.
**Table:** `scenario_opex_snapshots`
| Attribute | Type | Description |
| ------------------------ | --------------------------------- | ------------------------------------------------------- |
| id | Integer (PK) | Primary key |
| scenario_id | Integer (FK → scenarios.id) | Associated scenario |
| created_by_id | Integer (FK → users.id, nullable) | User that triggered persistence |
| calculation_source | String(64), nullable | Originating workflow identifier |
| calculated_at | DateTime | Timestamp the calculation completed |
| currency_code | String(3), nullable | Currency for totals |
| overall_annual | Numeric(18,2), nullable | Total annual opex |
| escalated_total | Numeric(18,2), nullable | Escalated cost across the evaluation horizon |
| annual_average | Numeric(18,2), nullable | Average annual cost over the horizon |
| evaluation_horizon_years | Integer, nullable | Number of years included in the timeline |
| escalation_pct | Numeric(12,6), nullable | Escalation percentage applied |
| apply_escalation | Boolean | Flag indicating whether escalation was applied |
| component_count | Integer, nullable | Number of normalized components captured |
| payload | JSON, nullable | Serialized component breakdown and calculation metadata |
| created_at | DateTime | Record creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `scenario`: Many-to-one with Scenario
- `created_by`: Many-to-one with User (nullable)
### SimulationParameter
Probability distribution settings for scenario simulations.
**Table:** `simulation_parameters`
| Attribute | Type | Description |
| ------------------ | --------------------------- | ------------------------ |
| id | Integer (PK) | Primary key |
| scenario_id | Integer (FK → scenarios.id) | Scenario foreign key |
| name | String(255) | Parameter name |
| distribution | DistributionType | Distribution type |
| variable | StochasticVariable | Stochastic variable type |
| resource_type | ResourceType | Resource type |
| mean_value | Numeric(18,4) | Mean value |
| standard_deviation | Numeric(18,4) | Standard deviation |
| minimum_value | Numeric(18,4) | Minimum value |
| maximum_value | Numeric(18,4) | Maximum value |
| unit | String(32) | Unit of measurement |
| configuration | JSON | Additional configuration |
| created_at | DateTime | Creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `scenario`: Many-to-one with Scenario
## Pricing Configuration
### PricingSettings
Persisted pricing defaults applied to scenario evaluations.
**Table:** `pricing_settings`
| Attribute | Type | Description |
| ------------------------ | ------------- | ----------------------------- |
| id | Integer (PK) | Primary key |
| name | String(128) | Unique settings name |
| slug | String(64) | Unique slug identifier |
| description | Text | Settings description |
| default_currency | String(3) | Default ISO currency code |
| default_payable_pct | Numeric(5,2) | Default payable percentage |
| moisture_threshold_pct | Numeric(5,2) | Moisture threshold percentage |
| moisture_penalty_per_pct | Numeric(14,4) | Moisture penalty per percent |
| metadata_payload | JSON | Additional metadata |
| created_at | DateTime | Creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `metal_overrides`: One-to-many with PricingMetalSettings
- `impurity_overrides`: One-to-many with PricingImpuritySettings
- `projects`: One-to-many with Project
### PricingMetalSettings
Contract-specific overrides for a particular metal.
**Table:** `pricing_metal_settings`
| Attribute | Type | Description |
| ------------------------ | ---------------------------------- | ---------------------------- |
| id | Integer (PK) | Primary key |
| pricing_settings_id | Integer (FK → pricing_settings.id) | Pricing settings foreign key |
| metal_code | String(32) | Metal code |
| payable_pct | Numeric(5,2) | Payable percentage |
| moisture_threshold_pct | Numeric(5,2) | Moisture threshold |
| moisture_penalty_per_pct | Numeric(14,4) | Moisture penalty |
| data | JSON | Additional data |
| created_at | DateTime | Creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `pricing_settings`: Many-to-one with PricingSettings
### PricingImpuritySettings
Impurity penalty thresholds associated with pricing settings.
**Table:** `pricing_impurity_settings`
| Attribute | Type | Description |
| ------------------- | ---------------------------------- | ---------------------------- |
| id | Integer (PK) | Primary key |
| pricing_settings_id | Integer (FK → pricing_settings.id) | Pricing settings foreign key |
| impurity_code | String(32) | Impurity code |
| threshold_ppm | Numeric(14,4) | Threshold in ppm |
| penalty_per_ppm | Numeric(14,4) | Penalty per ppm |
| notes | Text | Additional notes |
| created_at | DateTime | Creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `pricing_settings`: Many-to-one with PricingSettings