allucanget/calminer-docs
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add detailed SQLAlchemy models, navigation metadata, enumerations, Pydantic schemas, monitoring, and auditing documentation

Browse Source 
- Introduced SQLAlchemy models for user management, project management, financial inputs, and pricing configuration.
- Created navigation metadata tables for sidebar and top-level menus.
- Cataloged enumerations used across ORM models and Pydantic schemas.
- Documented Pydantic schemas for API request/response validation, including authentication, project, scenario, import, and export schemas.
- Added monitoring and auditing tables for performance metrics and import/export logs.
- Updated security documentation to reflect changes in data model references.
This commit is contained in:
zwitschi
2025-11-13 20:23:09 +01:00
parent 07e68a553d
commit 4dea0a9ae1
7 changed files with 842 additions and 749 deletions
Download Patch File Download Diff File Expand all files Collapse all files

369
architecture/08_concepts/02_data_model/01_sqlalchemy_models.md Normal file
View File

@@ -0,0 +1,369 @@
# SQLAlchemy Models
This reference describes the primary SQLAlchemy ORM models that underpin Calminer. It mirrors the original hierarchy from `02_data_model.md`, extracted so each domain area can evolve independently. See the [data model overview](../02_data_model.md) for context and navigation.
## User Management
### User
Represents authenticated platform users with optional elevated privileges.
**Table:** `users`
| Attribute | Type | Description |
| ------------- | ------------ | ------------------------- |
| id | Integer (PK) | Primary key |
| email | String(255) | Unique email address |
| username | String(128) | Unique username |
| password_hash | String(255) | Argon2 hashed password |
| is_active | Boolean | Account activation status |
| is_superuser | Boolean | Superuser privileges |
| last_login_at | DateTime | Last login timestamp |
| created_at | DateTime | Creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `role_assignments`: Many-to-many with Role via UserRole
### Role
Defines user roles for role-based access control (RBAC).
**Table:** `roles`
| Attribute | Type | Description |
| ------------ | ------------ | --------------------- |
| id | Integer (PK) | Primary key |
| name | String(64) | Unique role name |
| display_name | String(128) | Human-readable name |
| description | Text | Role description |
| created_at | DateTime | Creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `assignments`: One-to-many with UserRole
- `users`: Many-to-many with User (viewonly)
### UserRole
Association between users and roles with assignment metadata.
**Table:** `user_roles`
| Attribute | Type | Description |
| ---------- | ----------------------- | -------------------- |
| user_id | Integer (FK → users.id) | User foreign key |
| role_id | Integer (FK → roles.id) | Role foreign key |
| granted_at | DateTime | Assignment timestamp |
| granted_by | Integer (FK → users.id) | Granting user |
**Relationships:**
- `user`: Many-to-one with User
- `role`: Many-to-one with Role
- `granted_by_user`: Many-to-one with User
## Project Management
### Project
Top-level mining project grouping multiple scenarios.
**Table:** `projects`
| Attribute | Type | Description |
| ------------------- | ---------------------------------- | --------------------- |
| id | Integer (PK) | Primary key |
| name | String(255) | Unique project name |
| location | String(255) | Project location |
| operation_type | MiningOperationType | Mining operation type |
| description | Text | Project description |
| pricing_settings_id | Integer (FK → pricing_settings.id) | Pricing settings |
| created_at | DateTime | Creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `scenarios`: One-to-many with Scenario
- `pricing_settings`: Many-to-one with PricingSettings
### Scenario
A specific configuration of assumptions for a project.
**Table:** `scenarios`
| Attribute | Type | Description |
| ---------------- | -------------------------- | ------------------------- |
| id | Integer (PK) | Primary key |
| project_id | Integer (FK → projects.id) | Project foreign key |
| name | String(255) | Scenario name |
| description | Text | Scenario description |
| status | ScenarioStatus | Scenario lifecycle status |
| start_date | Date | Scenario start date |
| end_date | Date | Scenario end date |
| discount_rate | Numeric(5,2) | Discount rate percentage |
| currency | String(3) | ISO currency code |
| primary_resource | ResourceType | Primary resource type |
| created_at | DateTime | Creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `project`: Many-to-one with Project
- `financial_inputs`: One-to-many with FinancialInput
- `simulation_parameters`: One-to-many with SimulationParameter
### Projects → Scenarios → Profitability Calculations
Calminer organises feasibility data in a nested hierarchy. A project defines the overarching mining context and exposes a one-to-many `scenarios` collection. Each scenario captures a self-contained assumption set and anchors derived artefacts such as financial inputs, simulation parameters, and profitability snapshots. Profitability calculations execute at the scenario scope; when triggered, the workflow in `services/calculations.py` persists a `ScenarioProfitability` record and can optionally roll results up to project level by creating a `ProjectProfitability` snapshot. Consumers typically surface the most recent metrics via the `latest_profitability` helpers on both ORM models.
| Layer | ORM models | Pydantic schema(s) | Key relationships |
| -------------------------- | ----------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- |
| Project | `models.project.Project` | `schemas.project.ProjectRead` | `Project.scenarios`, `Project.profitability_snapshots`, `Project.latest_profitability` |
| Scenario | `models.scenario.Scenario` | `schemas.scenario.ScenarioRead` | `Scenario.project`, `Scenario.profitability_snapshots`, `Scenario.latest_profitability` |
| Profitability calculations | `models.profitability_snapshot.ProjectProfitability`, `models.profitability_snapshot.ScenarioProfitability` | `schemas.calculations.ProfitabilityCalculationRequest`, `schemas.calculations.ProfitabilityCalculationResult` | Persisted via `services.calculations.calculate_profitability`; aggregates scenario metrics into project snapshots |
Detailed CRUD endpoint behaviour for projects and scenarios is documented in `calminer-docs/api/README.md`.
### FinancialInput
Line-item financial assumption attached to a scenario.
**Table:** `financial_inputs`
| Attribute | Type | Description |
| -------------- | --------------------------- | -------------------------- |
| id | Integer (PK) | Primary key |
| scenario_id | Integer (FK → scenarios.id) | Scenario foreign key |
| name | String(255) | Input name |
| category | FinancialCategory | Financial category |
| cost_bucket | CostBucket | Cost bucket classification |
| amount | Numeric(18,2) | Monetary amount |
| currency | String(3) | ISO currency code |
| effective_date | Date | Effective date |
| notes | Text | Additional notes |
| created_at | DateTime | Creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `scenario`: Many-to-one with Scenario
## Project and Scenario Snapshots
### ProjectCapexSnapshot
Project-level snapshot capturing aggregated initial capital expenditure metrics.
**Table:** `project_capex_snapshots`
| Attribute | Type | Description |
| ---------------------- | --------------------------------- | ------------------------------------------------------- |
| id | Integer (PK) | Primary key |
| project_id | Integer (FK → projects.id) | Associated project |
| created_by_id | Integer (FK → users.id, nullable) | User that triggered persistence |
| calculation_source | String(64), nullable | Originating workflow identifier (UI, API, etc.) |
| calculated_at | DateTime | Timestamp the calculation completed |
| currency_code | String(3), nullable | Currency for totals |
| total_capex | Numeric(18,2), nullable | Aggregated capex before contingency |
| contingency_pct | Numeric(12,6), nullable | Applied contingency percentage |
| contingency_amount | Numeric(18,2), nullable | Monetary contingency amount |
| total_with_contingency | Numeric(18,2), nullable | Capex total after contingency |
| component_count | Integer, nullable | Number of normalized components captured |
| payload | JSON, nullable | Serialized component breakdown and calculation metadata |
| created_at | DateTime | Record creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `project`: Many-to-one with Project
- `created_by`: Many-to-one with User (nullable)
### ScenarioCapexSnapshot
Scenario-level snapshot storing detailed capex results.
**Table:** `scenario_capex_snapshots`
| Attribute | Type | Description |
| ---------------------- | --------------------------------- | ------------------------------------------------------- |
| id | Integer (PK) | Primary key |
| scenario_id | Integer (FK → scenarios.id) | Associated scenario |
| created_by_id | Integer (FK → users.id, nullable) | User that triggered persistence |
| calculation_source | String(64), nullable | Originating workflow identifier |
| calculated_at | DateTime | Timestamp the calculation completed |
| currency_code | String(3), nullable | Currency for totals |
| total_capex | Numeric(18,2), nullable | Aggregated capex before contingency |
| contingency_pct | Numeric(12,6), nullable | Applied contingency percentage |
| contingency_amount | Numeric(18,2), nullable | Monetary contingency amount |
| total_with_contingency | Numeric(18,2), nullable | Capex total after contingency |
| component_count | Integer, nullable | Number of normalized components captured |
| payload | JSON, nullable | Serialized component breakdown and calculation metadata |
| created_at | DateTime | Record creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `scenario`: Many-to-one with Scenario
- `created_by`: Many-to-one with User (nullable)
### ProjectOpexSnapshot
Project-level snapshot persisting recurring opex metrics.
**Table:** `project_opex_snapshots`
| Attribute | Type | Description |
| ------------------------ | --------------------------------- | ------------------------------------------------------- |
| id | Integer (PK) | Primary key |
| project_id | Integer (FK → projects.id) | Associated project |
| created_by_id | Integer (FK → users.id, nullable) | User that triggered persistence |
| calculation_source | String(64), nullable | Originating workflow identifier |
| calculated_at | DateTime | Timestamp the calculation completed |
| currency_code | String(3), nullable | Currency for totals |
| overall_annual | Numeric(18,2), nullable | Total annual opex |
| escalated_total | Numeric(18,2), nullable | Escalated cost across the evaluation horizon |
| annual_average | Numeric(18,2), nullable | Average annual cost over the horizon |
| evaluation_horizon_years | Integer, nullable | Number of years included in the timeline |
| escalation_pct | Numeric(12,6), nullable | Escalation percentage applied |
| apply_escalation | Boolean | Flag indicating whether escalation was applied |
| component_count | Integer, nullable | Number of normalized components captured |
| payload | JSON, nullable | Serialized component breakdown and calculation metadata |
| created_at | DateTime | Record creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `project`: Many-to-one with Project
- `created_by`: Many-to-one with User (nullable)
### ScenarioOpexSnapshot
Scenario-level snapshot persisting recurring opex metrics.
**Table:** `scenario_opex_snapshots`
| Attribute | Type | Description |
| ------------------------ | --------------------------------- | ------------------------------------------------------- |
| id | Integer (PK) | Primary key |
| scenario_id | Integer (FK → scenarios.id) | Associated scenario |
| created_by_id | Integer (FK → users.id, nullable) | User that triggered persistence |
| calculation_source | String(64), nullable | Originating workflow identifier |
| calculated_at | DateTime | Timestamp the calculation completed |
| currency_code | String(3), nullable | Currency for totals |
| overall_annual | Numeric(18,2), nullable | Total annual opex |
| escalated_total | Numeric(18,2), nullable | Escalated cost across the evaluation horizon |
| annual_average | Numeric(18,2), nullable | Average annual cost over the horizon |
| evaluation_horizon_years | Integer, nullable | Number of years included in the timeline |
| escalation_pct | Numeric(12,6), nullable | Escalation percentage applied |
| apply_escalation | Boolean | Flag indicating whether escalation was applied |
| component_count | Integer, nullable | Number of normalized components captured |
| payload | JSON, nullable | Serialized component breakdown and calculation metadata |
| created_at | DateTime | Record creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `scenario`: Many-to-one with Scenario
- `created_by`: Many-to-one with User (nullable)
### SimulationParameter
Probability distribution settings for scenario simulations.
**Table:** `simulation_parameters`
| Attribute | Type | Description |
| ------------------ | --------------------------- | ------------------------ |
| id | Integer (PK) | Primary key |
| scenario_id | Integer (FK → scenarios.id) | Scenario foreign key |
| name | String(255) | Parameter name |
| distribution | DistributionType | Distribution type |
| variable | StochasticVariable | Stochastic variable type |
| resource_type | ResourceType | Resource type |
| mean_value | Numeric(18,4) | Mean value |
| standard_deviation | Numeric(18,4) | Standard deviation |
| minimum_value | Numeric(18,4) | Minimum value |
| maximum_value | Numeric(18,4) | Maximum value |
| unit | String(32) | Unit of measurement |
| configuration | JSON | Additional configuration |
| created_at | DateTime | Creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `scenario`: Many-to-one with Scenario
## Pricing Configuration
### PricingSettings
Persisted pricing defaults applied to scenario evaluations.
**Table:** `pricing_settings`
| Attribute | Type | Description |
| ------------------------ | ------------- | ----------------------------- |
| id | Integer (PK) | Primary key |
| name | String(128) | Unique settings name |
| slug | String(64) | Unique slug identifier |
| description | Text | Settings description |
| default_currency | String(3) | Default ISO currency code |
| default_payable_pct | Numeric(5,2) | Default payable percentage |
| moisture_threshold_pct | Numeric(5,2) | Moisture threshold percentage |
| moisture_penalty_per_pct | Numeric(14,4) | Moisture penalty per percent |
| metadata_payload | JSON | Additional metadata |
| created_at | DateTime | Creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `metal_overrides`: One-to-many with PricingMetalSettings
- `impurity_overrides`: One-to-many with PricingImpuritySettings
- `projects`: One-to-many with Project
### PricingMetalSettings
Contract-specific overrides for a particular metal.
**Table:** `pricing_metal_settings`
| Attribute | Type | Description |
| ------------------------ | ---------------------------------- | ---------------------------- |
| id | Integer (PK) | Primary key |
| pricing_settings_id | Integer (FK → pricing_settings.id) | Pricing settings foreign key |
| metal_code | String(32) | Metal code |
| payable_pct | Numeric(5,2) | Payable percentage |
| moisture_threshold_pct | Numeric(5,2) | Moisture threshold |
| moisture_penalty_per_pct | Numeric(14,4) | Moisture penalty |
| data | JSON | Additional data |
| created_at | DateTime | Creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `pricing_settings`: Many-to-one with PricingSettings
### PricingImpuritySettings
Impurity penalty thresholds associated with pricing settings.
**Table:** `pricing_impurity_settings`
| Attribute | Type | Description |
| ------------------- | ---------------------------------- | ---------------------------- |
| id | Integer (PK) | Primary key |
| pricing_settings_id | Integer (FK → pricing_settings.id) | Pricing settings foreign key |
| impurity_code | String(32) | Impurity code |
| threshold_ppm | Numeric(14,4) | Threshold in ppm |
| penalty_per_ppm | Numeric(14,4) | Penalty per ppm |
| notes | Text | Additional notes |
| created_at | DateTime | Creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `pricing_settings`: Many-to-one with PricingSettings

65
architecture/08_concepts/02_data_model/02_navigation.md Normal file
View File

@@ -0,0 +1,65 @@
# Navigation Metadata
This page details the navigation metadata tables that drive the Calminer sidebar and top-level menus. It was split from `02_data_model.md` to isolate runtime navigation behaviour from the broader ORM catalogue.
## NavigationGroup
Represents a logical grouping of navigation links shown in the UI sidebar or header.
**Table:** `navigation_groups`
| Attribute | Type | Description |
| ------------ | ------------ | ----------------------------------------------- |
| id | Integer (PK) | Primary key |
| slug | String(64) | Unique slug identifier |
| title | String(128) | Display title |
| description | Text | Optional descriptive text |
| match_prefix | String(255) | URL prefix used to auto-expand in UI |
| weighting | Integer | Ordering weight |
| icon_name | String(64) | Optional icon identifier |
| feature_flag | String(64) | Optional feature flag for conditional rendering |
| created_at | DateTime | Creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `links`: One-to-many with NavigationLink
## NavigationLink
Individual navigation entry that may have an associated feature flag or role restriction.
**Table:** `navigation_links`
| Attribute | Type | Description |
| ------------- | ----------------------------------- | ---------------------------------------------------- |
| id | Integer (PK) | Primary key |
| group_id | Integer (FK → navigation_groups.id) | Parent navigation group |
| parent_id | Integer (FK → navigation_links.id) | Optional parent link (for nested menus) |
| slug | String(64) | Unique slug identifier |
| title | String(128) | Display title |
| description | Text | Optional descriptive text |
| href | String(255) | Static fallback URL |
| match_prefix | String(255) | URL prefix for expansion |
| feature_flag | String(64) | Optional feature flag for conditional rendering |
| required_role | String(64) | Optional role required to view the link |
| weighting | Integer | Ordering weight within the parent scope |
| icon_name | String(64) | Optional icon identifier |
| metadata | JSON | Additional configuration (e.g. legacy route aliases) |
| created_at | DateTime | Creation timestamp |
| updated_at | DateTime | Last update timestamp |
**Relationships:**
- `group`: Many-to-one with NavigationGroup
- `parent`: Many-to-one self-reference
- `children`: One-to-many self-reference
## Seeding and Runtime Consumption
- **Seed script:** `scripts/init_db.py` provisions baseline groups and links, including nested scenario calculators and role-gated admin entries.
- **Service layer:** `services/navigation.py` maps ORM entities into response DTOs, resolves contextual URLs for projects and scenarios, and applies feature flag as well as role filters.
- **API exposure:** `routes/navigation.py` serves the `/navigation/sidebar` endpoint, combining seeded data with runtime context to produce navigation trees consumed by the frontend.
- **Testing:** `tests/services/test_navigation_service.py` covers URL resolution and child filtering logic, while `tests/integration/test_navigation_sidebar_calculations.py` verifies scenario calculator entries in the API payload.
Refer to the [navigation service documentation](../../../../api/README.md#navigation) for endpoint-level behaviour.

83
architecture/08_concepts/02_data_model/03_enumerations.md Normal file
View File

@@ -0,0 +1,83 @@
# Enumerations
This page catalogs the enumerations shared by Calminer ORM models and Pydantic schemas. The entries mirror the original content from `02_data_model.md` so enum updates can be maintained independently of the broader data model reference.
## MiningOperationType
Supported mining operation categories.
- `OPEN_PIT`: Open pit mining
- `UNDERGROUND`: Underground mining
- `IN_SITU_LEACH`: In-situ leaching
- `PLACER`: Placer mining
- `QUARRY`: Quarry operations
- `MOUNTAINTOP_REMOVAL`: Mountaintop removal
- `OTHER`: Other operations
## ScenarioStatus
Lifecycle states for project scenarios.
- `DRAFT`: Draft status
- `ACTIVE`: Active status
- `ARCHIVED`: Archived status
## FinancialCategory
Enumeration of cost and revenue classifications.
- `CAPITAL_EXPENDITURE`: Capital expenditures
- `OPERATING_EXPENDITURE`: Operating expenditures
- `REVENUE`: Revenue
- `CONTINGENCY`: Contingency
- `OTHER`: Other
## DistributionType
Supported stochastic distribution families for simulations.
- `NORMAL`: Normal distribution
- `TRIANGULAR`: Triangular distribution
- `UNIFORM`: Uniform distribution
- `LOGNORMAL`: Lognormal distribution
- `CUSTOM`: Custom distribution
## ResourceType
Primary consumables and resources used in mining operations.
- `DIESEL`: Diesel fuel
- `ELECTRICITY`: Electrical power
- `WATER`: Process water
- `EXPLOSIVES`: Blasting agents
- `REAGENTS`: Processing reagents
- `LABOR`: Direct labor
- `EQUIPMENT_HOURS`: Equipment operating hours
- `TAILINGS_CAPACITY`: Tailings storage
## CostBucket
Granular cost buckets aligned with project accounting.
- `CAPITAL_INITIAL`: Initial capital
- `CAPITAL_SUSTAINING`: Sustaining capital
- `OPERATING_FIXED`: Fixed operating costs
- `OPERATING_VARIABLE`: Variable operating costs
- `MAINTENANCE`: Maintenance costs
- `RECLAMATION`: Reclamation costs
- `ROYALTIES`: Royalties
- `GENERAL_ADMIN`: General and administrative
## StochasticVariable
Domain variables that typically require probabilistic modelling.
- `ORE_GRADE`: Ore grade variability
- `RECOVERY_RATE`: Metallurgical recovery
- `METAL_PRICE`: Commodity price
- `OPERATING_COST`: Operating cost per tonne
- `CAPITAL_COST`: Capital cost
- `DISCOUNT_RATE`: Discount rate
- `THROUGHPUT`: Plant throughput
Refer back to the [data model overview](../02_data_model.md) when choosing the appropriate enum for new features.

267
architecture/08_concepts/02_data_model/04_pydantic.md Normal file
View File

@@ -0,0 +1,267 @@
# Pydantic Schemas
This page documents the Pydantic models used for request/response validation across the Calminer API surface. It was extracted from `02_data_model.md` so API-facing contracts can evolve separately from the ORM layer.
## Authentication Schemas (`schemas/auth.py`)
### RegistrationForm
Form model for user registration.
| Field | Type | Validation |
| ---------------- | ---- | ------------------------------- |
| username | str | 3-128 characters |
| email | str | Valid email format, 5-255 chars |
| password | str | 8-256 characters |
| confirm_password | str | Must match password |
### LoginForm
Form model for user login.
| Field | Type | Validation |
| -------- | ---- | ---------------- |
| username | str | 1-255 characters |
| password | str | 1-256 characters |
### PasswordResetRequestForm
Form model for password reset request.
| Field | Type | Validation |
| ----- | ---- | ------------------------------- |
| email | str | Valid email format, 5-255 chars |
### PasswordResetForm
Form model for password reset.
| Field | Type | Validation |
| ---------------- | ---- | ------------------- |
| token | str | Required |
| password | str | 8-256 characters |
| confirm_password | str | Must match password |
## Project Schemas (`schemas/project.py`)
### ProjectBase
Base schema for project data.
| Field | Type | Description |
| -------------- | ------------------- | --------------------- |
| name | str | Project name |
| location | str \| None | Project location |
| operation_type | MiningOperationType | Mining operation type |
| description | str \| None | Project description |
### ProjectCreate
Schema for creating projects (inherits ProjectBase).
### ProjectUpdate
Schema for updating projects.
| Field | Type | Description |
| -------------- | --------------------------- | --------------------- |
| name | str \| None | Project name |
| location | str \| None | Project location |
| operation_type | MiningOperationType \| None | Mining operation type |
| description | str \| None | Project description |
### ProjectRead
Schema for reading projects (inherits ProjectBase).
| Field | Type | Description |
| ---------- | -------- | ------------------ |
| id | int | Project ID |
| created_at | datetime | Creation timestamp |
| updated_at | datetime | Update timestamp |
## Scenario Schemas (`schemas/scenario.py`)
### ScenarioBase
Base schema for scenario data.
| Field | Type | Description |
| ---------------- | -------------------- | -------------------------------- |
| name | str | Scenario name |
| description | str \| None | Scenario description |
| status | ScenarioStatus | Scenario status (default: DRAFT) |
| start_date | date \| None | Start date |
| end_date | date \| None | End date |
| discount_rate | float \| None | Discount rate |
| currency | str \| None | ISO currency code |
| primary_resource | ResourceType \| None | Primary resource |
### ScenarioCreate
Schema for creating scenarios (inherits ScenarioBase).
### ScenarioUpdate
Schema for updating scenarios.
| Field | Type | Description |
| ---------------- | ---------------------- | -------------------- |
| name | str \| None | Scenario name |
| description | str \| None | Scenario description |
| status | ScenarioStatus \| None | Scenario status |
| start_date | date \| None | Start date |
| end_date | date \| None | End date |
| discount_rate | float \| None | Discount rate |
| currency | str \| None | ISO currency code |
| primary_resource | ResourceType \| None | Primary resource |
### ScenarioRead
Schema for reading scenarios (inherits ScenarioBase).
| Field | Type | Description |
| ---------- | -------- | ------------------ |
| id | int | Scenario ID |
| project_id | int | Project ID |
| created_at | datetime | Creation timestamp |
| updated_at | datetime | Update timestamp |
### ScenarioComparisonRequest
Schema for scenario comparison requests.
| Field | Type | Description |
| ------------ | --------- | --------------------------------------- |
| scenario_ids | list[int] | List of scenario IDs (minimum 2 unique) |
### ScenarioComparisonResponse
Schema for scenario comparison responses.
| Field | Type | Description |
| ---------- | ------------------ | --------------------- |
| project_id | int | Project ID |
| scenarios | list[ScenarioRead] | List of scenario data |
## Import Schemas (`schemas/imports.py`)
### ProjectImportRow
Schema for importing project data.
| Field | Type | Description |
| -------------- | ------------------- | ----------------------- |
| name | str | Project name (required) |
| location | str \| None | Project location |
| operation_type | MiningOperationType | Mining operation type |
| description | str \| None | Project description |
| created_at | datetime \| None | Creation timestamp |
| updated_at | datetime \| None | Update timestamp |
### ScenarioImportRow
Schema for importing scenario data.
| Field | Type | Description |
| ---------------- | -------------------- | -------------------------------- |
| project_name | str | Project name (required) |
| name | str | Scenario name (required) |
| status | ScenarioStatus | Scenario status (default: DRAFT) |
| start_date | date \| None | Start date |
| end_date | date \| None | End date |
| discount_rate | float \| None | Discount rate |
| currency | str \| None | ISO currency code |
| primary_resource | ResourceType \| None | Primary resource |
| description | str \| None | Scenario description |
| created_at | datetime \| None | Creation timestamp |
| updated_at | datetime \| None | Update timestamp |
## Export Schemas (`schemas/exports.py`)
### ExportFormat
Enumeration for export formats.
- `CSV`: CSV format
- `XLSX`: Excel format
### BaseExportRequest
Base schema for export requests.
| Field | Type | Description |
| ---------------- | ------------ | --------------------------------- |
| format | ExportFormat | Export format (default: CSV) |
| include_metadata | bool | Include metadata (default: False) |
### ProjectExportRequest
Schema for project export requests (inherits BaseExportRequest).
| Field | Type | Description |
| ------- | ---------------------------- | -------------- |
| filters | ProjectExportFilters \| None | Export filters |
### ScenarioExportRequest
Schema for scenario export requests (inherits BaseExportRequest).
| Field | Type | Description |
| ------- | ----------------------------- | -------------- |
| filters | ScenarioExportFilters \| None | Export filters |
### ExportTicket
Schema for export tickets.
| Field | Type | Description |
| -------- | -------------------------------- | ------------- |
| token | str | Export token |
| format | ExportFormat | Export format |
| resource | Literal["projects", "scenarios"] | Resource type |
### ExportResponse
Schema for export responses.
| Field | Type | Description |
| ------ | ------------ | ------------- |
| ticket | ExportTicket | Export ticket |
## Discrepancies Between Data Models and Pydantic Schemas
### Missing Pydantic Schemas
The following SQLAlchemy models do not have corresponding Pydantic schemas:
- `User` - Authentication handled through forms, not API schemas
- `Role` - Role management not exposed via API
- `UserRole` - User-role associations not exposed via API
- `FinancialInput` - Financial inputs managed through scenario context
- `SimulationParameter` - Simulation parameters managed through scenario context
- `PricingSettings` - Pricing settings managed through project context
- `PricingMetalSettings` - Pricing overrides managed through settings context
- `PricingImpuritySettings` - Pricing overrides managed through settings context
- `PerformanceMetric` - Metrics not exposed via API
- `ImportExportLog` - Audit logs not exposed via API
### Schema Differences
- **Project schemas** include `operation_type` as required enum, while the model allows null but defaults to `OTHER`
- **Scenario schemas** include currency normalization validation not present in the model validator
- **Import schemas** include extensive validation and coercion logic for CSV/Excel data parsing
- **Export schemas** reference filter classes (`ProjectExportFilters`, `ScenarioExportFilters`) not defined in this document
### Additional Validation
Pydantic schemas include additional validation beyond SQLAlchemy model constraints:
- Email format validation in auth forms
- Password confirmation matching
- Currency code normalization and validation
- Date range validation (end_date >= start_date)
- Required field validation for imports
- Enum value coercion with aliases for imports
Refer back to the [data model overview](../02_data_model.md) when aligning ORM entities with these schemas.

41
architecture/08_concepts/02_data_model/05_monitoring.md Normal file
View File

@@ -0,0 +1,41 @@
# Monitoring and Auditing
This page documents the monitoring- and auditing-related tables that were previously embedded in `02_data_model.md`. Separating them keeps operational telemetry isolated from the core project/scenario data model reference.
## PerformanceMetric
Tracks API performance metrics used by the monitoring pipeline.
**Table:** `performance_metrics`
| Attribute | Type | Description |
| ---------------- | ------------ | --------------------- |
| id | Integer (PK) | Primary key |
| timestamp | DateTime | Metric timestamp |
| metric_name | String | Metric name |
| value | Float | Metric value |
| labels | String | JSON string of labels |
| endpoint | String | API endpoint |
| method | String | HTTP method |
| status_code | Integer | HTTP status code |
| duration_seconds | Float | Request duration |
## ImportExportLog
Audit log for import and export operations initiated by users.
**Table:** `import_export_logs`
| Attribute | Type | Description |
| ---------- | ----------------------- | ------------------------------------- |
| id | Integer (PK) | Primary key |
| action | String(32) | Action type (preview, commit, export) |
| dataset | String(32) | Dataset type (projects, scenarios) |
| status | String(16) | Operation status |
| filename | String(255) | File name |
| row_count | Integer | Number of rows |
| detail | Text | Additional details |
| user_id | Integer (FK → users.id) | User foreign key |
| created_at | DateTime | Creation timestamp |
These tables power telemetry dashboards and audit trails but are not exposed via public API endpoints.