Add detailed SQLAlchemy models, navigation metadata, enumerations, Pydantic schemas, monitoring, and auditing documentation
- Introduced SQLAlchemy models for user management, project management, financial inputs, and pricing configuration. - Created navigation metadata tables for sidebar and top-level menus. - Cataloged enumerations used across ORM models and Pydantic schemas. - Documented Pydantic schemas for API request/response validation, including authentication, project, scenario, import, and export schemas. - Added monitoring and auditing tables for performance metrics and import/export logs. - Updated security documentation to reflect changes in data model references.
This commit is contained in:
267
architecture/08_concepts/02_data_model/04_pydantic.md
Normal file
267
architecture/08_concepts/02_data_model/04_pydantic.md
Normal file
@@ -0,0 +1,267 @@
|
||||
# Pydantic Schemas
|
||||
|
||||
This page documents the Pydantic models used for request/response validation across the Calminer API surface. It was extracted from `02_data_model.md` so API-facing contracts can evolve separately from the ORM layer.
|
||||
|
||||
## Authentication Schemas (`schemas/auth.py`)
|
||||
|
||||
### RegistrationForm
|
||||
|
||||
Form model for user registration.
|
||||
|
||||
| Field | Type | Validation |
|
||||
| ---------------- | ---- | ------------------------------- |
|
||||
| username | str | 3-128 characters |
|
||||
| email | str | Valid email format, 5-255 chars |
|
||||
| password | str | 8-256 characters |
|
||||
| confirm_password | str | Must match password |
|
||||
|
||||
### LoginForm
|
||||
|
||||
Form model for user login.
|
||||
|
||||
| Field | Type | Validation |
|
||||
| -------- | ---- | ---------------- |
|
||||
| username | str | 1-255 characters |
|
||||
| password | str | 1-256 characters |
|
||||
|
||||
### PasswordResetRequestForm
|
||||
|
||||
Form model for password reset request.
|
||||
|
||||
| Field | Type | Validation |
|
||||
| ----- | ---- | ------------------------------- |
|
||||
| email | str | Valid email format, 5-255 chars |
|
||||
|
||||
### PasswordResetForm
|
||||
|
||||
Form model for password reset.
|
||||
|
||||
| Field | Type | Validation |
|
||||
| ---------------- | ---- | ------------------- |
|
||||
| token | str | Required |
|
||||
| password | str | 8-256 characters |
|
||||
| confirm_password | str | Must match password |
|
||||
|
||||
## Project Schemas (`schemas/project.py`)
|
||||
|
||||
### ProjectBase
|
||||
|
||||
Base schema for project data.
|
||||
|
||||
| Field | Type | Description |
|
||||
| -------------- | ------------------- | --------------------- |
|
||||
| name | str | Project name |
|
||||
| location | str \| None | Project location |
|
||||
| operation_type | MiningOperationType | Mining operation type |
|
||||
| description | str \| None | Project description |
|
||||
|
||||
### ProjectCreate
|
||||
|
||||
Schema for creating projects (inherits ProjectBase).
|
||||
|
||||
### ProjectUpdate
|
||||
|
||||
Schema for updating projects.
|
||||
|
||||
| Field | Type | Description |
|
||||
| -------------- | --------------------------- | --------------------- |
|
||||
| name | str \| None | Project name |
|
||||
| location | str \| None | Project location |
|
||||
| operation_type | MiningOperationType \| None | Mining operation type |
|
||||
| description | str \| None | Project description |
|
||||
|
||||
### ProjectRead
|
||||
|
||||
Schema for reading projects (inherits ProjectBase).
|
||||
|
||||
| Field | Type | Description |
|
||||
| ---------- | -------- | ------------------ |
|
||||
| id | int | Project ID |
|
||||
| created_at | datetime | Creation timestamp |
|
||||
| updated_at | datetime | Update timestamp |
|
||||
|
||||
## Scenario Schemas (`schemas/scenario.py`)
|
||||
|
||||
### ScenarioBase
|
||||
|
||||
Base schema for scenario data.
|
||||
|
||||
| Field | Type | Description |
|
||||
| ---------------- | -------------------- | -------------------------------- |
|
||||
| name | str | Scenario name |
|
||||
| description | str \| None | Scenario description |
|
||||
| status | ScenarioStatus | Scenario status (default: DRAFT) |
|
||||
| start_date | date \| None | Start date |
|
||||
| end_date | date \| None | End date |
|
||||
| discount_rate | float \| None | Discount rate |
|
||||
| currency | str \| None | ISO currency code |
|
||||
| primary_resource | ResourceType \| None | Primary resource |
|
||||
|
||||
### ScenarioCreate
|
||||
|
||||
Schema for creating scenarios (inherits ScenarioBase).
|
||||
|
||||
### ScenarioUpdate
|
||||
|
||||
Schema for updating scenarios.
|
||||
|
||||
| Field | Type | Description |
|
||||
| ---------------- | ---------------------- | -------------------- |
|
||||
| name | str \| None | Scenario name |
|
||||
| description | str \| None | Scenario description |
|
||||
| status | ScenarioStatus \| None | Scenario status |
|
||||
| start_date | date \| None | Start date |
|
||||
| end_date | date \| None | End date |
|
||||
| discount_rate | float \| None | Discount rate |
|
||||
| currency | str \| None | ISO currency code |
|
||||
| primary_resource | ResourceType \| None | Primary resource |
|
||||
|
||||
### ScenarioRead
|
||||
|
||||
Schema for reading scenarios (inherits ScenarioBase).
|
||||
|
||||
| Field | Type | Description |
|
||||
| ---------- | -------- | ------------------ |
|
||||
| id | int | Scenario ID |
|
||||
| project_id | int | Project ID |
|
||||
| created_at | datetime | Creation timestamp |
|
||||
| updated_at | datetime | Update timestamp |
|
||||
|
||||
### ScenarioComparisonRequest
|
||||
|
||||
Schema for scenario comparison requests.
|
||||
|
||||
| Field | Type | Description |
|
||||
| ------------ | --------- | --------------------------------------- |
|
||||
| scenario_ids | list[int] | List of scenario IDs (minimum 2 unique) |
|
||||
|
||||
### ScenarioComparisonResponse
|
||||
|
||||
Schema for scenario comparison responses.
|
||||
|
||||
| Field | Type | Description |
|
||||
| ---------- | ------------------ | --------------------- |
|
||||
| project_id | int | Project ID |
|
||||
| scenarios | list[ScenarioRead] | List of scenario data |
|
||||
|
||||
## Import Schemas (`schemas/imports.py`)
|
||||
|
||||
### ProjectImportRow
|
||||
|
||||
Schema for importing project data.
|
||||
|
||||
| Field | Type | Description |
|
||||
| -------------- | ------------------- | ----------------------- |
|
||||
| name | str | Project name (required) |
|
||||
| location | str \| None | Project location |
|
||||
| operation_type | MiningOperationType | Mining operation type |
|
||||
| description | str \| None | Project description |
|
||||
| created_at | datetime \| None | Creation timestamp |
|
||||
| updated_at | datetime \| None | Update timestamp |
|
||||
|
||||
### ScenarioImportRow
|
||||
|
||||
Schema for importing scenario data.
|
||||
|
||||
| Field | Type | Description |
|
||||
| ---------------- | -------------------- | -------------------------------- |
|
||||
| project_name | str | Project name (required) |
|
||||
| name | str | Scenario name (required) |
|
||||
| status | ScenarioStatus | Scenario status (default: DRAFT) |
|
||||
| start_date | date \| None | Start date |
|
||||
| end_date | date \| None | End date |
|
||||
| discount_rate | float \| None | Discount rate |
|
||||
| currency | str \| None | ISO currency code |
|
||||
| primary_resource | ResourceType \| None | Primary resource |
|
||||
| description | str \| None | Scenario description |
|
||||
| created_at | datetime \| None | Creation timestamp |
|
||||
| updated_at | datetime \| None | Update timestamp |
|
||||
|
||||
## Export Schemas (`schemas/exports.py`)
|
||||
|
||||
### ExportFormat
|
||||
|
||||
Enumeration for export formats.
|
||||
|
||||
- `CSV`: CSV format
|
||||
- `XLSX`: Excel format
|
||||
|
||||
### BaseExportRequest
|
||||
|
||||
Base schema for export requests.
|
||||
|
||||
| Field | Type | Description |
|
||||
| ---------------- | ------------ | --------------------------------- |
|
||||
| format | ExportFormat | Export format (default: CSV) |
|
||||
| include_metadata | bool | Include metadata (default: False) |
|
||||
|
||||
### ProjectExportRequest
|
||||
|
||||
Schema for project export requests (inherits BaseExportRequest).
|
||||
|
||||
| Field | Type | Description |
|
||||
| ------- | ---------------------------- | -------------- |
|
||||
| filters | ProjectExportFilters \| None | Export filters |
|
||||
|
||||
### ScenarioExportRequest
|
||||
|
||||
Schema for scenario export requests (inherits BaseExportRequest).
|
||||
|
||||
| Field | Type | Description |
|
||||
| ------- | ----------------------------- | -------------- |
|
||||
| filters | ScenarioExportFilters \| None | Export filters |
|
||||
|
||||
### ExportTicket
|
||||
|
||||
Schema for export tickets.
|
||||
|
||||
| Field | Type | Description |
|
||||
| -------- | -------------------------------- | ------------- |
|
||||
| token | str | Export token |
|
||||
| format | ExportFormat | Export format |
|
||||
| resource | Literal["projects", "scenarios"] | Resource type |
|
||||
|
||||
### ExportResponse
|
||||
|
||||
Schema for export responses.
|
||||
|
||||
| Field | Type | Description |
|
||||
| ------ | ------------ | ------------- |
|
||||
| ticket | ExportTicket | Export ticket |
|
||||
|
||||
## Discrepancies Between Data Models and Pydantic Schemas
|
||||
|
||||
### Missing Pydantic Schemas
|
||||
|
||||
The following SQLAlchemy models do not have corresponding Pydantic schemas:
|
||||
|
||||
- `User` - Authentication handled through forms, not API schemas
|
||||
- `Role` - Role management not exposed via API
|
||||
- `UserRole` - User-role associations not exposed via API
|
||||
- `FinancialInput` - Financial inputs managed through scenario context
|
||||
- `SimulationParameter` - Simulation parameters managed through scenario context
|
||||
- `PricingSettings` - Pricing settings managed through project context
|
||||
- `PricingMetalSettings` - Pricing overrides managed through settings context
|
||||
- `PricingImpuritySettings` - Pricing overrides managed through settings context
|
||||
- `PerformanceMetric` - Metrics not exposed via API
|
||||
- `ImportExportLog` - Audit logs not exposed via API
|
||||
|
||||
### Schema Differences
|
||||
|
||||
- **Project schemas** include `operation_type` as required enum, while the model allows null but defaults to `OTHER`
|
||||
- **Scenario schemas** include currency normalization validation not present in the model validator
|
||||
- **Import schemas** include extensive validation and coercion logic for CSV/Excel data parsing
|
||||
- **Export schemas** reference filter classes (`ProjectExportFilters`, `ScenarioExportFilters`) not defined in this document
|
||||
|
||||
### Additional Validation
|
||||
|
||||
Pydantic schemas include additional validation beyond SQLAlchemy model constraints:
|
||||
|
||||
- Email format validation in auth forms
|
||||
- Password confirmation matching
|
||||
- Currency code normalization and validation
|
||||
- Date range validation (end_date >= start_date)
|
||||
- Required field validation for imports
|
||||
- Enum value coercion with aliases for imports
|
||||
|
||||
Refer back to the [data model overview](../02_data_model.md) when aligning ORM entities with these schemas.
|
||||
Reference in New Issue
Block a user