allucanget/calminer
2
0
Fork 0
Code Pull Requests Actions Packages Releases Activity

refactor: Enhance architecture documentation with detailed sections on purpose, constraints, runtime view, deployment, and key concepts; add implementation plan and update quickstart reference

Browse Source
This commit is contained in:
zwitschi
2025-10-23 16:59:15 +02:00
parent 76f92f8a7d
commit 8aee7b0d74
11 changed files with 634 additions and 72 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@@ -26,7 +26,9 @@ A range of features are implemented to support these functionalities.
## Documentation & quickstart ## Documentation & quickstart
This repository contains detailed developer and architecture documentation in the `docs/` folder. For a short quickstart, troubleshooting notes, migration/backfill instructions and the current implementation status, see [quickstart](docs/quickstart.md). This repository contains detailed developer and architecture documentation in the `docs/` folder.
[Quickstart](docs/quickstart.md) contains developer quickstart, migrations, testing and current status.
Key architecture documents: see [architecture](docs/architecture/README.md) for the arc42-based architecture documentation. Key architecture documents: see [architecture](docs/architecture/README.md) for the arc42-based architecture documentation.

26
docs/architecture/01_introduction_and_goals.md
View File

@@ -1,12 +1,30 @@
---
title: "01 — Introduction and Goals"
description: "System purpose, stakeholders, and high-level goals; project introduction and business/technical goals."
status: draft
---
# 01 — Introduction and Goals # 01 — Introduction and Goals
Status: skeleton ## Purpose
Describe the system purpose, stakeholders, and high-level goals. Fill this file with project introduction and business/technical goals. CalMiner aims to provide a comprehensive platform for mining project scenario analysis, enabling stakeholders to make informed decisions based on data-driven insights.
## Overview ## Stakeholders
CalMiner is a FastAPI application that collects mining project inputs, persists scenario-specific records, and surfaces aggregated insights. The platform targets Monte Carlo driven planning, with deterministic CRUD features in place and simulation logic staged for future work. - **Project Managers**: Require tools for scenario planning and risk assessment.
- **Data Analysts**: Need access to historical data and simulation results for analysis.
- **Executives**: Seek high-level insights and reporting for strategic decision-making.
## High-Level Goals
1. **Comprehensive Scenario Analysis**: Enable users to create and analyze multiple project scenarios to assess risks and opportunities.
2. **Data-Driven Decision Making**: Provide stakeholders with the insights needed to make informed decisions based on simulation results.
3. **User-Friendly Interface**: Ensure the platform is accessible and easy to use for all stakeholders, regardless of technical expertise.
## System Overview
FastAPI application that collects mining project inputs, persists scenario-specific records, and surfaces aggregated insights. The platform targets Monte Carlo driven planning, with deterministic CRUD features in place and simulation logic staged for future work.
Frontend components are server-rendered Jinja2 templates, with Chart.js powering the dashboard visualization. The backend leverages SQLAlchemy for ORM mapping to a PostgreSQL database. Frontend components are server-rendered Jinja2 templates, with Chart.js powering the dashboard visualization. The backend leverages SQLAlchemy for ORM mapping to a PostgreSQL database.

138
docs/architecture/02_architecture_constraints.md
View File

@@ -1,5 +1,139 @@
---
title: "02 — Architecture Constraints"
description: "Document imposed constraints: technical, organizational, regulatory, and environmental constraints that affect architecture decisions."
status: skeleton
---
# 02 — Architecture Constraints # 02 — Architecture Constraints
Status: skeleton ## Technical Constraints
Document imposed constraints: technical, organizational, regulatory, and environmental constraints that affect architecture decisions. > e.g., choice of FastAPI, PostgreSQL, SQLAlchemy, Chart.js, Jinja2 templates.
## Organizational Constraints
> e.g., team skillsets, development workflows, CI/CD pipelines.
## Regulatory Constraints
> e.g., data privacy laws, industry standards.
## Environmental Constraints
> e.g., deployment environments, cloud provider limitations.
## Performance Constraints
> e.g., response time requirements, scalability needs.
## Security Constraints
> e.g., authentication mechanisms, data encryption standards.
## Budgetary Constraints
> e.g., licensing costs, infrastructure budgets.
## Time Constraints
> e.g., project deadlines, release schedules.
## Interoperability Constraints
> e.g., integration with existing systems, third-party services.
## Maintainability Constraints
> e.g., code modularity, documentation standards.
## Usability Constraints
> e.g., user interface design principles, accessibility requirements.
## Data Constraints
> e.g., data storage formats, data retention policies.
## Deployment Constraints
> e.g., deployment environments, cloud provider limitations.
## Testing Constraints
> e.g., testing frameworks, test coverage requirements.
## Localization Constraints
> e.g., multi-language support, regional settings.
## Versioning Constraints
> e.g., API versioning strategies, backward compatibility.
## Monitoring Constraints
> e.g., logging standards, performance monitoring tools.
## Backup and Recovery Constraints
> e.g., data backup frequency, disaster recovery plans.
## Development Constraints
> e.g., coding languages, frameworks, libraries to be used or avoided.
## Collaboration Constraints
> e.g., communication tools, collaboration platforms.
## Documentation Constraints
> e.g., documentation tools, style guides.
## Training Constraints
> e.g., training programs, skill development initiatives.
## Support Constraints
> e.g., support channels, response time expectations.
## Legal Constraints
> e.g., compliance requirements, intellectual property considerations.
## Ethical Constraints
> e.g., ethical considerations in data usage, user privacy.
## Environmental Impact Constraints
> e.g., energy consumption considerations, sustainability goals.
## Innovation Constraints
> e.g., limitations on adopting new technologies, risk tolerance for experimentation.
## Cultural Constraints
> e.g., organizational culture, team dynamics affecting development practices.
## Stakeholder Constraints
> e.g., stakeholder expectations, communication preferences.
## Change Management Constraints
> e.g., processes for handling changes, version control practices.
## Resource Constraints
> e.g., availability of hardware, software, and human resources.
## Process Constraints
> e.g., development methodologies (Agile, Scrum), project management tools.
## Quality Constraints
> e.g., code quality standards, testing requirements.

37
docs/architecture/03_context_and_scope.md
View File

@@ -1,5 +1,38 @@
---
title: "03 — Context and Scope"
description: "Describe system context, external actors, and the scope of the architecture."
status: draft
---
# 03 — Context and Scope # 03 — Context and Scope
Status: skeleton ## System Context
Describe system context, external actors, and the scope of the architecture. The CalMiner system operates within the context of mining project management, providing tools for scenario analysis and decision support. It interacts with various data sources, including historical project data and real-time operational metrics.
## External Actors
- **Project Managers**: Utilize the platform for scenario planning and risk assessment.
- **Data Analysts**: Analyze simulation results and derive insights.
- **Executives**: Review high-level reports and dashboards for strategic decision-making.
## Scope of the Architecture
The architecture encompasses the following key areas:
1. **Data Ingestion**: Mechanisms for collecting and processing data from various sources.
2. **Data Storage**: Solutions for storing and managing historical and real-time data.
3. **Simulation Engine**: Core algorithms and models for scenario analysis.
3.1. **Modeling Framework**: Tools for defining and managing simulation models.
3.2. **Parameter Management**: Systems for handling input parameters and configurations.
3.3. **Execution Engine**: Infrastructure for running simulations and processing results.
3.4. **Result Storage**: Systems for storing simulation outputs for analysis and reporting.
4. **Financial Reporting**: Tools for generating reports and visualizations based on simulation outcomes.
5. **Risk Assessment**: Frameworks for identifying and evaluating potential project risks.
6. **Profitability Analysis**: Modules for calculating and analyzing project profitability metrics.
7. **User Interface**: Design and implementation of the user-facing components of the system.
8. **Security and Compliance**: Measures to ensure data security and regulatory compliance.
9. **Scalability and Performance**: Strategies for ensuring the system can handle increasing data volumes and user loads.
10. **Integration Points**: Interfaces for integrating with external systems and services.
11. **Monitoring and Logging**: Systems for tracking system performance and user activity.
12. **Maintenance and Support**: Processes for ongoing system maintenance and user support.

51
docs/architecture/04_solution_strategy.md
View File

@@ -1,20 +1,49 @@
---
title: "04 — Solution Strategy"
description: "High-level solution strategy describing major approaches, technology choices, and trade-offs."
status: draft
---
# 04 — Solution Strategy # 04 — Solution Strategy
Status: skeleton This section outlines the high-level solution strategy for implementing the CalMiner system, focusing on major approaches, technology choices, and trade-offs.
High-level solution strategy describing major approaches, technology choices, and trade-offs. ## Client-Server Architecture
## Monte Carlo engine & persistence - **Backend**: FastAPI serves as the backend framework, providing RESTful APIs for data management, simulation execution, and reporting. It leverages SQLAlchemy for ORM-based database interactions with PostgreSQL.
- **Frontend**: Server-rendered Jinja2 templates deliver dynamic HTML views, enhanced with Chart.js for interactive data visualizations. This approach balances performance and simplicity, avoiding the complexity of a full SPA.
- **Middleware**: Custom middleware handles JSON validation to ensure data integrity before processing requests.
- **Monte Carlo engine**: `services/simulation.py` will incorporate stochastic sampling (e.g., NumPy, SciPy) to populate `simulation_result` and feed reporting. ## Technology Choices
- **Persistence of simulation results**: plan to extend `/api/simulations/run` to persist iterations to `models/simulation_result` and provide a retrieval endpoint for historical runs.
## Simulation Roadmap - **FastAPI**: Chosen for its high performance, ease of use, and modern features like async support and automatic OpenAPI documentation.
- **PostgreSQL**: Selected for its robustness, scalability, and support for complex queries, making it suitable for handling the diverse data needs of mining project management.
- **SQLAlchemy**: Provides a flexible and powerful ORM layer, facilitating database interactions while maintaining code readability and maintainability.
- **Chart.js**: Utilized for its simplicity and effectiveness in rendering interactive charts, enhancing the user experience on the dashboard.
- **Jinja2**: Enables server-side rendering of HTML templates, allowing for dynamic content generation while keeping the frontend lightweight.
- **Pydantic**: Used for data validation and serialization, ensuring that incoming request payloads conform to expected schemas.
- **Docker**: Employed for containerization, ensuring consistent deployment across different environments and simplifying dependency management.
- **Redis**: Used as an in-memory data store to cache frequently accessed data, improving application performance and reducing database load.
- Implement stochastic sampling in `services/simulation.py` (e.g., NumPy random draws based on parameter distributions). ## Trade-offs
- Store iterations in `models/simulation_result.py` via `/api/simulations/run`.
- Feed persisted results into reporting for downstream analytics and historical comparisons.
### Status update (2025-10-21) - **Server-Rendered vs. SPA**: Opted for server-rendered templates over a single-page application (SPA) to reduce complexity and improve initial load times, at the cost of some interactivity.
- **Synchronous vs. Asynchronous**: While FastAPI supports async operations, the initial implementation focuses on synchronous request handling for simplicity, with plans to introduce async features as needed.
- **Monolithic vs. Microservices**: The initial architecture follows a monolithic approach for ease of development and deployment, with the possibility of refactoring into microservices as the system scales.
- **In-Memory Caching**: Implementing Redis for caching introduces additional infrastructure complexity but significantly enhances performance for read-heavy operations.
- **Database Choice**: PostgreSQL was chosen over NoSQL alternatives due to the structured nature of the data and the need for complex querying capabilities, despite potential scalability challenges.
- **Technology Familiarity**: Selected technologies align with the team's existing skill set to minimize the learning curve and accelerate development, even if some alternatives may offer marginally better performance or features.
- **Extensibility vs. Simplicity**: The architecture is designed to be extensible for future features (e.g., Monte Carlo simulation engine) while maintaining simplicity in the initial implementation to ensure timely delivery of core functionalities.
- A scaffolded simulation service (`services/simulation.py`) and `/api/simulations/run` route exist and return in-memory results. Persisting those iterations to `models/simulation_result` is scheduled for a follow-up change. ## Future Considerations
- **Scalability**: As the user base grows, consider transitioning to a microservices architecture and implementing load balancing strategies.
- **Asynchronous Processing**: Introduce asynchronous task queues (e.g., Celery) for long-running simulations to improve responsiveness.
- **Enhanced Frontend**: Explore the possibility of integrating a frontend framework (e.g., React or Vue.js) for more dynamic user interactions in future iterations.
- **Advanced Analytics**: Plan for integrating advanced analytics and machine learning capabilities to enhance simulation accuracy and reporting insights.
- **Security Enhancements**: Implement robust authentication and authorization mechanisms to protect sensitive data and ensure compliance with industry standards.
- **Continuous Integration/Continuous Deployment (CI/CD)**: Establish CI/CD pipelines to automate testing, building, and deployment processes for faster and more reliable releases.
- **Monitoring and Logging**: Integrate monitoring tools (e.g., Prometheus, Grafana) and centralized logging solutions (e.g., ELK stack) to track application performance and troubleshoot issues effectively.
- **User Feedback Loop**: Implement mechanisms for collecting user feedback to inform future development priorities and improve user experience.
- **Documentation**: Maintain comprehensive documentation for both developers and end-users to facilitate onboarding and effective use of the system.
- **Testing Strategy**: Develop a robust testing strategy, including unit, integration, and end-to-end tests, to ensure code quality and reliability as the system evolves.

21
docs/architecture/04_solution_strategy_extended.md → docs/architecture/04_solution_strategy/04_01_implementation_plan_20251020.md
View File

@@ -1,4 +1,4 @@
# Implementation Plan (extended) # Implementation Plan 2025-10-20
This file contains the implementation plan (MVP features, steps, and estimates). This file contains the implementation plan (MVP features, steps, and estimates).
@@ -107,23 +107,4 @@ This file contains the implementation plan (MVP features, steps, and estimates).
1. Write unit tests in `tests/unit/test_reporting.py`. 1. Write unit tests in `tests/unit/test_reporting.py`.
1. Enhance UI in `components/Dashboard.html` with charts. 1. Enhance UI in `components/Dashboard.html` with charts.
## MVP Feature Analysis (summary)
Goal: Identify core MVP features, acceptance criteria, and quick estimates.
### Edge cases to consider
- Large simulation runs (memory / timeouts) — use streaming, chunking, or background workers.
- DB migration and schema versioning.
- Authentication/authorization for scenario access.
### Next actionable items
1. Break Scenario Management into sub-issues (models, routes, tests, simple UI).
1. Scaffold Parameter Input & Validation (models/parameters.py, middleware, routes, tests).
1. Prototype the simulation engine with a small deterministic runner and unit tests.
1. Scaffold Monte Carlo Simulation endpoints (`services/simulation.py`, `routes/simulations.py`, tests).
1. Scaffold Reporting endpoints (`services/reporting.py`, `routes/reporting.py`, front-end Dashboard, tests).
1. Add CI job for tests and coverage.
See [UI and Style](docs/architecture/13_ui_and_style.md) for the UI template audit, layout guidance, and next steps. See [UI and Style](docs/architecture/13_ui_and_style.md) for the UI template audit, layout guidance, and next steps.

69
docs/architecture/05_building_block_view.md
View File

@@ -1,27 +1,12 @@
---
title: "05 — Building Block View"
description: "Explain the static structure: modules, components, services and their relationships."
status: draft
---
# 05 — Building Block View # 05 — Building Block View
Status: skeleton ## Architecture overview
Explain the static structure: modules, components, services and their relationships.
## System Components
- **FastAPI backend** (`main.py`, `routes/`): hosts REST endpoints for scenarios, parameters, costs, consumption, production, equipment, maintenance, simulations, and reporting. Each router encapsulates request/response schemas and DB access patterns, leveraging a shared dependency module (`routes/dependencies.get_db`) for SQLAlchemy session management.
- **Service layer** (`services/`): houses business logic. `services/reporting.py` produces statistical summaries, while `services/simulation.py` provides the Monte Carlo integration point.
- **Persistence** (`models/`, `config/database.py`): SQLAlchemy models map to PostgreSQL tables. Relationships connect scenarios to derived domain entities.
- **Presentation** (`templates/`, `components/`): server-rendered views extend a shared `base.html` layout with a persistent left sidebar, pull global styles from `static/css/main.css`, and surface data entry (scenario and parameter forms) alongside the Chart.js-powered dashboard.
- **Reusable partials** (`templates/partials/components.html`): macro library that standardises select inputs, feedback/empty states, and table wrappers so pages remain consistent while keeping DOM hooks stable for existing JavaScript modules.
- **Middleware** (`middleware/validation.py`): applies JSON validation before requests reach routers.
- **Testing** (`tests/unit/`): pytest suite covering route and service behavior, including UI rendering checks and negative-path router validation tests to ensure consistent HTTP error semantics. Playwright end-to-end coverage is planned for core smoke flows (dashboard load, scenario inputs, reporting) and will attach in CI once scaffolding is completed.
## Module Map (code)
- `scenario.py`: central scenario entity with relationships to cost, consumption, production, equipment, maintenance, and simulation results.
- `capex.py`, `opex.py`: financial expenditures tied to scenarios.
- `consumption.py`, `production_output.py`: operational data tables.
- `equipment.py`, `maintenance.py`: asset management models.
## Architecture overview (migrated)
This overview complements [architecture](docs/architecture/README.md) with a high-level map of CalMiner's module layout and request flow. This overview complements [architecture](docs/architecture/README.md) with a high-level map of CalMiner's module layout and request flow.
@@ -31,4 +16,42 @@ Refer to the detailed architecture chapters in `docs/architecture/`:
- Request flow & runtime interactions: [Runtime View](docs/architecture/06_runtime_view.md) - Request flow & runtime interactions: [Runtime View](docs/architecture/06_runtime_view.md)
- Simulation roadmap & strategy: [Solution Strategy](docs/architecture/04_solution_strategy.md) - Simulation roadmap & strategy: [Solution Strategy](docs/architecture/04_solution_strategy.md)
Currency normalization and backfill tooling have been added (see [backfill_currency.py](scripts/backfill_currency.py) and related migrations) to support canonical currency lookups across cost tables. ## System Components
### Backend
- **FastAPI application** (`main.py`): entry point that configures routers, middleware, and startup/shutdown events.
- **Routers** (`routes/`): modular route handlers for scenarios, parameters, costs, consumption, production, equipment, maintenance, simulations, and reporting. Each router defines RESTful endpoints, request/response schemas, and orchestrates service calls.
- leveraging a shared dependency module (`routes/dependencies.get_db`) for SQLAlchemy session management.
- **Models** (`models/`): SQLAlchemy ORM models representing database tables and relationships, encapsulating domain entities like Scenario, CapEx, OpEx, Consumption, ProductionOutput, Equipment, Maintenance, and SimulationResult.
- **Services** (`services/`): business logic layer that processes data, performs calculations, and interacts with models. Key services include reporting calculations and Monte Carlo simulation scaffolding.
- **Database** (`config/database.py`): sets up the SQLAlchemy engine and session management for PostgreSQL interactions.
### Frontend
- **Templates** (`templates/`): Jinja2 templates for server-rendered HTML views, extending a shared base layout with a persistent sidebar for navigation.
- **Static Assets** (`static/`): CSS and JavaScript files for styling and interactivity. Shared CSS variables in `static/css/main.css` define the color palette, while page-specific JS modules in `static/js/` handle dynamic behaviors.
- **Reusable partials** (`templates/partials/components.html`): macro library that standardises select inputs, feedback/empty states, and table wrappers so pages remain consistent while keeping DOM hooks stable for existing JavaScript modules.
### Middleware & Utilities
- **Middleware** (`middleware/validation.py`): applies JSON validation before requests reach routers.
- **Testing** (`tests/unit/`): pytest suite covering route and service behavior, including UI rendering checks and negative-path router validation tests to ensure consistent HTTP error semantics. Playwright end-to-end coverage is planned for core smoke flows (dashboard load, scenario inputs, reporting) and will attach in CI once scaffolding is completed.
## Module Map (code)
- `scenario.py`: central scenario entity with relationships to cost, consumption, production, equipment, maintenance, and simulation results.
- `capex.py`, `opex.py`: financial expenditures tied to scenarios.
- `consumption.py`, `production_output.py`: operational data tables.
- `equipment.py`, `maintenance.py`: asset management models.
- `simulation_result.py`: stores Monte Carlo iteration outputs.
## Service Layer
- `reporting.py`: computes aggregates (count, min/max, mean, median, percentiles, standard deviation, variance, tail-risk metrics) from simulation results.
- `simulation.py`: scaffolds Monte Carlo simulation logic (currently in-memory; persistence planned).
- `currency.py`: handles currency normalization for cost tables.
- `utils.py`: shared helper functions (e.g., statistical calculations).
- `validation.py`: JSON schema validation middleware.
- `database.py`: SQLAlchemy engine and session setup.
- `dependencies.py`: FastAPI dependency injection for DB sessions.

259
docs/architecture/06_runtime_view.md
View File

@@ -1,8 +1,179 @@
---
title: "06 — Runtime View"
description: "Describe runtime aspects: request flows, lifecycle of key interactions, and runtime components."
status: draft
---
# 06 — Runtime View # 06 — Runtime View
Status: skeleton ## Overview
Describe runtime aspects: request flows, lifecycle of key interactions, and runtime components. The runtime view focuses on the dynamic behavior of the CalMiner application during execution. It illustrates how various components interact to fulfill user requests, process data, and generate outputs. Key runtime scenarios include scenario management, parameter input handling, cost tracking, consumption tracking, production output recording, equipment management, maintenance logging, Monte Carlo simulations, and reporting.
## Request Flow
1. **User Interaction**: A user interacts with the web application through the UI, triggering actions such as creating a scenario, inputting parameters, or generating reports.
2. **API Request**: The frontend sends HTTP requests (GET, POST, PUT, DELETE) to the appropriate API endpoints defined in the `routes/` directory.
3. **Routing**: The FastAPI framework routes the incoming requests to the corresponding route handlers.
4. **Service Layer**: Route handlers invoke services from the `services/` directory to process the business logic.
5. **Database Interaction**: Services interact with the database via ORM models defined in the `models/` directory to perform CRUD operations.
6. **Response Generation**: After processing, services return data to the route handlers, which format the response (JSON or HTML) and send it back to the frontend.
7. **UI Update**: The frontend updates the UI based on the response, rendering new data or updating existing views.
8. **Reporting Pipeline**: For reporting, data is aggregated from various sources, processed to generate statistics, and presented in the dashboard using Chart.js.
9. **Monte Carlo Simulations**: Stochastic simulations are executed in the backend, generating probabilistic outcomes that are stored temporarily and used for risk analysis in reports.
10. **Error Handling**: Throughout the process, error handling mechanisms ensure that exceptions are caught and appropriate responses are sent back to the user.
Request flow diagram:
```mermaid
sequenceDiagram
participant User
participant Frontend
participant API
participant Service
participant Database
User->>Frontend: Interact with UI
Frontend->>API: Send HTTP Request
API->>Service: Route to Handler
Service->>Database: Perform CRUD Operation
Database-->>Service: Return Data
Service-->>API: Return Processed Data
API-->>Frontend: Send Response
Frontend-->>User: Update UI
participant Reporting
Service->>Reporting: Aggregate Data
Reporting-->>Service: Return Report Data
Service-->>API: Return Report Response
API-->>Frontend: Send Report Data
Frontend-->>User: Render Report
participant Simulation
Service->>Simulation: Execute Monte Carlo Simulation
Simulation-->>Service: Return Simulation Results
Service-->>API: Return Simulation Data
API-->>Frontend: Send Simulation Data
Frontend-->>User: Display Simulation Results
```
## Key Runtime Scenarios
### Scenario Management
1. User accesses the scenario list via the UI.
2. The frontend sends a GET request to `/api/scenarios`.
3. The `ScenarioService` retrieves scenarios from the database.
4. The response is rendered in the UI.
5. For scenario creation, the user submits a form, triggering a POST request to `/api/scenarios`, which the `ScenarioService` processes to create a new scenario in the database.
6. The UI updates to reflect the new scenario.
Scenario management diagram:
```mermaid
sequenceDiagram
participant User
participant Frontend
participant API
participant ScenarioService
participant Database
User->>Frontend: Access Scenario List
Frontend->>API: GET /api/scenarios
API->>ScenarioService: Route to Handler
ScenarioService->>Database: Retrieve Scenarios
Database-->>ScenarioService: Return Scenarios
ScenarioService-->>API: Return Scenario Data
API-->>Frontend: Send Response
Frontend-->>User: Render Scenario List
User->>Frontend: Submit New Scenario Form
Frontend->>API: POST /api/scenarios
API->>ScenarioService: Route to Handler
ScenarioService->>Database: Create New Scenario
Database-->>ScenarioService: Confirm Creation
ScenarioService-->>API: Return New Scenario Data
API-->>Frontend: Send Response
Frontend-->>User: Update UI with New Scenario
```
### Process Parameter Input
1. User navigates to the parameter input form.
2. The frontend fetches existing parameters via a GET request to `/api/parameters`.
3. The `ParameterService` retrieves parameters from the database.
4. The response is rendered in the UI.
5. For parameter updates, the user submits a form, triggering a PUT request to `/api/parameters/:id`, which the `ParameterService` processes to update the parameter in the database.
6. The UI updates to reflect the changes.
Parameter input diagram:
```mermaid
sequenceDiagram
participant User
participant Frontend
participant API
participant ParameterService
participant Database
User->>Frontend: Navigate to Parameter Input Form
Frontend->>API: GET /api/parameters
API->>ParameterService: Route to Handler
ParameterService->>Database: Retrieve Parameters
Database-->>ParameterService: Return Parameters
ParameterService-->>API: Return Parameter Data
API-->>Frontend: Send Response
Frontend-->>User: Render Parameter Form
User->>Frontend: Submit Parameter Update Form
Frontend->>API: PUT /api/parameters/:id
API->>ParameterService: Route to Handler
ParameterService->>Database: Update Parameter
Database-->>ParameterService: Confirm Update
ParameterService-->>API: Return Updated Parameter Data
API-->>Frontend: Send Response
Frontend-->>User: Update UI with Updated Parameter
```
### Cost Tracking
1. User accesses the cost tracking view.
2. The frontend sends a GET request to `/api/costs` to fetch existing cost records.
3. The `CostService` retrieves cost data from the database.
4. The response is rendered in the UI.
5. For cost updates, the user submits a form, triggering a PUT request to `/api/costs/:id`, which the `CostService` processes to update the cost record in the database.
6. The UI updates to reflect the changes.
Cost tracking diagram:
```mermaid
sequenceDiagram
participant User
participant Frontend
participant API
participant CostService
participant Database
User->>Frontend: Access Cost Tracking View
Frontend->>API: GET /api/costs
API->>CostService: Route to Handler
CostService->>Database: Retrieve Cost Records
Database-->>CostService: Return Cost Data
CostService-->>API: Return Cost Data
API-->>Frontend: Send Response
Frontend-->>User: Render Cost Tracking View
User->>Frontend: Submit Cost Update Form
Frontend->>API: PUT /api/costs/:id
API->>CostService: Route to Handler
CostService->>Database: Update Cost Record
Database-->>CostService: Confirm Update
CostService-->>API: Return Updated Cost Data
API-->>Frontend: Send Response
Frontend-->>User: Update UI with Updated Cost Data
```
## Reporting Pipeline and UI Integration ## Reporting Pipeline and UI Integration
@@ -31,3 +202,87 @@ Describe runtime aspects: request flows, lifecycle of key interactions, and runt
- `templates/Dashboard.html` posts the user-provided dataset to the summary endpoint, renders metric cards for each field, and charts the distribution using Chart.js. - `templates/Dashboard.html` posts the user-provided dataset to the summary endpoint, renders metric cards for each field, and charts the distribution using Chart.js.
- `SUMMARY_FIELDS` now includes variance, 5th/10th/90th/95th percentiles, and tail-risk metrics (VaR/Expected Shortfall at 95%); tooltip annotations surface the tail metrics alongside the percentile line chart. - `SUMMARY_FIELDS` now includes variance, 5th/10th/90th/95th percentiles, and tail-risk metrics (VaR/Expected Shortfall at 95%); tooltip annotations surface the tail metrics alongside the percentile line chart.
- Error handling surfaces HTTP failures inline so users can address malformed JSON or backend availability issues without leaving the page. - Error handling surfaces HTTP failures inline so users can address malformed JSON or backend availability issues without leaving the page.
Reporting pipeline diagram:
```mermaid
sequenceDiagram
participant User
participant Frontend
participant API
participant ReportingService
User->>Frontend: Input Data for Reporting
Frontend->>API: POST /api/reporting/summary
API->>ReportingService: Route to Handler
ReportingService->>ReportingService: Validate Payload
ReportingService->>ReportingService: Compute Statistics
ReportingService-->>API: Return Report Summary
API-->>Frontend: Send Report Summary
Frontend-->>User: Render Report Metrics and Charts
```
## Monte Carlo Simulation Execution
1. User initiates a Monte Carlo simulation via the UI.
2. The frontend sends a POST request to `/api/simulations/run` with simulation parameters.
3. The `SimulationService` executes the Monte Carlo logic, generating stochastic results.
4. The results are temporarily stored and returned to the frontend.
5. The UI displays the simulation results and allows users to trigger reporting based on these outcomes.
6. The reporting pipeline processes the simulation results as described above.
7. Error handling ensures that any issues during simulation execution are communicated back to the user.
8. Monte Carlo simulation diagram:
```mermaid
sequenceDiagram
participant User
participant Frontend
participant API
participant SimulationService
User->>Frontend: Input Simulation Parameters
Frontend->>API: POST /api/simulations/run
API->>SimulationService: Route to Handler
SimulationService->>SimulationService: Execute Monte Carlo Logic
SimulationService-->>API: Return Simulation Results
API-->>Frontend: Send Simulation Results
Frontend-->>User: Render Simulation Results
```
## Error Handling
Throughout the runtime processes, error handling mechanisms are implemented to catch exceptions and provide meaningful feedback to users. Common error scenarios include:
- Invalid input data
- Database connection issues
- Simulation execution errors
- Reporting calculation failures
- API endpoint unavailability
- Timeouts during long-running operations
- Unauthorized access attempts
- Data validation failures
- Resource not found errors
Error handling diagram:
```mermaid
sequenceDiagram
participant User
participant Frontend
participant API
participant Service
User->>Frontend: Perform Action
Frontend->>API: Send Request
API->>Service: Route to Handler
Service->>Service: Process Request
alt Success
Service-->>API: Return Data
API-->>Frontend: Send Response
Frontend-->>User: Update UI
else Error
Service-->>API: Return Error
API-->>Frontend: Send Error Response
Frontend-->>User: Display Error Message
end
```

55
docs/architecture/07_deployment_view.md
View File

@@ -1,8 +1,59 @@
---
title: "07 — Deployment View"
description: "Describe deployment topology, infrastructure components, and environments (dev/stage/prod)."
status: draft
---
# 07 — Deployment View # 07 — Deployment View
Status: skeleton ## Deployment Topology
Describe deployment topology, infrastructure components, and environments (dev/stage/prod). The CalMiner application is deployed using a multi-tier architecture consisting of the following layers:
1. **Client Layer**: This layer consists of web browsers that interact with the application through a user interface rendered by Jinja2 templates and enhanced with JavaScript (Chart.js for dashboards).
2. **Web Application Layer**: This layer hosts the FastAPI application, which handles API requests, business logic, and serves HTML templates. It communicates with the database layer for data persistence.
3. **Database Layer**: This layer consists of a PostgreSQL database that stores all application data, including scenarios, parameters, costs, consumption, production outputs, equipment, maintenance logs, and simulation results.
4. **Caching Layer**: This layer uses Redis to cache frequently accessed data and improve application performance.
## Infrastructure Components
The infrastructure components for the application include:
- **Web Server**: Hosts the FastAPI application and serves API endpoints.
- **Database Server**: PostgreSQL database for persisting application data.
- **Static File Server**: Serves static assets such as CSS, JavaScript, and image files.
- **Reverse Proxy (optional)**: An Nginx or Apache server can be used as a reverse proxy.
- **CI/CD Pipeline (optional)**: Automated deployment pipelines can be set up using tools like GitHub Actions.
- **Containerization (optional)**: Docker can be used to containerize the application.
- **Cloud Infrastructure (optional)**: The application can be deployed on cloud platforms.
## Environments
The application can be deployed in multiple environments to support development, testing, and production:
### Development Environment
The development environment is set up for local development and testing. It includes:
- Local PostgreSQL instance
- FastAPI server running in debug mode
### Testing Environment
The testing environment is set up for automated testing and quality assurance. It includes:
- Staging PostgreSQL instance
- FastAPI server running in testing mode
- Automated test suite (e.g., pytest) for running unit and integration tests
### Production Environment
The production environment is set up for serving live traffic and includes:
- Production PostgreSQL instance
- FastAPI server running in production mode
- Load balancer (e.g., Nginx) for distributing incoming requests
- Monitoring and logging tools for tracking application performance
## Integrations and Future Work (deployment-related) ## Integrations and Future Work (deployment-related)

42
docs/architecture/08_concepts.md
View File

@@ -1,8 +1,46 @@
---
title: "08 — Concepts"
description: "Document key concepts, domain models, and terminology used throughout the architecture documentation."
status: draft
---
# 08 — Concepts # 08 — Concepts
Status: skeleton ## Key Concepts
Document key concepts, domain models, and terminology used throughout the architecture documentation. ### Scenario
A `scenario` represents a distinct mining project configuration, encapsulating all relevant parameters, costs, consumption, production outputs, equipment, maintenance logs, and simulation results. Each scenario is independent, allowing users to model and analyze different mining strategies.
### Parameterization
Parameters are defined for each scenario to capture inputs such as resource consumption rates, production targets, cost factors, and equipment specifications. Parameters can have fixed values or be linked to probability distributions for stochastic simulations.
### Monte Carlo Simulation
The Monte Carlo simulation engine allows users to perform risk analysis by running multiple iterations of a scenario with varying input parameters based on defined probability distributions. This helps in understanding the range of possible outcomes and their associated probabilities.
## Domain Model
The domain model consists of the following key entities:
- `Scenario`: Represents a mining project configuration.
- `Parameter`: Input values for scenarios, which can be fixed or probabilistic.
- `Cost`: Tracks capital and operational expenditures.
- `Consumption`: Records resource usage.
- `ProductionOutput`: Captures production metrics.
- `Equipment`: Represents mining equipment associated with a scenario.
- `Maintenance`: Logs maintenance events for equipment.
- `SimulationResult`: Stores results from Monte Carlo simulations.
- `Distribution`: Defines probability distributions for stochastic parameters.
- `User`: Represents application users and their roles.
- `Report`: Generated reports summarizing scenario analyses.
- `Dashboard`: Visual representation of key performance indicators and metrics.
- `AuditLog`: Tracks changes and actions performed within the application.
- `Notification`: Alerts and messages related to scenario events and updates.
- `Tag`: Labels for categorizing scenarios and other entities.
- `Attachment`: Files associated with scenarios, such as documents or images.
- `Version`: Tracks different versions of scenarios and their configurations.
## Data Model Highlights ## Data Model Highlights

4
docs/architecture/README.md
View File

@@ -7,7 +7,7 @@ description: "arc42-based architecture documentation for the CalMiner project"
This folder mirrors the arc42 chapter structure (adapted to Markdown). This folder mirrors the arc42 chapter structure (adapted to Markdown).
Files: ## Files
- [01 Introduction and Goals](01_introduction_and_goals.md) - [01 Introduction and Goals](01_introduction_and_goals.md)
- [02 Architecture Constraints](02_architecture_constraints.md) - [02 Architecture Constraints](02_architecture_constraints.md)
@@ -24,5 +24,3 @@ Files:
- [13 UI and Style](13_ui_and_style.md) - [13 UI and Style](13_ui_and_style.md)
- [14 Testing & CI](14_testing_ci.md) - [14 Testing & CI](14_testing_ci.md)
- [15 Development Setup](15_development_setup.md) - [15 Development Setup](15_development_setup.md)
- [Quickstart](docs/quickstart.md) contains developer quickstart, migrations, testing and current status and will remain as a separate quickstart reference.