Refactor and enhance CalMiner application

- Updated README.md to reflect new features and usage instructions.
- Removed deprecated Dashboard.html component and integrated dashboard functionality directly into the main application.
- Revised architecture documentation for clarity and added module map and request flow diagrams.
- Enhanced maintenance model to include equipment association and cost tracking.
- Updated requirements.txt to include new dependencies (httpx, pandas, numpy).
- Improved consumption, maintenance, production, and reporting routes with better validation and response handling.
- Added unit tests for maintenance and production routes, ensuring proper CRUD operations and validation.
- Enhanced reporting service to calculate and return detailed summary statistics.
- Redesigned Dashboard.html for improved user experience and integrated Chart.js for visualizing simulation results.
This commit is contained in:
2025-10-20 20:53:55 +02:00
parent fee857637f
commit e73a987d25
19 changed files with 794 additions and 184 deletions

View File

@@ -2,40 +2,47 @@
## Overview
CalMiner is a web application for planning mining projects, estimating costs, returns, and profitability. It uses Monte Carlo simulations for risk analysis and supports multiple scenarios.
CalMiner is a FastAPI application that collects mining project inputs, persists scenario-specific records, and surfaces aggregated insights. The platform targets Monte Carlo driven planning, with deterministic CRUD features in place and simulation logic staged for future work.
## System Components
- **Frontend**: Web interface for user interaction (to be defined).
- **Backend**: Python API server (e.g., FastAPI) handling business logic.
- **Database**: PostgreSQL.
- **Configuration**: Environment variables and settings loaded via `python-dotenv` and stored in `config/` directory.
- **Simulation Engine**: Python-based Monte Carlo runs and stochastic calculations.
- **API Routes**: FastAPI routers defined in `routes/` for scenarios, simulations, consumptions, and reporting endpoints.
- **FastAPI backend** (`main.py`, `routes/`): hosts REST endpoints for scenarios, parameters, costs, consumption, production, equipment, maintenance, simulations, and reporting. Each router encapsulates request/response schemas and DB access patterns.
- **Service layer** (`services/`): houses business logic. `services/reporting.py` produces statistical summaries, while `services/simulation.py` provides the Monte Carlo integration point.
- **Persistence** (`models/`, `config/database.py`): SQLAlchemy models map to PostgreSQL tables in schema `bricsium_platform`. Relationships connect scenarios to derived domain entities.
- **Presentation** (`templates/`, `components/`): server-rendered views support data entry (scenario and parameter forms) and the dashboard visualization powered by Chart.js.
- **Middleware** (`middleware/validation.py`): applies JSON validation before requests reach routers.
- **Testing** (`tests/unit/`): pytest suite covering route and service behavior.
## Data Flow
## Runtime Flow
1. User inputs scenario parameters via frontend.
2. Backend validates and stores in database.
3. Simulation engine runs Monte Carlo iterations using stochastic variables.
4. Results stored in `simulation_result` table.
5. Frontend displays outputs like NPV, IRR, EBITDA.
1. Users navigate to form templates or API clients to manage scenarios, parameters, and operational data.
2. FastAPI routers validate payloads with Pydantic models, then delegate to SQLAlchemy sessions for persistence.
3. Simulation runs (placeholder `services/simulation.py`) will consume stored parameters to emit iteration results via `/api/simulations/run`.
4. Reporting requests POST simulation outputs to `/api/reporting/summary`; the reporting service calculates aggregates (count, min/max, mean, median, percentiles, standard deviation).
5. `templates/Dashboard.html` fetches summaries, renders metric cards, and plots distribution charts with Chart.js for stakeholder review.
## Database Architecture
## Data Model Highlights
- Schema: `bricsium_platform`
- Key tables include:
- `scenario`: central entity describing a mining scenario; owns relationships to cost, consumption, production, equipment, and maintenance tables.
- `capex`, `opex`: monetary tracking linked to scenarios.
- `consumption`: resource usage entries parameterized by scenario and description.
- `production_output`: production metrics per scenario.
- `equipment` and `maintenance`: equipment inventory and maintenance events with dates/costs.
- `simulation_result`: staging table for future Monte Carlo outputs (not yet populated by `run_simulation`).
- `scenario` (scenario metadata and parameters)
- `capex`, `opex` (capital and operational expenditures)
- `chemical_consumption`, `fuel_consumption`, `water_consumption`, `scrap_consumption`
- `production_output`, `equipment_operation`, `ore_batch`
- `exchange_rate`, `simulation_result`
Foreign keys secure referential integrity between domain tables and their scenarios, enabling per-scenario analytics.
- Relationships: Foreign keys link scenarios to parameters, consumptions, and simulation results.
## Integrations and Future Work
## Next Steps
- **Monte Carlo engine**: `services/simulation.py` will incorporate stochastic sampling (e.g., NumPy, SciPy) to populate `simulation_result` and feed reporting.
- **Persistence of results**: `/api/simulations/run` currently returns in-memory results; next iteration should persist to `simulation_result` and reference scenarios.
- **Authentication**: not yet implemented; all endpoints are open.
- **Deployment**: documentation focuses on local development; containerization and CI/CD pipelines remain to be defined.
- Define API endpoints.
- Implement simulation logic.
- Add authentication and user management.
For extended diagrams and setup instructions reference:
- [docs/development_setup.md](development_setup.md) — environment provisioning and tooling.
- [docs/testing.md](testing.md) — pytest workflow and coverage expectations.
- [docs/mvp.md](mvp.md) — roadmap and milestone scope.
- [docs/implementation_plan.md](implementation_plan.md) — feature breakdown aligned with the TODO tracker.
- [docs/architecture_overview.md](architecture_overview.md) — supplementary module map and request flow diagram.