Refactor and enhance CalMiner application
- Updated README.md to reflect new features and usage instructions. - Removed deprecated Dashboard.html component and integrated dashboard functionality directly into the main application. - Revised architecture documentation for clarity and added module map and request flow diagrams. - Enhanced maintenance model to include equipment association and cost tracking. - Updated requirements.txt to include new dependencies (httpx, pandas, numpy). - Improved consumption, maintenance, production, and reporting routes with better validation and response handling. - Added unit tests for maintenance and production routes, ensuring proper CRUD operations and validation. - Enhanced reporting service to calculate and return detailed summary statistics. - Redesigned Dashboard.html for improved user experience and integrated Chart.js for visualizing simulation results.
This commit is contained in:
@@ -2,40 +2,47 @@
|
||||
|
||||
## Overview
|
||||
|
||||
CalMiner is a web application for planning mining projects, estimating costs, returns, and profitability. It uses Monte Carlo simulations for risk analysis and supports multiple scenarios.
|
||||
CalMiner is a FastAPI application that collects mining project inputs, persists scenario-specific records, and surfaces aggregated insights. The platform targets Monte Carlo driven planning, with deterministic CRUD features in place and simulation logic staged for future work.
|
||||
|
||||
## System Components
|
||||
|
||||
- **Frontend**: Web interface for user interaction (to be defined).
|
||||
- **Backend**: Python API server (e.g., FastAPI) handling business logic.
|
||||
- **Database**: PostgreSQL.
|
||||
- **Configuration**: Environment variables and settings loaded via `python-dotenv` and stored in `config/` directory.
|
||||
- **Simulation Engine**: Python-based Monte Carlo runs and stochastic calculations.
|
||||
- **API Routes**: FastAPI routers defined in `routes/` for scenarios, simulations, consumptions, and reporting endpoints.
|
||||
- **FastAPI backend** (`main.py`, `routes/`): hosts REST endpoints for scenarios, parameters, costs, consumption, production, equipment, maintenance, simulations, and reporting. Each router encapsulates request/response schemas and DB access patterns.
|
||||
- **Service layer** (`services/`): houses business logic. `services/reporting.py` produces statistical summaries, while `services/simulation.py` provides the Monte Carlo integration point.
|
||||
- **Persistence** (`models/`, `config/database.py`): SQLAlchemy models map to PostgreSQL tables in schema `bricsium_platform`. Relationships connect scenarios to derived domain entities.
|
||||
- **Presentation** (`templates/`, `components/`): server-rendered views support data entry (scenario and parameter forms) and the dashboard visualization powered by Chart.js.
|
||||
- **Middleware** (`middleware/validation.py`): applies JSON validation before requests reach routers.
|
||||
- **Testing** (`tests/unit/`): pytest suite covering route and service behavior.
|
||||
|
||||
## Data Flow
|
||||
## Runtime Flow
|
||||
|
||||
1. User inputs scenario parameters via frontend.
|
||||
2. Backend validates and stores in database.
|
||||
3. Simulation engine runs Monte Carlo iterations using stochastic variables.
|
||||
4. Results stored in `simulation_result` table.
|
||||
5. Frontend displays outputs like NPV, IRR, EBITDA.
|
||||
1. Users navigate to form templates or API clients to manage scenarios, parameters, and operational data.
|
||||
2. FastAPI routers validate payloads with Pydantic models, then delegate to SQLAlchemy sessions for persistence.
|
||||
3. Simulation runs (placeholder `services/simulation.py`) will consume stored parameters to emit iteration results via `/api/simulations/run`.
|
||||
4. Reporting requests POST simulation outputs to `/api/reporting/summary`; the reporting service calculates aggregates (count, min/max, mean, median, percentiles, standard deviation).
|
||||
5. `templates/Dashboard.html` fetches summaries, renders metric cards, and plots distribution charts with Chart.js for stakeholder review.
|
||||
|
||||
## Database Architecture
|
||||
## Data Model Highlights
|
||||
|
||||
- Schema: `bricsium_platform`
|
||||
- Key tables include:
|
||||
- `scenario`: central entity describing a mining scenario; owns relationships to cost, consumption, production, equipment, and maintenance tables.
|
||||
- `capex`, `opex`: monetary tracking linked to scenarios.
|
||||
- `consumption`: resource usage entries parameterized by scenario and description.
|
||||
- `production_output`: production metrics per scenario.
|
||||
- `equipment` and `maintenance`: equipment inventory and maintenance events with dates/costs.
|
||||
- `simulation_result`: staging table for future Monte Carlo outputs (not yet populated by `run_simulation`).
|
||||
|
||||
- `scenario` (scenario metadata and parameters)
|
||||
- `capex`, `opex` (capital and operational expenditures)
|
||||
- `chemical_consumption`, `fuel_consumption`, `water_consumption`, `scrap_consumption`
|
||||
- `production_output`, `equipment_operation`, `ore_batch`
|
||||
- `exchange_rate`, `simulation_result`
|
||||
Foreign keys secure referential integrity between domain tables and their scenarios, enabling per-scenario analytics.
|
||||
|
||||
- Relationships: Foreign keys link scenarios to parameters, consumptions, and simulation results.
|
||||
## Integrations and Future Work
|
||||
|
||||
## Next Steps
|
||||
- **Monte Carlo engine**: `services/simulation.py` will incorporate stochastic sampling (e.g., NumPy, SciPy) to populate `simulation_result` and feed reporting.
|
||||
- **Persistence of results**: `/api/simulations/run` currently returns in-memory results; next iteration should persist to `simulation_result` and reference scenarios.
|
||||
- **Authentication**: not yet implemented; all endpoints are open.
|
||||
- **Deployment**: documentation focuses on local development; containerization and CI/CD pipelines remain to be defined.
|
||||
|
||||
- Define API endpoints.
|
||||
- Implement simulation logic.
|
||||
- Add authentication and user management.
|
||||
For extended diagrams and setup instructions reference:
|
||||
|
||||
- [docs/development_setup.md](development_setup.md) — environment provisioning and tooling.
|
||||
- [docs/testing.md](testing.md) — pytest workflow and coverage expectations.
|
||||
- [docs/mvp.md](mvp.md) — roadmap and milestone scope.
|
||||
- [docs/implementation_plan.md](implementation_plan.md) — feature breakdown aligned with the TODO tracker.
|
||||
- [docs/architecture_overview.md](architecture_overview.md) — supplementary module map and request flow diagram.
|
||||
|
||||
44
docs/architecture_overview.md
Normal file
44
docs/architecture_overview.md
Normal file
@@ -0,0 +1,44 @@
|
||||
# Architecture Overview
|
||||
|
||||
This overview complements `docs/architecture.md` with a high-level map of CalMiner's module layout and request flow.
|
||||
|
||||
## Module Map
|
||||
|
||||
- `main.py`: FastAPI entry point bootstrapping routers and middleware.
|
||||
- `models/`: SQLAlchemy declarative models for all database tables. Key modules:
|
||||
- `scenario.py`: central scenario entity with relationships to cost, consumption, production, equipment, maintenance, and simulation results.
|
||||
- `capex.py`, `opex.py`: financial expenditures tied to scenarios.
|
||||
- `consumption.py`, `production_output.py`: operational data tables.
|
||||
- `equipment.py`, `maintenance.py`: asset management models.
|
||||
- `routes/`: REST endpoints grouped by domain (scenarios, parameters, costs, consumption, production, equipment, maintenance, reporting, simulations, UI).
|
||||
- `services/`: business logic abstractions. `reporting.py` supplies summary statistics; `simulation.py` hosts the Monte Carlo extension point.
|
||||
- `middleware/validation.py`: request JSON validation prior to hitting routers.
|
||||
- `templates/`: Jinja2 templates for UI (scenario form, parameter input, dashboard).
|
||||
|
||||
## Request Flow
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
A[Browser / API Client] -->|HTTP| B[FastAPI Router]
|
||||
B --> C[Dependency Injection]
|
||||
C --> D[SQLAlchemy Session]
|
||||
B --> E[Service Layer]
|
||||
E --> D
|
||||
E --> F[Reporting / Simulation Logic]
|
||||
D --> G[PostgreSQL]
|
||||
F --> H[Summary Response]
|
||||
G --> H
|
||||
H --> A
|
||||
```
|
||||
|
||||
## Dashboard Interaction
|
||||
|
||||
1. User loads `/dashboard`, served by `templates/Dashboard.html`.
|
||||
2. Template fetches `/api/reporting/summary` with sample or user-provided simulation outputs.
|
||||
3. Response metrics populate the summary grid and Chart.js visualization.
|
||||
|
||||
## Simulation Roadmap
|
||||
|
||||
- Implement stochastic sampling in `services/simulation.py` (e.g., NumPy random draws based on parameter distributions).
|
||||
- Store iterations in `models/simulation_result.py` via `/api/simulations/run`.
|
||||
- Feed persisted results into reporting for downstream analytics and historical comparisons.
|
||||
Reference in New Issue
Block a user