calminer/README.md

# CalMiner

A web application to plan mining projects and estimate costs, returns and profitability.

Focuses on ore mining operations and covering parameters such as capital and operational expenditures, resource consumption, production output, and Monte Carlo simulations for risk analysis.

The system is designed to help mining companies make informed decisions by simulating various scenarios and analyzing potential outcomes based on stochastic variables.

A range of features are implemented to support these functionalities.

## Features

- **Scenario Management**: Manage multiple mining scenarios with independent parameter sets and outputs.
- **Process Parameters**: Define and persist process inputs via FastAPI endpoints and template-driven forms.
- **Cost Tracking**: Capture capital (`capex`) and operational (`opex`) expenditures per scenario.
- **Consumption Tracking**: Record resource consumption (chemicals, fuel, water, scrap) tied to scenarios.
- **Production Output**: Store production metrics such as tonnage, recovery, and revenue drivers.
- **Equipment Management**: Register scenario-specific equipment inventories.
- **Maintenance Logging**: Log maintenance events against equipment with dates and costs.
- **Reporting Dashboard**: Surface aggregated statistics for simulation outputs with an interactive Chart.js dashboard.
- **Monte Carlo Simulation (in progress)**: Services and routes are scaffolded for future stochastic analysis.

## Architecture

The architecture is documented in [docs/architecture.md](docs/architecture.md).

## Project Structure

The project is organized into several key directories:

- `models/`: Contains SQLAlchemy models representing database tables.
- `routes/`: Defines FastAPI routes for API endpoints.
- `services/`: Business logic and service layer.
- `components/`: Frontend components (to be defined).
- `config/`: Configuration files and settings.
- `middleware/`: Custom middleware for request/response processing.
- `tests/`: Unit and integration tests.
- `templates/`: Jinja2 HTML templates for server-side rendering.
- `docs/`: Documentation files.

Key files include:

- `main.py`: FastAPI application entry point.
- `.env`: Environment variables for configuration.
- `requirements.txt`: Python dependencies.

## Development

The development setup instructions are provided in [docs/development_setup.md](docs/development_setup.md).

To get started locally:

```powershell
# Clone the repository
git clone https://git.allucanget.biz/allucanget/calminer.git
cd calminer

# Create and activate a virtual environment
python -m venv .venv
.\.venv\Scripts\Activate.ps1

# Install dependencies
pip install -r requirements.txt

# Start the development server
uvicorn main:app --reload
```

## Usage Overview

- **API base URL**: `http://localhost:8000/api`
- **Key routes**:
  - `POST /api/scenarios/` create scenarios
  - `POST /api/parameters/` manage process parameters; payload supports optional `distribution_id` or inline `distribution_type`/`distribution_parameters` fields for simulation metadata
  - `POST /api/costs/capex` and `POST /api/costs/opex` capture project costs
  - `POST /api/consumption/` add consumption entries
  - `POST /api/production/` register production output
  - `POST /api/equipment/` create equipment records
  - `POST /api/maintenance/` log maintenance events
  - `POST /api/reporting/summary` aggregate simulation results, returning count, mean/median, min/max, standard deviation, variance, percentile bands (5/10/90/95), value-at-risk (95%) and expected shortfall (95%)

### Dashboard Preview

1. Start the FastAPI server and navigate to `/ui/dashboard` (ensure `routes/ui.py` exposes this template or add a router that serves `templates/Dashboard.html`).
2. Use the "Load Sample Data" button to populate the JSON textarea with demo results.
3. Select "Refresh Dashboard" to post the dataset to `/api/reporting/summary` and render the returned statistics and distribution chart.
4. Paste your own simulation outputs (array of objects containing a numeric `result` property) to visualize custom runs; the endpoint expects the same schema used by the reporting service.
5. If the summary endpoint is unavailable, the dashboard displays an inline error—refresh once the API is reachable.

## Testing

Testing guidelines and best practices are outlined in [docs/testing.md](docs/testing.md).

To execute the unit test suite:

```powershell
pytest
```

### Coverage Snapshot (2025-10-20)

- `pytest --cov=. --cov-report=term-missing` reports **95%** overall coverage across the project.
- Lower coverage hotspots to target next: `services/simulation.py` (79%), `middleware/validation.py` (78%), `routes/ui.py` (82%), and several API routers around lines 12-22 that create database sessions only.
- Deprecation cleanup migrated routes to Pydantic v2 patterns (`model_config = ConfigDict(...)`, `model_dump()`) and updated SQLAlchemy's `declarative_base`; reran `pytest` to confirm the suite passes without warnings.
- Coverage for route-heavy modules is primarily limited by error paths (e.g., bad request branches) that still need explicit tests.

## Database Objects

The database is composed of several tables that store different types of information.

- **CAPEX** — `capex`: Stores data on capital expenditures.
- **OPEX** — `opex`: Contains information on operational expenditures.
- **Chemical consumption** — `chemical_consumption`: Tracks the consumption of chemical reagents.
- **Fuel consumption** — `fuel_consumption`: Records the amount of fuel consumed.
- **Water consumption** — `water_consumption`: Monitors the use of water.
- **Scrap consumption** — `scrap_consumption`: Tracks the consumption of scrap materials.
- **Production output** — `production_output`: Stores data on production output, such as tons produced and recovery rates.
- **Equipment operation** — `equipment_operation`: Contains operational data for each piece of equipment.
- **Ore batch** — `ore_batch`: Stores information on ore batches, including their grade and other characteristics.
- **Exchange rate** — `exchange_rate`: Contains currency exchange rates.
- **Simulation result** — `simulation_result`: Stores the results of the Monte Carlo simulations.

## Static Parameters

These are values that are not expected to change frequently and are used for configuration purposes. Some examples include:

- **Currencies**: `currency_code`, `currency_name`.
- **Distribution types**: `distribution_name`.
- **Units**: `unit_name`, `unit_symbol`, `unit_system`, `conversion_to_base`.
- **Parameter categories**: `category_name`.
- **Material types**: `type_name`, `category`.
- **Chemical reagents**: `reagent_name`, `chemical_formula`.
- **Fuel**: `fuel_name`.
- **Water**: `water_type`.
- **Scrap material**: `scrap_name`.

## Variables

These are dynamic data points that are recorded over time and used in calculations and simulations. Some examples include:

- **CAPEX**: `amount`.
- **OPEX**: `amount`.
- **Chemical consumption**: `quantity`, `efficiency`, `waste_factor`.
- **Fuel consumption**: `quantity`.
- **Water consumption**: `quantity`.
- **Scrap consumption**: `quantity`.
- **Production output**: `tons_produced`, `recovery_rate`, `metal_content`, `metallurgical_loss`, `net_revenue`.
- **Equipment operation**: `hours_operated`, `downtime_hours`.
- **Ore batch**: `ore_grade`, `moisture`, `sulfur`, `chlorine`.
- **Exchange rate**: `rate`.
- **Parameter values**: `value`.
- **Simulation result**: NPV (`npv`), IRR (`irr`), EBITDA (`ebitda`), `net_revenue`.
- **Cementation parameters**: `temperature`, pH (`ph`), `reaction_time`, `copper_concentration`, `iron_surface_area`.
- **Precipitate product**: `density`, `melting_point`, `boiling_point`.