--- title: '07 — Deployment View' description: 'Describe deployment topology, infrastructure components, and environments (dev/stage/prod).' status: draft --- # 07 — Deployment View ## Deployment Topology The CalMiner application is deployed using a multi-tier architecture consisting of the following layers: 1. **Client Layer**: This layer consists of web browsers that interact with the application through a user interface rendered by Jinja2 templates and enhanced with JavaScript (Chart.js for dashboards). 2. **Web Application Layer**: This layer hosts the FastAPI application, which handles API requests, business logic, and serves HTML templates. It communicates with the database layer for data persistence. 3. **Database Layer**: This layer consists of a PostgreSQL database that stores all application data, including scenarios, parameters, costs, consumption, production outputs, equipment, maintenance logs, and simulation results. ```mermaid graph TD A[Client Layer] --> B[Web Application Layer] B --> C[Database Layer] ``` ## Infrastructure Components The infrastructure components for the application include: - **Reverse Proxy (optional)**: An Nginx or Apache server can be used as a reverse proxy. - **Containerization**: Docker images are generated via the repository `Dockerfile`, using a multi-stage build to keep the final runtime minimal. - **CI/CD Pipeline**: Automated pipelines (Gitea Actions) run tests, build/push Docker images, and trigger deployments. - **Gitea Actions Workflows**: Located under `.gitea/workflows/`, these workflows handle testing, building, pushing, and deploying the application. - **Gitea Action Runners**: Self-hosted runners execute the CI/CD workflows. - **Testing and Continuous Integration**: Automated tests ensure code quality before deployment, also documented in [Testing & CI](07_deployment/07_01_testing_ci.md.md). - **Docker Infrastructure**: Docker is used to containerize the application for consistent deployment across environments. - **Portainer**: Production deployment environment for managing Docker containers. - **Web Server**: Hosts the FastAPI application and serves API endpoints. - **Database Server**: PostgreSQL database for persisting application data. - **Static File Server**: Serves static assets such as CSS, JavaScript, and image files. - **Cloud Infrastructure (optional)**: The application can be deployed on cloud platforms. ```mermaid graph TD G[Git Repository] --> C[CI/CD Pipeline] C --> GAW[Gitea Action Workflows] GAW --> GAR[Gitea Action Runners] GAR --> T[Testing] GAR --> CI[Continuous Integration] T --> G CI --> G W[Web Server] --> DB[Database Server] RP[Reverse Proxy] --> W I((Internet)) <--> RP PO[Containerization] --> W C[CI/CD Pipeline] --> PO W --> S[Static File Server] S --> RP PO --> DB PO --> S ``` ## Environments The application can be deployed in multiple environments to support development, testing, and production. ```mermaid graph TD R[Repository] --> DEV[Development Environment] R[Repository] --> TEST[Testing Environment] R[Repository] --> PROD[Production Environment] DEV --> W_DEV[Web Server - Dev] DEV --> DB_DEV[Database Server - Dev] TEST --> W_TEST[Web Server - Test] TEST --> DB_TEST[Database Server - Test] PROD --> W_PROD[Web Server - Prod] PROD --> DB_PROD[Database Server - Prod] ``` ### Development Environment The development environment is set up for local development and testing. It includes: - Local PostgreSQL instance (docker compose recommended, script available at `docker-compose.postgres.yml`) - FastAPI server running in debug mode `docker-compose.dev.yml` encapsulates this topology: - `api` service mounts the repository for live reloads (`uvicorn --reload`) and depends on the database health check. - `db` service uses the Debian-based `postgres:16` image with UTF-8 locale configuration and persists data in `pg_data_dev`. - A shared `calminer_backend` bridge network keeps traffic contained; ports 8000/5432 are published for local tooling. See [docs/quickstart.md](../quickstart.md#compose-driven-development-stack) for command examples and volume maintenance tips. ### Testing Environment The testing environment is set up for automated testing and quality assurance. It includes: - Staging PostgreSQL instance - FastAPI server running in testing mode - Automated test suite (e.g., pytest) for running unit and integration tests `docker-compose.test.yml` provisions an ephemeral CI-like stack: - `tests` service builds the application image, installs `requirements-test.txt`, runs the database setup script (dry-run + apply), then executes pytest. - `api` service is available on port 8001 for manual verification against the test database. - `postgres` service seeds a disposable Postgres 16 instance with health checks and named volumes (`pg_data_test`, `pip_cache_test`). Typical commands mirror the CI workflow (`docker compose -f docker-compose.test.yml run --rm tests`); the [quickstart](../quickstart.md#compose-driven-test-stack) lists variations and teardown steps. ### Production Environment The production environment is set up for serving live traffic and includes: - Production PostgreSQL instance - FastAPI server running in production mode - Load balancer (Traefik) for distributing incoming requests - Monitoring and logging tools for tracking application performance #### Production docker compose topology - `docker-compose.prod.yml` defines the runtime topology for operator-managed deployments. - `api` service runs the FastAPI image with resource limits (`API_LIMIT_CPUS`, `API_LIMIT_MEMORY`) and a `/health` probe consumed by Traefik and the Compose health check. - `traefik` service (enabled via the `reverse-proxy` profile) terminates TLS using the ACME resolver configured by `TRAEFIK_ACME_EMAIL` and routes `CALMINER_DOMAIN` traffic to the API. - `postgres` service (enabled via the `local-db` profile) exists for edge deployments without managed PostgreSQL and persists data in the `pg_data_prod` volume while mounting `./backups` for operator snapshots. - All services join the configurable `CALMINER_NETWORK` (defaults to `calminer_backend`) to keep traffic isolated from host networks. Deployment workflow: 1. Copy `config/setup_production.env.example` to `config/setup_production.env` and populate domain, registry image tag, database credentials, and resource budgets. 2. Launch the stack with `docker compose --env-file config/setup_production.env -f docker-compose.prod.yml --profile reverse-proxy up -d` (append `--profile local-db` when hosting Postgres locally). 3. Run database migrations and seeding using `docker compose --env-file config/setup_production.env -f docker-compose.prod.yml run --rm api python scripts/setup_database.py --run-migrations --seed-data`. 4. Monitor container health via `docker compose -f docker-compose.prod.yml ps` or Traefik dashboards; the API health endpoint returns `{ "status": "ok" }` when ready. 5. Shut down with `docker compose -f docker-compose.prod.yml down` (volumes persist unless `-v` is supplied). ## Containerized Deployment Flow The Docker-based deployment path aligns with the solution strategy documented in [Solution Strategy](04_solution_strategy.md) and the CI practices captured in [Testing & CI](07_deployment/07_01_testing_ci.md.md). ### Image Build - The multi-stage `Dockerfile` installs dependencies in a builder layer (including system compilers and Python packages) and copies only the required runtime artifacts to the final image. - Build arguments are minimal; database configuration is supplied at runtime via granular variables (`DATABASE_DRIVER`, `DATABASE_HOST`, `DATABASE_PORT`, `DATABASE_USER`, `DATABASE_PASSWORD`, `DATABASE_NAME`, optional `DATABASE_SCHEMA`). Secrets and configuration should be passed via environment variables or an orchestrator. - The resulting image exposes port `8000` and starts `uvicorn main:app` (see main [README.md](../../README.md)). ### Runtime Environment - For single-node deployments, run the container alongside PostgreSQL/Redis using Docker Compose or an equivalent orchestrator. - A reverse proxy (Traefik) terminates TLS and forwards traffic to the container on port `8000`. - Migrations must be applied prior to rolling out a new image; automation can hook into the deploy step to run `scripts/run_migrations.py`. ### CI/CD Integration - Gitea Actions workflows reside under `.gitea/workflows/`. - `test.yml` executes the pytest suite using cached pip dependencies. - `build-and-push.yml` logs into the container registry, rebuilds the Docker image using GitHub Actions cache-backed layers, and pushes `latest` (and additional tags as required). - `deploy.yml` connects to the target host via SSH, pulls the pushed tag, stops any existing container, and launches the new version. - Required secrets: `REGISTRY_URL`, `REGISTRY_USERNAME`, `REGISTRY_PASSWORD`, `SSH_HOST`, `SSH_USERNAME`, `SSH_PRIVATE_KEY`. - Extend these workflows when introducing staging/blue-green deployments; keep cross-links with [Testing & CI](07_deployment/07_01_testing_ci.md.md) up to date. ## Integrations and Future Work (deployment-related) - **Persistence of results**: `/api/simulations/run` currently returns in-memory results; next iteration should persist to `simulation_result` and reference scenarios. - **Deployment**: implement infrastructure-as-code (e.g., Terraform/Ansible) to provision the hosting environment and maintain parity across dev/stage/prod.