feat: Refactor database configuration to use granular environment variables; update Docker and CI/CD workflows accordingly
Some checks failed
Build and Push Docker Image / build-and-push (push) Failing after 6s
Deploy to Server / deploy (push) Failing after 2s

This commit is contained in:
2025-10-23 19:17:24 +02:00
parent 8c3062fd80
commit 8dedfb8f26
15 changed files with 219 additions and 64 deletions

View File

@@ -38,7 +38,7 @@ Frontend components are server-rendered Jinja2 templates, with Chart.js powering
### Current implementation status (summary)
- Currency normalization, simulation scaffold, and reporting service exist; see [quickstart](docs/quickstart.md) for full status and migration instructions.
- Currency normalization, simulation scaffold, and reporting service exist; see [quickstart](../quickstart.md) for full status and migration instructions.
## MVP Features (migrated)

View File

@@ -6,7 +6,7 @@ This file contains the implementation plan (MVP features, steps, and estimates).
1. Connect to PostgreSQL database with schema `calminer`.
1. Create and activate a virtual environment and install dependencies via `requirements.txt`.
1. Define environment variables in `.env`, including `DATABASE_URL`.
1. Define database environment variables in `.env` (e.g., `DATABASE_DRIVER`, `DATABASE_HOST`, `DATABASE_PORT`, `DATABASE_USER`, `DATABASE_PASSWORD`, `DATABASE_NAME`, optional `DATABASE_SCHEMA`).
1. Configure FastAPI entrypoint in `main.py` to include routers.
## Feature: Scenario Management
@@ -107,4 +107,4 @@ This file contains the implementation plan (MVP features, steps, and estimates).
1. Write unit tests in `tests/unit/test_reporting.py`.
1. Enhance UI in `components/Dashboard.html` with charts.
See [UI and Style](docs/architecture/13_ui_and_style.md) for the UI template audit, layout guidance, and next steps.
See [UI and Style](../13_ui_and_style.md) for the UI template audit, layout guidance, and next steps.

View File

@@ -8,13 +8,13 @@ status: draft
## Architecture overview
This overview complements [architecture](docs/architecture/README.md) with a high-level map of CalMiner's module layout and request flow.
This overview complements [architecture](README.md) with a high-level map of CalMiner's module layout and request flow.
Refer to the detailed architecture chapters in `docs/architecture/`:
- Module map & components: [Building Block View](docs/architecture/05_building_block_view.md)
- Request flow & runtime interactions: [Runtime View](docs/architecture/06_runtime_view.md)
- Simulation roadmap & strategy: [Solution Strategy](docs/architecture/04_solution_strategy.md)
- Module map & components: [Building Block View](05_building_block_view.md)
- Request flow & runtime interactions: [Runtime View](06_runtime_view.md)
- Simulation roadmap & strategy: [Solution Strategy](04_solution_strategy.md)
## System Components

View File

@@ -64,8 +64,8 @@ The Docker-based deployment path aligns with the solution strategy documented in
### Image Build
- The multi-stage `Dockerfile` installs dependencies in a builder layer (including system compilers and Python packages) and copies only the required runtime artifacts to the final image.
- Build arguments are minimal; environment configuration (e.g., `DATABASE_URL`) is supplied at runtime. Secrets and configuration should be passed via environment variables or an orchestrator.
- The resulting image exposes port `8000` and starts `uvicorn main:app` (s. [README.md](../README.md)).
- Build arguments are minimal; database configuration is supplied at runtime via granular variables (`DATABASE_DRIVER`, `DATABASE_HOST`, `DATABASE_PORT`, `DATABASE_USER`, `DATABASE_PASSWORD`, `DATABASE_NAME`, optional `DATABASE_SCHEMA`). Secrets and configuration should be passed via environment variables or an orchestrator.
- The resulting image exposes port `8000` and starts `uvicorn main:app` (s. [README.md](../../README.md)).
### Runtime Environment
@@ -79,7 +79,7 @@ The Docker-based deployment path aligns with the solution strategy documented in
- `test.yml` executes the pytest suite using cached pip dependencies.
- `build-and-push.yml` logs into the container registry, rebuilds the Docker image using GitHub Actions cache-backed layers, and pushes `latest` (and additional tags as required).
- `deploy.yml` connects to the target host via SSH, pulls the pushed tag, stops any existing container, and launches the new version.
- Required secrets: `GITEA_REGISTRY`, `GITEA_USERNAME`, `GITEA_PASSWORD`, `SSH_HOST`, `SSH_USERNAME`, `SSH_PRIVATE_KEY`.
- Required secrets: `REGISTRY_URL`, `REGISTRY_USERNAME`, `REGISTRY_PASSWORD`, `SSH_HOST`, `SSH_USERNAME`, `SSH_PRIVATE_KEY`.
- Extend these workflows when introducing staging/blue-green deployments; keep cross-links with [14 — Testing & CI](14_testing_ci.md) up to date.
## Integrations and Future Work (deployment-related)

View File

@@ -24,7 +24,7 @@ CalMiner uses a combination of unit, integration, and end-to-end tests to ensure
- `test.yml` runs on every push with cached Python dependencies via `actions/cache@v3`.
- `build-and-push.yml` builds the Docker image with `docker/build-push-action@v2`, reusing GitHub Actions cache-backed layers, and pushes to the Gitea registry.
- `deploy.yml` connects to the target host (via `appleboy/ssh-action`) to pull the freshly pushed image and restart the container.
- Mandatory secrets: `GITEA_USERNAME`, `GITEA_PASSWORD`, `GITEA_REGISTRY`, `SSH_HOST`, `SSH_USERNAME`, `SSH_PRIVATE_KEY`.
- Mandatory secrets: `REGISTRY_USERNAME`, `REGISTRY_PASSWORD`, `REGISTRY_URL`, `SSH_HOST`, `SSH_USERNAME`, `SSH_PRIVATE_KEY`.
- Run tests on pull requests to shared branches; enforce coverage target ≥80% (pytest-cov).
### Running Tests

View File

@@ -10,7 +10,7 @@ This document outlines the local development environment and steps to get the pr
## Clone and Project Setup
```powershell
````powershell
# Clone the repository
git clone https://git.allucanget.biz/allucanget/calminer.git
cd calminer
@@ -36,28 +36,34 @@ pip install -r requirements.txt
```sql
CREATE USER calminer_user WITH PASSWORD 'your_password';
```
````
1. Create database:
```sql
````sql
CREATE DATABASE calminer;
```python
## Environment Variables
1. Copy `.env.example` to `.env` at project root.
1. Edit `.env` to set database connection string:
1. Edit `.env` to set database connection details:
```dotenv
DATABASE_URL=postgresql://<user>:<password>@localhost:5432/calminer
```
DATABASE_DRIVER=postgresql
DATABASE_HOST=localhost
DATABASE_PORT=5432
DATABASE_USER=calminer_user
DATABASE_PASSWORD=your_password
DATABASE_NAME=calminer
DATABASE_SCHEMA=public
````
1. The application uses `python-dotenv` to load these variables.
1. The application uses `python-dotenv` to load these variables. A legacy `DATABASE_URL` value is still accepted if the granular keys are omitted.
## Running the Application
```powershell
````powershell
# Start the FastAPI server
uvicorn main:app --reload
```python
@@ -66,6 +72,6 @@ uvicorn main:app --reload
```powershell
pytest
```
````
E2E tests use Playwright and a session-scoped `live_server` fixture that starts the app at `http://localhost:8001` for browser-driven tests.