calminer/docs/architecture/14_testing_ci.md

14 Testing, CI and Quality Assurance

This chapter centralizes the project's testing strategy, CI configuration, and quality targets.

Overview

CalMiner uses a combination of unit, integration, and end-to-end tests to ensure quality.

Frameworks

• Backend: pytest for unit and integration tests.
• Frontend: pytest with Playwright for E2E tests.
• Database: pytest fixtures with psycopg2 for DB tests.

Test Types

• Unit Tests: Test individual functions/modules.
• Integration Tests: Test API endpoints and DB interactions.
• E2E Tests: Playwright for full user flows.

CI/CD

• Use Gitea Actions for CI/CD; workflows live under .gitea/workflows/.
• test.yml runs on every push with cached Python dependencies via actions/cache@v3.
• build-and-push.yml builds the Docker image with docker/build-push-action@v2, reusing GitHub Actions cache-backed layers, and pushes to the Gitea registry.
• deploy.yml connects to the target host (via appleboy/ssh-action) to pull the freshly pushed image and restart the container.
• Mandatory secrets: REGISTRY_USERNAME, REGISTRY_PASSWORD, REGISTRY_URL, SSH_HOST, SSH_USERNAME, SSH_PRIVATE_KEY.
• Run tests on pull requests to shared branches; enforce coverage target ≥80% (pytest-cov).

Running Tests

• Unit: pytest tests/unit/
• E2E: pytest tests/e2e/
• All: pytest

Test Directory Structure

Organize tests under the tests/ directory mirroring the application structure:

tests/
  unit/
    test_<module>.py
  e2e/
    test_<flow>.py
  fixtures/
    conftest.py
```python

### Fixtures and Test Data

- Define reusable fixtures in `tests/fixtures/conftest.py`.
- Use temporary in-memory databases or isolated schemas for DB tests.
- Load sample data via fixtures for consistent test environments.
- Leverage the `seeded_ui_data` fixture in `tests/unit/conftest.py` to populate scenarios with related cost, maintenance, and simulation records for deterministic UI route checks.

### E2E (Playwright) Tests

The E2E test suite, located in `tests/e2e/`, uses Playwright to simulate user interactions in a live browser environment. These tests are designed to catch issues in the UI, frontend-backend integration, and overall application flow.

#### Fixtures

- `live_server`: A session-scoped fixture that launches the FastAPI application in a separate process, making it accessible to the browser.
- `playwright_instance`, `browser`, `page`: Standard `pytest-playwright` fixtures for managing the Playwright instance, browser, and individual pages.

#### Smoke Tests

- UI Page Loading: `test_smoke.py` contains a parameterized test that systematically navigates to all UI routes to ensure they load without errors, have the correct title, and display a primary heading.
- Form Submissions: Each major form in the application has a corresponding test file (e.g., `test_scenarios.py`, `test_costs.py`) that verifies: page loads, create item by filling the form, success message, and UI updates.

### Running E2E Tests

To run the Playwright tests:

```bash
pytest tests/e2e/

To run headed mode:

pytest tests/e2e/ --headed

Mocking and Dependency Injection

• Use unittest.mock to mock external dependencies.
• Inject dependencies via function parameters or FastAPI's dependency overrides in tests.

Code Coverage

• Install pytest-cov to generate coverage reports.
• Run with coverage: pytest --cov --cov-report=term (use --cov-report=html when visualizing hotspots).
• Target 95%+ overall coverage. Focus on historically low modules: services/simulation.py, services/reporting.py, middleware/validation.py, and routes/ui.py.
• Latest snapshot (2025-10-21): pytest --cov=. --cov-report=term-missing returns 91% overall coverage.

CI Integration

test.yml encapsulates the steps below:

• Check out the repository and set up Python 3.10.
• Restore the pip cache (keyed by requirements.txt).
• Install project dependencies and Playwright browsers (if needed for E2E).
• Run pytest (extend with --cov flags when enforcing coverage).

build-and-push.yml adds:

• Registry login using repository secrets.
• Docker image build/push with GHA cache storage (cache-from/cache-to set to type=gha).

deploy.yml handles:

• SSH into the deployment host.
• Pull the tagged image from the registry.
• Stop, remove, and relaunch the calminer container exposing port 8000.

When adding new workflows, mirror this structure to ensure secrets, caching, and deployment steps remain aligned with the production environment.