14 Testing, CI and Quality Assurance

This chapter centralizes the project's testing strategy, CI configuration, and quality targets.

Overview

CalMiner uses a combination of unit, integration, and end-to-end tests to ensure quality.

Frameworks

Backend: pytest for unit and integration tests.
Frontend: pytest with Playwright for E2E tests.
Database: pytest fixtures with psycopg2 for DB tests.

Test Types

Unit Tests: Test individual functions/modules.
Integration Tests: Test API endpoints and DB interactions.
E2E Tests: Playwright for full user flows.

CI/CD

Use Gitea Actions for CI/CD; workflows live under .gitea/workflows/.
test.yml runs on every push with cached Python dependencies via actions/cache@v3.
build-and-push.yml builds the Docker image with docker/build-push-action@v2, reusing GitHub Actions cache-backed layers, and pushes to the Gitea registry.
deploy.yml connects to the target host (via appleboy/ssh-action) to pull the freshly pushed image and restart the container.
Mandatory secrets: GITEA_USERNAME, GITEA_PASSWORD, GITEA_REGISTRY, SSH_HOST, SSH_USERNAME, SSH_PRIVATE_KEY.
Run tests on pull requests to shared branches; enforce coverage target ≥80% (pytest-cov).

Running Tests

Unit: pytest tests/unit/
E2E: pytest tests/e2e/
All: pytest

Test Directory Structure

Organize tests under the tests/ directory mirroring the application structure:

tests/
  unit/
    test_<module>.py
  e2e/
    test_<flow>.py
  fixtures/
    conftest.py
```python

### Fixtures and Test Data

- Define reusable fixtures in `tests/fixtures/conftest.py`.
- Use temporary in-memory databases or isolated schemas for DB tests.
- Load sample data via fixtures for consistent test environments.
- Leverage the `seeded_ui_data` fixture in `tests/unit/conftest.py` to populate scenarios with related cost, maintenance, and simulation records for deterministic UI route checks.

### E2E (Playwright) Tests

The E2E test suite, located in `tests/e2e/`, uses Playwright to simulate user interactions in a live browser environment. These tests are designed to catch issues in the UI, frontend-backend integration, and overall application flow.

#### Fixtures

- `live_server`: A session-scoped fixture that launches the FastAPI application in a separate process, making it accessible to the browser.
- `playwright_instance`, `browser`, `page`: Standard `pytest-playwright` fixtures for managing the Playwright instance, browser, and individual pages.

#### Smoke Tests

- UI Page Loading: `test_smoke.py` contains a parameterized test that systematically navigates to all UI routes to ensure they load without errors, have the correct title, and display a primary heading.
- Form Submissions: Each major form in the application has a corresponding test file (e.g., `test_scenarios.py`, `test_costs.py`) that verifies: page loads, create item by filling the form, success message, and UI updates.

### Running E2E Tests

To run the Playwright tests:

```bash
pytest tests/e2e/

To run headed mode:

pytest tests/e2e/ --headed

Mocking and Dependency Injection

Use unittest.mock to mock external dependencies.
Inject dependencies via function parameters or FastAPI's dependency overrides in tests.

Code Coverage

Install pytest-cov to generate coverage reports.
Run with coverage: pytest --cov --cov-report=term (use --cov-report=html when visualizing hotspots).
Target 95%+ overall coverage. Focus on historically low modules: services/simulation.py, services/reporting.py, middleware/validation.py, and routes/ui.py.
Latest snapshot (2025-10-21): pytest --cov=. --cov-report=term-missing returns 91% overall coverage.

CI Integration

test.yml encapsulates the steps below:

Check out the repository and set up Python 3.10.
Restore the pip cache (keyed by requirements.txt).
Install project dependencies and Playwright browsers (if needed for E2E).
Run pytest (extend with --cov flags when enforcing coverage).

build-and-push.yml adds:

Registry login using repository secrets.
Docker image build/push with GHA cache storage (cache-from/cache-to set to type=gha).

deploy.yml handles:

SSH into the deployment host.
Pull the tagged image from the registry.
Stop, remove, and relaunch the calminer container exposing port 8000.

When adding new workflows, mirror this structure to ensure secrets, caching, and deployment steps remain aligned with the production environment.

4.5 KiB Raw Blame History