118 lines
4.5 KiB
Markdown
118 lines
4.5 KiB
Markdown
# 14 Testing, CI and Quality Assurance
|
|
|
|
This chapter centralizes the project's testing strategy, CI configuration, and quality targets.
|
|
|
|
## Overview
|
|
|
|
CalMiner uses a combination of unit, integration, and end-to-end tests to ensure quality.
|
|
|
|
### Frameworks
|
|
|
|
- Backend: pytest for unit and integration tests.
|
|
- Frontend: pytest with Playwright for E2E tests.
|
|
- Database: pytest fixtures with psycopg2 for DB tests.
|
|
|
|
### Test Types
|
|
|
|
- Unit Tests: Test individual functions/modules.
|
|
- Integration Tests: Test API endpoints and DB interactions.
|
|
- E2E Tests: Playwright for full user flows.
|
|
|
|
### CI/CD
|
|
|
|
- Use Gitea Actions for CI/CD; workflows live under `.gitea/workflows/`.
|
|
- `test.yml` runs on every push with cached Python dependencies via `actions/cache@v3`.
|
|
- `build-and-push.yml` builds the Docker image with `docker/build-push-action@v2`, reusing GitHub Actions cache-backed layers, and pushes to the Gitea registry.
|
|
- `deploy.yml` connects to the target host (via `appleboy/ssh-action`) to pull the freshly pushed image and restart the container.
|
|
- Mandatory secrets: `REGISTRY_USERNAME`, `REGISTRY_PASSWORD`, `REGISTRY_URL`, `SSH_HOST`, `SSH_USERNAME`, `SSH_PRIVATE_KEY`.
|
|
- Run tests on pull requests to shared branches; enforce coverage target ≥80% (pytest-cov).
|
|
|
|
### Running Tests
|
|
|
|
- Unit: `pytest tests/unit/`
|
|
- E2E: `pytest tests/e2e/`
|
|
- All: `pytest`
|
|
|
|
### Test Directory Structure
|
|
|
|
Organize tests under the `tests/` directory mirroring the application structure:
|
|
|
|
````text
|
|
tests/
|
|
unit/
|
|
test_<module>.py
|
|
e2e/
|
|
test_<flow>.py
|
|
fixtures/
|
|
conftest.py
|
|
```python
|
|
|
|
### Fixtures and Test Data
|
|
|
|
- Define reusable fixtures in `tests/fixtures/conftest.py`.
|
|
- Use temporary in-memory databases or isolated schemas for DB tests.
|
|
- Load sample data via fixtures for consistent test environments.
|
|
- Leverage the `seeded_ui_data` fixture in `tests/unit/conftest.py` to populate scenarios with related cost, maintenance, and simulation records for deterministic UI route checks.
|
|
|
|
### E2E (Playwright) Tests
|
|
|
|
The E2E test suite, located in `tests/e2e/`, uses Playwright to simulate user interactions in a live browser environment. These tests are designed to catch issues in the UI, frontend-backend integration, and overall application flow.
|
|
|
|
#### Fixtures
|
|
|
|
- `live_server`: A session-scoped fixture that launches the FastAPI application in a separate process, making it accessible to the browser.
|
|
- `playwright_instance`, `browser`, `page`: Standard `pytest-playwright` fixtures for managing the Playwright instance, browser, and individual pages.
|
|
|
|
#### Smoke Tests
|
|
|
|
- UI Page Loading: `test_smoke.py` contains a parameterized test that systematically navigates to all UI routes to ensure they load without errors, have the correct title, and display a primary heading.
|
|
- Form Submissions: Each major form in the application has a corresponding test file (e.g., `test_scenarios.py`, `test_costs.py`) that verifies: page loads, create item by filling the form, success message, and UI updates.
|
|
|
|
### Running E2E Tests
|
|
|
|
To run the Playwright tests:
|
|
|
|
```bash
|
|
pytest tests/e2e/
|
|
````
|
|
|
|
To run headed mode:
|
|
|
|
```bash
|
|
pytest tests/e2e/ --headed
|
|
```
|
|
|
|
### Mocking and Dependency Injection
|
|
|
|
- Use `unittest.mock` to mock external dependencies.
|
|
- Inject dependencies via function parameters or FastAPI's dependency overrides in tests.
|
|
|
|
### Code Coverage
|
|
|
|
- Install `pytest-cov` to generate coverage reports.
|
|
- Run with coverage: `pytest --cov --cov-report=term` (use `--cov-report=html` when visualizing hotspots).
|
|
- Target 95%+ overall coverage. Focus on historically low modules: `services/simulation.py`, `services/reporting.py`, `middleware/validation.py`, and `routes/ui.py`.
|
|
- Latest snapshot (2025-10-21): `pytest --cov=. --cov-report=term-missing` returns **91%** overall coverage.
|
|
|
|
### CI Integration
|
|
|
|
`test.yml` encapsulates the steps below:
|
|
|
|
- Check out the repository and set up Python 3.10.
|
|
- Restore the pip cache (keyed by `requirements.txt`).
|
|
- Install project dependencies and Playwright browsers (if needed for E2E).
|
|
- Run `pytest` (extend with `--cov` flags when enforcing coverage).
|
|
|
|
`build-and-push.yml` adds:
|
|
|
|
- Registry login using repository secrets.
|
|
- Docker image build/push with GHA cache storage (`cache-from/cache-to` set to `type=gha`).
|
|
|
|
`deploy.yml` handles:
|
|
|
|
- SSH into the deployment host.
|
|
- Pull the tagged image from the registry.
|
|
- Stop, remove, and relaunch the `calminer` container exposing port 8000.
|
|
|
|
When adding new workflows, mirror this structure to ensure secrets, caching, and deployment steps remain aligned with the production environment.
|