feat: add audit events and runtime state snapshots to database

- Introduced new tables for audit events and runtime state snapshots in the database schema.
- Created data classes for AuditRecord and RuntimeStateRecord to represent the new entities.
- Implemented AuditRepository and RuntimeStateRepository for inserting and retrieving records.
- Enhanced the dashboard to include an audit trail section, displaying recent audit events.
- Added tests for the new audit repository and runtime lifecycle functionalities.
- Updated settings validation to ensure proper configuration for alerting features.
- Integrated alert notifications across various components, including execution sequencer and loss limits.
This commit is contained in:
2026-06-01 14:18:12 +02:00
parent b413c66ca4
commit c17f41aaf8
34 changed files with 2608 additions and 60 deletions
+56
View File
@@ -105,11 +105,14 @@ DUCKDB_PATH=./data/arbitrade.duckdb
FERNET_KEY=
KRAKEN_API_KEY=
KRAKEN_API_SECRET=
KRAKEN_API_KEY_PERMISSIONS=query,trade
```
Notes:
- Leave Kraken creds empty until Kraken integration lands.
- If Kraken creds are set, both key and secret are required.
- `KRAKEN_API_KEY_PERMISSIONS` must include `query,trade` and must not include withdrawal scope.
- `FERNET_KEY` optional. If empty, keyring-backed key generation used by secret helper.
- On Windows, app falls back to default `asyncio` loop. On non-Windows, `uvloop` installs automatically.
@@ -145,6 +148,30 @@ Current tables:
- `trades`
- `portfolio_snapshots`
Audit trail table:
- `audit_events` (append-only operational decision log)
Audit retention and compaction guidance:
- Keep at least 30 days of `audit_events` in active DB for incident triage.
- Archive older rows to a timestamped export file before deletion.
- Example monthly archive workflow:
```sql
COPY (
SELECT *
FROM audit_events
WHERE occurred_at < NOW() - INTERVAL 30 DAY
) TO 'data/audit_events_archive_YYYYMM.parquet' (FORMAT PARQUET);
DELETE FROM audit_events
WHERE occurred_at < NOW() - INTERVAL 30 DAY;
```
- Back up archive files and the main DuckDB file together.
- For production, run archive + backup as scheduled maintenance (cron/task scheduler).
## Quality Checks
Run tests:
@@ -171,6 +198,18 @@ Run mypy:
mypy src
```
Run dependency vulnerability audit:
```powershell
pip-audit --skip-editable
```
Run secret scan (worktree + git history):
```powershell
python scripts/security_scan.py
```
Install pre-commit hooks:
```powershell
@@ -282,3 +321,20 @@ uv pip install -e .[dev]
```
If DuckDB file missing, start app once or create `data/` directory manually.
## Security Hardening
Threat model notes:
- Primary risk surfaces: environment secrets, dashboard auth credentials, exchange API key scope, and dependency supply chain.
- Assumed attacker model: leaked repository content, leaked CI logs/artifacts, or unauthorized dashboard access.
- High-impact outcomes to prevent: credential exfiltration, unauthorized withdrawals, and unsafe live-trading control changes.
Hardening checklist:
- Use least-privilege Kraken API keys: query + trade only; never enable withdrawal.
- Rotate API keys immediately if secret scan flags a potential exposure.
- Keep dashboard auth enabled in non-local environments and avoid default/shared credentials.
- Run `pip-audit --skip-editable` in CI; treat vulnerability findings as release blockers.
- Run `python scripts/security_scan.py` before release and after major merges.
- Store secrets in environment/secret manager; never commit `.env` or key material.