feat: add audit events and runtime state snapshots to database
- Introduced new tables for audit events and runtime state snapshots in the database schema. - Created data classes for AuditRecord and RuntimeStateRecord to represent the new entities. - Implemented AuditRepository and RuntimeStateRepository for inserting and retrieving records. - Enhanced the dashboard to include an audit trail section, displaying recent audit events. - Added tests for the new audit repository and runtime lifecycle functionalities. - Updated settings validation to ensure proper configuration for alerting features. - Integrated alert notifications across various components, including execution sequencer and loss limits.
This commit is contained in:
@@ -105,11 +105,14 @@ DUCKDB_PATH=./data/arbitrade.duckdb
|
||||
FERNET_KEY=
|
||||
KRAKEN_API_KEY=
|
||||
KRAKEN_API_SECRET=
|
||||
KRAKEN_API_KEY_PERMISSIONS=query,trade
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- Leave Kraken creds empty until Kraken integration lands.
|
||||
- If Kraken creds are set, both key and secret are required.
|
||||
- `KRAKEN_API_KEY_PERMISSIONS` must include `query,trade` and must not include withdrawal scope.
|
||||
- `FERNET_KEY` optional. If empty, keyring-backed key generation used by secret helper.
|
||||
- On Windows, app falls back to default `asyncio` loop. On non-Windows, `uvloop` installs automatically.
|
||||
|
||||
@@ -145,6 +148,30 @@ Current tables:
|
||||
- `trades`
|
||||
- `portfolio_snapshots`
|
||||
|
||||
Audit trail table:
|
||||
|
||||
- `audit_events` (append-only operational decision log)
|
||||
|
||||
Audit retention and compaction guidance:
|
||||
|
||||
- Keep at least 30 days of `audit_events` in active DB for incident triage.
|
||||
- Archive older rows to a timestamped export file before deletion.
|
||||
- Example monthly archive workflow:
|
||||
|
||||
```sql
|
||||
COPY (
|
||||
SELECT *
|
||||
FROM audit_events
|
||||
WHERE occurred_at < NOW() - INTERVAL 30 DAY
|
||||
) TO 'data/audit_events_archive_YYYYMM.parquet' (FORMAT PARQUET);
|
||||
|
||||
DELETE FROM audit_events
|
||||
WHERE occurred_at < NOW() - INTERVAL 30 DAY;
|
||||
```
|
||||
|
||||
- Back up archive files and the main DuckDB file together.
|
||||
- For production, run archive + backup as scheduled maintenance (cron/task scheduler).
|
||||
|
||||
## Quality Checks
|
||||
|
||||
Run tests:
|
||||
@@ -171,6 +198,18 @@ Run mypy:
|
||||
mypy src
|
||||
```
|
||||
|
||||
Run dependency vulnerability audit:
|
||||
|
||||
```powershell
|
||||
pip-audit --skip-editable
|
||||
```
|
||||
|
||||
Run secret scan (worktree + git history):
|
||||
|
||||
```powershell
|
||||
python scripts/security_scan.py
|
||||
```
|
||||
|
||||
Install pre-commit hooks:
|
||||
|
||||
```powershell
|
||||
@@ -282,3 +321,20 @@ uv pip install -e .[dev]
|
||||
```
|
||||
|
||||
If DuckDB file missing, start app once or create `data/` directory manually.
|
||||
|
||||
## Security Hardening
|
||||
|
||||
Threat model notes:
|
||||
|
||||
- Primary risk surfaces: environment secrets, dashboard auth credentials, exchange API key scope, and dependency supply chain.
|
||||
- Assumed attacker model: leaked repository content, leaked CI logs/artifacts, or unauthorized dashboard access.
|
||||
- High-impact outcomes to prevent: credential exfiltration, unauthorized withdrawals, and unsafe live-trading control changes.
|
||||
|
||||
Hardening checklist:
|
||||
|
||||
- Use least-privilege Kraken API keys: query + trade only; never enable withdrawal.
|
||||
- Rotate API keys immediately if secret scan flags a potential exposure.
|
||||
- Keep dashboard auth enabled in non-local environments and avoid default/shared credentials.
|
||||
- Run `pip-audit --skip-editable` in CI; treat vulnerability findings as release blockers.
|
||||
- Run `python scripts/security_scan.py` before release and after major merges.
|
||||
- Store secrets in environment/secret manager; never commit `.env` or key material.
|
||||
|
||||
Reference in New Issue
Block a user