feat(reports): store and browse report artifacts

This commit is contained in:
zwitschi committed 2026-11-06 12:27:00 +01:00
1 parent 5967a43eb5
commit 6f2e479958
13 files changed
+875 -93

No files matched your search

+27 -5
View File
@@ -91,11 +91,10 @@ using opaque text as the only source of truth.
`limit`, and opaque `cursor` filters. Return a stable list of artifact
metadata: `id`, `source`, `scan_id`, `type`, `generated_at`, and available
export formats. Enforce a bounded default and maximum page size.
- Add `GET /api/reports/<artifact_id>` for the structured payload and
provenance metadata. Keep it separate from the existing
`GET /api/reports/<source>/<path>` route to avoid changing that route's
meaning during migration.
- Add an explicit export route for supported report types and formats
- Add `GET /api/report-artifacts/<artifact_id>` for the structured payload and
provenance metadata. Use a distinct path because `/api/reports/<source>`
already owns the one-segment route shape.
- Add `GET /api/report-artifacts/<artifact_id>/export` for supported report types and formats
(initially JSON and, where useful, CSV/TXT). Generate exports from canonical
tables and versioned artifact payloads; do not accept filesystem paths from
a database-backed export request.
@@ -118,3 +117,26 @@ as sensitive operational data: the existing web server must remain
localhost-only, payloads should include only fields needed by the report, and
new endpoints must not expose raw log data by default. Backups and exported
files inherit the same access and retention requirements as the database.
## Migration Implementation Status
- Schema v2 adds `report_artifacts` with source, scan link, report type,
generation time, payload version, original relative path, and SHA-256 hash.
- `python -m app.db.backfill --dry-run --reports-only` inventories supported
report files by source and format. The importer is idempotent, preserves
malformed JSON as text with a parse error, reports unreadable/oversized
files, and never modifies source files. It skips exports owned by canonical
tables (for example `requests.csv` and current blocklists).
- New analyzer text summaries and consolidated blocklist change reports are
written to SQLite and their existing files. Firewall snapshots continue to
be persisted in the normalized firewall tables.
- The Reports page lists database artifacts first, supports cursor pagination
and JSON/CSV/TXT downloads, and retains the existing file-backed routes as a
fallback for unmigrated files.
- Validation covers idempotency, malformed/oversized inputs, source/format
counts, backup/restore, dual-write payloads, and API fallback. A workspace
dry-run found 76 files: 60 eligible, 16 canonical exports skipped, and none
unmappable at the time of the check.
- No report files have been retired. Keep this gate closed until every listed
consumer has moved or has an explicit compatibility requirement, and a
production backup/restore and parity review has been approved.