intrusion-detection/REPORTS.md

Report Inventory

This inventory records the current report surface before moving report reads or writes into SQLite. Generated paths are rooted at REPORT_DIR (normally reports/). The SQLite tables listed below already exist in schema v1 unless noted otherwise.

Artifacts

Area	Producer and formats	Current role and consumers	Classification
copyparty/	Analyzer.write_report() creates timestamped .txt; write_data() creates per-IP CSV/JSON, denied-path CSV, flagged/banned IP TXT, requests.csv, authed_users.json, and mikrotik-block.rsc.	Reports API serves top-level text reports. requests.csv is imported by app.db.backfill; per-run data supports inspection and legacy workflows. Retention prunes generated files.	Request rows and per-scan IP stats are canonical in requests and ip_stats; rendered summaries and exports are derived.
npm/	Same analyzer outputs, plus user-agent and host summary CSV files.	Same consumers as copyparty. requests.csv is a legacy backfill input; retention prunes generated files.	Request rows and per-scan IP stats are canonical in requests and ip_stats; summaries and exports are derived.
app/data/	Consolidated scan writes blocklist_ipv4.txt and blocklist_ipv6.txt.	Blocklist UI/API reads these files; scan history reconstruction and firewall sync use them as current/compatibility inputs. Retention explicitly protects them.	Operational compatibility exports. Canonical blocked targets and events belong in blocklist_entries and blocklist_events; keep files until every consumer uses the DB and git-history contract is replaced.
app/ timestamped JSON	write_blocklist_report() emits scan/firewall change reports when the router is reachable.	Human-readable audit trail containing run delta, cumulative targets, router changes, dry-run state, and errors; retained under report retention.	Derived audit artifact. Preserve structured fields and source scan association in the DB before considering file retirement.
mikrotik/	MikrotikFirewall.save_state_report() writes timestamped JSON snapshots and report data; latest state is also written to data/mikrotik/*.json.	Firewall UI/API displays current state and snapshot history. app.db.backfill currently imports only the latest data/mikrotik snapshot; runtime snapshot persistence uses firewall_snapshots, firewall_address_list_entries, and firewall_rule_entries.	Firewall rows and snapshots are canonical in the DB once persisted. Keep JSON exports during migration and retain any git-diff inputs required by deployment/audit.

Consumers

• GET /api/reports/<source> lists top-level files in the selected report directory; GET /api/reports/<source>/<path> reads a file below it.
• The reports page consumes those endpoints. Scan status and dashboard helpers select the newest source and MikroTik report files for display.
• /api/blocklist reads the consolidated IPv4/IPv6 text files. Scan history reconstruction and firewall sync also consume those files alongside the SQLite ledger and git history.
• RouterOS .rsc exports and firewall synchronization consume generated targets; these are operational outputs, not disposable presentation files.
• app.db.backfill imports per-source requests.csv, blocklist history, and the latest common MikroTik JSON snapshot. It does not currently import all timestamped reports or all historical MikroTik snapshots.
• Report retention removes old generated artifacts after consolidated scans, subject to configured age/file-count limits; the consolidated blocklist files are protected.

Restructuring Implications

• Keep requests, ip_stats, blocklist tables, and firewall snapshot tables as the queryable source of truth for facts they already represent. Avoid copying those facts into a generic report blob.
• A future report-artifact record should store metadata and structured fields that have no canonical home yet. Rendered text/CSV/JSON should be generated from those records or remain an explicit export.
• Preserve source paths, generated timestamps, scan IDs, and hashes during import so DB records can be traced back to original files.
• Do not retire blocklist exports, RouterOS scripts, or source logs until their consumers and audit requirements have explicit replacements.