48 lines
4.3 KiB
Markdown
48 lines
4.3 KiB
Markdown
# Report Inventory
|
|
|
|
This inventory records the current report surface before moving report reads
|
|
or writes into SQLite. Generated paths are rooted at `REPORT_DIR` (normally
|
|
`reports/`). The SQLite tables listed below already exist in schema v1 unless
|
|
noted otherwise.
|
|
|
|
## Artifacts
|
|
|
|
| Area | Producer and formats | Current role and consumers | Classification |
|
|
| --- | --- | --- | --- |
|
|
| `copyparty/` | `Analyzer.write_report()` creates timestamped `.txt`; `write_data()` creates per-IP CSV/JSON, denied-path CSV, flagged/banned IP TXT, `requests.csv`, `authed_users.json`, and `mikrotik-block.rsc`. | Reports API serves top-level text reports. `requests.csv` is imported by `app.db.backfill`; per-run data supports inspection and legacy workflows. Retention prunes generated files. | Request rows and per-scan IP stats are canonical in `requests` and `ip_stats`; rendered summaries and exports are derived. |
|
|
| `npm/` | Same analyzer outputs, plus user-agent and host summary CSV files. | Same consumers as copyparty. `requests.csv` is a legacy backfill input; retention prunes generated files. | Request rows and per-scan IP stats are canonical in `requests` and `ip_stats`; summaries and exports are derived. |
|
|
| `app/data/` | Consolidated scan writes `blocklist_ipv4.txt` and `blocklist_ipv6.txt`. | Blocklist UI/API reads these files; scan history reconstruction and firewall sync use them as current/compatibility inputs. Retention explicitly protects them. | Operational compatibility exports. Canonical blocked targets and events belong in `blocklist_entries` and `blocklist_events`; keep files until every consumer uses the DB and git-history contract is replaced. |
|
|
| `app/` timestamped JSON | `write_blocklist_report()` emits scan/firewall change reports when the router is reachable. | Human-readable audit trail containing run delta, cumulative targets, router changes, dry-run state, and errors; retained under report retention. | Derived audit artifact. Preserve structured fields and source scan association in the DB before considering file retirement. |
|
|
| `mikrotik/` | `MikrotikFirewall.save_state_report()` writes timestamped JSON snapshots and report data; latest state is also written to `data/mikrotik/*.json`. | Firewall UI/API displays current state and snapshot history. `app.db.backfill` currently imports only the latest `data/mikrotik` snapshot; runtime snapshot persistence uses `firewall_snapshots`, `firewall_address_list_entries`, and `firewall_rule_entries`. | Firewall rows and snapshots are canonical in the DB once persisted. Keep JSON exports during migration and retain any git-diff inputs required by deployment/audit. |
|
|
|
|
## Consumers
|
|
|
|
- `GET /api/reports/<source>` lists top-level files in the selected report
|
|
directory; `GET /api/reports/<source>/<path>` reads a file below it.
|
|
- The reports page consumes those endpoints. Scan status and dashboard helpers
|
|
select the newest source and MikroTik report files for display.
|
|
- `/api/blocklist` reads the consolidated IPv4/IPv6 text files. Scan history
|
|
reconstruction and firewall sync also consume those files alongside the
|
|
SQLite ledger and git history.
|
|
- RouterOS `.rsc` exports and firewall synchronization consume generated
|
|
targets; these are operational outputs, not disposable presentation files.
|
|
- `app.db.backfill` imports per-source `requests.csv`, blocklist history, and
|
|
the latest common MikroTik JSON snapshot. It does not currently import all
|
|
timestamped reports or all historical MikroTik snapshots.
|
|
- Report retention removes old generated artifacts after consolidated scans,
|
|
subject to configured age/file-count limits; the consolidated blocklist
|
|
files are protected.
|
|
|
|
## Restructuring Implications
|
|
|
|
- Keep `requests`, `ip_stats`, blocklist tables, and firewall snapshot tables
|
|
as the queryable source of truth for facts they already represent. Avoid
|
|
copying those facts into a generic report blob.
|
|
- A future report-artifact record should store metadata and structured fields
|
|
that have no canonical home yet. Rendered text/CSV/JSON should be generated
|
|
from those records or remain an explicit export.
|
|
- Preserve source paths, generated timestamps, scan IDs, and hashes during
|
|
import so DB records can be traced back to original files.
|
|
- Do not retire blocklist exports, RouterOS scripts, or source logs until
|
|
their consumers and audit requirements have explicit replacements.
|