feat: Implement SQLite schema and migrations for intrusion detection system

- Added schema definition and migration logic in app/db/schema.py.
- Created tables for log sources, requests, IP stats, scans, blocklist, whitelist, and firewall rules.
- Introduced functions to apply migrations and ensure database integrity.

feat: Enhance scan functionality with database integration

- Added _sync_blocklist_to_db function to synchronize blocklist with SQLite.
- Updated run_scan function to handle database connections and scan records.
- Implemented error handling for database operations during scans.

feat: Introduce log analysis page and API endpoints

- Created analysis.html template for displaying log analysis results.
- Added routes for fetching top IPs, IP request timelines, and cross-source IPs.
- Integrated analysis data into the dashboard and status endpoints.

fix: Ensure whitelists are synchronized with the database

- Implemented sync_whitelists_to_db function to persist in-memory whitelists.
- Updated constants.py to define the path for the SQLite database.

refactor: Modularize analysis helpers for better maintainability

- Created _analysis_helpers function to encapsulate SQLite query logic.
- Updated routes.py to utilize new analysis helper functions for API responses.

app/utils/constants.py

@@ -53,6 +53,11 @@ MIKROTIK_DATA_DIR = os.path.join(DATA_DIR, "mikrotik")
 
 WHOIS_CACHE_PATH = os.path.join(DATA_DIR, "whois_cache.json")
 
+# Local SQLite database (see .github/instructions/DATABASE.md). The default
+# lives under data/ and holds log records, scan metadata, blocklist/whitelist
+# and firewall history. Override with IDS_DB_PATH (used by app/db).
+DATA_DB_PATH = os.environ.get("IDS_DB_PATH", os.path.join(DATA_DIR, "ids.db"))
+
 # Incremental-scan state: absolute log path -> last processed byte offset
 SCAN_STATE_PATH = os.path.join(DATA_DIR, "scan_state.json")
 

app/utils/whitelists.py

@@ -275,3 +275,20 @@ def refresh_whitelists(force: bool = False) -> bool:
     update_whitelists()
     _refreshed = True
     return True
+
+
+def sync_whitelists_to_db() -> None:
+    """Persist the current in-memory ``WHITELISTS`` into the SQLite store.
+
+    Best-effort (never raises): keeps the DB whitelist tables consistent with
+    the in-memory/provider state so lookups and analysis can rely on them.
+    """
+    try:
+        from ..db import get_connection
+        from ..db import repositories as repo
+        conn = get_connection()
+        for provider, cidrs in WHITELISTS.items():
+            repo.set_whitelist(conn, provider, cidrs,
+                               source="builtin/refresh")
+    except Exception as exc:  # pragma: no cover - defensive
+        print(f"WARN: could not sync whitelists to db ({exc})")