feat: Implement SQLite schema and migrations for intrusion detection system

- Added schema definition and migration logic in app/db/schema.py.
- Created tables for log sources, requests, IP stats, scans, blocklist, whitelist, and firewall rules.
- Introduced functions to apply migrations and ensure database integrity.

feat: Enhance scan functionality with database integration

- Added _sync_blocklist_to_db function to synchronize blocklist with SQLite.
- Updated run_scan function to handle database connections and scan records.
- Implemented error handling for database operations during scans.

feat: Introduce log analysis page and API endpoints

- Created analysis.html template for displaying log analysis results.
- Added routes for fetching top IPs, IP request timelines, and cross-source IPs.
- Integrated analysis data into the dashboard and status endpoints.

fix: Ensure whitelists are synchronized with the database

- Implemented sync_whitelists_to_db function to persist in-memory whitelists.
- Updated constants.py to define the path for the SQLite database.

refactor: Modularize analysis helpers for better maintainability

- Created _analysis_helpers function to encapsulate SQLite query logic.
- Updated routes.py to utilize new analysis helper functions for API responses.
This commit is contained in:
zwitschi committed 2026-10-24 19:48:20 +02:00
1 parent 0137264fe4
commit 6036fa8b1e
17 files changed
+1292 -5

No files matched your search

+62
View File
@@ -0,0 +1,62 @@
#!/usr/bin/env python3
"""Local SQLite database layer for the intrusion-detection system.
Owns the shared connection factory (WAL mode, busy timeout, single mutable
connection reused across the background-scan thread and web readers) and
applies schema migrations on open. Query helpers live in :mod:`repositories`.
"""
from __future__ import annotations
import os
import sqlite3
import threading
from ..utils.constants import DATA_DB_PATH
from .schema import apply_migrations
_connection: sqlite3.Connection | None = None
_lock = threading.RLock()
def connect(path: str | None = None) -> sqlite3.Connection:
"""Open an initialized SQLite connection at ``path``.
Enables WAL for concurrent readers during the single-writer background
scan and sets a busy timeout so short write contention does not raise.
Foreign keys are on and migrations are applied.
"""
db_path = path or DATA_DB_PATH
os.makedirs(os.path.dirname(os.path.abspath(db_path)) or ".", exist_ok=True)
conn = sqlite3.connect(
db_path,
check_same_thread=False, # background scan thread + web handlers
isolation_level=None, # autocommit; use explicit BEGIN for batches
)
conn.execute("PRAGMA journal_mode=WAL")
conn.execute("PRAGMA busy_timeout=5000")
conn.execute("PRAGMA synchronous=NORMAL")
conn.row_factory = sqlite3.Row
apply_migrations(conn)
return conn
def get_connection() -> sqlite3.Connection:
"""Return the shared, lazily-created connection (thread-safe)."""
global _connection
with _lock:
if _connection is None:
_connection = connect()
return _connection
def close() -> None:
"""Close the shared connection, if any."""
global _connection
with _lock:
if _connection is not None:
try:
_connection.close()
except sqlite3.Error:
pass
_connection = None