Compare commits

..
5 Commits
Author SHA1 Message Date
zwitschi 1e80be746e docs(readme): source of truth is SQLite; add Deployment & updating section
- Blocklist source of truth is the SQLite ledger, not git history.
- data/mikrotik is a scratch mirror, not a git audit trail.
- Removed the 'commit blocklist files' advice.
- Added 'Deployment & updating (git pull)' section covering deploy.sh,
  runtime isolation, and python -m app.db --migrate (D008).
2026-10-24 19:51:03 +02:00
zwitschi 5d81f617f0 feat(deploy): conflict-free 'git pull' update flow (D001/D004/D005)
- deploy.sh: clean-tree guard + git fetch && git pull --ff-only + optional
  dep reinstall + app.db --migrate + restart.
- restart-service.sh: supervisor-agnostic restart (systemd or fallback), waits
  for an in-flight scan.
- check_runtime_git.sh: proves no scan writes git-tracked runtime data.
- ids.service: systemd unit template.
- tests/test_deployment.py: stdlib unittest invariant tests for the deployment
  runtime-isolation guarantee.
2026-10-24 19:50:45 +02:00
zwitschi 4960eef312 feat(firewall): persist firewall snapshots to SQLite (F001/D007)
save_state_report now also writes the address-list + firewall-rule snapshot into
the app.db firewall_snapshots tables, making the audit trail independent of git.
Removes the 'git-visible data/mikrotik' framing.
2026-10-24 19:50:43 +02:00
zwitschi aef132fe94 feat(db): add 'python -m app.db' CLI for migrations
D006: idempotent --migrate, --info (schema version + row counts), and --path.
Used by deploy.sh to bring the DB up to date after a git pull without the web
server running.
2026-10-24 19:50:41 +02:00
zwitschi 7f392ef908 chore(data): stop tracking runtime-refreshed whitelist seeds
The Cloudflare ips-v4/v6.txt and paloalto/.gitkeep were git-tracked. The
Cloudflare files are overwritten at runtime by every whitelist refresh, so on
a deployed server they become locally-modified tracked files that make 'git pull'
conflict. Keep the read-only paloalto/scanner-ips.txt seed tracked; ignore the
rest (they are re-downloaded if absent; built-in ranges are the fallback).
2026-10-24 19:50:39 +02:00
13 changed files with 467 additions and 47 deletions

No files matched your search

+7 -5
View File
@@ -13,12 +13,14 @@ __pycache__/
# python virtual environment # python virtual environment
.venv/ .venv/
# app runtime data (feeds, caches, scan state - regenerated each run) # app runtime data (feeds, caches, scan state - regenerated each run).
# matched per-entry (not as a directory) so the whitelist seed files # All of this must stay UNTRACKED so ``git pull`` never conflicts with files a
# for Cloudflare and PaloAlto below can be re-included # scan has written since the last pull (see .github/instructions/DEPLOYMENT.md).
# The single allowed tracked seed is the read-only PaloAlto scanner-ips.txt
# (never rewritten at runtime); the Cloudflare ips-*.txt are runtime-refreshed
# downloads, so they are NOT tracked and are re-downloaded if absent.
app/data/* app/data/*
!app/data/cloudflare/ !app/data/paloalto/scanner-ips.txt
!app/data/paloalto/
# SQLite database (canonical store) + WAL/SHM sidecars # SQLite database (canonical store) + WAL/SHM sidecars
app/data/*.db app/data/*.db
+50 -12
View File
@@ -214,29 +214,36 @@ time and is never trimmed automatically.
it stops appearing in recent logs (for example after it leaves the NPM it stops appearing in recent logs (for example after it leaves the NPM
~30-day log-retention window, or once WHOIS subnet consolidation groups it ~30-day log-retention window, or once WHOIS subnet consolidation groups it
into a larger range). into a larger range).
- The **source of truth** for "all blocked IPs" is the git history of the - The **source of truth** for "all blocked IPs" is the **SQLite blocklist
consolidated blocklist files `reports/app/data/blocklist_ipv4.txt` and ledger** (`app.db` → `blocklist_entries` / `blocklist_events`), not git
`blocklist_ipv6.txt` (these files are git-tracked). After each scan the history. Each scan syncs the desired blocklist into the ledger; the router's
updated blocklist files should be committed so the history keeps growing. cumulative state is derived from it. `app/utils/blocklist_history.py` is a
read-only fallback only.
- The sync only ever **adds** entries. The only automatic removals are router - The sync only ever **adds** entries. The only automatic removals are router
entries that were never blocked by the system — i.e. absent from every git entries that were never blocked by the system — i.e. absent from the ledger
version of the blocklist files (manual / rogue entries). (manual / rogue entries).
- To explicitly unblock an IP, remove it manually via the web UI - To explicitly unblock an IP, remove it manually via the web UI
(`/blocklist` → remove) or directly on the router; the automated sync will (`/blocklist` → remove) or directly on the router; the automated sync will
not re-add entries that are not part of a scan. not re-add entries that are not part of a scan.
> **Deployment note:** runtime data (blocklists, reports, the DB) is **never
> committed to git**. Keeping it untracked is what makes `git pull` updates
> conflict-free — see [`DEPLOYMENT.md`](.github/instructions/DEPLOYMENT.md)
> and the `deploy/` scripts.
Each consolidated scan with `--update-firewall` also writes a timestamped Each consolidated scan with `--update-firewall` also writes a timestamped
change report to `reports/app/YYYY-MM-DD-hh-mm-ss.json`: the number and list change report to `reports/app/YYYY-MM-DD-hh-mm-ss.json`: the number and list
of entries added / removed (complete-set delta), the full cumulative blocked of entries added / removed (complete-set delta), the full cumulative blocked
list, and the router sync changes. list, and the router sync changes. It also persists a firewall snapshot into
the `firewall_snapshots` SQLite tables for long-term audit (F001).
### Report retention ### Report retention
Generated files under `reports/` are pruned automatically after each Generated files under `reports/` are pruned automatically after each
consolidated scan: every subdirectory keeps at most consolidated scan: every subdirectory keeps at most
`REPORT_RETENTION_MAX_FILES` files (default 10), all newer than `REPORT_RETENTION_MAX_FILES` files (default 10), all newer than
`REPORT_RETENTION_DAYS` days (default 30). The blocklist source of truth `REPORT_RETENTION_DAYS` days (default 30). The blocklist files
(`reports/app/data/blocklist_ipv4.txt` / `blocklist_ipv6.txt`) is always (`reports/app/data/blocklist_ipv4.txt` / `blocklist_ipv6.txt`) are always
protected and never pruned. Tune via `REPORT_RETENTION_DAYS` / protected and never pruned. Tune via `REPORT_RETENTION_DAYS` /
`REPORT_RETENTION_MAX_FILES`; `--dry-run` previews the cleanup without `REPORT_RETENTION_MAX_FILES`; `--dry-run` previews the cleanup without
deleting. deleting.
@@ -268,6 +275,35 @@ from the UI — applying changes is an explicit action. The main JSON endpoints
`/api/firewall/*`, `/api/whitelist`) are also usable directly from a REST `/api/firewall/*`, `/api/whitelist`) are also usable directly from a REST
client. client.
## Deployment & updating (`git pull`)
The app runs scans continuously (web UI + cron + CLI), and all of it is
intended to be updated on the server with a plain `git pull`. To make that
**safe** — i.e. never conflict with a file a scan wrote since the last pull —
runtime data is strictly **never committed to git**:
- `app/data/*`, `app/reports/*` and the SQLite DB (`app/data/ids.db`) are
gitignored. The only tracked file under `data/` is the read-only
`app/data/paloalto/scanner-ips.txt` seed.
- The Cloudflare `ips-*.txt` whitelist seeds are **not** tracked: they are
runtime-refreshed downloads and are re-fetched if absent (built-in ranges are
the fallback).
- History that must survive — the cumulative blocklist and firewall audit —
lives in the SQLite store, not in git revisions.
Update a deployed instance with:
```sh
sudo deploy/deploy.sh
```
See [`DEPLOYMENT.md`](.github/instructions/DEPLOYMENT.md) for the full plan,
and `deploy/` for the scripts (`deploy.sh`, `restart-service.sh`,
`check_runtime_git.sh`, `ids.service` systemd template). The wrapper
fast-forwards only (`git pull --ff-only`), refuses to run on an unclean tree,
reinstalls deps if `requirements.txt` changed, applies DB migrations
(`python -m app.db --migrate`), and restarts the `ids` unit.
## Output ## Output
Each script writes to its own report directory. `app.py` also writes Each script writes to its own report directory. `app.py` also writes
@@ -318,9 +354,11 @@ reports/
``` ```
The current router state (address lists + firewall rules) is also mirrored to The current router state (address lists + firewall rules) is also mirrored to
`data/mikrotik/` on every `--update-firewall` run. Those files are overwritten `data/mikrotik/` on every `--update-firewall` run as a scratch pointer to the
each run and are **git-visible**, so `git diff` shows how the firewall rules latest snapshot. Those files are overwritten each run and are **not** tracked;
and address lists change over time. the long-term audit trail is the SQLite `firewall_snapshots` tables plus the
timestamped reports under `reports/mikrotik/`. `git pull` is therefore never
blocked by firewall-state files.
## Detection ## Detection
-15
View File
@@ -1,15 +0,0 @@
173.245.48.0/20
103.21.244.0/22
103.22.200.0/22
103.31.4.0/22
141.101.64.0/18
108.162.192.0/18
190.93.240.0/20
188.114.96.0/20
197.234.240.0/22
198.41.128.0/17
162.158.0.0/15
104.16.0.0/13
104.24.0.0/14
172.64.0.0/13
131.0.72.0/22
-7
View File
@@ -1,7 +0,0 @@
2400:cb00::/32
2606:4700::/32
2803:f800::/32
2405:b500::/32
2405:8100::/32
2a06:98c0::/29
2c0f:f248::/32
-1
View File
@@ -1 +0,0 @@
$ = paloalto
+74
View File
@@ -0,0 +1,74 @@
#!/usr/bin/env python3
"""CLI entry point for the SQLite store.
Usage:
python -m app.db --migrate applies pending migrations (idempotent)
python -m app.db --info prints schema version + table counts, exits
python -m app.db --path prints the resolved DB path, exits
Used by ``deploy.sh`` (D004) to ensure the DB is up to date after a ``git pull``
without needing the web server up.
"""
from __future__ import annotations
import argparse
import sys
from ..utils.constants import DATA_DB_PATH
from .schema import SCHEMA_VERSION, apply_migrations
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(
prog="python -m app.db",
description="Intrusion-detection SQLite store utilities.")
parser.add_argument("--migrate", action="store_true",
help="apply pending schema migrations (idempotent)")
parser.add_argument("--info", action="store_true",
help="print schema version + table/row counts")
parser.add_argument("--path", action="store_true",
help="print the resolved DB path and exit")
args = parser.parse_args(argv)
if args.path:
print(DATA_DB_PATH)
return 0
from . import get_connection
conn = get_connection()
current = conn.execute("PRAGMA user_version").fetchone()[0]
if args.info:
counts = {}
tables = conn.execute(
"SELECT name FROM sqlite_master WHERE type='table' "
"AND name NOT LIKE 'sqlite_%' ORDER BY name").fetchall()
for t in tables:
name = t["name"]
try:
counts[name] = conn.execute(
f"SELECT COUNT(*) FROM \"{name}\"").fetchone()[0]
except Exception:
counts[name] = None
print(f"db_path: {DATA_DB_PATH}")
print(f"version: {current} (latest {SCHEMA_VERSION})")
print("tables:")
for name, n in counts.items():
print(f" {name:<32} {n} rows")
return 0
if args.migrate or current < SCHEMA_VERSION:
before = current
apply_migrations(conn)
print(f"schema version {before} -> "
f"{conn.execute('PRAGMA user_version').fetchone()[0]}")
return 0
print(f"schema is already up to date (version {current})")
return 0
if __name__ == "__main__":
sys.exit(main())
+31 -7
View File
@@ -143,9 +143,11 @@ class MikrotikFirewall:
"""Save current address lists and firewall rules for auditing. """Save current address lists and firewall rules for auditing.
Writes structured snapshots to ``<report_dir>/mikrotik/data/`` and to Writes structured snapshots to ``<report_dir>/mikrotik/data/`` and to
the common ``data/mikrotik/`` directory (overwritten every run so git the common ``data/mikrotik/`` directory (scratch mirror for debugging;
can diff changes to the firewall state over time), plus a timestamped **not** a git audit trail), plus a timestamped JSON report under
JSON report under ``<report_dir>/mikrotik/``. ``<report_dir>/mikrotik/``. The authoritative long-term audit trail is
the SQLite ``firewall_snapshots`` tables (see ``app.db``), which are
appended here regardless of the file writes.
""" """
state = { state = {
"router": self.router_ip, "router": self.router_ip,
@@ -174,11 +176,31 @@ class MikrotikFirewall:
os.path.join(data_dir, f"firewall_{version}.json"), os.path.join(data_dir, f"firewall_{version}.json"),
state["firewall_rules"][version]) state["firewall_rules"][version])
# Persist a SQLite snapshot (authoritative audit trail, independent of
# git). Best-effort: never breaks a router sync if the DB is down.
self._persist_snapshot_to_db(state)
timestamp = datetime.now().strftime("%Y-%m-%d-%H-%M-%S") timestamp = datetime.now().strftime("%Y-%m-%d-%H-%M-%S")
report_path = os.path.join(report_dir, "mikrotik", f"{timestamp}.json") report_path = os.path.join(report_dir, "mikrotik", f"{timestamp}.json")
self._write_json(report_path, state) self._write_json(report_path, state)
return report_path return report_path
@staticmethod
def _persist_snapshot_to_db(state: dict) -> None:
"""Write the firewall snapshot into the SQLite store (best-effort)."""
try:
from .db import get_connection
from .db import repositories as repo
repo.save_firewall_snapshot(
get_connection(),
address_lists_v4=state["address_lists"]["ipv4"],
address_lists_v6=state["address_lists"]["ipv6"],
rules_v4=state["firewall_rules"]["ipv4"],
rules_v6=state["firewall_rules"]["ipv6"],
)
except Exception as exc:
print(f"WARN: could not persist firewall snapshot to db ({exc})")
def sync_blocklist(self, blocked_ipv4: Iterable[str], def sync_blocklist(self, blocked_ipv4: Iterable[str],
blocked_ipv6: Iterable[str], blocked_ipv6: Iterable[str],
dry_run: bool = False, dry_run: bool = False,
@@ -186,10 +208,12 @@ class MikrotikFirewall:
"""Synchronize the Blocked list cumulatively and audit the result. """Synchronize the Blocked list cumulatively and audit the result.
The blocklist is grow-only: targets are added when present in the The blocklist is grow-only: targets are added when present in the
complete set (all git history of the blocklist files + the current complete cumulative set (git history of the blocklist files + the
working files + this scan's targets) and never removed automatically. current working files + this scan's targets) and never removed
Only router entries absent from the complete set — i.e. never blocked automatically. Only router entries absent from the complete set — i.e.
by the system — are pruned. See ``app.blocklist_history``. never blocked by the system — are pruned. The history now lives in the
SQLite blocklist ledger (``app.db`` ``blocklist_entries``); git history
remains a read-only fallback. See ``app.blocklist_history``.
""" """
desired_ipv4 = set(blocked_ipv4) desired_ipv4 = set(blocked_ipv4)
+13
View File
@@ -0,0 +1,13 @@
#!/usr/bin/env bash
#
# Deployment support for the intrusion-detection IDS.
#
# This directory is intentionally small and uses no external framework:
# deploy.sh conflict-free `git pull` update wrapper (D004)
# ids.service systemd unit template (D005)
# restart-service.sh supervisor-agnostic restart shim (D005)
# check_runtime_git.sh proves a scan never dirties tracked runtime files (D001)
#
# See .github/instructions/DEPLOYMENT.md for the full plan.
set -euo pipefail
+66
View File
@@ -0,0 +1,66 @@
#!/usr/bin/env bash
#
# D001 — runtime write-path guard.
#
# Proves the deployment invariant: after running a scan, `git status` shows
# NO tracked file modified by the scan — every runtime write lands in a
# gitignored location (data/, reports/, *.db). This is what makes `git pull`
# conflict-free on a server where scans keep running.
#
# If this fails, a scan wrote into a git-tracked path and `git pull` on the
# server can conflict. Investigate the printed files before deploying.
#
# Usage:
# deploy/check_runtime_git.sh [--live]
# --live additionally run a real scan first (requires log sources) and
# then assert the tree is still clean. Without --live it only
# verifies that whatever runtime files already exist are untracked.
set -euo pipefail
cd "$(dirname "$0")/.."
if [[ "${1:-}" == "--live" ]]; then
echo "== Running a real scan to exercise every write path =="
python app.py scan --quiet --dry-run || true
fi
echo "== Checking that runtime data is gitignored / untracked =="
# Paths a scan writes to. Any of these being tracked OR showing as (non-ignored)
# untracked means the invariant is broken and a git pull could conflict.
RUNTIME_PATHS=(
"app/data"
"app/reports"
"app/data/ids.db"
"app/data/ids.db-wal"
"app/data/ids.db-shm"
)
fail=0
# 1) No runtime path may be TRACKED, except the single allowed read-only
# PaloAlto seed. `git ls-files` reflects the index (including staged
# changes), so unborn commits don't cause false failures.
allowed="app/data/paloalto/scanner-ips.txt"
tracked=$(git ls-files app/data app/reports)
problem=$(printf '%s\n' "$tracked" | grep -xvF "$allowed" || true)
if [[ -n "$problem" ]]; then
echo "FAIL: unexpected tracked files under runtime dirs:" >&2
echo "$problem" >&2
fail=1
fi
# 2) Nothing may show as untracked-but-not-ignored under the runtime dirs, i.e.
# a scan wrote a file git would happily add (and later conflict on).
for p in "${RUNTIME_PATHS[@]}"; do
dirty=$(git status --porcelain --untracked-files=all -- "$p" 2>/dev/null || true)
if [[ -n "$dirty" ]]; then
echo "FAIL: runtime path has changes git cares about:" >&2
echo "$dirty" >&2
fail=1
fi
done
if [[ "$fail" -eq 0 ]]; then
echo "OK: no tracked runtime data; git pull will not conflict with scan writes."
else
echo "BROKEN: a scan wrote into git-tracked paths — fix before deploying." >&2
exit 1
fi
+75
View File
@@ -0,0 +1,75 @@
#!/usr/bin/env bash
#
# D004 — conflict-free deployment update wrapper.
#
# Updates a deployed instance with `git pull` WITHOUT ever hitting a merge
# conflict from files a scan wrote since the last pull. Guarantees:
#
# 1. Refuses to update over uncommitted tracked changes (clean-tree guard).
# 2. Fast-forwards only (--ff-only): never creates merge commits and never
# silently conflicts.
# 3. Runtime data (data/, reports/, *.db) is gitignored, so a running scan
# leaves no dirty tracked paths and never blocks the pull.
# 4. Reinstalls dependencies only if requirements.txt changed.
# 5. Applies DB migrations idempotently.
# 6. Restarts the app under its supervisor.
#
# Usage:
# sudo deploy/deploy.sh # update + restart (default env: production)
# ENV=production deploy/deploy.sh # use deploy/ids@production.service
# deploy/deploy.sh --no-restart # update deps+migrate but don't restart
set -euo pipefail
cd "$(dirname "$0")/.."
ENV="${ENV:-production}"
RESTART="${1:-restart}"
ROOT="$(pwd)"
PY="${PY:-python}"
echo "== Intrusion Detection deploy (env=$ENV) =="
# 1) Clean-tree guard. Runtime data is gitignored, so this only fails if a
# human edited tracked source without committing.
if ! git diff --quiet HEAD || ! git diff --cached --quiet; then
echo "REFUSING: working tree is not clean." >&2
echo "Commit or stash tracked changes, then re-run." >&2
echo " git status --short" >&2
exit 1
fi
# 2) Pre-flight: prove runtime isolation holds (fail fast before touching repo).
if "$ROOT/deploy/check_runtime_git.sh"; then
echo "== runtime isolation ok =="
else
echo "ABORT: runtime isolation check failed." >&2
exit 1
fi
# 3) Fast-forward only.
echo "== fetching upstream =="
git fetch origin
echo "== fast-forward pull =="
git pull --ff-only
# 4) Dependencies (only if the requirement set changed since the last pull).
if git diff --quiet HEAD@{1} HEAD -- requirements.txt; then
echo "== requirements.txt unchanged; skipping pip =="
else
echo "== requirements.txt changed; installing deps =="
command -v "$PY" >/dev/null && "$PY" -m pip install -r requirements.txt \
|| pip install -r requirements.txt
fi
# 5) Apply DB migrations (idempotent).
echo "== applying DB migrations =="
"$PY" -m app.db --migrate
# 6) Restart under the supervisor.
if [[ "$RESTART" == "--no-restart" ]]; then
echo "== --no-restart: update complete; restart manually =="
else
echo "== restarting service =="
"$ROOT/deploy/restart-service.sh" "$ENV"
fi
echo "== deploy complete =="
+41
View File
@@ -0,0 +1,41 @@
# systemd unit template (D005) — intrusion-detection web UI.
#
# Install:
# - Replace /path/to/intrusion-detection and the User/Group.
# - Copy to /etc/systemd/system/ids.service
# - sudo systemctl daemon-reload && sudo systemctl enable --now ids
#
# The app binds to 127.0.0.1:<WEB_PORT>; it must NOT be exposed to the LAN.
#
# Restart policy: automatically restart on crash. `deploy/deploy.sh` restarts
# this unit after a successful `git pull`. Scans run in a background thread, so
# one (serialized) scan at a time; deploy waits for the worker to be idle
# (see restart-service.sh) before restarting.
[Unit]
Description=Intrusion Detection web UI + IDS scanner
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
# Run as an unprivileged user; the app only needs read access to the log dirs
# it consumes (COPYPARTY_LOG_PATH / NPM_LOG_PATH) and write access to its own
# data/ + reports/ + ids.db.
User=ids
Group=ids
WorkingDirectory=/path/to/intrusion-detection
EnvironmentFile=-/etc/ids.env
ExecStart=/path/to/intrusion-detection/.venv/bin/python /path/to/intrusion-detection/app.py
Restart=on-failure
RestartSec=5
# Hardening
NoNewPrivileges=yes
PrivateTmp=yes
ProtectSystem=full
ProtectHome=no
ReadWritePaths=/path/to/intrusion-detection/data /path/to/intrusion-detection/reports /var/lib/ids
[Install]
WantedBy=multi-user.target
+53
View File
@@ -0,0 +1,53 @@
#!/usr/bin/env bash
#
# D005 — supervisor-agnostic restart shim.
#
# Restarts the intrusion-detection service after a deploy, preferring systemd
# (Linux), with a fallback message for other environments (Windows dev).
#
# It first waits for any in-flight background scan to finish so a checkout
# change never interleaves with a live scan (belt-and-suspenders; the runtime
# isolation guarantee already means scans only write gitignored files).
#
# Usage:
# deploy/restart-service.sh [production]
# ENV=production deploy/restart-service.sh
set -euo pipefail
cd "$(dirname "$0")/.."
ENV="${1:-${ENV:-production}}"
UNIT="ids@${ENV}.service"
# Wait for an in-flight scan to finish (~0..120s). We probe the web UI status
# endpoint if it is reachable; a failure to reach it just means no web UI is up.
web_host="${WEB_HOST:-127.0.0.1}"
web_port="${WEB_PORT:-54666}"
status_url="http://${web_host}:${web_port}/api/scan/status"
for _ in $(seq 1 60); do
running=$(curl -fsS --max-time 2 "$status_url" 2>/dev/null \
| python3 -c 'import sys,json;print(json.load(sys.stdin).get("running",False))' 2>/dev/null \
|| echo false)
[[ "$running" == "false" ]] && break
echo " waiting for running scan to finish..."
sleep 2
done
if command -v systemctl >/dev/null 2>&1; then
# Installed unit is <unit>.service.template; the enabled name includes env.
if systemctl list-unit-files "${UNIT}" >/dev/null 2>&1; then
echo "== systemctl restart ${UNIT} =="
systemctl restart "${UNIT}"
systemctl --no-pager status "${UNIT}" --lines=0 || true
elif systemctl list-unit-files "ids.service" >/dev/null 2>&1; then
echo "== systemctl restart ids.service =="
systemctl restart ids.service
systemctl --no-pager status ids.service --lines=0 || true
else
echo "No ids systemd unit found; restart manually." >&2
exit 2
fi
else
echo "No systemd available (Windows dev?). Restart the app manually." >&2
echo " python app.py # in a terminal, or via your dev runner" >&2
exit 2
fi
+57
View File
@@ -0,0 +1,57 @@
#!/usr/bin/env python3
"""Deployment invariant tests (D001).
Proves that running a scan never creates or modifies git-tracked runtime data,
which is what makes ``git pull`` conflict-free on a deployed server.
Run with:
python -m unittest discover -s tests -p 'test_*.py'
or (if pytest is installed):
python -m pytest tests/
"""
from __future__ import annotations
import os
import subprocess
import sys
import unittest
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
# The paloalto scanner-ips.txt seed is deliberately tracked (read-only); every
# other file under the runtime dirs must remain untracked.
ALLOWED_TRACKED_RUNTIME = {os.path.join("app", "data", "paloalto",
"scanner-ips.txt")}
def _git(*args: str) -> str:
return subprocess.run(["git", *args], cwd=ROOT, check=True,
capture_output=True, text=True).stdout
class RuntimeGitTest(unittest.TestCase):
def test_no_runtime_data_tracked(self):
"""No runtime file (other than the read-only paloalto seed) is tracked."""
tracked = _git("ls-files", "app/data", "app/reports").splitlines()
unexpected = [f for f in tracked
if f not in ALLOWED_TRACKED_RUNTIME]
self.assertEqual(
unexpected, [],
"git-tracked runtime files would make `git pull` conflict; "
"add them to .gitignore: %s" % unexpected)
def test_runtime_write_paths_ignored(self):
"""Every runtime write path is ignored by git (won't dirty the tree)."""
stats = subprocess.run(
["git", "check-ignore", "-v",
"app/data/scan_state.json", "app/data/scan_jobs.json",
"app/data/whois_cache.json", "app/data/ids.db",
"app/data/ids.db-wal", "app/reports/app/data/blocklist_ipv4.txt",
], cwd=ROOT, capture_output=True, text=True).stdout
self.assertIn("scan_state.json", stats)
self.assertIn("ids.db", stats)
self.assertIn("blocklist_ipv4.txt", stats)
if __name__ == "__main__":
unittest.main()