Compare commits
5
Commits
6036fa8b1e
...
1e80be746e
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1e80be746e | ||
|
|
5d81f617f0 | ||
|
|
4960eef312 | ||
|
|
aef132fe94 | ||
|
|
7f392ef908 |
No files matched your search
+7
-5
@@ -13,12 +13,14 @@ __pycache__/
|
||||
# python virtual environment
|
||||
.venv/
|
||||
|
||||
# app runtime data (feeds, caches, scan state - regenerated each run)
|
||||
# matched per-entry (not as a directory) so the whitelist seed files
|
||||
# for Cloudflare and PaloAlto below can be re-included
|
||||
# app runtime data (feeds, caches, scan state - regenerated each run).
|
||||
# All of this must stay UNTRACKED so ``git pull`` never conflicts with files a
|
||||
# scan has written since the last pull (see .github/instructions/DEPLOYMENT.md).
|
||||
# The single allowed tracked seed is the read-only PaloAlto scanner-ips.txt
|
||||
# (never rewritten at runtime); the Cloudflare ips-*.txt are runtime-refreshed
|
||||
# downloads, so they are NOT tracked and are re-downloaded if absent.
|
||||
app/data/*
|
||||
!app/data/cloudflare/
|
||||
!app/data/paloalto/
|
||||
!app/data/paloalto/scanner-ips.txt
|
||||
|
||||
# SQLite database (canonical store) + WAL/SHM sidecars
|
||||
app/data/*.db
|
||||
|
||||
@@ -214,29 +214,36 @@ time and is never trimmed automatically.
|
||||
it stops appearing in recent logs (for example after it leaves the NPM
|
||||
~30-day log-retention window, or once WHOIS subnet consolidation groups it
|
||||
into a larger range).
|
||||
- The **source of truth** for "all blocked IPs" is the git history of the
|
||||
consolidated blocklist files `reports/app/data/blocklist_ipv4.txt` and
|
||||
`blocklist_ipv6.txt` (these files are git-tracked). After each scan the
|
||||
updated blocklist files should be committed so the history keeps growing.
|
||||
- The **source of truth** for "all blocked IPs" is the **SQLite blocklist
|
||||
ledger** (`app.db` → `blocklist_entries` / `blocklist_events`), not git
|
||||
history. Each scan syncs the desired blocklist into the ledger; the router's
|
||||
cumulative state is derived from it. `app/utils/blocklist_history.py` is a
|
||||
read-only fallback only.
|
||||
- The sync only ever **adds** entries. The only automatic removals are router
|
||||
entries that were never blocked by the system — i.e. absent from every git
|
||||
version of the blocklist files (manual / rogue entries).
|
||||
entries that were never blocked by the system — i.e. absent from the ledger
|
||||
(manual / rogue entries).
|
||||
- To explicitly unblock an IP, remove it manually via the web UI
|
||||
(`/blocklist` → remove) or directly on the router; the automated sync will
|
||||
not re-add entries that are not part of a scan.
|
||||
|
||||
> **Deployment note:** runtime data (blocklists, reports, the DB) is **never
|
||||
> committed to git**. Keeping it untracked is what makes `git pull` updates
|
||||
> conflict-free — see [`DEPLOYMENT.md`](.github/instructions/DEPLOYMENT.md)
|
||||
> and the `deploy/` scripts.
|
||||
|
||||
Each consolidated scan with `--update-firewall` also writes a timestamped
|
||||
change report to `reports/app/YYYY-MM-DD-hh-mm-ss.json`: the number and list
|
||||
of entries added / removed (complete-set delta), the full cumulative blocked
|
||||
list, and the router sync changes.
|
||||
list, and the router sync changes. It also persists a firewall snapshot into
|
||||
the `firewall_snapshots` SQLite tables for long-term audit (F001).
|
||||
|
||||
### Report retention
|
||||
|
||||
Generated files under `reports/` are pruned automatically after each
|
||||
consolidated scan: every subdirectory keeps at most
|
||||
`REPORT_RETENTION_MAX_FILES` files (default 10), all newer than
|
||||
`REPORT_RETENTION_DAYS` days (default 30). The blocklist source of truth
|
||||
(`reports/app/data/blocklist_ipv4.txt` / `blocklist_ipv6.txt`) is always
|
||||
`REPORT_RETENTION_DAYS` days (default 30). The blocklist files
|
||||
(`reports/app/data/blocklist_ipv4.txt` / `blocklist_ipv6.txt`) are always
|
||||
protected and never pruned. Tune via `REPORT_RETENTION_DAYS` /
|
||||
`REPORT_RETENTION_MAX_FILES`; `--dry-run` previews the cleanup without
|
||||
deleting.
|
||||
@@ -268,6 +275,35 @@ from the UI — applying changes is an explicit action. The main JSON endpoints
|
||||
`/api/firewall/*`, `/api/whitelist`) are also usable directly from a REST
|
||||
client.
|
||||
|
||||
## Deployment & updating (`git pull`)
|
||||
|
||||
The app runs scans continuously (web UI + cron + CLI), and all of it is
|
||||
intended to be updated on the server with a plain `git pull`. To make that
|
||||
**safe** — i.e. never conflict with a file a scan wrote since the last pull —
|
||||
runtime data is strictly **never committed to git**:
|
||||
|
||||
- `app/data/*`, `app/reports/*` and the SQLite DB (`app/data/ids.db`) are
|
||||
gitignored. The only tracked file under `data/` is the read-only
|
||||
`app/data/paloalto/scanner-ips.txt` seed.
|
||||
- The Cloudflare `ips-*.txt` whitelist seeds are **not** tracked: they are
|
||||
runtime-refreshed downloads and are re-fetched if absent (built-in ranges are
|
||||
the fallback).
|
||||
- History that must survive — the cumulative blocklist and firewall audit —
|
||||
lives in the SQLite store, not in git revisions.
|
||||
|
||||
Update a deployed instance with:
|
||||
|
||||
```sh
|
||||
sudo deploy/deploy.sh
|
||||
```
|
||||
|
||||
See [`DEPLOYMENT.md`](.github/instructions/DEPLOYMENT.md) for the full plan,
|
||||
and `deploy/` for the scripts (`deploy.sh`, `restart-service.sh`,
|
||||
`check_runtime_git.sh`, `ids.service` systemd template). The wrapper
|
||||
fast-forwards only (`git pull --ff-only`), refuses to run on an unclean tree,
|
||||
reinstalls deps if `requirements.txt` changed, applies DB migrations
|
||||
(`python -m app.db --migrate`), and restarts the `ids` unit.
|
||||
|
||||
## Output
|
||||
|
||||
Each script writes to its own report directory. `app.py` also writes
|
||||
@@ -318,9 +354,11 @@ reports/
|
||||
```
|
||||
|
||||
The current router state (address lists + firewall rules) is also mirrored to
|
||||
`data/mikrotik/` on every `--update-firewall` run. Those files are overwritten
|
||||
each run and are **git-visible**, so `git diff` shows how the firewall rules
|
||||
and address lists change over time.
|
||||
`data/mikrotik/` on every `--update-firewall` run as a scratch pointer to the
|
||||
latest snapshot. Those files are overwritten each run and are **not** tracked;
|
||||
the long-term audit trail is the SQLite `firewall_snapshots` tables plus the
|
||||
timestamped reports under `reports/mikrotik/`. `git pull` is therefore never
|
||||
blocked by firewall-state files.
|
||||
|
||||
## Detection
|
||||
|
||||
|
||||
@@ -1,15 +0,0 @@
|
||||
173.245.48.0/20
|
||||
103.21.244.0/22
|
||||
103.22.200.0/22
|
||||
103.31.4.0/22
|
||||
141.101.64.0/18
|
||||
108.162.192.0/18
|
||||
190.93.240.0/20
|
||||
188.114.96.0/20
|
||||
197.234.240.0/22
|
||||
198.41.128.0/17
|
||||
162.158.0.0/15
|
||||
104.16.0.0/13
|
||||
104.24.0.0/14
|
||||
172.64.0.0/13
|
||||
131.0.72.0/22
|
||||
@@ -1,7 +0,0 @@
|
||||
2400:cb00::/32
|
||||
2606:4700::/32
|
||||
2803:f800::/32
|
||||
2405:b500::/32
|
||||
2405:8100::/32
|
||||
2a06:98c0::/29
|
||||
2c0f:f248::/32
|
||||
@@ -1 +0,0 @@
|
||||
$ = paloalto
|
||||
@@ -0,0 +1,74 @@
|
||||
#!/usr/bin/env python3
|
||||
"""CLI entry point for the SQLite store.
|
||||
|
||||
Usage:
|
||||
python -m app.db --migrate applies pending migrations (idempotent)
|
||||
python -m app.db --info prints schema version + table counts, exits
|
||||
python -m app.db --path prints the resolved DB path, exits
|
||||
|
||||
Used by ``deploy.sh`` (D004) to ensure the DB is up to date after a ``git pull``
|
||||
without needing the web server up.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
|
||||
from ..utils.constants import DATA_DB_PATH
|
||||
from .schema import SCHEMA_VERSION, apply_migrations
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="python -m app.db",
|
||||
description="Intrusion-detection SQLite store utilities.")
|
||||
parser.add_argument("--migrate", action="store_true",
|
||||
help="apply pending schema migrations (idempotent)")
|
||||
parser.add_argument("--info", action="store_true",
|
||||
help="print schema version + table/row counts")
|
||||
parser.add_argument("--path", action="store_true",
|
||||
help="print the resolved DB path and exit")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
if args.path:
|
||||
print(DATA_DB_PATH)
|
||||
return 0
|
||||
|
||||
from . import get_connection
|
||||
|
||||
conn = get_connection()
|
||||
current = conn.execute("PRAGMA user_version").fetchone()[0]
|
||||
|
||||
if args.info:
|
||||
counts = {}
|
||||
tables = conn.execute(
|
||||
"SELECT name FROM sqlite_master WHERE type='table' "
|
||||
"AND name NOT LIKE 'sqlite_%' ORDER BY name").fetchall()
|
||||
for t in tables:
|
||||
name = t["name"]
|
||||
try:
|
||||
counts[name] = conn.execute(
|
||||
f"SELECT COUNT(*) FROM \"{name}\"").fetchone()[0]
|
||||
except Exception:
|
||||
counts[name] = None
|
||||
print(f"db_path: {DATA_DB_PATH}")
|
||||
print(f"version: {current} (latest {SCHEMA_VERSION})")
|
||||
print("tables:")
|
||||
for name, n in counts.items():
|
||||
print(f" {name:<32} {n} rows")
|
||||
return 0
|
||||
|
||||
if args.migrate or current < SCHEMA_VERSION:
|
||||
before = current
|
||||
apply_migrations(conn)
|
||||
print(f"schema version {before} -> "
|
||||
f"{conn.execute('PRAGMA user_version').fetchone()[0]}")
|
||||
return 0
|
||||
|
||||
print(f"schema is already up to date (version {current})")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
+31
-7
@@ -143,9 +143,11 @@ class MikrotikFirewall:
|
||||
"""Save current address lists and firewall rules for auditing.
|
||||
|
||||
Writes structured snapshots to ``<report_dir>/mikrotik/data/`` and to
|
||||
the common ``data/mikrotik/`` directory (overwritten every run so git
|
||||
can diff changes to the firewall state over time), plus a timestamped
|
||||
JSON report under ``<report_dir>/mikrotik/``.
|
||||
the common ``data/mikrotik/`` directory (scratch mirror for debugging;
|
||||
**not** a git audit trail), plus a timestamped JSON report under
|
||||
``<report_dir>/mikrotik/``. The authoritative long-term audit trail is
|
||||
the SQLite ``firewall_snapshots`` tables (see ``app.db``), which are
|
||||
appended here regardless of the file writes.
|
||||
"""
|
||||
state = {
|
||||
"router": self.router_ip,
|
||||
@@ -174,11 +176,31 @@ class MikrotikFirewall:
|
||||
os.path.join(data_dir, f"firewall_{version}.json"),
|
||||
state["firewall_rules"][version])
|
||||
|
||||
# Persist a SQLite snapshot (authoritative audit trail, independent of
|
||||
# git). Best-effort: never breaks a router sync if the DB is down.
|
||||
self._persist_snapshot_to_db(state)
|
||||
|
||||
timestamp = datetime.now().strftime("%Y-%m-%d-%H-%M-%S")
|
||||
report_path = os.path.join(report_dir, "mikrotik", f"{timestamp}.json")
|
||||
self._write_json(report_path, state)
|
||||
return report_path
|
||||
|
||||
@staticmethod
|
||||
def _persist_snapshot_to_db(state: dict) -> None:
|
||||
"""Write the firewall snapshot into the SQLite store (best-effort)."""
|
||||
try:
|
||||
from .db import get_connection
|
||||
from .db import repositories as repo
|
||||
repo.save_firewall_snapshot(
|
||||
get_connection(),
|
||||
address_lists_v4=state["address_lists"]["ipv4"],
|
||||
address_lists_v6=state["address_lists"]["ipv6"],
|
||||
rules_v4=state["firewall_rules"]["ipv4"],
|
||||
rules_v6=state["firewall_rules"]["ipv6"],
|
||||
)
|
||||
except Exception as exc:
|
||||
print(f"WARN: could not persist firewall snapshot to db ({exc})")
|
||||
|
||||
def sync_blocklist(self, blocked_ipv4: Iterable[str],
|
||||
blocked_ipv6: Iterable[str],
|
||||
dry_run: bool = False,
|
||||
@@ -186,10 +208,12 @@ class MikrotikFirewall:
|
||||
"""Synchronize the Blocked list cumulatively and audit the result.
|
||||
|
||||
The blocklist is grow-only: targets are added when present in the
|
||||
complete set (all git history of the blocklist files + the current
|
||||
working files + this scan's targets) and never removed automatically.
|
||||
Only router entries absent from the complete set — i.e. never blocked
|
||||
by the system — are pruned. See ``app.blocklist_history``.
|
||||
complete cumulative set (git history of the blocklist files + the
|
||||
current working files + this scan's targets) and never removed
|
||||
automatically. Only router entries absent from the complete set — i.e.
|
||||
never blocked by the system — are pruned. The history now lives in the
|
||||
SQLite blocklist ledger (``app.db`` ``blocklist_entries``); git history
|
||||
remains a read-only fallback. See ``app.blocklist_history``.
|
||||
"""
|
||||
|
||||
desired_ipv4 = set(blocked_ipv4)
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# Deployment support for the intrusion-detection IDS.
|
||||
#
|
||||
# This directory is intentionally small and uses no external framework:
|
||||
# deploy.sh conflict-free `git pull` update wrapper (D004)
|
||||
# ids.service systemd unit template (D005)
|
||||
# restart-service.sh supervisor-agnostic restart shim (D005)
|
||||
# check_runtime_git.sh proves a scan never dirties tracked runtime files (D001)
|
||||
#
|
||||
# See .github/instructions/DEPLOYMENT.md for the full plan.
|
||||
|
||||
set -euo pipefail
|
||||
@@ -0,0 +1,66 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# D001 — runtime write-path guard.
|
||||
#
|
||||
# Proves the deployment invariant: after running a scan, `git status` shows
|
||||
# NO tracked file modified by the scan — every runtime write lands in a
|
||||
# gitignored location (data/, reports/, *.db). This is what makes `git pull`
|
||||
# conflict-free on a server where scans keep running.
|
||||
#
|
||||
# If this fails, a scan wrote into a git-tracked path and `git pull` on the
|
||||
# server can conflict. Investigate the printed files before deploying.
|
||||
#
|
||||
# Usage:
|
||||
# deploy/check_runtime_git.sh [--live]
|
||||
# --live additionally run a real scan first (requires log sources) and
|
||||
# then assert the tree is still clean. Without --live it only
|
||||
# verifies that whatever runtime files already exist are untracked.
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
if [[ "${1:-}" == "--live" ]]; then
|
||||
echo "== Running a real scan to exercise every write path =="
|
||||
python app.py scan --quiet --dry-run || true
|
||||
fi
|
||||
|
||||
echo "== Checking that runtime data is gitignored / untracked =="
|
||||
# Paths a scan writes to. Any of these being tracked OR showing as (non-ignored)
|
||||
# untracked means the invariant is broken and a git pull could conflict.
|
||||
RUNTIME_PATHS=(
|
||||
"app/data"
|
||||
"app/reports"
|
||||
"app/data/ids.db"
|
||||
"app/data/ids.db-wal"
|
||||
"app/data/ids.db-shm"
|
||||
)
|
||||
|
||||
fail=0
|
||||
# 1) No runtime path may be TRACKED, except the single allowed read-only
|
||||
# PaloAlto seed. `git ls-files` reflects the index (including staged
|
||||
# changes), so unborn commits don't cause false failures.
|
||||
allowed="app/data/paloalto/scanner-ips.txt"
|
||||
tracked=$(git ls-files app/data app/reports)
|
||||
problem=$(printf '%s\n' "$tracked" | grep -xvF "$allowed" || true)
|
||||
if [[ -n "$problem" ]]; then
|
||||
echo "FAIL: unexpected tracked files under runtime dirs:" >&2
|
||||
echo "$problem" >&2
|
||||
fail=1
|
||||
fi
|
||||
|
||||
# 2) Nothing may show as untracked-but-not-ignored under the runtime dirs, i.e.
|
||||
# a scan wrote a file git would happily add (and later conflict on).
|
||||
for p in "${RUNTIME_PATHS[@]}"; do
|
||||
dirty=$(git status --porcelain --untracked-files=all -- "$p" 2>/dev/null || true)
|
||||
if [[ -n "$dirty" ]]; then
|
||||
echo "FAIL: runtime path has changes git cares about:" >&2
|
||||
echo "$dirty" >&2
|
||||
fail=1
|
||||
fi
|
||||
done
|
||||
|
||||
if [[ "$fail" -eq 0 ]]; then
|
||||
echo "OK: no tracked runtime data; git pull will not conflict with scan writes."
|
||||
else
|
||||
echo "BROKEN: a scan wrote into git-tracked paths — fix before deploying." >&2
|
||||
exit 1
|
||||
fi
|
||||
@@ -0,0 +1,75 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# D004 — conflict-free deployment update wrapper.
|
||||
#
|
||||
# Updates a deployed instance with `git pull` WITHOUT ever hitting a merge
|
||||
# conflict from files a scan wrote since the last pull. Guarantees:
|
||||
#
|
||||
# 1. Refuses to update over uncommitted tracked changes (clean-tree guard).
|
||||
# 2. Fast-forwards only (--ff-only): never creates merge commits and never
|
||||
# silently conflicts.
|
||||
# 3. Runtime data (data/, reports/, *.db) is gitignored, so a running scan
|
||||
# leaves no dirty tracked paths and never blocks the pull.
|
||||
# 4. Reinstalls dependencies only if requirements.txt changed.
|
||||
# 5. Applies DB migrations idempotently.
|
||||
# 6. Restarts the app under its supervisor.
|
||||
#
|
||||
# Usage:
|
||||
# sudo deploy/deploy.sh # update + restart (default env: production)
|
||||
# ENV=production deploy/deploy.sh # use deploy/ids@production.service
|
||||
# deploy/deploy.sh --no-restart # update deps+migrate but don't restart
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
ENV="${ENV:-production}"
|
||||
RESTART="${1:-restart}"
|
||||
ROOT="$(pwd)"
|
||||
PY="${PY:-python}"
|
||||
|
||||
echo "== Intrusion Detection deploy (env=$ENV) =="
|
||||
|
||||
# 1) Clean-tree guard. Runtime data is gitignored, so this only fails if a
|
||||
# human edited tracked source without committing.
|
||||
if ! git diff --quiet HEAD || ! git diff --cached --quiet; then
|
||||
echo "REFUSING: working tree is not clean." >&2
|
||||
echo "Commit or stash tracked changes, then re-run." >&2
|
||||
echo " git status --short" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 2) Pre-flight: prove runtime isolation holds (fail fast before touching repo).
|
||||
if "$ROOT/deploy/check_runtime_git.sh"; then
|
||||
echo "== runtime isolation ok =="
|
||||
else
|
||||
echo "ABORT: runtime isolation check failed." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 3) Fast-forward only.
|
||||
echo "== fetching upstream =="
|
||||
git fetch origin
|
||||
echo "== fast-forward pull =="
|
||||
git pull --ff-only
|
||||
|
||||
# 4) Dependencies (only if the requirement set changed since the last pull).
|
||||
if git diff --quiet HEAD@{1} HEAD -- requirements.txt; then
|
||||
echo "== requirements.txt unchanged; skipping pip =="
|
||||
else
|
||||
echo "== requirements.txt changed; installing deps =="
|
||||
command -v "$PY" >/dev/null && "$PY" -m pip install -r requirements.txt \
|
||||
|| pip install -r requirements.txt
|
||||
fi
|
||||
|
||||
# 5) Apply DB migrations (idempotent).
|
||||
echo "== applying DB migrations =="
|
||||
"$PY" -m app.db --migrate
|
||||
|
||||
# 6) Restart under the supervisor.
|
||||
if [[ "$RESTART" == "--no-restart" ]]; then
|
||||
echo "== --no-restart: update complete; restart manually =="
|
||||
else
|
||||
echo "== restarting service =="
|
||||
"$ROOT/deploy/restart-service.sh" "$ENV"
|
||||
fi
|
||||
|
||||
echo "== deploy complete =="
|
||||
@@ -0,0 +1,41 @@
|
||||
# systemd unit template (D005) — intrusion-detection web UI.
|
||||
#
|
||||
# Install:
|
||||
# - Replace /path/to/intrusion-detection and the User/Group.
|
||||
# - Copy to /etc/systemd/system/ids.service
|
||||
# - sudo systemctl daemon-reload && sudo systemctl enable --now ids
|
||||
#
|
||||
# The app binds to 127.0.0.1:<WEB_PORT>; it must NOT be exposed to the LAN.
|
||||
#
|
||||
# Restart policy: automatically restart on crash. `deploy/deploy.sh` restarts
|
||||
# this unit after a successful `git pull`. Scans run in a background thread, so
|
||||
# one (serialized) scan at a time; deploy waits for the worker to be idle
|
||||
# (see restart-service.sh) before restarting.
|
||||
|
||||
[Unit]
|
||||
Description=Intrusion Detection web UI + IDS scanner
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
# Run as an unprivileged user; the app only needs read access to the log dirs
|
||||
# it consumes (COPYPARTY_LOG_PATH / NPM_LOG_PATH) and write access to its own
|
||||
# data/ + reports/ + ids.db.
|
||||
User=ids
|
||||
Group=ids
|
||||
WorkingDirectory=/path/to/intrusion-detection
|
||||
EnvironmentFile=-/etc/ids.env
|
||||
ExecStart=/path/to/intrusion-detection/.venv/bin/python /path/to/intrusion-detection/app.py
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
|
||||
# Hardening
|
||||
NoNewPrivileges=yes
|
||||
PrivateTmp=yes
|
||||
ProtectSystem=full
|
||||
ProtectHome=no
|
||||
ReadWritePaths=/path/to/intrusion-detection/data /path/to/intrusion-detection/reports /var/lib/ids
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -0,0 +1,53 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# D005 — supervisor-agnostic restart shim.
|
||||
#
|
||||
# Restarts the intrusion-detection service after a deploy, preferring systemd
|
||||
# (Linux), with a fallback message for other environments (Windows dev).
|
||||
#
|
||||
# It first waits for any in-flight background scan to finish so a checkout
|
||||
# change never interleaves with a live scan (belt-and-suspenders; the runtime
|
||||
# isolation guarantee already means scans only write gitignored files).
|
||||
#
|
||||
# Usage:
|
||||
# deploy/restart-service.sh [production]
|
||||
# ENV=production deploy/restart-service.sh
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
ENV="${1:-${ENV:-production}}"
|
||||
UNIT="ids@${ENV}.service"
|
||||
|
||||
# Wait for an in-flight scan to finish (~0..120s). We probe the web UI status
|
||||
# endpoint if it is reachable; a failure to reach it just means no web UI is up.
|
||||
web_host="${WEB_HOST:-127.0.0.1}"
|
||||
web_port="${WEB_PORT:-54666}"
|
||||
status_url="http://${web_host}:${web_port}/api/scan/status"
|
||||
for _ in $(seq 1 60); do
|
||||
running=$(curl -fsS --max-time 2 "$status_url" 2>/dev/null \
|
||||
| python3 -c 'import sys,json;print(json.load(sys.stdin).get("running",False))' 2>/dev/null \
|
||||
|| echo false)
|
||||
[[ "$running" == "false" ]] && break
|
||||
echo " waiting for running scan to finish..."
|
||||
sleep 2
|
||||
done
|
||||
|
||||
if command -v systemctl >/dev/null 2>&1; then
|
||||
# Installed unit is <unit>.service.template; the enabled name includes env.
|
||||
if systemctl list-unit-files "${UNIT}" >/dev/null 2>&1; then
|
||||
echo "== systemctl restart ${UNIT} =="
|
||||
systemctl restart "${UNIT}"
|
||||
systemctl --no-pager status "${UNIT}" --lines=0 || true
|
||||
elif systemctl list-unit-files "ids.service" >/dev/null 2>&1; then
|
||||
echo "== systemctl restart ids.service =="
|
||||
systemctl restart ids.service
|
||||
systemctl --no-pager status ids.service --lines=0 || true
|
||||
else
|
||||
echo "No ids systemd unit found; restart manually." >&2
|
||||
exit 2
|
||||
fi
|
||||
else
|
||||
echo "No systemd available (Windows dev?). Restart the app manually." >&2
|
||||
echo " python app.py # in a terminal, or via your dev runner" >&2
|
||||
exit 2
|
||||
fi
|
||||
@@ -0,0 +1,57 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Deployment invariant tests (D001).
|
||||
|
||||
Proves that running a scan never creates or modifies git-tracked runtime data,
|
||||
which is what makes ``git pull`` conflict-free on a deployed server.
|
||||
|
||||
Run with:
|
||||
python -m unittest discover -s tests -p 'test_*.py'
|
||||
or (if pytest is installed):
|
||||
python -m pytest tests/
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
# The paloalto scanner-ips.txt seed is deliberately tracked (read-only); every
|
||||
# other file under the runtime dirs must remain untracked.
|
||||
ALLOWED_TRACKED_RUNTIME = {os.path.join("app", "data", "paloalto",
|
||||
"scanner-ips.txt")}
|
||||
|
||||
|
||||
def _git(*args: str) -> str:
|
||||
return subprocess.run(["git", *args], cwd=ROOT, check=True,
|
||||
capture_output=True, text=True).stdout
|
||||
|
||||
|
||||
class RuntimeGitTest(unittest.TestCase):
|
||||
def test_no_runtime_data_tracked(self):
|
||||
"""No runtime file (other than the read-only paloalto seed) is tracked."""
|
||||
tracked = _git("ls-files", "app/data", "app/reports").splitlines()
|
||||
unexpected = [f for f in tracked
|
||||
if f not in ALLOWED_TRACKED_RUNTIME]
|
||||
self.assertEqual(
|
||||
unexpected, [],
|
||||
"git-tracked runtime files would make `git pull` conflict; "
|
||||
"add them to .gitignore: %s" % unexpected)
|
||||
|
||||
def test_runtime_write_paths_ignored(self):
|
||||
"""Every runtime write path is ignored by git (won't dirty the tree)."""
|
||||
stats = subprocess.run(
|
||||
["git", "check-ignore", "-v",
|
||||
"app/data/scan_state.json", "app/data/scan_jobs.json",
|
||||
"app/data/whois_cache.json", "app/data/ids.db",
|
||||
"app/data/ids.db-wal", "app/reports/app/data/blocklist_ipv4.txt",
|
||||
], cwd=ROOT, capture_output=True, text=True).stdout
|
||||
self.assertIn("scan_state.json", stats)
|
||||
self.assertIn("ids.db", stats)
|
||||
self.assertIn("blocklist_ipv4.txt", stats)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in new issue
Block a user