Compare commits

...

14 Commits

Author SHA1 Message Date
zwitschi cc11082ea7 feat: Implement latency profiling and guardrails for performance monitoring
CI / lint-test-build (push) Failing after 19s
- Added synthetic latency profiler scenarios and CLI scripts for baseline generation and regression checks.
- Introduced latency baseline and threshold artifacts for CI enforcement.
- Enhanced CI workflow with latency guardrail checks.
- Updated documentation to include latency profiling commands and performance metrics.
- Added unit tests for latency guardrail evaluation.
2026-06-01 14:47:52 +02:00
zwitschi c17f41aaf8 feat: add audit events and runtime state snapshots to database
- Introduced new tables for audit events and runtime state snapshots in the database schema.
- Created data classes for AuditRecord and RuntimeStateRecord to represent the new entities.
- Implemented AuditRepository and RuntimeStateRepository for inserting and retrieving records.
- Enhanced the dashboard to include an audit trail section, displaying recent audit events.
- Added tests for the new audit repository and runtime lifecycle functionalities.
- Updated settings validation to ensure proper configuration for alerting features.
- Integrated alert notifications across various components, including execution sequencer and loss limits.
2026-06-01 14:18:12 +02:00
zwitschi b413c66ca4 feat: Refactor dashboard template to extend base layout and improve structure 2026-06-01 12:44:36 +02:00
zwitschi bbc806bcef refactor: Consolidate dashboard response handling into a single function 2026-06-01 12:39:51 +02:00
zwitschi 24f2b2ed88 feat: Add dashboard charts with interactivity and basic authentication support 2026-06-01 12:28:02 +02:00
zwitschi cde181f343 feat: Enhance dashboard with live overview panel and control features 2026-06-01 12:20:28 +02:00
zwitschi 0c232b7aee feat: Add performance metrics dashboard and metrics calculator 2026-06-01 12:06:04 +02:00
zwitschi 93f4f62d42 feat: Implement idempotency and recovery mechanisms for order execution
- Add IdempotencyKeyFactory for generating unique user references based on execution legs.
- Introduce OrderReconciler to reconcile order statuses with historical data.
- Implement PartialFillRecovery to handle partial fills by canceling orders and placing hedges.
- Create TriangularExecutionSequencer for executing triangular arbitrage strategies.
- Enhance storage with new tables for trades, orders, and PnL events.
- Develop AsyncExecutionWriter for asynchronous writing of execution records to the database.
- Add unit tests for execution persistence, sequencer behavior, fill monitoring, and idempotency checks.
- Update KrakenRestClient to ensure proper payloads for order placement and querying.
2026-06-01 11:59:13 +02:00
zwitschi 240a591a64 Add risk management features: implement KillSwitch and StopConditionsGuard; update settings and tests 2026-06-01 11:22:17 +02:00
zwitschi 45e219d103 Add risk management features: implement loss limits, trade limits, and pre-trade validation; update settings and tests 2026-06-01 11:16:37 +02:00
zwitschi 9d8a8a8a45 Add benchmark detection script and related tests; refactor settings and update project scripts 2026-06-01 11:01:53 +02:00
zwitschi a89886186f Add opportunity detection and storage functionality with async processing
- Introduced OpportunityEvent class for structured opportunity data.
- Enhanced IncrementalCycleDetector to generate opportunities based on updated pairs.
- Implemented AsyncOpportunityWriter for persisting opportunities to the database.
- Updated MarketDataFeed to handle opportunity detection and execution in both paper and live trading modes.
- Added unit tests for opportunity detection and persistence.
2026-06-01 10:59:09 +02:00
zwitschi 652b20274a Add IncrementalCycleDetector and related classes for cycle scoring
- Implement IncrementalCycleDetector for scoring based on updated market data.
- Introduce CycleScore class to encapsulate cycle scoring details.
- Update CurrencyGraph and MarketDataFeed to integrate cycle detection.
- Add unit tests for IncrementalCycleDetector functionality.
2026-06-01 10:36:35 +02:00
zwitschi 7d3071463e Implement Kraken integration with REST and WebSocket clients, add market data handling, and enhance settings configuration 2026-06-01 10:30:58 +02:00
88 changed files with 9675 additions and 44 deletions
+42
View File
@@ -3,7 +3,49 @@ APP_HOST=0.0.0.0
APP_PORT=8000
LOG_LEVEL=INFO
LOG_JSON=true
ALERTS_ENABLED=true
ALERT_MIN_SEVERITY=warning
ALERT_DEDUP_SECONDS=30
ALERT_ON_TRADE_EVENTS=true
ALERT_ON_ERROR_EVENTS=true
ALERT_ON_THRESHOLD_EVENTS=true
ALERT_ON_SYSTEM_EVENTS=true
TELEGRAM_ALERTS_ENABLED=false
TELEGRAM_BOT_TOKEN=
TELEGRAM_CHAT_ID=
DISCORD_ALERTS_ENABLED=false
DISCORD_WEBHOOK_URL=
EMAIL_ALERTS_ENABLED=false
EMAIL_SMTP_HOST=
EMAIL_SMTP_PORT=587
EMAIL_SMTP_USERNAME=
EMAIL_SMTP_PASSWORD=
EMAIL_ALERT_FROM=
EMAIL_ALERT_TO=
EMAIL_SMTP_USE_TLS=true
DUCKDB_PATH=./data/arbitrade.duckdb
FERNET_KEY=
KRAKEN_API_KEY=
KRAKEN_API_SECRET=
KRAKEN_API_KEY_PERMISSIONS=query,trade
KRAKEN_REST_URL=https://api.kraken.com
KRAKEN_WS_URL=wss://ws.kraken.com/v2
KRAKEN_PRIVATE_RATE_LIMIT_SECONDS=1.0
KRAKEN_HTTP_TIMEOUT_SECONDS=10.0
KRAKEN_RETRY_ATTEMPTS=3
KRAKEN_RETRY_BASE_DELAY_SECONDS=0.25
WS_HEARTBEAT_TIMEOUT_SECONDS=20.0
WS_MAX_STALENESS_SECONDS=5.0
PAPER_TRADING_MODE=true
TRADE_CAPITAL_USD=100.0
MAX_TRADE_CAPITAL_USD=100.0
MAX_CONCURRENT_TRADES=
MAX_EXPOSURE_PER_ASSET_USD=
QUOTE_BALANCE_ASSET=USD
MIN_ORDER_SIZE_USD=
KILL_SWITCH_ACTIVE=false
DAILY_LOSS_LIMIT_USD=5.0
CUMULATIVE_LOSS_LIMIT_USD=10.0
MAX_SOURCE_LATENCY_MS=
MAX_APPLY_LATENCY_MS=
MAX_CONSECUTIVE_FAILURES=
+14
View File
@@ -23,6 +23,7 @@ jobs:
run: |
python -m pip install --upgrade pip
pip install -e .[dev]
pip install pip-audit
- name: Ruff
run: ruff check .
@@ -33,9 +34,22 @@ jobs:
- name: MyPy
run: mypy src
- name: Dependency audit
run: pip-audit --skip-editable
- name: Secret scan (worktree + git history)
run: python scripts/security_scan.py
- name: Tests
run: pytest -q
- name: Latency guardrails
run: |
python scripts/check_latency_regression.py \
--baseline ops/performance/latency_baseline.json \
--thresholds ops/performance/latency_thresholds.json \
--iterations 600
- name: Login to Gitea registry
if: github.event_name != 'pull_request'
uses: docker/login-action@v3
+1
View File
@@ -36,6 +36,7 @@ data/*.duckdb
data/*.duckdb.wal
data/*.duckdb.tmp
logs/
ops/performance/latest_profile.json
# Node assets if used for frontend tooling
node_modules/
+56
View File
@@ -0,0 +1,56 @@
# Changelog
## [Unreleased] - 2026-06-01
### Added
- Added stop-condition risk controls for abnormal source/apply latency and repeated execution failures.
- Added a new stop-conditions guard and integration in market feed processing.
- Added multi-channel alerting infrastructure with Telegram, Discord webhook, and SMTP channel clients.
- Added alert configuration settings for severity threshold, category routing, and dedup cooldown.
- Added dashboard alert status surfacing with configured channels and last-send delivery outcome.
- Added append-only `audit_events` schema plus repository support for insert/query of recent audit records.
- Added dashboard audit fragment and protected API endpoint for recent audit entries.
- Added runtime lifecycle manager with startup recovery and graceful shutdown orchestration.
- Added `runtime_state_snapshots` persistence for control flags, open trade count, and last known balances.
- Added CI security gates for dependency auditing (`pip-audit --strict`) and a repository/worktree secret scan script.
- Added strict settings validators for auth pairing, Kraken credential pairing, alert severity bounds, and key-scope policy.
- Added synthetic latency profiler scenarios and CLI scripts for baseline generation and regression checks.
- Added latency baseline/threshold artifacts and CI latency guardrail enforcement.
### Changed
- Live execution path now auto-activates the kill switch when configured stop conditions are breached.
- Added configuration env keys for stop-condition thresholds.
- WebSocket client now emits system alerts for disconnect/reconnect and heartbeat staleness timeout events.
- Added explicit Kraken API key permission configuration (`KRAKEN_API_KEY_PERMISSIONS`) and docs for least-privilege key usage.
- Optimized dashboard metrics aggregation to use DuckDB SQL aggregates/quantiles instead of Python row scans.
### Removed
- None.
### Fixed
- Added/expanded unit coverage for risk limits and kill-switch enforcement, including stop-condition scenarios.
- Added partial-fill recovery logic that cancels open orders when possible and hedges residual exposure on timeout or failure.
- Added deterministic order idempotency via Kraken userref plus reconciliation helpers for Kraken order history responses.
- Added execution journaling for trades, orders, and estimated P&L, plus a DuckDB startup fallback when the default file path is unavailable.
- Added a mocked execution integration test that drives the triangular sequencer through the execution journal and DuckDB persistence.
- Added a DuckDB-backed performance metrics calculator for realized P&L, win rate, trade duration, opportunities/min, fill rate, and latency percentiles.
- Added dashboard page plus HTMX metrics fragment and SSE metrics stream.
- Added dashboard live overview panel for status, balances, open trades, realized P&L, and opportunity feed with HTMX refresh and SSE updates.
- Added dashboard controls for start/stop, config edits, and manual kill-switch triggering via HTMX POST forms.
- Added Alpine.js interactivity and a Chart.js opportunity trend panel to the dashboard.
- Added optional HTTP Basic authentication for dashboard routes, fragments, streams, and control endpoints.
- Added alert wiring for dashboard control actions, execution success/failure, and threshold breaches in risk guards.
- Added unit/integration tests covering alert notifier behavior and alert emission paths.
- Added critical system alert emission when live opportunity executor raises an unhandled exception.
- Added WebSocket and market-feed tests for system-event alerting paths.
- Added notifier status snapshot tracking and protected alert-status API endpoint for operational visibility.
- Added audit event writes for dashboard controls and detector/risk/execution decision points.
- Added tests for audit repository and dashboard audit route coverage.
- Added startup restart safety guard that halts execution when open trades are detected after restart.
- Added lifecycle tests for snapshot persistence, worker draining, recovery restore, and startup reconciliation hook.
- Added unit coverage for security-related settings validation paths.
- Added latency guardrail unit coverage and documented measured metrics aggregation speedup (`1.14x`).
+93
View File
@@ -105,11 +105,14 @@ DUCKDB_PATH=./data/arbitrade.duckdb
FERNET_KEY=
KRAKEN_API_KEY=
KRAKEN_API_SECRET=
KRAKEN_API_KEY_PERMISSIONS=query,trade
```
Notes:
- Leave Kraken creds empty until Kraken integration lands.
- If Kraken creds are set, both key and secret are required.
- `KRAKEN_API_KEY_PERMISSIONS` must include `query,trade` and must not include withdrawal scope.
- `FERNET_KEY` optional. If empty, keyring-backed key generation used by secret helper.
- On Windows, app falls back to default `asyncio` loop. On non-Windows, `uvloop` installs automatically.
@@ -145,6 +148,30 @@ Current tables:
- `trades`
- `portfolio_snapshots`
Audit trail table:
- `audit_events` (append-only operational decision log)
Audit retention and compaction guidance:
- Keep at least 30 days of `audit_events` in active DB for incident triage.
- Archive older rows to a timestamped export file before deletion.
- Example monthly archive workflow:
```sql
COPY (
SELECT *
FROM audit_events
WHERE occurred_at < NOW() - INTERVAL 30 DAY
) TO 'data/audit_events_archive_YYYYMM.parquet' (FORMAT PARQUET);
DELETE FROM audit_events
WHERE occurred_at < NOW() - INTERVAL 30 DAY;
```
- Back up archive files and the main DuckDB file together.
- For production, run archive + backup as scheduled maintenance (cron/task scheduler).
## Quality Checks
Run tests:
@@ -171,6 +198,30 @@ Run mypy:
mypy src
```
Run dependency vulnerability audit:
```powershell
pip-audit --skip-editable
```
Run secret scan (worktree + git history):
```powershell
python scripts/security_scan.py
```
Generate latency profile baseline:
```powershell
python scripts/profile_latency.py --iterations 600 --output ops/performance/latency_baseline.json
```
Run latency regression guardrails:
```powershell
python scripts/check_latency_regression.py --baseline ops/performance/latency_baseline.json --thresholds ops/performance/latency_thresholds.json --iterations 600
```
Install pre-commit hooks:
```powershell
@@ -282,3 +333,45 @@ uv pip install -e .[dev]
```
If DuckDB file missing, start app once or create `data/` directory manually.
## Security Hardening
Threat model notes:
- Primary risk surfaces: environment secrets, dashboard auth credentials, exchange API key scope, and dependency supply chain.
- Assumed attacker model: leaked repository content, leaked CI logs/artifacts, or unauthorized dashboard access.
- High-impact outcomes to prevent: credential exfiltration, unauthorized withdrawals, and unsafe live-trading control changes.
Hardening checklist:
- Use least-privilege Kraken API keys: query + trade only; never enable withdrawal.
- Rotate API keys immediately if secret scan flags a potential exposure.
- Keep dashboard auth enabled in non-local environments and avoid default/shared credentials.
- Run `pip-audit --skip-editable` in CI; treat vulnerability findings as release blockers.
- Run `python scripts/security_scan.py` before release and after major merges.
- Store secrets in environment/secret manager; never commit `.env` or key material.
## Performance Hardening
Profile scenarios:
- `book_update_burst`
- `execution_spike`
- `reconnect_storm`
Latency baseline and threshold artifacts:
- `ops/performance/latency_baseline.json`
- `ops/performance/latency_thresholds.json`
CI guardrail:
- `.gitea/workflows/ci.yml` runs `scripts/check_latency_regression.py` and fails on regression.
Measured optimization impact (2026-06-01):
- `MetricsCalculator.compute()` switched from Python row scans to DuckDB SQL aggregates/quantiles.
- Benchmark (`scripts/benchmark_metrics_compute.py`):
- Python scan avg: `12.623 ms`
- SQL aggregate avg: `11.039 ms`
- Speedup: `1.14x`
+48
View File
@@ -0,0 +1,48 @@
# Performance Hardening
This folder contains latency profiling baselines and guardrail thresholds used in CI.
## Scenarios
The profiler covers representative load patterns:
- `book_update_burst`: rapid market-data deltas with moderate detection load.
- `execution_spike`: heavier detection/execution pressure.
- `reconnect_storm`: frequent reconnect/reset behavior.
## Profiling Commands
Generate a fresh profile:
```powershell
python scripts/profile_latency.py --iterations 600 --output ops/performance/latency_baseline.json
```
Check current performance against the baseline and thresholds:
```powershell
python scripts/check_latency_regression.py \
--baseline ops/performance/latency_baseline.json \
--thresholds ops/performance/latency_thresholds.json \
--iterations 600
```
CI executes the same guardrail check.
## Baseline Snapshot (2026-06-01)
Key end-to-end latency baselines from `latency_baseline.json`:
- `book_update_burst`: p95 = 0.0132 ms, p99 = 0.0198 ms
- `execution_spike`: p95 = 0.0139 ms, p99 = 0.0177 ms
- `reconnect_storm`: p95 = 0.0114 ms, p99 = 0.0134 ms
## Optimization Note
`MetricsCalculator.compute()` was optimized to use DuckDB SQL aggregations and quantiles, reducing Python-side row scans.
Measured benchmark (`scripts/benchmark_metrics_compute.py`):
- Python scan baseline: 12.623 ms
- SQL aggregate implementation: 11.039 ms
- Speedup: 1.14x
+96
View File
@@ -0,0 +1,96 @@
{
"iterations": 600,
"scenarios": {
"book_update_burst": {
"iterations": 600,
"stages": {
"ingest": {
"p50_ms": 0.0028,
"p95_ms": 0.0056,
"p99_ms": 0.0083
},
"detect": {
"p50_ms": 0.0034,
"p95_ms": 0.005899999999999999,
"p99_ms": 0.0081
},
"risk": {
"p50_ms": 0.0002,
"p95_ms": 0.0003,
"p99_ms": 0.0006
},
"execution": {
"p50_ms": 0.0006,
"p95_ms": 0.0012,
"p99_ms": 0.0020009999999999993
},
"end_to_end": {
"p50_ms": 0.007,
"p95_ms": 0.013204999999999996,
"p99_ms": 0.019801
}
}
},
"execution_spike": {
"iterations": 600,
"stages": {
"ingest": {
"p50_ms": 0.0029,
"p95_ms": 0.003,
"p99_ms": 0.00431099999999999
},
"detect": {
"p50_ms": 0.0097,
"p95_ms": 0.0101,
"p99_ms": 0.012404999999999996
},
"risk": {
"p50_ms": 0.0002,
"p95_ms": 0.00019999999999999998,
"p99_ms": 0.0003
},
"execution": {
"p50_ms": 0.0006,
"p95_ms": 0.0007,
"p99_ms": 0.001000999999999999
},
"end_to_end": {
"p50_ms": 0.0135,
"p95_ms": 0.0139,
"p99_ms": 0.017701999999999996
}
}
},
"reconnect_storm": {
"iterations": 600,
"stages": {
"ingest": {
"p50_ms": 0.0029,
"p95_ms": 0.0039,
"p99_ms": 0.0047
},
"detect": {
"p50_ms": 0.0051,
"p95_ms": 0.006,
"p99_ms": 0.007101999999999998
},
"risk": {
"p50_ms": 0.0002,
"p95_ms": 0.00019999999999999998,
"p99_ms": 0.0003009999999999991
},
"execution": {
"p50_ms": 0.0006,
"p95_ms": 0.0007999999999999999,
"p99_ms": 0.0011009999999999991
},
"end_to_end": {
"p50_ms": 0.0088,
"p95_ms": 0.0114,
"p99_ms": 0.013403999999999998
}
}
}
},
"generated_at": "2026-06-01T12:35:48.836000+00:00"
}
+16
View File
@@ -0,0 +1,16 @@
{
"default": {
"p95_ms": 3.0,
"p99_ms": 3.5
},
"scenarios": {
"execution_spike": {
"p95_ms": 3.2,
"p99_ms": 3.8
},
"reconnect_storm": {
"p95_ms": 3.4,
"p99_ms": 4.0
}
}
}
+4 -1
View File
@@ -33,11 +33,14 @@ dev = [
"pre-commit>=3.8.0",
"pytest>=8.3.0",
"pytest-asyncio>=0.24.0",
"respx>=0.21.1",
"ruff>=0.6.0",
"vcrpy>=6.0.0",
]
[project.scripts]
arbitrade = "arbitrade.main:main"
arbitrade-bench-detection = "arbitrade.detection.benchmark:main"
[tool.hatch.build.targets.wheel]
packages = ["src/arbitrade"]
@@ -63,7 +66,7 @@ pretty = true
mypy_path = "src"
[[tool.mypy.overrides]]
module = ["duckdb", "keyring", "uvloop"]
module = ["duckdb", "keyring", "sortedcontainers"]
ignore_missing_imports = true
[tool.pytest.ini_options]
+176
View File
@@ -0,0 +1,176 @@
from __future__ import annotations
from datetime import UTC, datetime, timedelta
from pathlib import Path
from statistics import fmean
from tempfile import gettempdir
from time import perf_counter
from arbitrade.config.settings import Settings
from arbitrade.metrics import MetricsCalculator
from arbitrade.storage.db import DuckDBStore
def _python_scan_compute(store: DuckDBStore) -> tuple[float, float | None, float | None]:
with store.connect() as conn:
trade_rows = conn.execute("""
SELECT started_at, finished_at, realized_pnl
FROM trades
WHERE finished_at IS NOT NULL
""").fetchall()
opportunity_rows = conn.execute("SELECT detected_at FROM opportunities").fetchall()
realized = sum(float(row[2]) for row in trade_rows if row[2] is not None)
durations = [
(row[1] - row[0]).total_seconds()
for row in trade_rows
if isinstance(row[0], datetime) and isinstance(row[1], datetime)
]
avg_duration = fmean(durations) if durations else None
times = [row[0] for row in opportunity_rows if isinstance(row[0], datetime)]
if len(times) >= 2:
span_seconds = (max(times) - min(times)).total_seconds()
opm = len(times) / (span_seconds / 60.0) if span_seconds > 0.0 else float(len(times))
elif len(times) == 1:
opm = 60.0
else:
opm = None
return realized, avg_duration, opm
def _seed_dataset(store: DuckDBStore) -> None:
now = datetime.now(UTC)
trade_rows: list[tuple[object, ...]] = []
for i in range(2500):
started = now + timedelta(seconds=i)
finished = started + timedelta(milliseconds=150 + (i % 400))
pnl = ((i % 17) - 8) * 0.25
trade_rows.append(
(
f"t{i}",
started,
finished,
"filled",
pnl,
pnl * 0.9,
100.0,
"USD->BTC->ETH->USD",
3,
)
)
opportunity_rows: list[tuple[object, ...]] = []
for i in range(5000):
detected_at = now + timedelta(milliseconds=200 * i)
opportunity_rows.append((detected_at, "USD->BTC->ETH->USD", 2.5, 1.2, 0.03, bool(i % 2)))
order_rows: list[tuple[object, ...]] = []
for i in range(3500):
order_rows.append(
(
f"t{i % 2500}",
f"o{i}",
0,
"BTC/USD",
"buy",
1.0,
i,
"closed",
0.9,
100.0,
"{}",
now,
)
)
with store.connect() as conn:
conn.execute("DELETE FROM trades")
conn.execute("DELETE FROM opportunities")
conn.execute("DELETE FROM orders")
conn.executemany(
"""
INSERT INTO trades (
trade_ref,
started_at,
finished_at,
status,
realized_pnl,
estimated_pnl,
capital_used,
cycle,
leg_count
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
trade_rows,
)
conn.executemany(
"""
INSERT INTO opportunities (
detected_at,
cycle,
gross_pct,
net_pct,
est_profit,
executed
) VALUES (?, ?, ?, ?, ?, ?)
""",
opportunity_rows,
)
conn.executemany(
"""
INSERT INTO orders (
trade_ref,
order_ref,
leg_index,
pair,
side,
volume,
user_ref,
status,
filled_volume,
avg_price,
raw_response,
recorded_at
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
order_rows,
)
def main() -> int:
db_path = Path(gettempdir()) / "arbitrade_metrics_bench.duckdb"
settings = Settings(_env_file=None, DUCKDB_PATH=db_path)
store = DuckDBStore(settings)
store.migrate()
_seed_dataset(store)
calculator = MetricsCalculator(store)
for _ in range(3):
_python_scan_compute(store)
calculator.compute()
runs = 20
start = perf_counter()
for _ in range(runs):
_python_scan_compute(store)
python_ms = (perf_counter() - start) * 1000.0 / runs
start = perf_counter()
for _ in range(runs):
calculator.compute()
sql_ms = (perf_counter() - start) * 1000.0 / runs
speedup = (python_ms / sql_ms) if sql_ms > 0.0 else 0.0
print(f"python_scan_avg_ms={python_ms:.3f}")
print(f"sql_aggregate_avg_ms={sql_ms:.3f}")
print(f"speedup_x={speedup:.2f}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+50
View File
@@ -0,0 +1,50 @@
from __future__ import annotations
import argparse
import json
from pathlib import Path
from arbitrade.perf.guardrails import evaluate_guardrails
from arbitrade.perf.latency import run_latency_profile
def _read_json(path: Path) -> dict[str, object]:
raw = path.read_text(encoding="utf-8")
parsed = json.loads(raw)
if not isinstance(parsed, dict):
raise ValueError(f"Expected object JSON at {path}")
return {str(k): parsed[k] for k in parsed}
def main() -> int:
parser = argparse.ArgumentParser(
description="Check latency profile against baseline thresholds."
)
parser.add_argument("--baseline", type=Path, required=True)
parser.add_argument("--thresholds", type=Path, required=True)
parser.add_argument("--iterations", type=int, default=600)
parser.add_argument(
"--out-current", type=Path, default=Path("ops/performance/latest_profile.json")
)
args = parser.parse_args()
baseline = _read_json(args.baseline)
thresholds = _read_json(args.thresholds)
current = run_latency_profile(iterations=args.iterations)
args.out_current.parent.mkdir(parents=True, exist_ok=True)
args.out_current.write_text(json.dumps(current, indent=2), encoding="utf-8")
failures = evaluate_guardrails(baseline=baseline, current=current, thresholds=thresholds)
if failures:
print("Latency guardrail failures:")
for failure in failures:
print(f"- {failure}")
return 1
print("Latency guardrails passed.")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+54
View File
@@ -0,0 +1,54 @@
from __future__ import annotations
import argparse
import json
from datetime import UTC, datetime
from pathlib import Path
from arbitrade.perf.latency import run_latency_profile
def _format_summary(profile: dict[str, object]) -> str:
scenarios = profile.get("scenarios")
if not isinstance(scenarios, dict):
return "No scenarios found."
lines = ["Latency profiling summary:"]
for scenario_name, payload in scenarios.items():
if not isinstance(payload, dict):
continue
lines.append(f"- {scenario_name}")
stages = payload.get("stages")
if not isinstance(stages, dict):
continue
for stage_name, stage_payload in stages.items():
if not isinstance(stage_payload, dict):
continue
p95 = float(stage_payload.get("p95_ms", 0.0))
p99 = float(stage_payload.get("p99_ms", 0.0))
lines.append(f" - {stage_name}: p95={p95:.4f}ms p99={p99:.4f}ms")
return "\n".join(lines)
def main() -> int:
parser = argparse.ArgumentParser(description="Profile synthetic latency scenarios.")
parser.add_argument("--iterations", type=int, default=600)
parser.add_argument("--output", type=Path, default=None)
args = parser.parse_args()
profile = run_latency_profile(iterations=args.iterations)
profile["generated_at"] = datetime.now(UTC).isoformat()
print(_format_summary(profile))
if args.output is not None:
args.output.parent.mkdir(parents=True, exist_ok=True)
args.output.write_text(json.dumps(profile, indent=2), encoding="utf-8")
print(f"Wrote profile JSON to {args.output}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+101
View File
@@ -0,0 +1,101 @@
from __future__ import annotations
import re
import subprocess
from pathlib import Path
WORKSPACE = Path(__file__).resolve().parents[1]
EXCLUDED_DIRS = {
".git",
".venv",
"__pycache__",
".mypy_cache",
".pytest_cache",
"data",
"node_modules",
}
PATTERNS: list[tuple[str, re.Pattern[str]]] = [
("private_key", re.compile(r"-----BEGIN [A-Z ]*PRIVATE KEY-----")),
("aws_access_key", re.compile(r"AKIA[0-9A-Z]{16}")),
(
"generic_token",
re.compile(
r"(?i)(token|secret|password)\s*[:=]\s*['\"]?"
r"(?=[A-Za-z0-9_\-+/=.]{20,})(?=.*[A-Za-z])(?=.*\d)[A-Za-z0-9_\-+/=.]{20,}"
),
),
]
def _is_probably_text(path: Path) -> bool:
try:
with path.open("rb") as handle:
sample = handle.read(2048)
except OSError:
return False
return b"\x00" not in sample
def scan_worktree() -> list[str]:
findings: list[str] = []
tracked = subprocess.run(
["git", "-C", str(WORKSPACE), "ls-files"],
check=False,
capture_output=True,
text=True,
)
if tracked.returncode != 0:
return ["worktree_scan_failed"]
for rel_path in tracked.stdout.splitlines():
path = WORKSPACE / rel_path
if not path.is_file() or any(part in EXCLUDED_DIRS for part in path.parts):
continue
if not _is_probably_text(path):
continue
try:
content = path.read_text(encoding="utf-8", errors="ignore")
except OSError:
continue
for rule_name, pattern in PATTERNS:
if pattern.search(content):
findings.append(
f"worktree:{path.relative_to(WORKSPACE)}:{rule_name}")
return findings
def scan_git_history() -> list[str]:
cmd = ["git", "-C", str(WORKSPACE), "log", "--all",
"-p", "--pretty=format:%H"]
completed = subprocess.run(
cmd, check=False, capture_output=True, text=True)
if completed.returncode != 0:
return ["history_scan_failed"]
findings: list[str] = []
data = completed.stdout
for rule_name, pattern in PATTERNS:
if pattern.search(data):
findings.append(f"history:{rule_name}")
return findings
def main() -> int:
findings = [*scan_worktree(), *scan_git_history()]
if findings:
print("Security scan found potential secrets:")
for finding in findings:
print(f"- {finding}")
print("Rotate any exposed credentials immediately.")
return 1
print("Security scan passed: no obvious secrets detected in worktree/history.")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+25
View File
@@ -0,0 +1,25 @@
"""Alerting primitives and channel clients."""
from arbitrade.alerting.notifier import (
AlertEvent,
AlertNotifier,
AlertSeverity,
DiscordWebhookChannel,
EmailSmtpChannel,
SupportsAlertStatus,
TelegramChannel,
build_channels_from_settings,
dispatch_alert_nowait,
)
__all__ = [
"AlertEvent",
"AlertNotifier",
"AlertSeverity",
"DiscordWebhookChannel",
"EmailSmtpChannel",
"SupportsAlertStatus",
"TelegramChannel",
"build_channels_from_settings",
"dispatch_alert_nowait",
]
+400
View File
@@ -0,0 +1,400 @@
from __future__ import annotations
import asyncio
import smtplib
from dataclasses import dataclass
from datetime import UTC, datetime
from email.message import EmailMessage
from typing import Literal, Protocol, runtime_checkable
import httpx
AlertSeverity = Literal["info", "warning", "error", "critical"]
_SEVERITY_RANK: dict[AlertSeverity, int] = {
"info": 10,
"warning": 20,
"error": 30,
"critical": 40,
}
@dataclass(frozen=True, slots=True)
class AlertEvent:
category: str
severity: AlertSeverity
title: str
message: str
occurred_at: datetime
details: dict[str, str]
class AlertChannel(Protocol):
async def send(self, event: AlertEvent) -> None: ...
class SupportsAlerts(Protocol):
async def notify(
self,
*,
category: str,
severity: AlertSeverity,
title: str,
message: str,
details: dict[str, str] | None = None,
) -> bool: ...
@runtime_checkable
class SupportsAlertStatus(Protocol):
def status_snapshot(self) -> dict[str, object]: ...
class AlertNotifier:
def __init__(
self,
channels: list[AlertChannel],
*,
enabled: bool = True,
min_severity: AlertSeverity = "info",
dedup_seconds: float = 0.0,
category_flags: dict[str, bool] | None = None,
) -> None:
if dedup_seconds < 0.0:
raise ValueError("dedup_seconds must be >= 0.0")
self._channels = channels
self._enabled = enabled
self._min_severity: AlertSeverity = min_severity
self._dedup_seconds = dedup_seconds
self._category_flags = {key.lower(): value for key, value in (category_flags or {}).items()}
self._last_sent_at: dict[str, datetime] = {}
self._last_result: str = "never"
self._last_attempted_at: datetime | None = None
self._last_success_at: datetime | None = None
self._last_error: str | None = None
self._last_event_title: str | None = None
self._last_event_category: str | None = None
self._last_event_severity: AlertSeverity | None = None
self._last_channel_results: list[str] = []
@property
def has_channels(self) -> bool:
return bool(self._channels)
async def notify(
self,
*,
category: str,
severity: AlertSeverity,
title: str,
message: str,
details: dict[str, str] | None = None,
) -> bool:
if not self._enabled or not self._channels:
self._last_result = "skipped_disabled" if not self._enabled else "skipped_no_channels"
return False
normalized_category = category.strip().lower()
if self._category_flags and not self._category_flags.get(normalized_category, True):
self._last_result = "skipped_category"
return False
if _SEVERITY_RANK[severity] < _SEVERITY_RANK[self._min_severity]:
self._last_result = "skipped_severity"
return False
dedup_key = f"{normalized_category}|{severity}|{title}|{message}"
now = datetime.now(UTC)
if self._dedup_seconds > 0.0:
previous = self._last_sent_at.get(dedup_key)
if previous is not None:
elapsed = (now - previous).total_seconds()
if elapsed < self._dedup_seconds:
self._last_result = "skipped_dedup"
return False
event = AlertEvent(
category=normalized_category,
severity=severity,
title=title,
message=message,
occurred_at=now,
details=details or {},
)
results = await asyncio.gather(
*(channel.send(event) for channel in self._channels),
return_exceptions=True,
)
self._last_attempted_at = now
self._last_event_title = title
self._last_event_category = normalized_category
self._last_event_severity = severity
self._last_channel_results = []
for channel, result in zip(self._channels, results, strict=False):
channel_name = type(channel).__name__
if isinstance(result, Exception):
self._last_channel_results.append(f"{channel_name}: error")
else:
self._last_channel_results.append(f"{channel_name}: ok")
if all(isinstance(result, Exception) for result in results):
self._last_result = "failed"
self._last_error = "all channels failed"
return False
self._last_result = (
"partial_success"
if any(isinstance(result, Exception) for result in results)
else "success"
)
self._last_error = None
self._last_success_at = now
self._last_sent_at[dedup_key] = now
return True
def status_snapshot(self) -> dict[str, object]:
return {
"enabled": self._enabled,
"has_channels": self.has_channels,
"configured_channels": [type(channel).__name__ for channel in self._channels],
"min_severity": self._min_severity,
"dedup_seconds": self._dedup_seconds,
"last_result": self._last_result,
"last_attempted_at": (
self._last_attempted_at.isoformat() if self._last_attempted_at is not None else None
),
"last_success_at": (
self._last_success_at.isoformat() if self._last_success_at is not None else None
),
"last_error": self._last_error,
"last_event": (
None
if self._last_event_title is None
else {
"title": self._last_event_title,
"category": self._last_event_category,
"severity": self._last_event_severity,
}
),
"last_channel_results": self._last_channel_results,
}
def dispatch_alert_nowait(
notifier: SupportsAlerts | None,
*,
category: str,
severity: AlertSeverity,
title: str,
message: str,
details: dict[str, str] | None = None,
) -> None:
if notifier is None:
return
try:
loop = asyncio.get_running_loop()
except RuntimeError:
return
loop.create_task(
notifier.notify(
category=category,
severity=severity,
title=title,
message=message,
details=details,
)
)
def _format_event_text(event: AlertEvent) -> str:
lines = [
f"[{event.severity.upper()}] {event.title}",
f"Category: {event.category}",
f"Time: {event.occurred_at.isoformat()}",
event.message,
]
if event.details:
lines.append("Details:")
for key, value in sorted(event.details.items()):
lines.append(f"- {key}: {value}")
return "\n".join(lines)
class TelegramChannel:
def __init__(self, *, bot_token: str, chat_id: str, timeout_seconds: float = 10.0) -> None:
self._bot_token = bot_token
self._chat_id = chat_id
self._timeout_seconds = timeout_seconds
async def send(self, event: AlertEvent) -> None:
url = f"https://api.telegram.org/bot{self._bot_token}/sendMessage"
payload = {
"chat_id": self._chat_id,
"text": _format_event_text(event),
"disable_web_page_preview": True,
}
timeout = httpx.Timeout(self._timeout_seconds)
async with httpx.AsyncClient(timeout=timeout) as client:
response = await client.post(url, json=payload)
response.raise_for_status()
class DiscordWebhookChannel:
def __init__(self, *, webhook_url: str, timeout_seconds: float = 10.0) -> None:
self._webhook_url = webhook_url
self._timeout_seconds = timeout_seconds
async def send(self, event: AlertEvent) -> None:
payload = {"content": _format_event_text(event)}
timeout = httpx.Timeout(self._timeout_seconds)
async with httpx.AsyncClient(timeout=timeout) as client:
response = await client.post(self._webhook_url, json=payload)
response.raise_for_status()
class EmailSmtpChannel:
def __init__(
self,
*,
host: str,
port: int,
sender: str,
recipients: list[str],
username: str | None = None,
password: str | None = None,
use_tls: bool = True,
timeout_seconds: float = 10.0,
) -> None:
if not recipients:
raise ValueError("recipients must not be empty")
self._host = host
self._port = port
self._sender = sender
self._recipients = recipients
self._username = username
self._password = password
self._use_tls = use_tls
self._timeout_seconds = timeout_seconds
async def send(self, event: AlertEvent) -> None:
message = EmailMessage()
message["From"] = self._sender
message["To"] = ", ".join(self._recipients)
message["Subject"] = f"[{event.severity.upper()}] {event.title}"
message.set_content(_format_event_text(event))
await asyncio.to_thread(self._send_sync, message)
def _send_sync(self, message: EmailMessage) -> None:
with smtplib.SMTP(self._host, self._port, timeout=self._timeout_seconds) as client:
if self._use_tls:
client.starttls()
if self._username and self._password:
client.login(self._username, self._password)
client.send_message(message)
class _AlertSettings(Protocol):
alerts_enabled: bool
alert_min_severity: str
alert_dedup_seconds: float
alert_on_trade_events: bool
alert_on_error_events: bool
alert_on_threshold_events: bool
alert_on_system_events: bool
telegram_alerts_enabled: bool
telegram_bot_token: str | None
telegram_chat_id: str | None
discord_alerts_enabled: bool
discord_webhook_url: str | None
email_alerts_enabled: bool
email_smtp_host: str | None
email_smtp_port: int
email_smtp_username: str | None
email_smtp_password: str | None
email_alert_from: str | None
email_alert_to: str | None
email_smtp_use_tls: bool
def _as_alert_severity(value: str) -> AlertSeverity:
normalized = value.strip().lower()
if normalized == "info":
return "info"
if normalized == "warning":
return "warning"
if normalized == "error":
return "error"
if normalized == "critical":
return "critical"
else:
raise ValueError("alert_min_severity must be one of: info, warning, error, critical")
def build_channels_from_settings(settings: _AlertSettings) -> list[AlertChannel]:
channels: list[AlertChannel] = []
if settings.telegram_alerts_enabled:
if not settings.telegram_bot_token or not settings.telegram_chat_id:
raise ValueError("telegram alerts require bot token and chat id")
channels.append(
TelegramChannel(
bot_token=settings.telegram_bot_token,
chat_id=settings.telegram_chat_id,
)
)
if settings.discord_alerts_enabled:
if not settings.discord_webhook_url:
raise ValueError("discord alerts require webhook url")
channels.append(DiscordWebhookChannel(webhook_url=settings.discord_webhook_url))
if settings.email_alerts_enabled:
if not settings.email_smtp_host:
raise ValueError("email alerts require SMTP host")
if not settings.email_alert_from:
raise ValueError("email alerts require sender address")
if not settings.email_alert_to:
raise ValueError("email alerts require recipient list")
recipients = [
address.strip() for address in settings.email_alert_to.split(",") if address.strip()
]
channels.append(
EmailSmtpChannel(
host=settings.email_smtp_host,
port=settings.email_smtp_port,
sender=settings.email_alert_from,
recipients=recipients,
username=settings.email_smtp_username,
password=settings.email_smtp_password,
use_tls=settings.email_smtp_use_tls,
)
)
return channels
def build_notifier_from_settings(settings: _AlertSettings) -> AlertNotifier:
severity = _as_alert_severity(settings.alert_min_severity)
channels = build_channels_from_settings(settings)
category_flags = {
"trade": settings.alert_on_trade_events,
"error": settings.alert_on_error_events,
"threshold": settings.alert_on_threshold_events,
"system": settings.alert_on_system_events,
}
return AlertNotifier(
channels,
enabled=settings.alerts_enabled,
min_severity=severity,
dedup_seconds=settings.alert_dedup_seconds,
category_flags=category_flags,
)
+26 -2
View File
@@ -1,11 +1,19 @@
from __future__ import annotations
from collections.abc import AsyncIterator
from contextlib import asynccontextmanager
from fastapi import FastAPI
from arbitrade.api.routes import router
from arbitrade.alerting.notifier import build_notifier_from_settings
from arbitrade.api.control_state import DashboardControlState
from arbitrade.api.routes import public_router, router
from arbitrade.config.settings import Settings
from arbitrade.logging_setup import configure_logging
from arbitrade.metrics import MetricsCalculator
from arbitrade.runtime.lifecycle import graceful_shutdown, restore_runtime_state
from arbitrade.storage.db import DuckDBStore
from arbitrade.storage.repositories import AuditRepository, RuntimeStateRepository
def create_app(settings: Settings) -> FastAPI:
@@ -14,6 +22,22 @@ def create_app(settings: Settings) -> FastAPI:
db = DuckDBStore(settings)
db.migrate()
app = FastAPI(title="arbitrade", version="0.1.0")
@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncIterator[None]:
await restore_runtime_state(app)
yield
await graceful_shutdown(app)
app = FastAPI(title="arbitrade", version="0.1.0", lifespan=lifespan)
app.state.settings = settings
app.state.store = db
app.state.metrics = MetricsCalculator(db)
app.state.audit_repository = AuditRepository(db)
app.state.runtime_state_repository = RuntimeStateRepository(db)
app.state.alert_notifier = build_notifier_from_settings(settings)
app.state.dashboard_controls = DashboardControlState(
is_running=not settings.kill_switch_active,
)
app.include_router(public_router)
app.include_router(router)
return app
+38
View File
@@ -0,0 +1,38 @@
from __future__ import annotations
from secrets import compare_digest
from typing import Annotated
from fastapi import Depends, HTTPException, Request, status
from fastapi.security import HTTPBasic, HTTPBasicCredentials
dashboard_basic_auth = HTTPBasic(auto_error=False)
def require_dashboard_auth(
request: Request,
credentials: Annotated[HTTPBasicCredentials | None, Depends(dashboard_basic_auth)],
) -> None:
settings = request.app.state.settings
username = settings.dashboard_auth_username
password = settings.dashboard_auth_password
if username is None and password is None:
return
if username is None or password is None:
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Dashboard auth misconfigured",
)
if (
credentials is None
or not compare_digest(credentials.username, username)
or not compare_digest(credentials.password, password)
):
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Not authenticated",
headers={"WWW-Authenticate": 'Basic realm="Arbitrade Dashboard"'},
)
+16
View File
@@ -0,0 +1,16 @@
from __future__ import annotations
from dataclasses import dataclass, field
from datetime import UTC, datetime
from arbitrade.risk.kill_switch import KillSwitch
@dataclass(slots=True)
class DashboardControlState:
is_running: bool = True
kill_switch: KillSwitch = field(default_factory=KillSwitch)
updated_at: datetime = field(default_factory=lambda: datetime.now(UTC))
def mark_updated(self) -> None:
self.updated_at = datetime.now(UTC)
+582 -11
View File
@@ -1,28 +1,599 @@
from __future__ import annotations
import json
from collections.abc import AsyncIterator
from datetime import UTC, datetime
from pathlib import Path
from typing import cast
from urllib.parse import parse_qs
from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse, JSONResponse
import duckdb
from fastapi import APIRouter, Depends, Request
from fastapi.responses import HTMLResponse, JSONResponse, StreamingResponse
from fastapi.templating import Jinja2Templates
router = APIRouter()
templates = Jinja2Templates(directory="web/templates")
from arbitrade.alerting.notifier import SupportsAlerts, SupportsAlertStatus
from arbitrade.api.auth import require_dashboard_auth
from arbitrade.api.control_state import DashboardControlState
from arbitrade.storage.repositories import AuditRecord, AuditRepository
router = APIRouter(dependencies=[Depends(require_dashboard_auth)])
public_router = APIRouter()
templates = Jinja2Templates(
directory=str(Path(__file__).resolve().parents[3] / "web" / "templates")
)
@router.get("/", response_class=HTMLResponse)
async def home(request: Request) -> HTMLResponse:
def _format_metric(value: float | None, *, precision: int = 2, suffix: str = "") -> str:
if value is None:
return ""
return f"{value:.{precision}f}{suffix}"
def _dashboard_metrics(request: Request) -> dict[str, str]:
metrics = request.app.state.metrics.compute()
return {
"realized_pnl": _format_metric(metrics.realized_pnl_usd, precision=2, suffix=" USD"),
"win_rate": _format_metric(
metrics.win_rate * 100.0 if metrics.win_rate is not None else None,
precision=1,
suffix="%",
),
"avg_trade_duration": _format_metric(
metrics.avg_trade_duration_seconds, precision=1, suffix=" s"
),
"opportunities_per_minute": _format_metric(
metrics.opportunities_per_minute, precision=1, suffix=" /min"
),
"fill_rate": _format_metric(
metrics.fill_rate * 100.0 if metrics.fill_rate is not None else None,
precision=1,
suffix="%",
),
"latency_p50": _format_metric(metrics.latency_p50_seconds, precision=3, suffix=" s"),
"latency_p95": _format_metric(metrics.latency_p95_seconds, precision=3, suffix=" s"),
"latency_p99": _format_metric(metrics.latency_p99_seconds, precision=3, suffix=" s"),
"generated_at": datetime.now(UTC).isoformat(),
}
def _table_columns(conn: duckdb.DuckDBPyConnection, table_name: str) -> set[str]:
rows = conn.execute(f"PRAGMA table_info('{table_name}')").fetchall()
return {str(row[1]) for row in rows}
def _dashboard_overview(request: Request) -> dict[str, object]:
store = request.app.state.store
with store.connect() as conn:
trade_columns = _table_columns(conn, "trades")
trade_ref_expr = "trade_ref" if "trade_ref" in trade_columns else "CAST(id AS VARCHAR)"
cycle_expr = "cycle" if "cycle" in trade_columns else "NULL"
if "finished_at" in trade_columns:
open_trade_filter = "finished_at IS NULL"
else:
open_trade_filter = "LOWER(status) NOT IN ('filled', 'closed', 'cancelled', 'canceled')"
portfolio_row = conn.execute("""
SELECT balances, total_value_usd
FROM portfolio_snapshots
ORDER BY snapshot_at DESC
LIMIT 1
""").fetchone()
open_trades = conn.execute(f"""
SELECT {trade_ref_expr}, status, started_at, {cycle_expr}
FROM trades
WHERE {open_trade_filter}
ORDER BY started_at DESC
LIMIT 5
""").fetchall()
pnl_total_row = conn.execute("""
SELECT COALESCE(SUM(COALESCE(realized_pnl, 0)), 0)
FROM trades
""").fetchone()
latest_opportunities = conn.execute("""
SELECT cycle, net_pct, est_profit, detected_at
FROM opportunities
ORDER BY detected_at DESC
LIMIT 5
""").fetchall()
balances_value = ""
total_value = ""
if portfolio_row is not None:
balances_raw, total_value_raw = portfolio_row
balances_value = str(balances_raw) if balances_raw is not None else ""
if total_value_raw is not None:
total_value = f"{float(total_value_raw):.2f} USD"
open_trade_rows = [
{
"trade_ref": str(row[0]),
"status": str(row[1]),
"started_at": row[2].isoformat() if isinstance(row[2], datetime) else "",
"cycle": str(row[3]) if row[3] is not None else "",
}
for row in open_trades
]
opportunity_rows = [
{
"cycle": str(row[0]),
"net_pct": f"{float(row[1]):.2f}%" if row[1] is not None else "",
"est_profit": f"{float(row[2]):.2f} USD" if row[2] is not None else "",
"detected_at": row[3].isoformat() if isinstance(row[3], datetime) else "",
}
for row in latest_opportunities
]
return {
"status": "live",
"generated_at": datetime.now(UTC).isoformat(),
"balances": balances_value,
"total_value": total_value,
"open_trade_count": len(open_trade_rows),
"open_trades": open_trade_rows,
"realized_pnl_total": f"{float(pnl_total_row[0]):.2f} USD" if pnl_total_row else "",
"opportunities": opportunity_rows,
}
def _dashboard_charts(request: Request) -> dict[str, object]:
store = request.app.state.store
with store.connect() as conn:
opportunity_rows = conn.execute("""
SELECT detected_at, cycle, net_pct, est_profit
FROM opportunities
ORDER BY detected_at DESC
LIMIT 10
""").fetchall()
chart_rows = list(reversed(opportunity_rows))
labels = [
row[0].isoformat() if isinstance(row[0], datetime) else f"opportunity-{index + 1}"
for index, row in enumerate(chart_rows)
]
net_pct_values = [float(row[2]) if row[2] is not None else 0.0 for row in chart_rows]
est_profit_values = [float(row[3]) if row[3] is not None else 0.0 for row in chart_rows]
cycles = [str(row[1]) for row in chart_rows]
return {
"labels": labels,
"net_pct_values": net_pct_values,
"est_profit_values": est_profit_values,
"cycles": cycles,
"has_chart_data": bool(chart_rows),
"generated_at": datetime.now(UTC).isoformat(),
}
def _dashboard_controls_state(request: Request) -> DashboardControlState:
return cast(DashboardControlState, request.app.state.dashboard_controls)
def _audit_repository(request: Request) -> AuditRepository | None:
repository = getattr(request.app.state, "audit_repository", None)
return cast(AuditRepository | None, repository)
def _record_audit(
request: Request,
*,
actor: str,
event_type: str,
decision: str,
payload: dict[str, object] | None = None,
) -> None:
repository = _audit_repository(request)
if repository is None:
return
correlation_id = request.headers.get("x-request-id")
repository.insert(
AuditRecord(
occurred_at=datetime.now(UTC),
actor=actor,
event_type=event_type,
decision=decision,
payload=None if payload is None else {str(key): payload[key] for key in payload},
correlation_id=correlation_id,
)
)
def _dashboard_audit(request: Request, *, limit: int = 15) -> dict[str, object]:
repository = _audit_repository(request)
if repository is None:
return {
"entries": [],
"generated_at": datetime.now(UTC).isoformat(),
}
records = repository.list_recent(limit=limit)
entries: list[dict[str, str]] = []
for record in records:
payload_text = ""
if record.payload:
payload_text = json.dumps(record.payload)
entries.append(
{
"occurred_at": record.occurred_at.isoformat(),
"actor": record.actor,
"event_type": record.event_type,
"decision": record.decision,
"payload": payload_text,
"correlation_id": record.correlation_id or "",
}
)
return {
"entries": entries,
"generated_at": datetime.now(UTC).isoformat(),
}
def _alert_notifier(request: Request) -> SupportsAlerts | None:
notifier = getattr(request.app.state, "alert_notifier", None)
return cast(SupportsAlerts | None, notifier)
def _alert_status_snapshot(request: Request) -> dict[str, object]:
notifier = getattr(request.app.state, "alert_notifier", None)
if isinstance(notifier, SupportsAlertStatus):
return notifier.status_snapshot()
return {
"enabled": False,
"has_channels": False,
"configured_channels": [],
"min_severity": "",
"dedup_seconds": 0.0,
"last_result": "unavailable",
"last_attempted_at": None,
"last_success_at": None,
"last_error": None,
"last_event": None,
"last_channel_results": [],
}
def _dashboard_controls(request: Request) -> dict[str, object]:
controls = _dashboard_controls_state(request)
settings = request.app.state.settings
alert_status = _alert_status_snapshot(request)
last_event = alert_status.get("last_event")
last_event_title = ""
if isinstance(last_event, dict):
title_value = last_event.get("title")
if isinstance(title_value, str):
last_event_title = title_value
configured_channels = alert_status.get("configured_channels")
channels_display = ""
if isinstance(configured_channels, list) and configured_channels:
channels_display = ", ".join(str(channel) for channel in configured_channels)
dedup_seconds_raw = alert_status.get("dedup_seconds", 0.0)
dedup_seconds = float(dedup_seconds_raw) if isinstance(dedup_seconds_raw, int | float) else 0.0
return {
"execution_status": "running" if controls.is_running else "stopped",
"kill_switch_status": "active" if controls.kill_switch.is_active else "inactive",
"kill_switch_reason": controls.kill_switch.reason or "",
"paper_trading_mode": "enabled" if settings.paper_trading_mode else "disabled",
"trade_capital_usd": f"{float(settings.trade_capital_usd):.2f} USD",
"trade_capital_usd_value": f"{float(settings.trade_capital_usd):.2f}",
"max_trade_capital_usd": (
""
if settings.max_trade_capital_usd is None
else f"{float(settings.max_trade_capital_usd):.2f} USD"
),
"max_trade_capital_usd_value": (
""
if settings.max_trade_capital_usd is None
else f"{float(settings.max_trade_capital_usd):.2f}"
),
"max_concurrent_trades": (
"" if settings.max_concurrent_trades is None else str(settings.max_concurrent_trades)
),
"max_concurrent_trades_value": (
"" if settings.max_concurrent_trades is None else str(settings.max_concurrent_trades)
),
"alerts_enabled": "enabled" if bool(alert_status.get("enabled", False)) else "disabled",
"alerts_channels": channels_display,
"alerts_min_severity": str(alert_status.get("min_severity", "")),
"alerts_dedup_seconds": f"{dedup_seconds:.0f}",
"alerts_last_result": str(alert_status.get("last_result", "unavailable")),
"alerts_last_attempted_at": str(alert_status.get("last_attempted_at") or ""),
"alerts_last_success_at": str(alert_status.get("last_success_at") or ""),
"alerts_last_event_title": last_event_title,
"alerts_last_error": str(alert_status.get("last_error") or ""),
"alerts_last_channel_results": [
str(item) for item in cast(list[object], alert_status.get("last_channel_results", []))
],
"updated_at": controls.updated_at.isoformat(),
"start_endpoint": "/dashboard/control/start",
"stop_endpoint": "/dashboard/control/stop",
"kill_switch_endpoint": "/dashboard/control/kill-switch",
"config_endpoint": "/dashboard/control/config",
"chart_endpoint": "/dashboard/fragment/charts",
}
def _parse_form_body(body: bytes) -> dict[str, str]:
parsed = parse_qs(body.decode("utf-8"), keep_blank_values=True)
return {key: values[-1] for key, values in parsed.items() if values}
def _form_bool(value: str | None) -> bool:
if value is None:
return False
return value.lower() in {"1", "true", "yes", "on"}
async def _dashboard_response(
request: Request, template_name: str = "dashboard.html"
) -> HTMLResponse:
return templates.TemplateResponse(
request=request,
name="health.html",
name=template_name,
context={
"status": "ok",
"time": datetime.now(UTC).isoformat(),
"title": "Arbitrade Health",
"title": "Arbitrade Dashboard",
"request": request,
"metrics_endpoint": "/dashboard/fragment/metrics",
"overview_endpoint": "/dashboard/fragment/overview",
"controls_endpoint": "/dashboard/fragment/controls",
"charts_endpoint": "/dashboard/fragment/charts",
"audit_endpoint": "/dashboard/fragment/audit",
"stream_endpoint": "/dashboard/stream/metrics",
"overview_stream_endpoint": "/dashboard/stream/overview",
},
)
@router.get("/health", response_class=JSONResponse)
@router.get("/", response_class=HTMLResponse)
async def home(request: Request) -> HTMLResponse:
return await _dashboard_response(request)
@router.get("/dashboard", response_class=HTMLResponse)
async def dashboard(request: Request) -> HTMLResponse:
return await _dashboard_response(request)
@router.get("/dashboard/fragment/metrics", response_class=HTMLResponse)
async def dashboard_metrics(request: Request) -> HTMLResponse:
return templates.TemplateResponse(
request=request,
name="partials/metrics.html",
context={"request": request, **_dashboard_metrics(request)},
)
@router.get("/dashboard/fragment/overview", response_class=HTMLResponse)
async def dashboard_overview(request: Request) -> HTMLResponse:
return templates.TemplateResponse(
request=request,
name="partials/overview.html",
context={"request": request, **_dashboard_overview(request)},
)
@router.get("/dashboard/fragment/controls", response_class=HTMLResponse)
async def dashboard_controls(request: Request) -> HTMLResponse:
return templates.TemplateResponse(
request=request,
name="partials/controls.html",
context={"request": request, **_dashboard_controls(request)},
)
@router.get("/dashboard/fragment/charts", response_class=HTMLResponse)
async def dashboard_charts(request: Request) -> HTMLResponse:
return templates.TemplateResponse(
request=request,
name="partials/charts.html",
context={"request": request, **_dashboard_charts(request)},
)
@router.get("/dashboard/fragment/audit", response_class=HTMLResponse)
async def dashboard_audit(request: Request) -> HTMLResponse:
return templates.TemplateResponse(
request=request,
name="partials/audit.html",
context={"request": request, **_dashboard_audit(request)},
)
@router.get("/dashboard/api/alerts/status", response_class=JSONResponse)
async def dashboard_alert_status(request: Request) -> JSONResponse:
return JSONResponse(_alert_status_snapshot(request))
@router.get("/dashboard/api/audit/recent", response_class=JSONResponse)
async def dashboard_audit_recent(request: Request) -> JSONResponse:
return JSONResponse(_dashboard_audit(request, limit=25))
@router.post("/dashboard/control/start", response_class=HTMLResponse)
async def dashboard_control_start(request: Request) -> HTMLResponse:
controls = _dashboard_controls_state(request)
controls.is_running = True
controls.mark_updated()
notifier = _alert_notifier(request)
if notifier is not None:
await notifier.notify(
category="system",
severity="info",
title="Execution started",
message="Dashboard control started execution.",
)
_record_audit(
request,
actor="dashboard_user",
event_type="dashboard.control.start",
decision="approved",
payload={"execution_status": "running"},
)
return templates.TemplateResponse(
request=request,
name="partials/controls.html",
context={"request": request, **_dashboard_controls(request)},
)
@router.post("/dashboard/control/stop", response_class=HTMLResponse)
async def dashboard_control_stop(request: Request) -> HTMLResponse:
controls = _dashboard_controls_state(request)
controls.is_running = False
controls.mark_updated()
notifier = _alert_notifier(request)
if notifier is not None:
await notifier.notify(
category="system",
severity="warning",
title="Execution stopped",
message="Dashboard control stopped execution.",
)
_record_audit(
request,
actor="dashboard_user",
event_type="dashboard.control.stop",
decision="approved",
payload={"execution_status": "stopped"},
)
return templates.TemplateResponse(
request=request,
name="partials/controls.html",
context={"request": request, **_dashboard_controls(request)},
)
@router.post("/dashboard/control/kill-switch", response_class=HTMLResponse)
async def dashboard_control_kill_switch(request: Request) -> HTMLResponse:
controls = _dashboard_controls_state(request)
form = _parse_form_body(await request.body())
reason = form.get("reason") or "manual"
controls.kill_switch.activate(reason=reason)
controls.is_running = False
controls.mark_updated()
notifier = _alert_notifier(request)
if notifier is not None:
await notifier.notify(
category="threshold",
severity="critical",
title="Kill switch activated",
message="Kill switch triggered from dashboard control.",
details={"reason": reason},
)
_record_audit(
request,
actor="dashboard_user",
event_type="dashboard.control.kill_switch",
decision="approved",
payload={"reason": reason},
)
return templates.TemplateResponse(
request=request,
name="partials/controls.html",
context={"request": request, **_dashboard_controls(request)},
)
@router.post("/dashboard/control/config", response_class=HTMLResponse)
async def dashboard_control_config(request: Request) -> HTMLResponse:
controls = _dashboard_controls_state(request)
settings = request.app.state.settings
form = _parse_form_body(await request.body())
if "trade_capital_usd" in form and form["trade_capital_usd"]:
settings.trade_capital_usd = float(form["trade_capital_usd"])
if "max_trade_capital_usd" in form:
max_trade_capital_value = form["max_trade_capital_usd"].strip()
settings.max_trade_capital_usd = (
float(max_trade_capital_value) if max_trade_capital_value else None
)
if "max_concurrent_trades" in form:
max_concurrent_value = form["max_concurrent_trades"].strip()
settings.max_concurrent_trades = int(max_concurrent_value) if max_concurrent_value else None
settings.paper_trading_mode = _form_bool(form.get("paper_trading_mode"))
controls.mark_updated()
notifier = _alert_notifier(request)
if notifier is not None:
await notifier.notify(
category="system",
severity="info",
title="Runtime config updated",
message="Dashboard control updated runtime risk and execution settings.",
details={
"trade_capital_usd": f"{settings.trade_capital_usd}",
"max_trade_capital_usd": (
"none"
if settings.max_trade_capital_usd is None
else f"{settings.max_trade_capital_usd}"
),
"max_concurrent_trades": (
"none"
if settings.max_concurrent_trades is None
else f"{settings.max_concurrent_trades}"
),
"paper_trading_mode": "true" if settings.paper_trading_mode else "false",
},
)
_record_audit(
request,
actor="dashboard_user",
event_type="dashboard.control.config",
decision="approved",
payload={
"trade_capital_usd": settings.trade_capital_usd,
"max_trade_capital_usd": settings.max_trade_capital_usd,
"max_concurrent_trades": settings.max_concurrent_trades,
"paper_trading_mode": settings.paper_trading_mode,
},
)
return templates.TemplateResponse(
request=request,
name="partials/controls.html",
context={"request": request, **_dashboard_controls(request)},
)
@router.get("/dashboard/stream/metrics")
async def dashboard_metrics_stream(request: Request) -> StreamingResponse:
fragment = (
templates.get_template("partials/metrics.html")
.render(
request=request,
**_dashboard_metrics(request),
)
.strip()
.replace("\n", "")
)
async def _event_stream() -> AsyncIterator[bytes]:
payload = json.dumps(fragment)
yield f"event: metrics\ndata: {payload}\n\n".encode()
return StreamingResponse(_event_stream(), media_type="text/event-stream")
@router.get("/dashboard/stream/overview")
async def dashboard_overview_stream(request: Request) -> StreamingResponse:
fragment = (
templates.get_template("partials/overview.html")
.render(request=request, **_dashboard_overview(request))
.strip()
.replace("\n", "")
)
async def _event_stream() -> AsyncIterator[bytes]:
payload = json.dumps(fragment)
yield f"event: overview\ndata: {payload}\n\n".encode()
return StreamingResponse(_event_stream(), media_type="text/event-stream")
@public_router.get("/health", response_class=JSONResponse)
async def health() -> JSONResponse:
return JSONResponse({"status": "ok", "service": "arbitrade"})
+122 -2
View File
@@ -3,12 +3,17 @@ from __future__ import annotations
from functools import lru_cache
from pathlib import Path
from pydantic import Field
from pydantic import Field, field_validator, model_validator
from pydantic_settings import BaseSettings, SettingsConfigDict
class Settings(BaseSettings):
model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8", extra="ignore")
model_config = SettingsConfigDict(
env_file=".env",
env_file_encoding="utf-8",
extra="ignore",
env_ignore_empty=True,
)
app_env: str = Field(default="dev", alias="APP_ENV")
app_host: str = Field(default="0.0.0.0", alias="APP_HOST")
@@ -17,10 +22,125 @@ class Settings(BaseSettings):
log_level: str = Field(default="INFO", alias="LOG_LEVEL")
log_json: bool = Field(default=True, alias="LOG_JSON")
dashboard_auth_username: str | None = Field(
default=None,
alias="DASHBOARD_AUTH_USERNAME",
)
dashboard_auth_password: str | None = Field(
default=None,
alias="DASHBOARD_AUTH_PASSWORD",
)
alerts_enabled: bool = Field(default=True, alias="ALERTS_ENABLED")
alert_min_severity: str = Field(default="warning", alias="ALERT_MIN_SEVERITY")
alert_dedup_seconds: float = Field(default=30.0, alias="ALERT_DEDUP_SECONDS")
alert_on_trade_events: bool = Field(default=True, alias="ALERT_ON_TRADE_EVENTS")
alert_on_error_events: bool = Field(default=True, alias="ALERT_ON_ERROR_EVENTS")
alert_on_threshold_events: bool = Field(default=True, alias="ALERT_ON_THRESHOLD_EVENTS")
alert_on_system_events: bool = Field(default=True, alias="ALERT_ON_SYSTEM_EVENTS")
telegram_alerts_enabled: bool = Field(default=False, alias="TELEGRAM_ALERTS_ENABLED")
telegram_bot_token: str | None = Field(default=None, alias="TELEGRAM_BOT_TOKEN")
telegram_chat_id: str | None = Field(default=None, alias="TELEGRAM_CHAT_ID")
discord_alerts_enabled: bool = Field(default=False, alias="DISCORD_ALERTS_ENABLED")
discord_webhook_url: str | None = Field(default=None, alias="DISCORD_WEBHOOK_URL")
email_alerts_enabled: bool = Field(default=False, alias="EMAIL_ALERTS_ENABLED")
email_smtp_host: str | None = Field(default=None, alias="EMAIL_SMTP_HOST")
email_smtp_port: int = Field(default=587, alias="EMAIL_SMTP_PORT")
email_smtp_username: str | None = Field(default=None, alias="EMAIL_SMTP_USERNAME")
email_smtp_password: str | None = Field(default=None, alias="EMAIL_SMTP_PASSWORD")
email_alert_from: str | None = Field(default=None, alias="EMAIL_ALERT_FROM")
email_alert_to: str | None = Field(default=None, alias="EMAIL_ALERT_TO")
email_smtp_use_tls: bool = Field(default=True, alias="EMAIL_SMTP_USE_TLS")
duckdb_path: Path = Field(default=Path("./data/arbitrade.duckdb"), alias="DUCKDB_PATH")
kraken_rest_url: str = Field(default="https://api.kraken.com", alias="KRAKEN_REST_URL")
kraken_ws_url: str = Field(default="wss://ws.kraken.com/v2", alias="KRAKEN_WS_URL")
kraken_private_rate_limit_seconds: float = Field(
default=1.0, alias="KRAKEN_PRIVATE_RATE_LIMIT_SECONDS"
)
kraken_http_timeout_seconds: float = Field(default=10.0, alias="KRAKEN_HTTP_TIMEOUT_SECONDS")
kraken_retry_attempts: int = Field(default=3, alias="KRAKEN_RETRY_ATTEMPTS")
kraken_retry_base_delay_seconds: float = Field(
default=0.25, alias="KRAKEN_RETRY_BASE_DELAY_SECONDS"
)
kraken_api_key: str | None = Field(default=None, alias="KRAKEN_API_KEY")
kraken_api_secret: str | None = Field(default=None, alias="KRAKEN_API_SECRET")
kraken_api_key_permissions: str = Field(
default="query,trade",
alias="KRAKEN_API_KEY_PERMISSIONS",
)
ws_heartbeat_timeout_seconds: float = Field(default=20.0, alias="WS_HEARTBEAT_TIMEOUT_SECONDS")
ws_max_staleness_seconds: float = Field(default=5.0, alias="WS_MAX_STALENESS_SECONDS")
paper_trading_mode: bool = Field(default=True, alias="PAPER_TRADING_MODE")
trade_capital_usd: float = Field(default=100.0, alias="TRADE_CAPITAL_USD")
max_trade_capital_usd: float = Field(default=100.0, alias="MAX_TRADE_CAPITAL_USD")
max_concurrent_trades: int | None = Field(default=None, alias="MAX_CONCURRENT_TRADES")
max_exposure_per_asset_usd: float | None = Field(
default=None,
alias="MAX_EXPOSURE_PER_ASSET_USD",
)
quote_balance_asset: str = Field(default="USD", alias="QUOTE_BALANCE_ASSET")
min_order_size_usd: float | None = Field(default=None, alias="MIN_ORDER_SIZE_USD")
kill_switch_active: bool = Field(default=False, alias="KILL_SWITCH_ACTIVE")
daily_loss_limit_usd: float | None = Field(default=None, alias="DAILY_LOSS_LIMIT_USD")
cumulative_loss_limit_usd: float | None = Field(default=None, alias="CUMULATIVE_LOSS_LIMIT_USD")
max_source_latency_ms: float | None = Field(default=None, alias="MAX_SOURCE_LATENCY_MS")
max_apply_latency_ms: float | None = Field(default=None, alias="MAX_APPLY_LATENCY_MS")
max_consecutive_failures: int | None = Field(default=None, alias="MAX_CONSECUTIVE_FAILURES")
fernet_key: str | None = Field(default=None, alias="FERNET_KEY")
@field_validator("app_env")
@classmethod
def _validate_app_env(cls, value: str) -> str:
normalized = value.strip().lower()
if normalized not in {"dev", "test", "prod"}:
raise ValueError("APP_ENV must be one of: dev, test, prod")
return normalized
@field_validator("log_level")
@classmethod
def _validate_log_level(cls, value: str) -> str:
normalized = value.strip().upper()
if normalized not in {"DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"}:
raise ValueError("LOG_LEVEL must be one of: DEBUG, INFO, WARNING, ERROR, CRITICAL")
return normalized
@field_validator("alert_min_severity")
@classmethod
def _validate_alert_severity(cls, value: str) -> str:
normalized = value.strip().lower()
if normalized not in {"info", "warning", "error", "critical"}:
raise ValueError("ALERT_MIN_SEVERITY must be one of: info, warning, error, critical")
return normalized
@model_validator(mode="after")
def _validate_security_constraints(self) -> Settings:
if bool(self.dashboard_auth_username) ^ bool(self.dashboard_auth_password):
raise ValueError("dashboard auth requires both username and password")
if bool(self.kraken_api_key) ^ bool(self.kraken_api_secret):
raise ValueError("Kraken API auth requires both API key and secret")
permissions = {
token.strip().lower()
for token in self.kraken_api_key_permissions.split(",")
if token.strip()
}
if permissions and ("query" not in permissions or "trade" not in permissions):
raise ValueError("KRAKEN_API_KEY_PERMISSIONS must include query and trade")
if "withdraw" in permissions or "withdrawals" in permissions:
raise ValueError("KRAKEN_API_KEY_PERMISSIONS must not include withdrawal scope")
if self.alert_dedup_seconds < 0.0:
raise ValueError("ALERT_DEDUP_SECONDS must be >= 0")
return self
@lru_cache(maxsize=1)
def get_settings() -> Settings:
+12
View File
@@ -0,0 +1,12 @@
"""Arbitrage detection package."""
from arbitrade.detection.engine import CycleScore, IncrementalCycleDetector, OpportunityEvent
from arbitrade.detection.graph import CurrencyGraph, TriangularCycle
__all__ = [
"CurrencyGraph",
"TriangularCycle",
"CycleScore",
"OpportunityEvent",
"IncrementalCycleDetector",
]
+113
View File
@@ -0,0 +1,113 @@
from __future__ import annotations
import argparse
import statistics
import time
from dataclasses import asdict, dataclass
import orjson
from arbitrade.detection.engine import IncrementalCycleDetector
from arbitrade.detection.graph import CurrencyGraph
from arbitrade.exchange.models import BookLevel
from arbitrade.market_data.order_book import OrderBook
@dataclass(frozen=True, slots=True)
class DetectionBenchmarkResult:
iterations: int
total_ms: float
avg_ms: float
p50_ms: float
p95_ms: float
max_ms: float
target_ms: float
@property
def meets_target(self) -> bool:
return self.p95_ms <= self.target_ms
def _make_book(*, bid: float, ask: float) -> OrderBook:
book = OrderBook()
book.apply_bids([BookLevel(price=bid, volume=10.0)])
book.apply_asks([BookLevel(price=ask, volume=10.0)])
return book
def _build_detector_and_books() -> tuple[IncrementalCycleDetector, dict[str, OrderBook]]:
asset_pairs = {
"XXBTZUSD": {"wsname": "BTC/USD"},
"XETHXXBT": {"wsname": "ETH/BTC"},
"XETHZUSD": {"wsname": "ETH/USD"},
}
graph = CurrencyGraph.from_kraken_asset_pairs(asset_pairs)
cycles = graph.triangular_cycles()
index = graph.index_cycles_by_pair(cycles)
detector = IncrementalCycleDetector(
index,
fee_rate=0.001,
min_profit_threshold=0.001,
max_depth_levels=5,
max_book_age_seconds=10.0,
)
books = {
"BTC/USD": _make_book(bid=99.9, ask=100.0),
"ETH/BTC": _make_book(bid=0.049, ask=0.05),
"ETH/USD": _make_book(bid=5.2, ask=5.21),
}
return detector, books
def run_incremental_detection_benchmark(
*,
iterations: int = 50_000,
target_ms: float = 1.0,
) -> DetectionBenchmarkResult:
if iterations <= 0:
raise ValueError("iterations must be > 0")
detector, books = _build_detector_and_books()
samples_ms: list[float] = []
started_ns = time.perf_counter_ns()
for _ in range(iterations):
t0_ns = time.perf_counter_ns()
detector.score_updated_pair("ETH/BTC", books)
elapsed_ms = (time.perf_counter_ns() - t0_ns) / 1_000_000
samples_ms.append(elapsed_ms)
total_ms = (time.perf_counter_ns() - started_ns) / 1_000_000
return DetectionBenchmarkResult(
iterations=iterations,
total_ms=total_ms,
avg_ms=statistics.fmean(samples_ms),
p50_ms=statistics.quantiles(samples_ms, n=100)[49],
p95_ms=statistics.quantiles(samples_ms, n=100)[94],
max_ms=max(samples_ms),
target_ms=target_ms,
)
def main() -> None:
parser = argparse.ArgumentParser(description="Benchmark incremental detection latency")
parser.add_argument("--iterations", type=int, default=50_000)
parser.add_argument("--target-ms", type=float, default=1.0)
args = parser.parse_args()
result = run_incremental_detection_benchmark(
iterations=args.iterations,
target_ms=args.target_ms,
)
payload = {
**asdict(result),
"meets_target": result.meets_target,
}
print(orjson.dumps(payload).decode("utf-8"))
if __name__ == "__main__":
main()
+295
View File
@@ -0,0 +1,295 @@
from __future__ import annotations
from collections.abc import Mapping
from dataclasses import dataclass
from datetime import UTC, datetime
from arbitrade.detection.graph import TriangularCycle
from arbitrade.exchange.models import BookLevel
from arbitrade.market_data.order_book import OrderBook
def _normalize_pair_symbol(symbol: str) -> str:
if "/" not in symbol:
return symbol.upper()
base, quote = symbol.split("/", 1)
return f"{base.upper()}/{quote.upper()}"
@dataclass(frozen=True, slots=True)
class CycleScore:
cycle: TriangularCycle
gross_rate: float
net_rate: float
min_profit_threshold: float
updated_pair: str
scored_at: datetime
@property
def is_profitable(self) -> bool:
return (self.net_rate - 1.0) >= self.min_profit_threshold
@dataclass(frozen=True, slots=True)
class OpportunityEvent:
detected_at: datetime
cycle: str
updated_pair: str
gross_rate: float
net_rate: float
gross_pct: float
net_pct: float
est_profit: float
allocated_capital: float = 1.0
@classmethod
def from_cycle_score(cls, score: CycleScore, base_capital: float = 1.0) -> OpportunityEvent:
gross_pct = (score.gross_rate - 1.0) * 100.0
net_pct = (score.net_rate - 1.0) * 100.0
est_profit = (score.net_rate - 1.0) * base_capital
a, b, c = score.cycle.currencies
cycle = f"{a}->{b}->{c}->{a}"
return cls(
detected_at=score.scored_at,
cycle=cycle,
updated_pair=score.updated_pair,
gross_rate=score.gross_rate,
net_rate=score.net_rate,
gross_pct=gross_pct,
net_pct=net_pct,
est_profit=est_profit,
allocated_capital=base_capital,
)
class IncrementalCycleDetector:
def __init__(
self,
cycles_by_pair: Mapping[str, list[TriangularCycle]],
*,
fee_rate: float = 0.0,
max_depth_levels: int = 10,
min_profit_threshold: float = 0.0,
min_order_size_by_pair: Mapping[str, float] | None = None,
max_book_age_seconds: float | None = None,
) -> None:
self._cycles_by_pair = {
_normalize_pair_symbol(pair): list(cycles) for pair, cycles in cycles_by_pair.items()
}
self._fee_multiplier = 1.0 - fee_rate
self._max_depth_levels = max_depth_levels
self._min_profit_threshold = min_profit_threshold
self._max_book_age_seconds = max_book_age_seconds
self._min_order_size_by_pair = {
_normalize_pair_symbol(pair): float(min_size)
for pair, min_size in (min_order_size_by_pair or {}).items()
}
if self._fee_multiplier < 0.0:
raise ValueError("fee_rate must be <= 1.0")
if self._max_depth_levels <= 0:
raise ValueError("max_depth_levels must be > 0")
if self._min_profit_threshold < 0.0:
raise ValueError("min_profit_threshold must be >= 0.0")
if self._max_book_age_seconds is not None and self._max_book_age_seconds <= 0.0:
raise ValueError("max_book_age_seconds must be > 0.0")
for min_size in self._min_order_size_by_pair.values():
if min_size <= 0.0:
raise ValueError("minimum order size must be > 0.0")
def score_updated_pair(
self,
updated_pair: str,
books: Mapping[str, OrderBook],
) -> list[CycleScore]:
normalized_pair = _normalize_pair_symbol(updated_pair)
impacted_cycles = self._cycles_by_pair.get(normalized_pair, [])
normalized_books = {_normalize_pair_symbol(symbol): book for symbol, book in books.items()}
scores: list[CycleScore] = []
scored_at = datetime.now(UTC)
for cycle in impacted_cycles:
rates = self._score_cycle(cycle, normalized_books, scored_at)
if rates is None:
continue
gross_rate, net_rate = rates
if (net_rate - 1.0) < self._min_profit_threshold:
continue
scores.append(
CycleScore(
cycle=cycle,
gross_rate=gross_rate,
net_rate=net_rate,
min_profit_threshold=self._min_profit_threshold,
updated_pair=normalized_pair,
scored_at=scored_at,
)
)
return scores
def opportunities_for_updated_pair(
self,
updated_pair: str,
books: Mapping[str, OrderBook],
*,
base_capital: float = 1.0,
) -> list[OpportunityEvent]:
if base_capital <= 0.0:
raise ValueError("base_capital must be > 0.0")
scores = self.score_updated_pair(updated_pair, books)
return [OpportunityEvent.from_cycle_score(score, base_capital) for score in scores]
def _score_cycle(
self,
cycle: TriangularCycle,
books: Mapping[str, OrderBook],
scored_at: datetime,
) -> tuple[float, float] | None:
if not self._is_cycle_fresh(cycle, books, scored_at):
return None
a, b, c = cycle.currencies
gross_amount = 1.0
gross_ab = self._convert(gross_amount, a, b, cycle, books)
if gross_ab is None:
return None
net_ab = gross_ab * self._fee_multiplier
gross_bc = self._convert(gross_ab, b, c, cycle, books)
if gross_bc is None:
return None
net_bc = self._convert(net_ab, b, c, cycle, books)
if net_bc is None:
return None
net_bc *= self._fee_multiplier
gross_ca = self._convert(gross_bc, c, a, cycle, books)
if gross_ca is None:
return None
net_ca = self._convert(net_bc, c, a, cycle, books)
if net_ca is None:
return None
net_ca *= self._fee_multiplier
return gross_ca, net_ca
def _is_cycle_fresh(
self,
cycle: TriangularCycle,
books: Mapping[str, OrderBook],
scored_at: datetime,
) -> bool:
if self._max_book_age_seconds is None:
return True
for pair in cycle.pairs:
normalized_pair = _normalize_pair_symbol(pair)
book = books.get(normalized_pair)
if book is None:
return False
age_seconds = (scored_at - book.updated_at).total_seconds()
if age_seconds > self._max_book_age_seconds:
return False
return True
@staticmethod
def _pair_for_edge(cycle: TriangularCycle, from_currency: str, to_currency: str) -> str | None:
for pair in cycle.pairs:
if "/" not in pair:
continue
base, quote = pair.split("/", 1)
base = base.upper()
quote = quote.upper()
if {base, quote} == {from_currency, to_currency}:
return f"{base}/{quote}"
return None
def _convert(
self,
amount: float,
from_currency: str,
to_currency: str,
cycle: TriangularCycle,
books: Mapping[str, OrderBook],
) -> float | None:
pair = self._pair_for_edge(cycle, from_currency, to_currency)
if pair is None:
return None
book = books.get(pair)
if book is None:
return None
bids, asks = book.top_levels(depth=self._max_depth_levels)
base, quote = pair.split("/", 1)
base = base.upper()
quote = quote.upper()
if from_currency == base and to_currency == quote:
quote_out = self._sell_base_for_quote(amount, bids)
if quote_out is None:
return None
if not self._is_min_order_size_satisfied(pair, amount):
return None
return quote_out
if from_currency == quote and to_currency == base:
base_out = self._buy_base_with_quote(amount, asks)
if base_out is None:
return None
if not self._is_min_order_size_satisfied(pair, base_out):
return None
return base_out
return None
def _is_min_order_size_satisfied(self, pair: str, base_amount: float) -> bool:
min_size = self._min_order_size_by_pair.get(pair)
if min_size is None:
return True
return base_amount >= min_size
@staticmethod
def _sell_base_for_quote(amount_base: float, bids: list[BookLevel]) -> float | None:
remaining = amount_base
quote_out = 0.0
for level in bids:
if remaining <= 0.0:
break
if level.price <= 0.0 or level.volume <= 0.0:
continue
executed = min(remaining, level.volume)
quote_out += executed * level.price
remaining -= executed
if remaining > 0.0:
return None
return quote_out
@staticmethod
def _buy_base_with_quote(amount_quote: float, asks: list[BookLevel]) -> float | None:
remaining_quote = amount_quote
base_out = 0.0
for level in asks:
if remaining_quote <= 0.0:
break
if level.price <= 0.0 or level.volume <= 0.0:
continue
level_quote_capacity = level.volume * level.price
spend = min(remaining_quote, level_quote_capacity)
base_out += spend / level.price
remaining_quote -= spend
if remaining_quote > 0.0:
return None
return base_out
+90
View File
@@ -0,0 +1,90 @@
from __future__ import annotations
from dataclasses import dataclass
from typing import Any
@dataclass(frozen=True, slots=True)
class TriangularCycle:
currencies: tuple[str, str, str]
pairs: tuple[str, str, str]
def _canonical_pair(base: str, quote: str) -> str:
return f"{base}/{quote}"
class CurrencyGraph:
def __init__(self) -> None:
self._adjacency: dict[str, set[str]] = {}
self._pair_by_direction: dict[tuple[str, str], str] = {}
@property
def adjacency(self) -> dict[str, set[str]]:
return self._adjacency
@property
def pair_by_direction(self) -> dict[tuple[str, str], str]:
return self._pair_by_direction
def add_pair(self, base: str, quote: str, pair_symbol: str | None = None) -> None:
normalized_base = base.upper()
normalized_quote = quote.upper()
symbol = pair_symbol or _canonical_pair(normalized_base, normalized_quote)
self._adjacency.setdefault(normalized_base, set()).add(normalized_quote)
self._adjacency.setdefault(normalized_quote, set()).add(normalized_base)
self._pair_by_direction[(normalized_base, normalized_quote)] = symbol
self._pair_by_direction[(normalized_quote, normalized_base)] = symbol
@classmethod
def from_kraken_asset_pairs(cls, asset_pairs: dict[str, Any]) -> CurrencyGraph:
graph = cls()
for value in asset_pairs.values():
if not isinstance(value, dict):
continue
wsname = value.get("wsname")
if isinstance(wsname, str) and "/" in wsname:
base, quote = wsname.split("/", 1)
graph.add_pair(base, quote, wsname)
continue
raw_base = value.get("base")
raw_quote = value.get("quote")
if isinstance(raw_base, str) and isinstance(raw_quote, str):
graph.add_pair(raw_base, raw_quote)
return graph
def triangular_cycles(self) -> list[TriangularCycle]:
found: dict[tuple[str, str, str], TriangularCycle] = {}
for a, neighbors_a in self._adjacency.items():
for b in neighbors_a:
if a >= b:
continue
neighbors_b = self._adjacency.get(b, set())
for c in neighbors_b:
if b >= c:
continue
if a not in self._adjacency.get(c, set()):
continue
p_ab = self._pair_by_direction[(a, b)]
p_bc = self._pair_by_direction[(b, c)]
p_ca = self._pair_by_direction[(c, a)]
key = (a, b, c)
found[key] = TriangularCycle(currencies=key, pairs=(p_ab, p_bc, p_ca))
return list(found.values())
@staticmethod
def index_cycles_by_pair(cycles: list[TriangularCycle]) -> dict[str, list[TriangularCycle]]:
index: dict[str, list[TriangularCycle]] = {}
for cycle in cycles:
for pair in cycle.pairs:
index.setdefault(pair, []).append(cycle)
return index
+1
View File
@@ -0,0 +1 @@
"""Kraken exchange integration package."""
+281
View File
@@ -0,0 +1,281 @@
from __future__ import annotations
import asyncio
import time
from typing import Any
from urllib.parse import urlencode
import httpx
import structlog
from arbitrade.config.settings import Settings
from arbitrade.exchange.models import KrakenApiResult, LatencySample
from arbitrade.exchange.signing import sign_kraken_private_path
_LOG = structlog.get_logger(__name__)
def _result_dict(payload: dict[str, Any]) -> dict[str, Any]:
result = payload.get("result", {})
if isinstance(result, dict):
return result
return {}
class KrakenRestClient:
def __init__(self, settings: Settings) -> None:
self._settings = settings
self._client = httpx.AsyncClient(
base_url=settings.kraken_rest_url,
timeout=settings.kraken_http_timeout_seconds,
limits=httpx.Limits(max_keepalive_connections=10, max_connections=50),
headers={"User-Agent": "arbitrade/0.1.0"},
)
self._private_lock = asyncio.Lock()
issues = self.validate_compliance()
if issues:
_LOG.warning("kraken_compliance_issues", issues=issues)
else:
_LOG.info("kraken_compliance_ok")
def validate_compliance(self) -> list[str]:
issues: list[str] = []
if not self._settings.kraken_rest_url.startswith("https://"):
issues.append("KRAKEN_REST_URL should use https://")
if self._settings.kraken_private_rate_limit_seconds < 1.0:
issues.append("KRAKEN_PRIVATE_RATE_LIMIT_SECONDS below 1.0 may violate Kraken limits")
if self._settings.kraken_retry_attempts < 1:
issues.append("KRAKEN_RETRY_ATTEMPTS must be >= 1")
if self._settings.kraken_retry_base_delay_seconds < 0:
issues.append("KRAKEN_RETRY_BASE_DELAY_SECONDS must be >= 0")
return issues
async def close(self) -> None:
await self._client.aclose()
async def warm_connection_pool(self) -> None:
await self.server_time()
async def _request_with_retry(
self,
endpoint: str,
params: dict[str, Any] | None = None,
) -> KrakenApiResult:
attempts = self._settings.kraken_retry_attempts
delay = self._settings.kraken_retry_base_delay_seconds
params = params or {}
for attempt in range(1, attempts + 1):
t0 = time.perf_counter()
try:
response = await self._client.get(endpoint, params=params)
response.raise_for_status()
payload = response.json()
if payload.get("error"):
raise RuntimeError(f"Kraken error: {payload['error']}")
latency = (time.perf_counter() - t0) * 1000
_LOG.info(
"kraken_rest_request_ok",
endpoint=endpoint,
attempt=attempt,
latency_ms=latency,
sample=LatencySample.now("rest_request", latency_ms=latency).latency_ms,
)
return KrakenApiResult(endpoint=endpoint, payload=payload)
except Exception as exc:
latency = (time.perf_counter() - t0) * 1000
_LOG.warning(
"kraken_rest_request_failed",
endpoint=endpoint,
attempt=attempt,
latency_ms=latency,
error=str(exc),
)
if attempt >= attempts:
raise
await asyncio.sleep(delay * (2 ** (attempt - 1)))
raise RuntimeError("unreachable retry loop")
async def _private_post_with_retry(
self,
endpoint: str,
data: dict[str, str] | None = None,
) -> KrakenApiResult:
api_key = self._settings.kraken_api_key
api_secret = self._settings.kraken_api_secret
if not api_key or not api_secret:
raise RuntimeError("Missing Kraken API credentials for private endpoint")
attempts = self._settings.kraken_retry_attempts
delay = self._settings.kraken_retry_base_delay_seconds
for attempt in range(1, attempts + 1):
t0 = time.perf_counter()
try:
nonce = str(int(time.time() * 1000))
payload = {"nonce": nonce}
if data is not None:
payload.update(data)
encoded = urlencode(payload)
signature = sign_kraken_private_path(endpoint, nonce, encoded, api_secret)
response = await self._client.post(
endpoint,
data=payload,
headers={
"API-Key": api_key,
"API-Sign": signature,
},
)
response.raise_for_status()
body = response.json()
if body.get("error"):
raise RuntimeError(f"Kraken error: {body['error']}")
latency = (time.perf_counter() - t0) * 1000
_LOG.info(
"kraken_private_rest_request_ok",
endpoint=endpoint,
attempt=attempt,
latency_ms=latency,
sample=LatencySample.now("private_rest_request", latency_ms=latency).latency_ms,
)
return KrakenApiResult(endpoint=endpoint, payload=body)
except Exception as exc:
latency = (time.perf_counter() - t0) * 1000
_LOG.warning(
"kraken_private_rest_request_failed",
endpoint=endpoint,
attempt=attempt,
latency_ms=latency,
error=str(exc),
)
if attempt >= attempts:
raise
await asyncio.sleep(delay * (2 ** (attempt - 1)))
raise RuntimeError("unreachable retry loop")
async def server_time(self) -> dict[str, Any]:
result = await self._request_with_retry("/0/public/Time")
return _result_dict(result.payload)
async def assets(self) -> dict[str, Any]:
result = await self._request_with_retry("/0/public/Assets")
return _result_dict(result.payload)
async def asset_pairs(self) -> dict[str, Any]:
result = await self._request_with_retry("/0/public/AssetPairs")
return _result_dict(result.payload)
async def _throttled_private_call(
self,
endpoint: str,
data: dict[str, str] | None = None,
) -> dict[str, Any]:
async with self._private_lock:
result = await self._private_post_with_retry(endpoint, data=data)
await asyncio.sleep(self._settings.kraken_private_rate_limit_seconds)
return _result_dict(result.payload)
async def balances(self) -> dict[str, Any]:
return await self._throttled_private_call("/0/private/Balance")
async def place_market_order(
self,
*,
pair: str,
side: str,
volume: float,
user_ref: int | None = None,
) -> dict[str, Any]:
normalized_side = side.lower()
if normalized_side not in {"buy", "sell"}:
raise ValueError("side must be 'buy' or 'sell'")
if volume <= 0.0:
raise ValueError("volume must be > 0.0")
if user_ref is not None and user_ref < 0:
raise ValueError("user_ref must be >= 0")
data = {
"pair": pair,
"type": normalized_side,
"ordertype": "market",
"volume": str(volume),
}
if user_ref is not None:
data["userref"] = str(user_ref)
return await self._throttled_private_call(
"/0/private/AddOrder",
data=data,
)
async def place_limit_order(
self,
*,
pair: str,
side: str,
volume: float,
price: float,
user_ref: int | None = None,
) -> dict[str, Any]:
normalized_side = side.lower()
if normalized_side not in {"buy", "sell"}:
raise ValueError("side must be 'buy' or 'sell'")
if volume <= 0.0:
raise ValueError("volume must be > 0.0")
if price <= 0.0:
raise ValueError("price must be > 0.0")
if user_ref is not None and user_ref < 0:
raise ValueError("user_ref must be >= 0")
data = {
"pair": pair,
"type": normalized_side,
"ordertype": "limit",
"price": str(price),
"volume": str(volume),
}
if user_ref is not None:
data["userref"] = str(user_ref)
return await self._throttled_private_call(
"/0/private/AddOrder",
data=data,
)
async def query_order(
self,
*,
order_id: str,
include_trades: bool = True,
) -> dict[str, Any]:
if not order_id.strip():
raise ValueError("order_id must be non-empty")
return await self._throttled_private_call(
"/0/private/QueryOrders",
data={
"txid": order_id,
"trades": "true" if include_trades else "false",
},
)
async def cancel_order(self, *, order_id: str) -> dict[str, Any]:
if not order_id.strip():
raise ValueError("order_id must be non-empty")
return await self._throttled_private_call(
"/0/private/CancelOrder",
data={"txid": order_id},
)
+177
View File
@@ -0,0 +1,177 @@
from __future__ import annotations
import asyncio
import time
from collections.abc import AsyncIterator
from dataclasses import dataclass
from datetime import UTC, datetime
from typing import Any
import orjson
import structlog
import websockets
from arbitrade.alerting.notifier import AlertSeverity, SupportsAlerts
from arbitrade.config.settings import Settings
from arbitrade.exchange.models import BookDelta, BookLevel
_LOG = structlog.get_logger(__name__)
@dataclass(slots=True)
class WsMessage:
received_at: datetime
payload: dict[str, Any]
class KrakenWsClient:
def __init__(self, settings: Settings, *, alert_notifier: SupportsAlerts | None = None) -> None:
self._settings = settings
self._last_message_at: datetime | None = None
self._stop = asyncio.Event()
self._alert_notifier = alert_notifier
self._has_connected_once = False
self._was_disconnected = False
@property
def is_stale(self) -> bool:
if self._last_message_at is None:
return True
return (
datetime.now(UTC) - self._last_message_at
).total_seconds() > self._settings.ws_max_staleness_seconds
async def stop(self) -> None:
self._stop.set()
async def connect_stream(self) -> AsyncIterator[WsMessage]:
delay = 1.0
while not self._stop.is_set():
try:
async with websockets.connect(
self._settings.kraken_ws_url, max_size=2_000_000
) as ws:
_LOG.info("kraken_ws_connected", url=self._settings.kraken_ws_url)
if self._has_connected_once and self._was_disconnected:
await self._notify(
category="system",
severity="info",
title="WebSocket reconnected",
message="Kraken WebSocket connection restored.",
details={"url": self._settings.kraken_ws_url},
)
self._has_connected_once = True
self._was_disconnected = False
delay = 1.0
async for raw in self._recv_loop(ws):
yield raw
except Exception as exc:
_LOG.warning("kraken_ws_disconnected", error=str(exc), reconnect_in=delay)
self._was_disconnected = True
await self._notify(
category="system",
severity="warning",
title="WebSocket disconnected",
message="Kraken WebSocket disconnected, reconnect scheduled.",
details={
"error": str(exc),
"reconnect_in_seconds": f"{delay}",
},
)
await asyncio.sleep(delay)
delay = min(delay * 2, 30.0)
async def _recv_loop(self, ws: Any) -> AsyncIterator[WsMessage]:
while not self._stop.is_set():
t0 = time.perf_counter()
try:
raw = await asyncio.wait_for(
ws.recv(), timeout=self._settings.ws_heartbeat_timeout_seconds
)
except TimeoutError:
await self._notify(
category="system",
severity="critical",
title="WebSocket staleness abort",
message="No WebSocket heartbeat within configured timeout; reconnecting.",
details={
"heartbeat_timeout_seconds": (
f"{self._settings.ws_heartbeat_timeout_seconds}"
),
},
)
raise
parse_start = time.perf_counter()
payload = orjson.loads(raw)
self._last_message_at = datetime.now(UTC)
_LOG.debug(
"kraken_ws_message",
recv_latency_ms=(parse_start - t0) * 1000,
parse_latency_ms=(time.perf_counter() - parse_start) * 1000,
)
if isinstance(payload, dict):
yield WsMessage(received_at=self._last_message_at, payload=payload)
async def _notify(
self,
*,
category: str,
severity: AlertSeverity,
title: str,
message: str,
details: dict[str, str] | None = None,
) -> None:
if self._alert_notifier is None:
return
await self._alert_notifier.notify(
category=category,
severity=severity,
title=title,
message=message,
details=details,
)
@staticmethod
def parse_book_delta(message: dict[str, Any]) -> BookDelta | None:
# Kraken v2 book update shape can vary by channel; keep parser defensive.
channel = str(message.get("channel", ""))
if "book" not in channel:
return None
symbol = str(message.get("symbol", ""))
data = message.get("data")
if not isinstance(data, list) or not data:
return None
first = data[0]
if not isinstance(first, dict):
return None
bids = [
BookLevel(price=float(level["price"]), volume=float(level["qty"]))
for level in first.get("bids", [])
if isinstance(level, dict) and "price" in level and "qty" in level
]
asks = [
BookLevel(price=float(level["price"]), volume=float(level["qty"]))
for level in first.get("asks", [])
if isinstance(level, dict) and "price" in level and "qty" in level
]
checksum: int | None = None
raw_checksum = first.get("checksum")
if isinstance(raw_checksum, int):
checksum = raw_checksum
source_timestamp_ms: int | None = None
if isinstance(first.get("timestamp"), int):
source_timestamp_ms = first["timestamp"]
return BookDelta(
symbol=symbol,
bids=bids,
asks=asks,
checksum=checksum,
source_timestamp_ms=source_timestamp_ms,
)
+37
View File
@@ -0,0 +1,37 @@
from __future__ import annotations
from dataclasses import dataclass
from datetime import UTC, datetime
from typing import Any
@dataclass(slots=True)
class KrakenApiResult:
endpoint: str
payload: dict[str, Any]
@dataclass(slots=True)
class LatencySample:
stage: str
at: datetime
latency_ms: float
@classmethod
def now(cls, stage: str, latency_ms: float) -> LatencySample:
return cls(stage=stage, at=datetime.now(UTC), latency_ms=latency_ms)
@dataclass(slots=True)
class BookLevel:
price: float
volume: float
@dataclass(slots=True)
class BookDelta:
symbol: str
bids: list[BookLevel]
asks: list[BookLevel]
checksum: int | None = None
source_timestamp_ms: int | None = None
+14
View File
@@ -0,0 +1,14 @@
from __future__ import annotations
import base64
import hashlib
import hmac
from functools import lru_cache
@lru_cache(maxsize=2048)
def sign_kraken_private_path(path: str, nonce: str, post_data: str, api_secret: str) -> str:
message = nonce.encode("utf-8") + post_data.encode("utf-8")
sha256 = hashlib.sha256(message).digest()
mac = hmac.new(base64.b64decode(api_secret), path.encode("utf-8") + sha256, hashlib.sha512)
return base64.b64encode(mac.digest()).decode("utf-8")
+32
View File
@@ -0,0 +1,32 @@
"""Trade execution helpers."""
from arbitrade.execution.fill_monitor import (
FillMonitor,
FillMonitorResult,
OrderFillState,
)
from arbitrade.execution.idempotency import (
IdempotencyKeyFactory,
OrderReconciler,
ReconciliationReport,
)
from arbitrade.execution.recovery import PartialFillRecovery, RecoveryAction
from arbitrade.execution.sequencer import (
ExecutionLeg,
TriangularExecutionResult,
TriangularExecutionSequencer,
)
__all__ = [
"ExecutionLeg",
"OrderFillState",
"FillMonitorResult",
"FillMonitor",
"IdempotencyKeyFactory",
"ReconciliationReport",
"OrderReconciler",
"RecoveryAction",
"PartialFillRecovery",
"TriangularExecutionResult",
"TriangularExecutionSequencer",
]
+133
View File
@@ -0,0 +1,133 @@
from __future__ import annotations
import asyncio
import time
from collections.abc import Callable
from dataclasses import dataclass
from datetime import UTC, datetime
from typing import Any, Protocol
class SupportsOrderStatusPolling(Protocol):
async def query_order(
self, *, order_id: str, include_trades: bool = True
) -> dict[str, Any]: ...
@dataclass(frozen=True, slots=True)
class OrderFillState:
order_id: str
status: str
filled_volume: float | None
avg_price: float | None
updated_at: datetime
source: str
@property
def is_terminal(self) -> bool:
return self.status in {"closed", "canceled", "expired"}
@dataclass(frozen=True, slots=True)
class FillMonitorResult:
order_id: str
timed_out: bool
terminal_state: OrderFillState | None
last_state: OrderFillState | None
elapsed_seconds: float
class FillMonitor:
def __init__(
self,
poll_client: SupportsOrderStatusPolling,
*,
poll_interval_seconds: float = 0.5,
max_wait_seconds: float = 10.0,
ws_status_provider: Callable[[str], OrderFillState | None] | None = None,
) -> None:
if poll_interval_seconds <= 0.0:
raise ValueError("poll_interval_seconds must be > 0.0")
if max_wait_seconds <= 0.0:
raise ValueError("max_wait_seconds must be > 0.0")
self._poll_client = poll_client
self._poll_interval_seconds = poll_interval_seconds
self._max_wait_seconds = max_wait_seconds
self._ws_status_provider = ws_status_provider
@staticmethod
def _to_float(value: Any) -> float | None:
if value is None:
return None
try:
return float(value)
except (TypeError, ValueError):
return None
@classmethod
def _state_from_payload(
cls, order_id: str, payload: dict[str, Any], *, source: str
) -> OrderFillState:
status = str(payload.get("status", "unknown")).lower()
return OrderFillState(
order_id=order_id,
status=status,
filled_volume=cls._to_float(payload.get("vol_exec")),
avg_price=cls._to_float(payload.get("price") or payload.get("avg_price")),
updated_at=datetime.now(UTC),
source=source,
)
@classmethod
def _extract_order_payload(cls, order_id: str, response: dict[str, Any]) -> dict[str, Any]:
if order_id in response and isinstance(response[order_id], dict):
payload = response[order_id]
return {str(key): value for key, value in payload.items()}
return response
async def wait_for_terminal_fill(self, order_id: str) -> FillMonitorResult:
if not order_id.strip():
raise ValueError("order_id must be non-empty")
started = time.monotonic()
last_state: OrderFillState | None = None
while True:
elapsed = time.monotonic() - started
if elapsed >= self._max_wait_seconds:
return FillMonitorResult(
order_id=order_id,
timed_out=True,
terminal_state=None,
last_state=last_state,
elapsed_seconds=elapsed,
)
if self._ws_status_provider is not None:
ws_state = self._ws_status_provider(order_id)
if ws_state is not None:
last_state = ws_state
if ws_state.is_terminal:
return FillMonitorResult(
order_id=order_id,
timed_out=False,
terminal_state=ws_state,
last_state=ws_state,
elapsed_seconds=elapsed,
)
response = await self._poll_client.query_order(order_id=order_id, include_trades=True)
payload = self._extract_order_payload(order_id, response)
polled_state = self._state_from_payload(order_id, payload, source="rest_poll")
last_state = polled_state
if polled_state.is_terminal:
return FillMonitorResult(
order_id=order_id,
timed_out=False,
terminal_state=polled_state,
last_state=polled_state,
elapsed_seconds=time.monotonic() - started,
)
await asyncio.sleep(self._poll_interval_seconds)
+105
View File
@@ -0,0 +1,105 @@
from __future__ import annotations
import hashlib
from dataclasses import dataclass
from typing import Any, Protocol
from arbitrade.detection.engine import OpportunityEvent
from arbitrade.execution.sequencer import ExecutionLeg
class SupportsOrderHistoryLookup(Protocol):
async def query_order(
self, *, order_id: str, include_trades: bool = True
) -> dict[str, Any]: ...
@dataclass(frozen=True, slots=True)
class ReconciliationReport:
order_id: str
user_ref: int
status: str
filled_volume: float | None
avg_price: float | None
is_terminal: bool
matches_request: bool
raw_payload: dict[str, Any]
class IdempotencyKeyFactory:
def user_ref_for_leg(self, event: OpportunityEvent, leg: ExecutionLeg, leg_index: int) -> int:
material = "|".join(
[
event.cycle,
event.updated_pair,
leg.from_currency,
leg.to_currency,
leg.pair,
leg.side,
f"{leg.volume:.12f}",
str(leg_index),
]
).encode("utf-8")
digest = hashlib.sha256(material).digest()
value = int.from_bytes(digest[:8], "big") % 2_147_483_647
return value or 1
class OrderReconciler:
def __init__(self, history_client: SupportsOrderHistoryLookup) -> None:
self._history_client = history_client
@staticmethod
def _to_float(value: Any) -> float | None:
if value is None:
return None
try:
return float(value)
except (TypeError, ValueError):
return None
@staticmethod
def _extract_payload(order_id: str, response: dict[str, Any]) -> dict[str, Any]:
if order_id in response and isinstance(response[order_id], dict):
payload = response[order_id]
return {str(key): value for key, value in payload.items()}
return response
async def reconcile_order(
self,
*,
order_id: str,
user_ref: int,
expected_pair: str,
expected_side: str,
expected_volume: float,
) -> ReconciliationReport:
if not order_id.strip():
raise ValueError("order_id must be non-empty")
response = await self._history_client.query_order(order_id=order_id, include_trades=True)
payload = self._extract_payload(order_id, response)
status = str(payload.get("status", "unknown")).lower()
filled_volume = self._to_float(payload.get("vol_exec"))
avg_price = self._to_float(payload.get("price") or payload.get("avg_price"))
reported_pair = str(payload.get("pair", expected_pair))
reported_side = str(payload.get("type", expected_side)).lower()
matches_request = (
reported_pair == expected_pair
and reported_side == expected_side.lower()
and (
expected_volume <= 0.0 or filled_volume is None or filled_volume <= expected_volume
)
and payload.get("userref") in {None, str(user_ref), user_ref}
)
return ReconciliationReport(
order_id=order_id,
user_ref=user_ref,
status=status,
filled_volume=filled_volume,
avg_price=avg_price,
is_terminal=status in {"closed", "canceled", "expired"},
matches_request=matches_request,
raw_payload=payload,
)
+98
View File
@@ -0,0 +1,98 @@
from __future__ import annotations
from dataclasses import dataclass
from typing import Any, Protocol
from arbitrade.execution.fill_monitor import FillMonitorResult, OrderFillState
class SupportsOrderLifecycle(Protocol):
async def cancel_order(self, *, order_id: str) -> dict[str, Any]: ...
async def place_market_order(
self, *, pair: str, side: str, volume: float
) -> dict[str, Any]: ...
@dataclass(frozen=True, slots=True)
class RecoveryAction:
order_id: str
canceled: bool
hedged: bool
hedge_pair: str | None = None
hedge_side: str | None = None
hedge_volume: float | None = None
cancel_response: dict[str, Any] | None = None
hedge_response: dict[str, Any] | None = None
reason: str | None = None
class PartialFillRecovery:
def __init__(self, rest_client: SupportsOrderLifecycle) -> None:
self._rest_client = rest_client
@staticmethod
def _counter_side(side: str) -> str:
normalized = side.lower()
if normalized == "buy":
return "sell"
if normalized == "sell":
return "buy"
raise ValueError("side must be 'buy' or 'sell'")
@staticmethod
def _residual_volume(terminal_state: OrderFillState | None, requested_volume: float) -> float:
if requested_volume <= 0.0:
raise ValueError("requested_volume must be > 0.0")
if terminal_state is None or terminal_state.filled_volume is None:
return requested_volume
residual = requested_volume - terminal_state.filled_volume
return residual if residual > 0.0 else 0.0
async def recover_partial_fill(
self,
*,
order_id: str,
pair: str,
side: str,
requested_volume: float,
fill_result: FillMonitorResult,
) -> RecoveryAction:
if not order_id.strip():
raise ValueError("order_id must be non-empty")
cancel_response: dict[str, Any] | None = None
hedge_response: dict[str, Any] | None = None
hedged = False
canceled = False
reason = None
state = fill_result.terminal_state or fill_result.last_state
residual_volume = self._residual_volume(state, requested_volume)
if state is not None and state.status in {"open", "partial"}:
cancel_response = await self._rest_client.cancel_order(order_id=order_id)
canceled = True
reason = f"canceled_{state.status}_order"
if residual_volume > 0.0 and fill_result.timed_out:
hedge_response = await self._rest_client.place_market_order(
pair=pair,
side=self._counter_side(side),
volume=residual_volume,
)
hedged = True
if reason is None:
reason = "hedged_timed_out_order"
return RecoveryAction(
order_id=order_id,
canceled=canceled,
hedged=hedged,
hedge_pair=pair if hedged else None,
hedge_side=self._counter_side(side) if hedged else None,
hedge_volume=residual_volume if hedged else None,
cancel_response=cancel_response,
hedge_response=hedge_response,
reason=reason,
)
+288
View File
@@ -0,0 +1,288 @@
from __future__ import annotations
from collections.abc import Callable, Sequence
from dataclasses import dataclass
from datetime import UTC, datetime
from typing import Any, Protocol
from arbitrade.alerting.notifier import SupportsAlerts
from arbitrade.detection.engine import OpportunityEvent
from arbitrade.storage.executions import AsyncExecutionWriter
from arbitrade.storage.repositories import (
AuditRecord,
AuditRepository,
OrderRecord,
PnLRecord,
TradeRecord,
)
class SupportsOrderPlacement(Protocol):
async def place_market_order(
self, *, pair: str, side: str, volume: float
) -> dict[str, Any]: ...
@dataclass(frozen=True, slots=True)
class ExecutionLeg:
from_currency: str
to_currency: str
pair: str
side: str
volume: float
@dataclass(frozen=True, slots=True)
class TriangularExecutionResult:
success: bool
requested_legs: tuple[ExecutionLeg, ...]
completed_legs: int
responses: tuple[dict[str, Any], ...]
failure_reason: str | None = None
class TriangularExecutionSequencer:
def __init__(
self,
rest_client: SupportsOrderPlacement,
*,
available_pairs: Sequence[str],
volume_for_leg: Callable[[OpportunityEvent, ExecutionLeg, int], float] | None = None,
execution_writer: AsyncExecutionWriter | None = None,
alert_notifier: SupportsAlerts | None = None,
audit_repository: AuditRepository | None = None,
) -> None:
self._rest_client = rest_client
self._available_pairs = {self._normalize_pair(pair) for pair in available_pairs}
self._volume_for_leg = volume_for_leg or self._default_volume_for_leg
self._execution_writer = execution_writer
self._alert_notifier = alert_notifier
self._audit_repository = audit_repository
@staticmethod
def _normalize_pair(pair: str) -> str:
normalized = pair.strip().upper().replace("-", "/")
if "/" not in normalized:
return normalized
base, quote = normalized.split("/", 1)
return f"{base}/{quote}"
@staticmethod
def _default_volume_for_leg(event: OpportunityEvent, _leg: ExecutionLeg, _idx: int) -> float:
if event.allocated_capital <= 0.0:
raise ValueError("allocated_capital must be > 0.0")
return event.allocated_capital
def _resolve_leg(self, from_currency: str, to_currency: str, volume: float) -> ExecutionLeg:
from_cur = from_currency.upper()
to_cur = to_currency.upper()
buy_pair = f"{to_cur}/{from_cur}"
if buy_pair in self._available_pairs:
return ExecutionLeg(
from_currency=from_cur,
to_currency=to_cur,
pair=buy_pair,
side="buy",
volume=volume,
)
sell_pair = f"{from_cur}/{to_cur}"
if sell_pair in self._available_pairs:
return ExecutionLeg(
from_currency=from_cur,
to_currency=to_cur,
pair=sell_pair,
side="sell",
volume=volume,
)
raise ValueError(f"No tradable pair for leg {from_cur}->{to_cur}")
def _build_legs(self, event: OpportunityEvent) -> tuple[ExecutionLeg, ...]:
currencies = [part.strip().upper() for part in event.cycle.split("->") if part.strip()]
if len(currencies) < 4 or currencies[0] != currencies[-1]:
raise ValueError("cycle must be a closed triangular path like A->B->C->A")
if len(currencies) != 4:
raise ValueError("cycle must contain exactly three unique currencies")
legs: list[ExecutionLeg] = []
for idx in range(3):
from_currency = currencies[idx]
to_currency = currencies[idx + 1]
placeholder_leg = ExecutionLeg(
from_currency=from_currency,
to_currency=to_currency,
pair="",
side="buy",
volume=0.0,
)
volume = self._volume_for_leg(event, placeholder_leg, idx)
if volume <= 0.0:
raise ValueError("volume_for_leg must return a positive volume")
legs.append(self._resolve_leg(from_currency, to_currency, volume))
return tuple(legs)
@staticmethod
def _trade_ref_for_event(event: OpportunityEvent) -> str:
material = (
f"{event.cycle}|{event.updated_pair}|"
f"{event.detected_at.timestamp():.6f}|"
f"{event.allocated_capital:.12f}"
)
return material.encode("utf-8").hex()[:32]
@staticmethod
def _order_ref_from_response(response: dict[str, Any], default: str) -> str:
txid = response.get("txid")
if isinstance(txid, list) and txid:
return str(txid[0])
if isinstance(txid, str) and txid.strip():
return txid
return default
async def execute(self, event: OpportunityEvent) -> TriangularExecutionResult:
legs = self._build_legs(event)
responses: list[dict[str, Any]] = []
trade_ref = self._trade_ref_for_event(event)
started_at = datetime.now(UTC)
for idx, leg in enumerate(legs):
try:
response = await self._rest_client.place_market_order(
pair=leg.pair,
side=leg.side,
volume=leg.volume,
)
except Exception as exc:
if self._audit_repository is not None:
self._audit_repository.insert(
AuditRecord(
occurred_at=datetime.now(UTC),
actor="execution_engine",
event_type="execution.trade.failed",
decision="rejected",
payload={
"cycle": event.cycle,
"failed_leg_index": idx,
"error": str(exc),
},
correlation_id=trade_ref,
)
)
if self._alert_notifier is not None:
await self._alert_notifier.notify(
category="error",
severity="error",
title="Trade execution failed",
message="Triangular execution failed before completing all legs.",
details={
"cycle": event.cycle,
"failed_leg_index": str(idx),
"error": str(exc),
},
)
if self._execution_writer is not None:
await self._execution_writer.enqueue(
TradeRecord(
trade_ref=trade_ref,
started_at=started_at,
finished_at=datetime.now(UTC),
status="failed",
realized_pnl=None,
estimated_pnl=event.est_profit,
capital_used=event.allocated_capital,
cycle=event.cycle,
leg_count=len(legs),
)
)
return TriangularExecutionResult(
success=False,
requested_legs=legs,
completed_legs=idx,
responses=tuple(responses),
failure_reason=str(exc),
)
responses.append(response)
if self._execution_writer is not None:
order_ref = self._order_ref_from_response(response, f"leg-{idx}")
await self._execution_writer.enqueue(
OrderRecord(
trade_ref=trade_ref,
order_ref=order_ref,
leg_index=idx,
pair=leg.pair,
side=leg.side,
volume=leg.volume,
user_ref=None,
status=str(response.get("status", "submitted")),
filled_volume=None,
avg_price=None,
raw_response=response,
recorded_at=datetime.now(UTC),
)
)
if self._execution_writer is not None:
await self._execution_writer.enqueue(
TradeRecord(
trade_ref=trade_ref,
started_at=started_at,
finished_at=datetime.now(UTC),
status="filled",
realized_pnl=None,
estimated_pnl=event.est_profit,
capital_used=event.allocated_capital,
cycle=event.cycle,
leg_count=len(legs),
)
)
await self._execution_writer.enqueue(
PnLRecord(
trade_ref=trade_ref,
recorded_at=datetime.now(UTC),
kind="estimated",
pnl_usd=event.est_profit,
source="triangular_sequencer",
)
)
if self._alert_notifier is not None:
await self._alert_notifier.notify(
category="trade",
severity="warning" if event.est_profit < 0.0 else "info",
title="Trade execution completed",
message="Triangular execution completed all requested legs.",
details={
"cycle": event.cycle,
"completed_legs": str(len(legs)),
"estimated_pnl_usd": f"{event.est_profit}",
},
)
if self._audit_repository is not None:
self._audit_repository.insert(
AuditRecord(
occurred_at=datetime.now(UTC),
actor="execution_engine",
event_type="execution.trade.completed",
decision="approved",
payload={
"cycle": event.cycle,
"completed_legs": len(legs),
"estimated_pnl_usd": event.est_profit,
},
correlation_id=trade_ref,
)
)
return TriangularExecutionResult(
success=True,
requested_legs=legs,
completed_legs=len(legs),
responses=tuple(responses),
)
+1
View File
@@ -0,0 +1 @@
"""Market data ingestion and book cache package."""
+485
View File
@@ -0,0 +1,485 @@
from __future__ import annotations
import time
from collections.abc import Awaitable, Callable, Mapping
from dataclasses import dataclass
from datetime import UTC, datetime
import structlog
from arbitrade.alerting.notifier import SupportsAlerts, dispatch_alert_nowait
from arbitrade.detection.engine import IncrementalCycleDetector, OpportunityEvent
from arbitrade.exchange.kraken_ws import KrakenWsClient
from arbitrade.market_data.order_book import OrderBook
from arbitrade.risk.kill_switch import KillSwitch
from arbitrade.risk.loss_limits import LossLimitGuard
from arbitrade.risk.pre_trade import PreTradeValidator
from arbitrade.risk.stop_conditions import StopConditionsGuard
from arbitrade.risk.trade_limits import TradeLimitsGuard
from arbitrade.storage.market_snapshots import AsyncMarketSnapshotWriter, MarketSnapshot
from arbitrade.storage.opportunities import AsyncOpportunityWriter
from arbitrade.storage.repositories import AuditRecord, AuditRepository
_LOG = structlog.get_logger(__name__)
@dataclass(frozen=True, slots=True)
class ExecutionOutcome:
realized_pnl: float | None = None
close_trade: bool = True
class MarketDataFeed:
def __init__(
self,
ws_client: KrakenWsClient,
snapshot_writer: AsyncMarketSnapshotWriter,
detector: IncrementalCycleDetector | None = None,
opportunity_writer: AsyncOpportunityWriter | None = None,
paper_trading_mode: bool = True,
opportunity_executor: (
Callable[[OpportunityEvent], Awaitable[ExecutionOutcome | float | None]] | None
) = None,
trade_capital: float = 1.0,
max_trade_capital: float | None = None,
loss_limit_guard: LossLimitGuard | None = None,
trade_limits_guard: TradeLimitsGuard | None = None,
pre_trade_validator: PreTradeValidator | None = None,
balance_provider: Callable[[], Mapping[str, float]] | None = None,
quote_balance_asset: str = "USD",
kill_switch: KillSwitch | None = None,
stop_conditions_guard: StopConditionsGuard | None = None,
alert_notifier: SupportsAlerts | None = None,
audit_repository: AuditRepository | None = None,
) -> None:
self._ws_client = ws_client
self._snapshot_writer = snapshot_writer
self._books: dict[str, OrderBook] = {}
self._detector = detector
self._opportunity_writer = opportunity_writer
self._paper_trading_mode = paper_trading_mode
self._opportunity_executor = opportunity_executor
self._trade_capital = trade_capital
self._max_trade_capital = max_trade_capital
self._loss_limit_guard = loss_limit_guard
self._trade_limits_guard = trade_limits_guard
self._pre_trade_validator = pre_trade_validator
self._balance_provider = balance_provider
self._quote_balance_asset = quote_balance_asset.upper()
self._kill_switch = kill_switch
self._stop_conditions_guard = stop_conditions_guard
self._alert_notifier = alert_notifier
self._audit_repository = audit_repository
if self._trade_capital <= 0.0:
raise ValueError("trade_capital must be > 0.0")
if self._max_trade_capital is not None and self._max_trade_capital <= 0.0:
raise ValueError("max_trade_capital must be > 0.0")
@property
def books(self) -> dict[str, OrderBook]:
return self._books
def _effective_trade_capital(self) -> float:
if self._max_trade_capital is None:
return self._trade_capital
return min(self._trade_capital, self._max_trade_capital)
@staticmethod
def _exposure_for_event(event: OpportunityEvent) -> dict[str, float]:
currencies = [part for part in event.cycle.split("->") if part]
if len(currencies) < 2:
return {}
start = currencies[0]
exposure_assets = {currency for currency in currencies[1:] if currency != start}
return {asset: event.allocated_capital for asset in exposure_assets}
async def run(self) -> None:
async for message in self._ws_client.connect_stream():
parse_start = time.perf_counter()
delta = self._ws_client.parse_book_delta(message.payload)
if delta is None:
continue
book = self._books.setdefault(delta.symbol, OrderBook())
book.apply_bids(delta.bids)
book.apply_asks(delta.asks)
checksum_ok = True
if delta.checksum is not None:
checksum_ok = book.compute_checksum() == delta.checksum
apply_latency_ms = (time.perf_counter() - parse_start) * 1000
source_latency_ms: float | None = None
if delta.source_timestamp_ms is not None:
source_latency_ms = datetime.now(UTC).timestamp() * 1000 - float(
delta.source_timestamp_ms
)
_LOG.info(
"book_delta_applied",
symbol=delta.symbol,
bids=len(delta.bids),
asks=len(delta.asks),
checksum_ok=checksum_ok,
apply_latency_ms=apply_latency_ms,
source_latency_ms=source_latency_ms,
)
if self._stop_conditions_guard is not None:
self._stop_conditions_guard.observe_latency(
source_latency_ms=source_latency_ms,
apply_latency_ms=apply_latency_ms,
)
if self._stop_conditions_guard.is_halted:
if self._kill_switch is not None and not self._kill_switch.is_active:
self._kill_switch.activate(
reason=self._stop_conditions_guard.halted_reason
or "stop_conditions_halted",
)
_LOG.warning(
"stop_condition_halt_triggered",
reason=self._stop_conditions_guard.halted_reason,
symbol=delta.symbol,
)
if self._audit_repository is not None:
self._audit_repository.insert(
AuditRecord(
occurred_at=datetime.now(UTC),
actor="risk_manager",
event_type="risk.stop_condition_halt",
decision="rejected",
payload={
"reason": self._stop_conditions_guard.halted_reason
or "unknown",
"symbol": delta.symbol,
},
)
)
if self._detector is not None:
opportunities = self._detector.opportunities_for_updated_pair(
delta.symbol,
self._books,
base_capital=self._effective_trade_capital(),
)
_LOG.debug(
"incremental_opportunity_scores",
symbol=delta.symbol,
opportunities=len(opportunities),
)
for event in opportunities:
if self._audit_repository is not None:
self._audit_repository.insert(
AuditRecord(
occurred_at=datetime.now(UTC),
actor="detector",
event_type="detector.opportunity",
decision="scored",
payload={
"cycle": event.cycle,
"updated_pair": event.updated_pair,
"net_pct": event.net_pct,
"est_profit": event.est_profit,
},
)
)
_LOG.info(
"opportunity_detected",
cycle=event.cycle,
updated_pair=event.updated_pair,
gross_pct=event.gross_pct,
net_pct=event.net_pct,
est_profit=event.est_profit,
mode="paper" if self._paper_trading_mode else "live",
)
if self._opportunity_writer is not None:
await self._opportunity_writer.enqueue(event)
if self._paper_trading_mode:
_LOG.info(
"paper_trade_simulated",
cycle=event.cycle,
updated_pair=event.updated_pair,
net_pct=event.net_pct,
)
if self._audit_repository is not None:
self._audit_repository.insert(
AuditRecord(
occurred_at=datetime.now(UTC),
actor="execution_engine",
event_type="execution.paper_trade",
decision="skipped",
payload={
"cycle": event.cycle,
"updated_pair": event.updated_pair,
},
)
)
continue
if self._opportunity_executor is None:
_LOG.warning(
"live_trade_skipped_no_executor",
cycle=event.cycle,
updated_pair=event.updated_pair,
)
if self._audit_repository is not None:
self._audit_repository.insert(
AuditRecord(
occurred_at=datetime.now(UTC),
actor="execution_engine",
event_type="execution.live_trade",
decision="rejected",
payload={
"reason": "missing_executor",
"cycle": event.cycle,
},
)
)
continue
if self._kill_switch is not None and self._kill_switch.is_active:
_LOG.warning(
"live_trade_skipped_kill_switch",
cycle=event.cycle,
updated_pair=event.updated_pair,
reason=self._kill_switch.reason,
)
if self._audit_repository is not None:
self._audit_repository.insert(
AuditRecord(
occurred_at=datetime.now(UTC),
actor="risk_manager",
event_type="risk.kill_switch",
decision="rejected",
payload={
"reason": self._kill_switch.reason or "manual",
"cycle": event.cycle,
},
)
)
continue
if (
self._stop_conditions_guard is not None
and self._stop_conditions_guard.is_halted
):
_LOG.warning(
"live_trade_skipped_stop_condition_halt",
cycle=event.cycle,
updated_pair=event.updated_pair,
reason=self._stop_conditions_guard.halted_reason,
)
if self._audit_repository is not None:
self._audit_repository.insert(
AuditRecord(
occurred_at=datetime.now(UTC),
actor="risk_manager",
event_type="risk.stop_condition",
decision="rejected",
payload={
"reason": self._stop_conditions_guard.halted_reason
or "halted",
"cycle": event.cycle,
},
)
)
continue
if self._loss_limit_guard is not None and self._loss_limit_guard.is_halted:
_LOG.warning(
"live_trade_skipped_loss_limit_halted",
cycle=event.cycle,
updated_pair=event.updated_pair,
reason=self._loss_limit_guard.halted_reason,
)
if self._audit_repository is not None:
self._audit_repository.insert(
AuditRecord(
occurred_at=datetime.now(UTC),
actor="risk_manager",
event_type="risk.loss_limit",
decision="rejected",
payload={
"reason": self._loss_limit_guard.halted_reason or "halted",
"cycle": event.cycle,
},
)
)
continue
if self._pre_trade_validator is not None and self._balance_provider is not None:
required_balances = {self._quote_balance_asset: event.allocated_capital}
balances = {
asset.upper(): amount
for asset, amount in self._balance_provider().items()
}
if not self._pre_trade_validator.validate(
balances_by_asset=balances,
required_by_asset=required_balances,
):
_LOG.warning(
"live_trade_skipped_pre_trade_validation",
cycle=event.cycle,
updated_pair=event.updated_pair,
required_by_asset=required_balances,
)
if self._audit_repository is not None:
self._audit_repository.insert(
AuditRecord(
occurred_at=datetime.now(UTC),
actor="risk_manager",
event_type="risk.pre_trade_validation",
decision="rejected",
payload={
"cycle": event.cycle,
"required_by_asset": {
key: required_balances[key]
for key in required_balances
},
},
)
)
continue
exposure_by_asset = self._exposure_for_event(event)
if (
self._trade_limits_guard is not None
and not self._trade_limits_guard.is_trade_allowed(exposure_by_asset)
):
_LOG.warning(
"live_trade_skipped_trade_limits",
cycle=event.cycle,
updated_pair=event.updated_pair,
exposure_by_asset=exposure_by_asset,
)
if self._audit_repository is not None:
self._audit_repository.insert(
AuditRecord(
occurred_at=datetime.now(UTC),
actor="risk_manager",
event_type="risk.trade_limits",
decision="rejected",
payload={
"cycle": event.cycle,
"exposure_by_asset": {
key: exposure_by_asset[key] for key in exposure_by_asset
},
},
)
)
continue
if self._trade_limits_guard is not None:
self._trade_limits_guard.open_trade(exposure_by_asset)
try:
outcome = await self._opportunity_executor(event)
except Exception as exc:
if self._trade_limits_guard is not None:
self._trade_limits_guard.close_trade(exposure_by_asset)
dispatch_alert_nowait(
self._alert_notifier,
category="system",
severity="critical",
title="Critical execution exception",
message="Unhandled exception raised by opportunity executor.",
details={
"cycle": event.cycle,
"updated_pair": event.updated_pair,
"error": str(exc),
},
)
if self._stop_conditions_guard is not None:
self._stop_conditions_guard.register_failure()
if self._stop_conditions_guard.is_halted:
if (
self._kill_switch is not None
and not self._kill_switch.is_active
):
self._kill_switch.activate(
reason=self._stop_conditions_guard.halted_reason
or "stop_conditions_halted",
)
_LOG.warning(
"stop_condition_halt_triggered",
reason=self._stop_conditions_guard.halted_reason,
cycle=event.cycle,
updated_pair=event.updated_pair,
)
_LOG.exception(
"live_trade_execution_failed",
cycle=event.cycle,
updated_pair=event.updated_pair,
)
if self._audit_repository is not None:
self._audit_repository.insert(
AuditRecord(
occurred_at=datetime.now(UTC),
actor="execution_engine",
event_type="execution.live_trade",
decision="error",
payload={
"cycle": event.cycle,
"updated_pair": event.updated_pair,
"error": str(exc),
},
)
)
continue
if self._stop_conditions_guard is not None:
self._stop_conditions_guard.register_success()
realized_pnl: float | None
close_trade = True
if isinstance(outcome, ExecutionOutcome):
realized_pnl = outcome.realized_pnl
close_trade = outcome.close_trade
else:
realized_pnl = outcome
if realized_pnl is not None and self._loss_limit_guard is not None:
self._loss_limit_guard.register_realized_pnl(realized_pnl)
if self._loss_limit_guard.is_halted:
_LOG.warning(
"loss_limit_halt_triggered",
reason=self._loss_limit_guard.halted_reason,
cumulative_pnl=self._loss_limit_guard.cumulative_pnl,
)
if self._trade_limits_guard is not None and close_trade:
self._trade_limits_guard.close_trade(exposure_by_asset)
if self._audit_repository is not None:
self._audit_repository.insert(
AuditRecord(
occurred_at=datetime.now(UTC),
actor="execution_engine",
event_type="execution.live_trade",
decision="approved",
payload={
"cycle": event.cycle,
"updated_pair": event.updated_pair,
"realized_pnl": realized_pnl,
"close_trade": close_trade,
},
)
)
await self._snapshot_writer.enqueue(
MarketSnapshot(
snapshot_at=datetime.now(UTC),
symbol=delta.symbol,
source="kraken_ws",
payload=message.payload,
latency_ms=source_latency_ms,
)
)
+104
View File
@@ -0,0 +1,104 @@
from __future__ import annotations
import re
from collections.abc import Iterable
from dataclasses import dataclass
from datetime import UTC, datetime
from sortedcontainers import SortedDict
from arbitrade.exchange.models import BookLevel
ZERO_CLEAN_RE = re.compile(r"^0+", re.ASCII)
def _normalize_price_for_checksum(value: float) -> str:
text = f"{value:.10f}".replace(".", "")
text = text.rstrip("0")
stripped = ZERO_CLEAN_RE.sub("", text)
return stripped or "0"
def _normalize_volume_for_checksum(value: float) -> str:
text = f"{value:.10f}".replace(".", "")
text = text.rstrip("0")
stripped = ZERO_CLEAN_RE.sub("", text)
return stripped or "0"
@dataclass(slots=True)
class BookView:
best_bid: BookLevel | None
best_ask: BookLevel | None
updated_at: datetime
class OrderBook:
def __init__(self) -> None:
self._bids: SortedDict[float, float] = SortedDict()
self._asks: SortedDict[float, float] = SortedDict()
self._updated_at: datetime = datetime.now(UTC)
@property
def updated_at(self) -> datetime:
return self._updated_at
def apply_bids(self, updates: Iterable[BookLevel]) -> None:
for level in updates:
if level.volume <= 0:
self._bids.pop(level.price, None)
else:
self._bids[level.price] = level.volume
self._updated_at = datetime.now(UTC)
def apply_asks(self, updates: Iterable[BookLevel]) -> None:
for level in updates:
if level.volume <= 0:
self._asks.pop(level.price, None)
else:
self._asks[level.price] = level.volume
self._updated_at = datetime.now(UTC)
def best_bid(self) -> BookLevel | None:
if not self._bids:
return None
price = self._bids.peekitem(-1)[0]
return BookLevel(price=price, volume=self._bids[price])
def best_ask(self) -> BookLevel | None:
if not self._asks:
return None
price = self._asks.peekitem(0)[0]
return BookLevel(price=price, volume=self._asks[price])
def snapshot(self) -> BookView:
return BookView(
best_bid=self.best_bid(),
best_ask=self.best_ask(),
updated_at=self._updated_at,
)
def top_levels(self, depth: int = 10) -> tuple[list[BookLevel], list[BookLevel]]:
bid_keys = list(self._bids.keys())
ask_keys = list(self._asks.keys())
bids = [
BookLevel(price=price, volume=self._bids[price])
for price in reversed(bid_keys[-depth:])
]
asks = [BookLevel(price=price, volume=self._asks[price]) for price in ask_keys[:depth]]
return bids, asks
def compute_checksum(self, depth: int = 10) -> int:
bids, asks = self.top_levels(depth)
combined: list[str] = []
for level in bids:
combined.append(_normalize_price_for_checksum(level.price))
combined.append(_normalize_volume_for_checksum(level.volume))
for level in asks:
combined.append(_normalize_price_for_checksum(level.price))
combined.append(_normalize_volume_for_checksum(level.volume))
import zlib
return zlib.crc32("".join(combined).encode("utf-8"))
+132
View File
@@ -0,0 +1,132 @@
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime
from arbitrade.storage.db import DuckDBStore
@dataclass(frozen=True, slots=True)
class PerformanceMetrics:
realized_pnl_usd: float
win_rate: float | None
avg_trade_duration_seconds: float | None
opportunities_per_minute: float | None
fill_rate: float | None
latency_p50_seconds: float | None
latency_p95_seconds: float | None
latency_p99_seconds: float | None
class MetricsCalculator:
def __init__(self, store: DuckDBStore) -> None:
self._store = store
def compute(self) -> PerformanceMetrics:
with self._store.connect() as conn:
trade_metrics = conn.execute("""
SELECT
COALESCE(SUM(COALESCE(realized_pnl, 0)), 0) AS realized_pnl_usd,
COUNT(*) AS total_trades,
SUM(CASE WHEN realized_pnl > 0 THEN 1 ELSE 0 END) AS winning_trades,
AVG(EPOCH(finished_at) - EPOCH(started_at)) AS avg_trade_duration_seconds,
quantile_cont(
EPOCH(finished_at) - EPOCH(started_at),
0.50
) AS latency_p50_seconds,
quantile_cont(
EPOCH(finished_at) - EPOCH(started_at),
0.95
) AS latency_p95_seconds,
quantile_cont(
EPOCH(finished_at) - EPOCH(started_at),
0.99
) AS latency_p99_seconds
FROM trades
WHERE finished_at IS NOT NULL
""").fetchone()
opportunity_metrics = conn.execute("""
SELECT
COUNT(*) AS opportunity_count,
MIN(detected_at) AS first_detected_at,
MAX(detected_at) AS last_detected_at
FROM opportunities
""").fetchone()
fill_metrics = conn.execute("""
SELECT AVG(filled_volume / volume) AS fill_rate
FROM orders
WHERE volume > 0 AND filled_volume IS NOT NULL
""").fetchone()
realized_pnl_usd = (
float(trade_metrics[0]) if trade_metrics and trade_metrics[0] is not None else 0.0
)
total_trades = (
int(trade_metrics[1]) if trade_metrics and trade_metrics[1] is not None else 0
)
winning_trades = (
int(trade_metrics[2]) if trade_metrics and trade_metrics[2] is not None else 0
)
win_rate = winning_trades / total_trades if total_trades > 0 else None
avg_trade_duration_seconds = (
float(trade_metrics[3]) if trade_metrics and trade_metrics[3] is not None else None
)
opportunity_count = (
int(opportunity_metrics[0])
if opportunity_metrics is not None and opportunity_metrics[0] is not None
else 0
)
first_detected_at = (
opportunity_metrics[1]
if opportunity_metrics is not None and isinstance(opportunity_metrics[1], datetime)
else None
)
last_detected_at = (
opportunity_metrics[2]
if opportunity_metrics is not None and isinstance(opportunity_metrics[2], datetime)
else None
)
opportunities_per_minute: float | None
if (
opportunity_count >= 2
and first_detected_at is not None
and last_detected_at is not None
):
span_seconds = (last_detected_at - first_detected_at).total_seconds()
opportunities_per_minute = (
opportunity_count / (span_seconds / 60.0)
if span_seconds > 0.0
else float(opportunity_count)
)
elif opportunity_count == 1:
opportunities_per_minute = 60.0
else:
opportunities_per_minute = None
fill_rate = float(fill_metrics[0]) if fill_metrics and fill_metrics[0] is not None else None
latency_p50_seconds = (
float(trade_metrics[4]) if trade_metrics and trade_metrics[4] is not None else None
)
latency_p95_seconds = (
float(trade_metrics[5]) if trade_metrics and trade_metrics[5] is not None else None
)
latency_p99_seconds = (
float(trade_metrics[6]) if trade_metrics and trade_metrics[6] is not None else None
)
return PerformanceMetrics(
realized_pnl_usd=realized_pnl_usd,
win_rate=win_rate,
avg_trade_duration_seconds=avg_trade_duration_seconds,
opportunities_per_minute=opportunities_per_minute,
fill_rate=fill_rate,
latency_p50_seconds=latency_p50_seconds,
latency_p95_seconds=latency_p95_seconds,
latency_p99_seconds=latency_p99_seconds,
)
+4
View File
@@ -0,0 +1,4 @@
from arbitrade.perf.guardrails import evaluate_guardrails
from arbitrade.perf.latency import run_latency_profile
__all__ = ["run_latency_profile", "evaluate_guardrails"]
+80
View File
@@ -0,0 +1,80 @@
from __future__ import annotations
def evaluate_guardrails(
*,
baseline: dict[str, object],
current: dict[str, object],
thresholds: dict[str, object],
) -> list[str]:
failures: list[str] = []
baseline_scenarios = baseline.get("scenarios")
current_scenarios = current.get("scenarios")
if not isinstance(baseline_scenarios, dict) or not isinstance(current_scenarios, dict):
return ["invalid profile payload: missing scenarios map"]
default_thresholds = thresholds.get("default")
if not isinstance(default_thresholds, dict):
default_thresholds = {"p95_ms": 2.5, "p99_ms": 3.0}
scenario_thresholds = thresholds.get("scenarios")
if not isinstance(scenario_thresholds, dict):
scenario_thresholds = {}
for scenario, baseline_payload in baseline_scenarios.items():
current_payload = current_scenarios.get(scenario)
if not isinstance(baseline_payload, dict) or not isinstance(current_payload, dict):
failures.append(f"missing scenario in current profile: {scenario}")
continue
baseline_stages = baseline_payload.get("stages")
current_stages = current_payload.get("stages")
if not isinstance(baseline_stages, dict) or not isinstance(current_stages, dict):
failures.append(f"missing stages map for scenario: {scenario}")
continue
scenario_config = scenario_thresholds.get(scenario)
if not isinstance(scenario_config, dict):
scenario_config = {}
for stage, baseline_stage in baseline_stages.items():
current_stage = current_stages.get(stage)
if not isinstance(baseline_stage, dict) or not isinstance(current_stage, dict):
failures.append(f"missing stage in current profile: {scenario}.{stage}")
continue
for percentile_key in ("p95_ms", "p99_ms"):
threshold_ratio_raw = scenario_config.get(
percentile_key,
default_thresholds.get(percentile_key, 3.0),
)
threshold_ratio = (
float(threshold_ratio_raw)
if isinstance(threshold_ratio_raw, int | float)
else 3.0
)
base_value_raw = baseline_stage.get(percentile_key)
current_value_raw = current_stage.get(percentile_key)
if not isinstance(base_value_raw, int | float) or not isinstance(
current_value_raw, int | float
):
failures.append(
f"invalid percentile value: {scenario}.{stage}.{percentile_key}"
)
continue
base_value = float(base_value_raw)
current_value = float(current_value_raw)
# Avoid divide-by-zero while still preserving strict checks.
max_allowed = max(base_value * threshold_ratio, 0.001)
if current_value > max_allowed:
failures.append(
f"latency regression: {scenario}.{stage}.{percentile_key} "
f"current={current_value:.4f}ms "
f"baseline={base_value:.4f}ms "
f"allowed={max_allowed:.4f}ms"
)
return failures
+195
View File
@@ -0,0 +1,195 @@
from __future__ import annotations
from collections.abc import Callable
from dataclasses import dataclass
from time import perf_counter_ns
import orjson
@dataclass(frozen=True, slots=True)
class PercentileSummary:
p50_ms: float
p95_ms: float
p99_ms: float
@dataclass(frozen=True, slots=True)
class ScenarioProfile:
scenario: str
iterations: int
stages: dict[str, PercentileSummary]
def _percentile(samples: list[float], percentile: float) -> float:
if not samples:
return 0.0
ordered = sorted(samples)
if percentile <= 0.0:
return ordered[0]
if percentile >= 100.0:
return ordered[-1]
rank = (len(ordered) - 1) * (percentile / 100.0)
lower = int(rank)
upper = min(lower + 1, len(ordered) - 1)
weight = rank - lower
return ordered[lower] * (1.0 - weight) + ordered[upper] * weight
def _summarize(samples: list[float]) -> PercentileSummary:
return PercentileSummary(
p50_ms=_percentile(samples, 50.0),
p95_ms=_percentile(samples, 95.0),
p99_ms=_percentile(samples, 99.0),
)
def _ingest_stage(raw_payload: bytes, state: dict[str, float]) -> None:
parsed = orjson.loads(raw_payload)
bids = parsed.get("bids", [])
asks = parsed.get("asks", [])
for price, volume in bids[:4]:
state[str(price)] = float(volume)
for price, volume in asks[:4]:
state[str(price)] = float(volume)
def _detect_stage(values: list[float], cycles: int) -> float:
best = 0.0
size = len(values)
for idx in range(cycles):
a = values[idx % size]
b = values[(idx + 3) % size]
c = values[(idx + 7) % size]
gross = (a / b) * c
net = gross * 0.9975
if net > best:
best = net
return best
def _risk_stage(net_edge: float, capital: float) -> float:
if net_edge < 1.0002:
return 0.0
if capital > 500.0:
capital = 500.0
return capital * min(net_edge - 1.0, 0.02)
def _execution_stage(planned_pnl: float, order_id: int) -> None:
payload = {
"order_id": order_id,
"planned_pnl": planned_pnl,
"legs": [
{"pair": "BTC/USD", "side": "buy", "qty": 0.01},
{"pair": "ETH/BTC", "side": "buy", "qty": 0.1},
{"pair": "ETH/USD", "side": "sell", "qty": 0.1},
],
}
_ = orjson.dumps(payload)
def _run_scenario(
name: str,
iterations: int,
detect_cycles: int,
reconnect_every: int,
) -> ScenarioProfile:
payloads = [
orjson.dumps(
{
"symbol": "BTC/USD",
"bids": [[100000.0 + i, 0.2 + (i % 5) * 0.01] for i in range(12)],
"asks": [[100001.0 + i, 0.2 + (i % 7) * 0.01] for i in range(12)],
}
)
for _ in range(5)
]
value_series = [1.0 + (idx % 31) * 0.0007 for idx in range(128)]
order_state: dict[str, float] = {}
ingest_ms: list[float] = []
detect_ms: list[float] = []
risk_ms: list[float] = []
execution_ms: list[float] = []
end_to_end_ms: list[float] = []
for idx in range(iterations):
start_ns = perf_counter_ns()
payload = payloads[idx % len(payloads)]
t0 = perf_counter_ns()
_ingest_stage(payload, order_state)
if reconnect_every > 0 and idx > 0 and idx % reconnect_every == 0:
order_state.clear()
t1 = perf_counter_ns()
net_edge = _detect_stage(value_series, detect_cycles)
t2 = perf_counter_ns()
planned = _risk_stage(net_edge, capital=100.0 + (idx % 50))
t3 = perf_counter_ns()
_execution_stage(planned, order_id=idx)
t4 = perf_counter_ns()
ingest_ms.append((t1 - t0) / 1_000_000.0)
detect_ms.append((t2 - t1) / 1_000_000.0)
risk_ms.append((t3 - t2) / 1_000_000.0)
execution_ms.append((t4 - t3) / 1_000_000.0)
end_to_end_ms.append((t4 - start_ns) / 1_000_000.0)
return ScenarioProfile(
scenario=name,
iterations=iterations,
stages={
"ingest": _summarize(ingest_ms),
"detect": _summarize(detect_ms),
"risk": _summarize(risk_ms),
"execution": _summarize(execution_ms),
"end_to_end": _summarize(end_to_end_ms),
},
)
def run_latency_profile(iterations: int = 600) -> dict[str, object]:
scenarios: list[Callable[[], ScenarioProfile]] = [
lambda: _run_scenario(
name="book_update_burst",
iterations=iterations,
detect_cycles=32,
reconnect_every=0,
),
lambda: _run_scenario(
name="execution_spike",
iterations=iterations,
detect_cycles=96,
reconnect_every=0,
),
lambda: _run_scenario(
name="reconnect_storm",
iterations=iterations,
detect_cycles=48,
reconnect_every=20,
),
]
result: dict[str, object] = {"iterations": iterations, "scenarios": {}}
scenario_map = result["scenarios"]
assert isinstance(scenario_map, dict)
for scenario in scenarios:
profile = scenario()
scenario_map[profile.scenario] = {
"iterations": profile.iterations,
"stages": {
stage: {
"p50_ms": summary.p50_ms,
"p95_ms": summary.p95_ms,
"p99_ms": summary.p99_ms,
}
for stage, summary in profile.stages.items()
},
}
return result
+15
View File
@@ -0,0 +1,15 @@
"""Risk management helpers."""
from arbitrade.risk.kill_switch import KillSwitch
from arbitrade.risk.loss_limits import LossLimitGuard
from arbitrade.risk.pre_trade import PreTradeValidator
from arbitrade.risk.stop_conditions import StopConditionsGuard
from arbitrade.risk.trade_limits import TradeLimitsGuard
__all__ = [
"LossLimitGuard",
"TradeLimitsGuard",
"PreTradeValidator",
"KillSwitch",
"StopConditionsGuard",
]
+23
View File
@@ -0,0 +1,23 @@
from __future__ import annotations
class KillSwitch:
def __init__(self, *, active: bool = False, reason: str | None = None) -> None:
self._active = active
self._reason = reason or ("manual" if active else None)
@property
def is_active(self) -> bool:
return self._active
@property
def reason(self) -> str | None:
return self._reason
def activate(self, *, reason: str = "manual") -> None:
self._active = True
self._reason = reason
def deactivate(self) -> None:
self._active = False
self._reason = None
+90
View File
@@ -0,0 +1,90 @@
from __future__ import annotations
from datetime import UTC, date, datetime
from arbitrade.alerting.notifier import SupportsAlerts, dispatch_alert_nowait
class LossLimitGuard:
def __init__(
self,
*,
daily_loss_limit: float | None = None,
cumulative_loss_limit: float | None = None,
alert_notifier: SupportsAlerts | None = None,
) -> None:
self._daily_loss_limit = daily_loss_limit
self._cumulative_loss_limit = cumulative_loss_limit
if self._daily_loss_limit is not None and self._daily_loss_limit <= 0.0:
raise ValueError("daily_loss_limit must be > 0.0")
if self._cumulative_loss_limit is not None and self._cumulative_loss_limit <= 0.0:
raise ValueError("cumulative_loss_limit must be > 0.0")
self._cumulative_pnl = 0.0
self._daily_pnl: dict[date, float] = {}
self._halted_reason: str | None = None
self._alert_notifier = alert_notifier
@property
def cumulative_pnl(self) -> float:
return self._cumulative_pnl
@property
def halted_reason(self) -> str | None:
return self._halted_reason
@property
def is_halted(self) -> bool:
return self._halted_reason is not None
def daily_pnl(self, day: date) -> float:
return self._daily_pnl.get(day, 0.0)
def register_realized_pnl(self, pnl: float, *, at: datetime | None = None) -> None:
if self.is_halted:
return
timestamp = at or datetime.now(UTC)
day_key = timestamp.date()
self._cumulative_pnl += pnl
self._daily_pnl[day_key] = self._daily_pnl.get(day_key, 0.0) + pnl
if (
self._daily_loss_limit is not None
and self._daily_pnl[day_key] <= -self._daily_loss_limit
):
self._halted_reason = "daily_loss_limit_breached"
dispatch_alert_nowait(
self._alert_notifier,
category="threshold",
severity="critical",
title="Daily loss limit breached",
message="Trading halted because daily realized PnL crossed configured loss limit.",
details={
"daily_pnl": f"{self._daily_pnl[day_key]}",
"daily_loss_limit": f"{self._daily_loss_limit}",
},
)
return
if (
self._cumulative_loss_limit is not None
and self._cumulative_pnl <= -self._cumulative_loss_limit
):
self._halted_reason = "cumulative_loss_limit_breached"
dispatch_alert_nowait(
self._alert_notifier,
category="threshold",
severity="critical",
title="Cumulative loss limit breached",
message=(
"Trading halted because cumulative realized PnL crossed "
"configured loss limit."
),
details={
"cumulative_pnl": f"{self._cumulative_pnl}",
"cumulative_loss_limit": f"{self._cumulative_loss_limit}",
},
)
+43
View File
@@ -0,0 +1,43 @@
from __future__ import annotations
from collections.abc import Mapping
class PreTradeValidator:
def __init__(
self,
*,
min_order_size_by_asset: Mapping[str, float] | None = None,
) -> None:
self._min_order_size_by_asset = {
asset.upper(): float(value) for asset, value in (min_order_size_by_asset or {}).items()
}
for value in self._min_order_size_by_asset.values():
if value <= 0.0:
raise ValueError("minimum order size must be > 0.0")
def validate(
self,
*,
balances_by_asset: Mapping[str, float],
required_by_asset: Mapping[str, float],
) -> bool:
# Minimum order size checks first to fail fast on structural invalid sizes.
for asset, required in required_by_asset.items():
if required <= 0.0:
continue
min_size = self._min_order_size_by_asset.get(asset.upper())
if min_size is not None and required < min_size:
return False
# Balance checks ensure required quantity is currently available.
for asset, required in required_by_asset.items():
if required <= 0.0:
continue
available = balances_by_asset.get(asset.upper(), 0.0)
if available < required:
return False
return True
+109
View File
@@ -0,0 +1,109 @@
from __future__ import annotations
from arbitrade.alerting.notifier import SupportsAlerts, dispatch_alert_nowait
class StopConditionsGuard:
def __init__(
self,
*,
max_source_latency_ms: float | None = None,
max_apply_latency_ms: float | None = None,
max_consecutive_failures: int | None = None,
alert_notifier: SupportsAlerts | None = None,
) -> None:
if max_source_latency_ms is not None and max_source_latency_ms <= 0.0:
raise ValueError("max_source_latency_ms must be > 0.0")
if max_apply_latency_ms is not None and max_apply_latency_ms <= 0.0:
raise ValueError("max_apply_latency_ms must be > 0.0")
if max_consecutive_failures is not None and max_consecutive_failures <= 0:
raise ValueError("max_consecutive_failures must be > 0")
self._max_source_latency_ms = max_source_latency_ms
self._max_apply_latency_ms = max_apply_latency_ms
self._max_consecutive_failures = max_consecutive_failures
self._consecutive_failures = 0
self._halted_reason: str | None = None
self._alert_notifier = alert_notifier
@property
def halted_reason(self) -> str | None:
return self._halted_reason
@property
def is_halted(self) -> bool:
return self._halted_reason is not None
@property
def consecutive_failures(self) -> int:
return self._consecutive_failures
def observe_latency(
self,
*,
source_latency_ms: float | None,
apply_latency_ms: float,
) -> None:
if self.is_halted:
return
if (
self._max_source_latency_ms is not None
and source_latency_ms is not None
and source_latency_ms > self._max_source_latency_ms
):
self._halted_reason = "source_latency_limit_breached"
dispatch_alert_nowait(
self._alert_notifier,
category="threshold",
severity="critical",
title="Source latency limit breached",
message="Trading halted because source latency exceeded configured limit.",
details={
"source_latency_ms": f"{source_latency_ms}",
"max_source_latency_ms": f"{self._max_source_latency_ms}",
},
)
return
if self._max_apply_latency_ms is not None and apply_latency_ms > self._max_apply_latency_ms:
self._halted_reason = "apply_latency_limit_breached"
dispatch_alert_nowait(
self._alert_notifier,
category="threshold",
severity="critical",
title="Apply latency limit breached",
message="Trading halted because apply latency exceeded configured limit.",
details={
"apply_latency_ms": f"{apply_latency_ms}",
"max_apply_latency_ms": f"{self._max_apply_latency_ms}",
},
)
def register_failure(self) -> None:
if self.is_halted:
return
self._consecutive_failures += 1
if (
self._max_consecutive_failures is not None
and self._consecutive_failures >= self._max_consecutive_failures
):
self._halted_reason = "consecutive_failures_limit_breached"
dispatch_alert_nowait(
self._alert_notifier,
category="threshold",
severity="critical",
title="Consecutive failures limit breached",
message="Trading halted because consecutive failures exceeded configured limit.",
details={
"consecutive_failures": f"{self._consecutive_failures}",
"max_consecutive_failures": f"{self._max_consecutive_failures}",
},
)
def register_success(self) -> None:
if self.is_halted:
return
self._consecutive_failures = 0
+98
View File
@@ -0,0 +1,98 @@
from __future__ import annotations
from collections.abc import Mapping
from arbitrade.alerting.notifier import SupportsAlerts, dispatch_alert_nowait
class TradeLimitsGuard:
def __init__(
self,
*,
max_concurrent_trades: int | None = None,
max_exposure_per_asset: float | None = None,
alert_notifier: SupportsAlerts | None = None,
) -> None:
if max_concurrent_trades is not None and max_concurrent_trades <= 0:
raise ValueError("max_concurrent_trades must be > 0")
if max_exposure_per_asset is not None and max_exposure_per_asset <= 0.0:
raise ValueError("max_exposure_per_asset must be > 0.0")
self._max_concurrent_trades = max_concurrent_trades
self._max_exposure_per_asset = max_exposure_per_asset
self._active_trades = 0
self._asset_exposure: dict[str, float] = {}
self._alert_notifier = alert_notifier
@property
def active_trades(self) -> int:
return self._active_trades
def exposure_for_asset(self, asset: str) -> float:
return self._asset_exposure.get(asset.upper(), 0.0)
def is_trade_allowed(self, exposure_by_asset: Mapping[str, float]) -> bool:
if (
self._max_concurrent_trades is not None
and self._active_trades >= self._max_concurrent_trades
):
dispatch_alert_nowait(
self._alert_notifier,
category="threshold",
severity="warning",
title="Concurrent trade limit reached",
message="Trade rejected by concurrent trade cap.",
details={
"active_trades": f"{self._active_trades}",
"max_concurrent_trades": f"{self._max_concurrent_trades}",
},
)
return False
if self._max_exposure_per_asset is None:
return True
for asset, exposure in exposure_by_asset.items():
if exposure <= 0.0:
continue
key = asset.upper()
next_exposure = self._asset_exposure.get(key, 0.0) + exposure
if next_exposure > self._max_exposure_per_asset:
dispatch_alert_nowait(
self._alert_notifier,
category="threshold",
severity="warning",
title="Asset exposure limit reached",
message="Trade rejected by per-asset exposure cap.",
details={
"asset": key,
"next_exposure": f"{next_exposure}",
"max_exposure_per_asset": f"{self._max_exposure_per_asset}",
},
)
return False
return True
def open_trade(self, exposure_by_asset: Mapping[str, float]) -> None:
self._active_trades += 1
for asset, exposure in exposure_by_asset.items():
if exposure <= 0.0:
continue
key = asset.upper()
self._asset_exposure[key] = self._asset_exposure.get(key, 0.0) + exposure
def close_trade(self, exposure_by_asset: Mapping[str, float]) -> None:
if self._active_trades > 0:
self._active_trades -= 1
for asset, exposure in exposure_by_asset.items():
if exposure <= 0.0:
continue
key = asset.upper()
current = self._asset_exposure.get(key, 0.0)
next_exposure = max(current - exposure, 0.0)
if next_exposure == 0.0:
self._asset_exposure.pop(key, None)
else:
self._asset_exposure[key] = next_exposure
+15
View File
@@ -0,0 +1,15 @@
"""Runtime lifecycle and recovery helpers."""
from arbitrade.runtime.lifecycle import (
RuntimeRecoveryReport,
graceful_shutdown,
persist_runtime_snapshot,
restore_runtime_state,
)
__all__ = [
"RuntimeRecoveryReport",
"graceful_shutdown",
"persist_runtime_snapshot",
"restore_runtime_state",
]
+224
View File
@@ -0,0 +1,224 @@
from __future__ import annotations
import inspect
from dataclasses import dataclass
from datetime import UTC, datetime
from typing import Any, cast
from fastapi import FastAPI
from arbitrade.api.control_state import DashboardControlState
from arbitrade.storage.db import DuckDBStore
from arbitrade.storage.repositories import (
AuditRecord,
AuditRepository,
RuntimeStateRecord,
RuntimeStateRepository,
)
@dataclass(slots=True)
class RuntimeRecoveryReport:
restored_from_snapshot: bool
snapshot_at: str | None
open_trades_detected: int
restart_guard_active: bool
def _controls(app: FastAPI) -> DashboardControlState:
return cast(DashboardControlState, app.state.dashboard_controls)
def _store(app: FastAPI) -> DuckDBStore:
return cast(DuckDBStore, app.state.store)
def _audit_repository(app: FastAPI) -> AuditRepository | None:
repository = getattr(app.state, "audit_repository", None)
return repository if isinstance(repository, AuditRepository) else None
def _runtime_repository(app: FastAPI) -> RuntimeStateRepository | None:
repository = getattr(app.state, "runtime_state_repository", None)
return repository if isinstance(repository, RuntimeStateRepository) else None
def _open_trade_count(store: DuckDBStore) -> int:
with store.connect() as conn:
row = conn.execute("""
SELECT COUNT(*)
FROM trades
WHERE finished_at IS NULL
""").fetchone()
return int(row[0]) if row is not None else 0
def _latest_balances(store: DuckDBStore) -> dict[str, Any] | None:
with store.connect() as conn:
row = conn.execute("""
SELECT balances
FROM portfolio_snapshots
ORDER BY snapshot_at DESC
LIMIT 1
""").fetchone()
if row is None or row[0] is None:
return None
raw_balances = row[0]
if isinstance(raw_balances, str):
return {"raw": raw_balances}
return {"raw": str(raw_balances)}
def _record_audit(
app: FastAPI,
*,
event_type: str,
decision: str,
payload: dict[str, Any] | None = None,
) -> None:
repository = _audit_repository(app)
if repository is None:
return
repository.insert(
AuditRecord(
occurred_at=datetime.now(UTC),
actor="runtime",
event_type=event_type,
decision=decision,
payload=payload,
correlation_id=None,
)
)
async def _run_startup_reconciler(app: FastAPI) -> None:
reconciler = getattr(app.state, "startup_reconciler", None)
if reconciler is None:
return
reconcile_member = getattr(reconciler, "reconcile_open_trades", None)
if reconcile_member is None or not callable(reconcile_member):
return
result = reconcile_member()
if inspect.isawaitable(result):
await result
def persist_runtime_snapshot(app: FastAPI, *, note: str | None = None) -> RuntimeStateRecord | None:
repository = _runtime_repository(app)
if repository is None:
return None
controls = _controls(app)
store = _store(app)
snapshot = RuntimeStateRecord(
snapshot_at=datetime.now(UTC),
is_running=controls.is_running,
kill_switch_active=controls.kill_switch.is_active,
kill_switch_reason=controls.kill_switch.reason,
open_trade_count=_open_trade_count(store),
last_known_balances=_latest_balances(store),
note=note,
)
repository.insert(snapshot)
return snapshot
async def restore_runtime_state(app: FastAPI) -> RuntimeRecoveryReport:
controls = _controls(app)
store = _store(app)
runtime_repository = _runtime_repository(app)
restored_from_snapshot = False
snapshot_at: str | None = None
latest = runtime_repository.latest() if runtime_repository is not None else None
if latest is not None:
restored_from_snapshot = True
snapshot_at = latest.snapshot_at.isoformat()
controls.is_running = latest.is_running
if latest.kill_switch_active:
controls.kill_switch.activate(
reason=latest.kill_switch_reason or "recovered")
else:
controls.kill_switch.deactivate()
controls.mark_updated()
open_trades = _open_trade_count(store)
restart_guard_active = False
if open_trades > 0:
controls.is_running = False
if not controls.kill_switch.is_active:
controls.kill_switch.activate(
reason="recovery_open_trades_detected")
controls.mark_updated()
restart_guard_active = True
report = RuntimeRecoveryReport(
restored_from_snapshot=restored_from_snapshot,
snapshot_at=snapshot_at,
open_trades_detected=open_trades,
restart_guard_active=restart_guard_active,
)
app.state.recovery_report = report
_record_audit(
app,
event_type="runtime.startup_recovery",
decision="applied",
payload={
"restored_from_snapshot": restored_from_snapshot,
"open_trades_detected": open_trades,
"restart_guard_active": restart_guard_active,
},
)
await _run_startup_reconciler(app)
return report
async def drain_background_workers(app: FastAPI) -> None:
workers: list[object] = []
declared = getattr(app.state, "background_workers", None)
if isinstance(declared, list):
workers.extend(declared)
for attr_name in ("execution_writer", "opportunity_writer", "snapshot_writer"):
worker = getattr(app.state, attr_name, None)
if worker is not None:
workers.append(worker)
seen: set[int] = set()
for worker in workers:
worker_id = id(worker)
if worker_id in seen:
continue
seen.add(worker_id)
stop_member = getattr(worker, "stop", None)
if stop_member is None or not callable(stop_member):
continue
result = stop_member()
if inspect.isawaitable(result):
await result
async def graceful_shutdown(app: FastAPI) -> None:
controls = _controls(app)
controls.is_running = False
controls.mark_updated()
_record_audit(
app,
event_type="runtime.shutdown",
decision="initiated",
payload={"execution_status": "stopped"},
)
await drain_background_workers(app)
persist_runtime_snapshot(app, note="graceful_shutdown")
+1
View File
@@ -0,0 +1 @@
"""Storage helpers."""
+71 -2
View File
@@ -5,9 +5,12 @@ from contextlib import contextmanager
from pathlib import Path
import duckdb
import structlog
from arbitrade.config.settings import Settings
_LOG = structlog.get_logger(__name__)
SCHEMA_SQL = """
CREATE TABLE IF NOT EXISTS schema_migrations (
version INTEGER PRIMARY KEY,
@@ -26,11 +29,40 @@ CREATE TABLE IF NOT EXISTS opportunities (
CREATE TABLE IF NOT EXISTS trades (
id UUID DEFAULT uuid(),
trade_ref VARCHAR NOT NULL,
started_at TIMESTAMP NOT NULL,
finished_at TIMESTAMP,
status VARCHAR NOT NULL,
realized_pnl DOUBLE,
capital_used DOUBLE
estimated_pnl DOUBLE,
capital_used DOUBLE,
cycle VARCHAR,
leg_count INTEGER
);
CREATE TABLE IF NOT EXISTS orders (
id UUID DEFAULT uuid(),
trade_ref VARCHAR NOT NULL,
order_ref VARCHAR NOT NULL,
leg_index INTEGER NOT NULL,
pair VARCHAR NOT NULL,
side VARCHAR NOT NULL,
volume DOUBLE NOT NULL,
user_ref INTEGER,
status VARCHAR,
filled_volume DOUBLE,
avg_price DOUBLE,
raw_response JSON,
recorded_at TIMESTAMP NOT NULL
);
CREATE TABLE IF NOT EXISTS pnl_events (
id UUID DEFAULT uuid(),
trade_ref VARCHAR NOT NULL,
recorded_at TIMESTAMP NOT NULL,
kind VARCHAR NOT NULL,
pnl_usd DOUBLE NOT NULL,
source VARCHAR NOT NULL
);
CREATE TABLE IF NOT EXISTS portfolio_snapshots (
@@ -38,6 +70,34 @@ CREATE TABLE IF NOT EXISTS portfolio_snapshots (
balances JSON,
total_value_usd DOUBLE
);
CREATE TABLE IF NOT EXISTS market_snapshots (
snapshot_at TIMESTAMP NOT NULL,
symbol VARCHAR NOT NULL,
source VARCHAR NOT NULL,
payload JSON NOT NULL,
latency_ms DOUBLE
);
CREATE TABLE IF NOT EXISTS audit_events (
id UUID DEFAULT uuid(),
occurred_at TIMESTAMP NOT NULL,
actor VARCHAR NOT NULL,
event_type VARCHAR NOT NULL,
decision VARCHAR NOT NULL,
payload JSON,
correlation_id VARCHAR
);
CREATE TABLE IF NOT EXISTS runtime_state_snapshots (
snapshot_at TIMESTAMP NOT NULL,
is_running BOOLEAN NOT NULL,
kill_switch_active BOOLEAN NOT NULL,
kill_switch_reason VARCHAR,
open_trade_count INTEGER NOT NULL,
last_known_balances JSON,
note VARCHAR
);
"""
@@ -45,10 +105,19 @@ class DuckDBStore:
def __init__(self, settings: Settings) -> None:
self._db_path = Path(settings.duckdb_path)
self._db_path.parent.mkdir(parents=True, exist_ok=True)
self._use_memory_fallback = False
@contextmanager
def connect(self) -> Iterator[duckdb.DuckDBPyConnection]:
conn = duckdb.connect(str(self._db_path))
try:
conn = duckdb.connect(str(self._db_path))
except duckdb.IOException:
if not self._use_memory_fallback:
_LOG.warning(
"duckdb_path_unavailable_falling_back_to_memory", path=str(self._db_path)
)
self._use_memory_fallback = True
conn = duckdb.connect(":memory:")
try:
yield conn
finally:
+66
View File
@@ -0,0 +1,66 @@
from __future__ import annotations
import asyncio
import structlog
from arbitrade.storage.repositories import (
OrderRecord,
OrderRepository,
PnLRecord,
PnLRepository,
TradeRecord,
TradeRepository,
)
_LOG = structlog.get_logger(__name__)
class AsyncExecutionWriter:
def __init__(
self,
trade_repository: TradeRepository,
order_repository: OrderRepository,
pnl_repository: PnLRepository,
max_queue_size: int = 50_000,
) -> None:
self._trade_repository = trade_repository
self._order_repository = order_repository
self._pnl_repository = pnl_repository
self._queue: asyncio.Queue[TradeRecord | OrderRecord | PnLRecord] = asyncio.Queue(
maxsize=max_queue_size
)
self._task: asyncio.Task[None] | None = None
self._stop = asyncio.Event()
async def start(self) -> None:
if self._task is None or self._task.done():
self._stop.clear()
self._task = asyncio.create_task(self._run(), name="execution-writer")
async def stop(self) -> None:
self._stop.set()
if self._task is not None:
await self._task
async def enqueue(self, record: TradeRecord | OrderRecord | PnLRecord) -> None:
await self._queue.put(record)
async def _run(self) -> None:
while not (self._stop.is_set() and self._queue.empty()):
try:
record = await asyncio.wait_for(self._queue.get(), timeout=0.5)
except TimeoutError:
continue
try:
if isinstance(record, TradeRecord):
self._trade_repository.insert(record)
elif isinstance(record, OrderRecord):
self._order_repository.insert(record)
else:
self._pnl_repository.insert(record)
except Exception as exc:
_LOG.error("execution_write_failed", error=str(exc))
finally:
self._queue.task_done()
+64
View File
@@ -0,0 +1,64 @@
from __future__ import annotations
import asyncio
from dataclasses import dataclass
from datetime import datetime
from typing import Any
import structlog
from arbitrade.storage.repositories import MarketSnapshotRecord, MarketSnapshotRepository
_LOG = structlog.get_logger(__name__)
@dataclass(slots=True)
class MarketSnapshot:
snapshot_at: datetime
symbol: str
source: str
payload: dict[str, Any]
latency_ms: float | None
class AsyncMarketSnapshotWriter:
def __init__(self, repository: MarketSnapshotRepository, max_queue_size: int = 50_000) -> None:
self._repository = repository
self._queue: asyncio.Queue[MarketSnapshot] = asyncio.Queue(maxsize=max_queue_size)
self._task: asyncio.Task[None] | None = None
self._stop = asyncio.Event()
async def start(self) -> None:
if self._task is None or self._task.done():
self._stop.clear()
self._task = asyncio.create_task(self._run(), name="market-snapshot-writer")
async def stop(self) -> None:
self._stop.set()
if self._task is not None:
await self._task
async def enqueue(self, snapshot: MarketSnapshot) -> None:
await self._queue.put(snapshot)
async def _run(self) -> None:
while not (self._stop.is_set() and self._queue.empty()):
try:
item = await asyncio.wait_for(self._queue.get(), timeout=0.5)
except TimeoutError:
continue
try:
self._repository.insert(
MarketSnapshotRecord(
snapshot_at=item.snapshot_at,
symbol=item.symbol,
source=item.source,
payload=item.payload,
latency_ms=item.latency_ms,
)
)
except Exception as exc:
_LOG.error("market_snapshot_write_failed", error=str(exc), symbol=item.symbol)
finally:
self._queue.task_done()
+58
View File
@@ -0,0 +1,58 @@
from __future__ import annotations
import asyncio
import structlog
from arbitrade.detection.engine import OpportunityEvent
from arbitrade.storage.repositories import OpportunityRecord, OpportunityRepository
_LOG = structlog.get_logger(__name__)
class AsyncOpportunityWriter:
def __init__(self, repository: OpportunityRepository, max_queue_size: int = 50_000) -> None:
self._repository = repository
self._queue: asyncio.Queue[OpportunityEvent] = asyncio.Queue(maxsize=max_queue_size)
self._task: asyncio.Task[None] | None = None
self._stop = asyncio.Event()
async def start(self) -> None:
if self._task is None or self._task.done():
self._stop.clear()
self._task = asyncio.create_task(self._run(), name="opportunity-writer")
async def stop(self) -> None:
self._stop.set()
if self._task is not None:
await self._task
async def enqueue(self, event: OpportunityEvent) -> None:
await self._queue.put(event)
async def _run(self) -> None:
while not (self._stop.is_set() and self._queue.empty()):
try:
event = await asyncio.wait_for(self._queue.get(), timeout=0.5)
except TimeoutError:
continue
try:
self._repository.insert(
OpportunityRecord(
detected_at=event.detected_at,
cycle=event.cycle,
gross_pct=event.gross_pct,
net_pct=event.net_pct,
est_profit=event.est_profit,
)
)
except Exception as exc:
_LOG.error(
"opportunity_write_failed",
error=str(exc),
cycle=event.cycle,
updated_pair=event.updated_pair,
)
finally:
self._queue.task_done()
+378
View File
@@ -0,0 +1,378 @@
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime
from typing import Any
import orjson
from arbitrade.storage.db import DuckDBStore
@dataclass(slots=True)
class MarketSnapshotRecord:
snapshot_at: datetime
symbol: str
source: str
payload: dict[str, Any]
latency_ms: float | None
@dataclass(slots=True)
class OpportunityRecord:
detected_at: datetime
cycle: str
gross_pct: float
net_pct: float
est_profit: float
executed: bool = False
@dataclass(slots=True)
class TradeRecord:
trade_ref: str
started_at: datetime
finished_at: datetime | None
status: str
realized_pnl: float | None
estimated_pnl: float | None
capital_used: float | None
cycle: str | None = None
leg_count: int | None = None
@dataclass(slots=True)
class OrderRecord:
trade_ref: str
order_ref: str
leg_index: int
pair: str
side: str
volume: float
user_ref: int | None
status: str | None
filled_volume: float | None
avg_price: float | None
raw_response: dict[str, Any]
recorded_at: datetime
@dataclass(slots=True)
class PnLRecord:
trade_ref: str
recorded_at: datetime
kind: str
pnl_usd: float
source: str
@dataclass(slots=True)
class AuditRecord:
occurred_at: datetime
actor: str
event_type: str
decision: str
payload: dict[str, Any] | None = None
correlation_id: str | None = None
@dataclass(slots=True)
class RuntimeStateRecord:
snapshot_at: datetime
is_running: bool
kill_switch_active: bool
kill_switch_reason: str | None
open_trade_count: int
last_known_balances: dict[str, Any] | None = None
note: str | None = None
class MarketSnapshotRepository:
def __init__(self, store: DuckDBStore) -> None:
self._store = store
def insert(self, record: MarketSnapshotRecord) -> None:
with self._store.connect() as conn:
conn.execute(
"""
INSERT INTO market_snapshots (snapshot_at, symbol, source, payload, latency_ms)
VALUES (?, ?, ?, ?, ?)
""",
[
record.snapshot_at,
record.symbol,
record.source,
orjson.dumps(record.payload).decode("utf-8"),
record.latency_ms,
],
)
class OpportunityRepository:
def __init__(self, store: DuckDBStore) -> None:
self._store = store
def insert(self, record: OpportunityRecord) -> None:
with self._store.connect() as conn:
conn.execute(
"""
INSERT INTO opportunities (
detected_at,
cycle,
gross_pct,
net_pct,
est_profit,
executed
)
VALUES (?, ?, ?, ?, ?, ?)
""",
[
record.detected_at,
record.cycle,
record.gross_pct,
record.net_pct,
record.est_profit,
record.executed,
],
)
class TradeRepository:
def __init__(self, store: DuckDBStore) -> None:
self._store = store
def insert(self, record: TradeRecord) -> None:
with self._store.connect() as conn:
conn.execute(
"""
INSERT INTO trades (
trade_ref,
started_at,
finished_at,
status,
realized_pnl,
estimated_pnl,
capital_used,
cycle,
leg_count
)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
[
record.trade_ref,
record.started_at,
record.finished_at,
record.status,
record.realized_pnl,
record.estimated_pnl,
record.capital_used,
record.cycle,
record.leg_count,
],
)
class OrderRepository:
def __init__(self, store: DuckDBStore) -> None:
self._store = store
def insert(self, record: OrderRecord) -> None:
with self._store.connect() as conn:
conn.execute(
"""
INSERT INTO orders (
trade_ref,
order_ref,
leg_index,
pair,
side,
volume,
user_ref,
status,
filled_volume,
avg_price,
raw_response,
recorded_at
)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
[
record.trade_ref,
record.order_ref,
record.leg_index,
record.pair,
record.side,
record.volume,
record.user_ref,
record.status,
record.filled_volume,
record.avg_price,
orjson.dumps(record.raw_response).decode("utf-8"),
record.recorded_at,
],
)
class PnLRepository:
def __init__(self, store: DuckDBStore) -> None:
self._store = store
def insert(self, record: PnLRecord) -> None:
with self._store.connect() as conn:
conn.execute(
"""
INSERT INTO pnl_events (
trade_ref,
recorded_at,
kind,
pnl_usd,
source
)
VALUES (?, ?, ?, ?, ?)
""",
[
record.trade_ref,
record.recorded_at,
record.kind,
record.pnl_usd,
record.source,
],
)
class AuditRepository:
def __init__(self, store: DuckDBStore) -> None:
self._store = store
def insert(self, record: AuditRecord) -> None:
with self._store.connect() as conn:
conn.execute(
"""
INSERT INTO audit_events (
occurred_at,
actor,
event_type,
decision,
payload,
correlation_id
)
VALUES (?, ?, ?, ?, ?, ?)
""",
[
record.occurred_at,
record.actor,
record.event_type,
record.decision,
(
None
if record.payload is None
else orjson.dumps(record.payload).decode("utf-8")
),
record.correlation_id,
],
)
def list_recent(self, *, limit: int = 25) -> list[AuditRecord]:
with self._store.connect() as conn:
rows = conn.execute(
"""
SELECT occurred_at, actor, event_type, decision, payload, correlation_id
FROM audit_events
ORDER BY occurred_at DESC
LIMIT ?
""",
[limit],
).fetchall()
records: list[AuditRecord] = []
for row in rows:
payload: dict[str, Any] | None = None
raw_payload = row[4]
if isinstance(raw_payload, str) and raw_payload:
decoded = orjson.loads(raw_payload)
if isinstance(decoded, dict):
payload = {str(k): decoded[k] for k in decoded}
records.append(
AuditRecord(
occurred_at=row[0],
actor=str(row[1]),
event_type=str(row[2]),
decision=str(row[3]),
payload=payload,
correlation_id=str(row[5]) if row[5] is not None else None,
)
)
return records
class RuntimeStateRepository:
def __init__(self, store: DuckDBStore) -> None:
self._store = store
def insert(self, record: RuntimeStateRecord) -> None:
with self._store.connect() as conn:
conn.execute(
"""
INSERT INTO runtime_state_snapshots (
snapshot_at,
is_running,
kill_switch_active,
kill_switch_reason,
open_trade_count,
last_known_balances,
note
)
VALUES (?, ?, ?, ?, ?, ?, ?)
""",
[
record.snapshot_at,
record.is_running,
record.kill_switch_active,
record.kill_switch_reason,
record.open_trade_count,
(
None
if record.last_known_balances is None
else orjson.dumps(record.last_known_balances).decode("utf-8")
),
record.note,
],
)
def latest(self) -> RuntimeStateRecord | None:
with self._store.connect() as conn:
row = conn.execute("""
SELECT
snapshot_at,
is_running,
kill_switch_active,
kill_switch_reason,
open_trade_count,
last_known_balances,
note
FROM runtime_state_snapshots
ORDER BY snapshot_at DESC
LIMIT 1
""").fetchone()
if row is None:
return None
balances: dict[str, Any] | None = None
raw_balances = row[5]
if isinstance(raw_balances, str) and raw_balances:
decoded = orjson.loads(raw_balances)
if isinstance(decoded, dict):
balances = {str(key): decoded[key] for key in decoded}
return RuntimeStateRecord(
snapshot_at=row[0],
is_running=bool(row[1]),
kill_switch_active=bool(row[2]),
kill_switch_reason=str(row[3]) if row[3] is not None else None,
open_trade_count=int(row[4]),
last_known_balances=balances,
note=str(row[6]) if row[6] is not None else None,
)
+321
View File
@@ -0,0 +1,321 @@
from __future__ import annotations
from datetime import UTC, datetime, timedelta
from typing import Any
import httpx
from arbitrade.api.app import create_app
from arbitrade.config.settings import Settings
class _FakeAlertNotifier:
def __init__(self) -> None:
self.events: list[dict[str, Any]] = []
async def notify(
self,
*,
category: str,
severity: str,
title: str,
message: str,
details: dict[str, str] | None = None,
) -> bool:
self.events.append(
{
"category": category,
"severity": severity,
"title": title,
"message": message,
"details": details or {},
}
)
return True
def _seed_metrics_data(app) -> None:
store = app.state.store
started = datetime.now(UTC)
finished = started + timedelta(seconds=20)
with store.connect() as conn:
conn.execute(
"""
INSERT INTO trades (
trade_ref,
started_at,
finished_at,
status,
realized_pnl,
estimated_pnl,
capital_used,
cycle,
leg_count
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
[
"trade-1",
started,
finished,
"filled",
15.0,
10.0,
100.0,
"USD->BTC->ETH->USD",
3,
],
)
conn.execute(
"""
INSERT INTO opportunities (
detected_at,
cycle,
gross_pct,
net_pct,
est_profit,
executed
) VALUES (?, ?, ?, ?, ?, ?)
""",
[started, "USD->BTC->ETH->USD", 4.0, 3.0, 0.03, True],
)
conn.execute(
"""
INSERT INTO orders (
trade_ref,
order_ref,
leg_index,
pair,
side,
volume,
user_ref,
status,
filled_volume,
avg_price,
raw_response,
recorded_at
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
[
"trade-1",
"order-1",
0,
"BTC/USD",
"buy",
2.0,
100,
"closed",
2.0,
100.0,
"{}",
started,
],
)
conn.execute(
"""
INSERT INTO portfolio_snapshots (
snapshot_at,
balances,
total_value_usd
) VALUES (?, ?, ?)
""",
[started, '{"USD": 1000.0, "BTC": 0.25}', 1250.0],
)
conn.execute(
"""
INSERT INTO trades (
trade_ref,
started_at,
finished_at,
status,
realized_pnl,
estimated_pnl,
capital_used,
cycle,
leg_count
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
[
"trade-open",
started,
None,
"open",
None,
5.0,
50.0,
"USD->BTC->ETH->USD",
3,
],
)
async def test_dashboard_page_and_fragment_and_sse(tmp_path) -> None:
app = create_app(Settings(DUCKDB_PATH=tmp_path / "dash.duckdb"))
_seed_metrics_data(app)
transport = httpx.ASGITransport(app=app)
async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
page = await client.get("/")
fragment = await client.get("/dashboard/fragment/metrics")
stream = await client.get("/dashboard/stream/metrics")
overview = await client.get("/dashboard/fragment/overview")
overview_stream = await client.get("/dashboard/stream/overview")
controls = await client.get("/dashboard/fragment/controls")
charts = await client.get("/dashboard/fragment/charts")
audit = await client.get("/dashboard/fragment/audit")
assert page.status_code == 200
assert "EventSource" in page.text
assert "alpinejs" in page.text.lower()
assert "Chart.js" in page.text or "chart.umd.min.js" in page.text
assert 'hx-get="/dashboard/fragment/metrics"' in page.text
assert 'hx-get="/dashboard/fragment/controls"' in page.text
assert 'hx-get="/dashboard/fragment/charts"' in page.text
assert 'hx-get="/dashboard/fragment/audit"' in page.text
assert fragment.status_code == 200
assert "Realized P&amp;L" in fragment.text
assert "15.00 USD" in fragment.text
assert "100.0%" in fragment.text
assert stream.status_code == 200
assert stream.headers["content-type"].startswith("text/event-stream")
assert "event: metrics" in stream.text
assert "Realized P&amp;L" in stream.text
assert overview.status_code == 200
assert "live" in overview.text
assert "Balances Snapshot" in overview.text
assert "Open Trades" in overview.text
assert "Opportunity Feed" in overview.text
assert "1250.00 USD" in overview.text
assert "trade-open" in overview.text
assert overview_stream.status_code == 200
assert overview_stream.headers["content-type"].startswith("text/event-stream")
assert "event: overview" in overview_stream.text
assert "trade-open" in overview_stream.text
assert controls.status_code == 200
assert "Runtime Status" in controls.text
assert ">running<" in controls.text
assert "Alerting" in controls.text
assert "Last result" in controls.text
assert "Paper trading mode" in controls.text
assert "Trade capital USD" in controls.text
assert charts.status_code == 200
assert "Opportunity Trend" in charts.text
assert "opportunity-chart" in charts.text
assert "Hide chart" in charts.text or "Show chart" in charts.text
assert audit.status_code == 200
assert "Audit Trail" in audit.text
async def test_dashboard_controls_update_runtime_state_and_config(tmp_path) -> None:
app = create_app(Settings(DUCKDB_PATH=tmp_path / "controls.duckdb"))
transport = httpx.ASGITransport(app=app)
async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
stop_response = await client.post("/dashboard/control/stop")
start_response = await client.post("/dashboard/control/start")
kill_response = await client.post(
"/dashboard/control/kill-switch",
data={"reason": "manual"},
)
config_response = await client.post(
"/dashboard/control/config",
data={
"trade_capital_usd": "250.50",
"max_trade_capital_usd": "300.00",
"max_concurrent_trades": "4",
"paper_trading_mode": "on",
},
)
assert stop_response.status_code == 200
assert ">stopped<" in stop_response.text
assert start_response.status_code == 200
assert ">running<" in start_response.text
assert kill_response.status_code == 200
assert ">active<" in kill_response.text
assert "manual" in kill_response.text
assert config_response.status_code == 200
assert "250.50 USD" in config_response.text
assert "300.00 USD" in config_response.text
assert "4" in config_response.text
assert app.state.settings.trade_capital_usd == 250.5
assert app.state.settings.max_trade_capital_usd == 300.0
assert app.state.settings.max_concurrent_trades == 4
assert app.state.settings.paper_trading_mode is True
transport = httpx.ASGITransport(app=app)
async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
audit_recent = await client.get("/dashboard/api/audit/recent")
assert audit_recent.status_code == 200
entries = audit_recent.json()["entries"]
assert len(entries) >= 4
assert any(entry["event_type"] == "dashboard.control.stop" for entry in entries)
assert any(entry["event_type"] == "dashboard.control.start" for entry in entries)
assert any(entry["event_type"] == "dashboard.control.kill_switch" for entry in entries)
assert any(entry["event_type"] == "dashboard.control.config" for entry in entries)
async def test_dashboard_controls_emit_alerts(tmp_path) -> None:
app = create_app(Settings(DUCKDB_PATH=tmp_path / "alerts.duckdb"))
fake_notifier = _FakeAlertNotifier()
app.state.alert_notifier = fake_notifier
transport = httpx.ASGITransport(app=app)
async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
await client.post("/dashboard/control/start")
await client.post("/dashboard/control/stop")
await client.post("/dashboard/control/kill-switch", data={"reason": "manual-test"})
assert len(fake_notifier.events) == 3
assert fake_notifier.events[0]["title"] == "Execution started"
assert fake_notifier.events[1]["title"] == "Execution stopped"
assert fake_notifier.events[2]["title"] == "Kill switch activated"
assert fake_notifier.events[2]["details"]["reason"] == "manual-test"
async def test_dashboard_requires_basic_auth_when_configured(tmp_path) -> None:
app = create_app(
Settings(
DUCKDB_PATH=tmp_path / "auth.duckdb",
DASHBOARD_AUTH_USERNAME="admin",
DASHBOARD_AUTH_PASSWORD="secret",
)
)
transport = httpx.ASGITransport(app=app)
async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
unauthenticated = await client.get("/dashboard/fragment/overview")
authenticated = await client.get(
"/dashboard/fragment/overview",
auth=("admin", "secret"),
)
health = await client.get("/health")
assert unauthenticated.status_code == 401
assert unauthenticated.headers["www-authenticate"] == 'Basic realm="Arbitrade Dashboard"'
assert authenticated.status_code == 200
assert health.status_code == 200
async def test_dashboard_alert_status_api_exposes_notifier_snapshot(tmp_path) -> None:
app = create_app(Settings(DUCKDB_PATH=tmp_path / "alerts-status.duckdb"))
transport = httpx.ASGITransport(app=app)
async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
response = await client.get("/dashboard/api/alerts/status")
assert response.status_code == 200
payload = response.json()
assert payload["enabled"] is True
assert "configured_channels" in payload
assert "last_result" in payload
+131
View File
@@ -0,0 +1,131 @@
from __future__ import annotations
from dataclasses import dataclass, field
import pytest
from arbitrade.alerting.notifier import AlertEvent, AlertNotifier
@dataclass(slots=True)
class _FakeChannel:
events: list[AlertEvent] = field(default_factory=list)
fail: bool = False
async def send(self, event: AlertEvent) -> None:
if self.fail:
raise RuntimeError("channel send failed")
self.events.append(event)
@pytest.mark.asyncio
async def test_alert_notifier_sends_event_when_enabled() -> None:
channel = _FakeChannel()
notifier = AlertNotifier([channel], enabled=True, min_severity="info")
sent = await notifier.notify(
category="trade",
severity="info",
title="Trade complete",
message="Completed all legs.",
)
assert sent is True
assert len(channel.events) == 1
assert channel.events[0].category == "trade"
@pytest.mark.asyncio
async def test_alert_notifier_respects_severity_and_category_filters() -> None:
channel = _FakeChannel()
notifier = AlertNotifier(
[channel],
enabled=True,
min_severity="error",
category_flags={"trade": False, "error": True},
)
low = await notifier.notify(
category="error",
severity="warning",
title="Low",
message="Ignored by severity.",
)
filtered = await notifier.notify(
category="trade",
severity="critical",
title="Trade",
message="Ignored by category.",
)
high = await notifier.notify(
category="error",
severity="critical",
title="High",
message="Delivered.",
)
assert low is False
assert filtered is False
assert high is True
assert len(channel.events) == 1
assert channel.events[0].title == "High"
@pytest.mark.asyncio
async def test_alert_notifier_applies_dedup_window() -> None:
channel = _FakeChannel()
notifier = AlertNotifier([channel], dedup_seconds=60.0)
first = await notifier.notify(
category="error",
severity="error",
title="Burst",
message="Same message",
)
second = await notifier.notify(
category="error",
severity="error",
title="Burst",
message="Same message",
)
assert first is True
assert second is False
assert len(channel.events) == 1
@pytest.mark.asyncio
async def test_alert_notifier_returns_false_when_all_channels_fail() -> None:
notifier = AlertNotifier([_FakeChannel(fail=True), _FakeChannel(fail=True)])
sent = await notifier.notify(
category="error",
severity="critical",
title="Failure",
message="Both channels fail.",
)
assert sent is False
@pytest.mark.asyncio
async def test_alert_notifier_exposes_status_snapshot_for_dashboard() -> None:
channel = _FakeChannel()
notifier = AlertNotifier([channel], enabled=True, min_severity="info", dedup_seconds=30.0)
await notifier.notify(
category="system",
severity="warning",
title="Reconnect",
message="Socket restored.",
)
status = notifier.status_snapshot()
assert status["enabled"] is True
assert status["has_channels"] is True
assert status["configured_channels"] == ["_FakeChannel"]
assert status["last_result"] == "success"
assert status["last_attempted_at"] is not None
assert status["last_success_at"] is not None
assert status["last_event"] is not None
+34
View File
@@ -0,0 +1,34 @@
from __future__ import annotations
from datetime import UTC, datetime
from arbitrade.config.settings import Settings
from arbitrade.storage.db import DuckDBStore
from arbitrade.storage.repositories import AuditRecord, AuditRepository
def test_audit_repository_inserts_and_lists_recent(tmp_path) -> None:
settings = Settings(_env_file=None, DUCKDB_PATH=tmp_path / "audit.duckdb")
store = DuckDBStore(settings)
store.migrate()
repository = AuditRepository(store)
repository.insert(
AuditRecord(
occurred_at=datetime.now(UTC),
actor="dashboard_user",
event_type="dashboard.control.start",
decision="approved",
payload={"execution_status": "running"},
correlation_id="req-1",
)
)
recent = repository.list_recent(limit=5)
assert len(recent) == 1
assert recent[0].actor == "dashboard_user"
assert recent[0].event_type == "dashboard.control.start"
assert recent[0].decision == "approved"
assert recent[0].payload == {"execution_status": "running"}
assert recent[0].correlation_id == "req-1"
+48
View File
@@ -0,0 +1,48 @@
from arbitrade.detection.graph import CurrencyGraph
def test_currency_graph_from_kraken_pairs_builds_adjacency() -> None:
asset_pairs = {
"XXBTZUSD": {"wsname": "BTC/USD"},
"XETHXXBT": {"wsname": "ETH/BTC"},
"XETHZUSD": {"wsname": "ETH/USD"},
}
graph = CurrencyGraph.from_kraken_asset_pairs(asset_pairs)
assert "USD" in graph.adjacency
assert "BTC" in graph.adjacency["USD"]
assert "ETH" in graph.adjacency["USD"]
def test_triangular_cycles_detected_once() -> None:
asset_pairs = {
"XXBTZUSD": {"wsname": "BTC/USD"},
"XETHXXBT": {"wsname": "ETH/BTC"},
"XETHZUSD": {"wsname": "ETH/USD"},
}
graph = CurrencyGraph.from_kraken_asset_pairs(asset_pairs)
cycles = graph.triangular_cycles()
assert len(cycles) == 1
cycle = cycles[0]
assert cycle.currencies == ("BTC", "ETH", "USD")
assert set(cycle.pairs) == {"BTC/USD", "ETH/BTC", "ETH/USD"}
def test_cycles_indexed_by_pair() -> None:
asset_pairs = {
"XXBTZUSD": {"wsname": "BTC/USD"},
"XETHXXBT": {"wsname": "ETH/BTC"},
"XETHZUSD": {"wsname": "ETH/USD"},
}
graph = CurrencyGraph.from_kraken_asset_pairs(asset_pairs)
cycles = graph.triangular_cycles()
index = graph.index_cycles_by_pair(cycles)
assert "BTC/USD" in index
assert "ETH/BTC" in index
assert "ETH/USD" in index
assert len(index["BTC/USD"]) == 1
+19
View File
@@ -0,0 +1,19 @@
import pytest
from arbitrade.detection.benchmark import run_incremental_detection_benchmark
def test_incremental_detection_benchmark_returns_metrics() -> None:
result = run_incremental_detection_benchmark(iterations=500)
assert result.iterations == 500
assert result.total_ms > 0.0
assert result.avg_ms > 0.0
assert result.p50_ms > 0.0
assert result.p95_ms > 0.0
assert result.max_ms >= result.p95_ms
def test_incremental_detection_benchmark_rejects_invalid_iterations() -> None:
with pytest.raises(ValueError, match="iterations"):
run_incremental_detection_benchmark(iterations=0)
@@ -0,0 +1,117 @@
from __future__ import annotations
import pytest
from arbitrade.detection.engine import IncrementalCycleDetector
from arbitrade.detection.graph import CurrencyGraph
from arbitrade.exchange.models import BookLevel
from arbitrade.market_data.order_book import OrderBook
def _make_book(*, bid: float, ask: float) -> OrderBook:
book = OrderBook()
book.apply_bids([BookLevel(price=bid, volume=2_000.0)])
book.apply_asks([BookLevel(price=ask, volume=2_000.0)])
return book
def _make_book_levels(
*, bids: list[tuple[float, float]], asks: list[tuple[float, float]]
) -> OrderBook:
book = OrderBook()
book.apply_bids([BookLevel(price=price, volume=volume) for price, volume in bids])
book.apply_asks([BookLevel(price=price, volume=volume) for price, volume in asks])
return book
def test_synthetic_single_cycle_known_exact_rates() -> None:
asset_pairs = {
"XXBTZUSD": {"wsname": "BTC/USD"},
"XETHXXBT": {"wsname": "ETH/BTC"},
"XETHZUSD": {"wsname": "ETH/USD"},
}
graph = CurrencyGraph.from_kraken_asset_pairs(asset_pairs)
detector = IncrementalCycleDetector(
graph.index_cycles_by_pair(graph.triangular_cycles()),
fee_rate=0.001,
max_depth_levels=1,
)
books = {
"BTC/USD": _make_book(bid=99.9, ask=100.0),
"ETH/BTC": _make_book(bid=0.049, ask=0.05),
"ETH/USD": _make_book(bid=5.20, ask=5.21),
}
scores = detector.score_updated_pair("ETH/BTC", books)
assert len(scores) == 1
score = scores[0]
# Known path result: 1 USD -> 0.01 BTC -> 0.2 ETH -> 1.04 USD gross.
assert score.gross_rate == pytest.approx(1.04)
assert score.net_rate == pytest.approx(1.04 * (1 - 0.001) ** 3)
def test_synthetic_two_cycles_known_filtering_and_pair_index() -> None:
asset_pairs = {
"XXBTZUSD": {"wsname": "BTC/USD"},
"XETHXXBT": {"wsname": "ETH/BTC"},
"XETHZUSD": {"wsname": "ETH/USD"},
"XLTCXXBT": {"wsname": "LTC/BTC"},
"XLTCZUSD": {"wsname": "LTC/USD"},
}
graph = CurrencyGraph.from_kraken_asset_pairs(asset_pairs)
detector = IncrementalCycleDetector(
graph.index_cycles_by_pair(graph.triangular_cycles()),
min_profit_threshold=0.02,
max_depth_levels=3,
)
books = {
# ETH cycle expected to pass threshold (~4% gross)
"BTC/USD": _make_book(bid=99.9, ask=100.0),
"ETH/BTC": _make_book(bid=0.049, ask=0.05),
"ETH/USD": _make_book(bid=5.20, ask=5.21),
# LTC cycle expected to fail threshold (<2% gross)
"LTC/BTC": _make_book(bid=0.01005, ask=0.0101),
"LTC/USD": _make_book(bid=1.01, ask=1.011),
}
scores = detector.score_updated_pair("BTC/USD", books)
assert len(scores) == 1
assert scores[0].cycle.currencies == ("BTC", "ETH", "USD")
assert (scores[0].net_rate - 1.0) >= 0.02
def test_synthetic_depth_scenario_known_no_result_when_not_fillable() -> None:
asset_pairs = {
"XXBTZUSD": {"wsname": "BTC/USD"},
"XETHXXBT": {"wsname": "ETH/BTC"},
"XETHZUSD": {"wsname": "ETH/USD"},
}
graph = CurrencyGraph.from_kraken_asset_pairs(asset_pairs)
detector = IncrementalCycleDetector(
graph.index_cycles_by_pair(graph.triangular_cycles()),
max_depth_levels=2,
)
books = {
# Not enough ask-side BTC/USD capacity inside top-2 levels for initial USD->BTC conversion.
"BTC/USD": _make_book_levels(
bids=[(99.9, 10.0)],
asks=[(100.0, 0.003), (101.0, 0.003)],
),
"ETH/BTC": _make_book_levels(
bids=[(0.049, 10.0)],
asks=[(0.05, 10.0)],
),
"ETH/USD": _make_book_levels(
bids=[(5.20, 10.0)],
asks=[(5.21, 10.0)],
),
}
scores = detector.score_updated_pair("BTC/USD", books)
assert scores == []
+89
View File
@@ -0,0 +1,89 @@
from __future__ import annotations
from dataclasses import dataclass
from datetime import UTC, datetime
import pytest
from arbitrade.config.settings import Settings
from arbitrade.detection.engine import OpportunityEvent
from arbitrade.execution.sequencer import TriangularExecutionSequencer
from arbitrade.storage.db import DuckDBStore
from arbitrade.storage.executions import AsyncExecutionWriter
from arbitrade.storage.repositories import OrderRepository, PnLRepository, TradeRepository
@dataclass(slots=True)
class _FakeRestClient:
calls: int = 0
async def place_market_order(self, *, pair: str, side: str, volume: float) -> dict[str, object]:
self.calls += 1
return {"txid": [f"tx-{self.calls}"], "status": "submitted"}
def _sample_event() -> OpportunityEvent:
return OpportunityEvent(
detected_at=datetime.now(UTC),
cycle="USD->BTC->ETH->USD",
updated_pair="BTC/USD",
gross_rate=1.04,
net_rate=1.03,
gross_pct=4.0,
net_pct=3.0,
est_profit=0.03,
)
@pytest.mark.asyncio
async def test_execution_writer_persists_trade_order_and_pnl(tmp_path) -> None:
settings = Settings(_env_file=None, DUCKDB_PATH=tmp_path / "exec.duckdb")
store = DuckDBStore(settings)
store.migrate()
writer = AsyncExecutionWriter(
TradeRepository(store),
OrderRepository(store),
PnLRepository(store),
max_queue_size=10,
)
await writer.start()
client = _FakeRestClient()
sequencer = TriangularExecutionSequencer(
client,
available_pairs=["BTC/USD", "ETH/BTC", "ETH/USD"],
execution_writer=writer,
)
result = await sequencer.execute(_sample_event())
await writer.stop()
assert result.success
assert client.calls == 3
with store.connect() as conn:
trades = conn.execute(
"SELECT trade_ref, status, estimated_pnl, capital_used, cycle, leg_count FROM trades"
).fetchall()
orders = conn.execute(
"SELECT trade_ref, order_ref, leg_index, pair, side, volume, status "
"FROM orders ORDER BY leg_index"
).fetchall()
pnls = conn.execute("SELECT trade_ref, kind, pnl_usd, source FROM pnl_events").fetchall()
assert len(trades) == 1
assert trades[0][1] == "filled"
assert trades[0][2] == 0.03
assert trades[0][3] == 1.0
assert trades[0][4] == "USD->BTC->ETH->USD"
assert trades[0][5] == 3
assert len(orders) == 3
assert orders[0][2] == 0
assert orders[1][2] == 1
assert orders[2][2] == 2
assert orders[0][6] == "submitted"
assert len(pnls) == 1
assert pnls[0][1] == "estimated"
assert pnls[0][2] == 0.03
+128
View File
@@ -0,0 +1,128 @@
from __future__ import annotations
from dataclasses import dataclass, field
from datetime import UTC, datetime
from typing import Any
import pytest
from arbitrade.detection.engine import OpportunityEvent
from arbitrade.execution.sequencer import TriangularExecutionSequencer
@dataclass(slots=True)
class _FakeAlertNotifier:
events: list[dict[str, str]] = field(default_factory=list)
async def notify(
self,
*,
category: str,
severity: str,
title: str,
message: str,
details: dict[str, str] | None = None,
) -> bool:
self.events.append(
{
"category": category,
"severity": severity,
"title": title,
"message": message,
**(details or {}),
}
)
return True
@dataclass(slots=True)
class _FakeRestClient:
fail_at_call: int | None = None
calls: list[dict[str, Any]] = field(default_factory=list)
async def place_market_order(self, *, pair: str, side: str, volume: float) -> dict[str, Any]:
call_number = len(self.calls) + 1
if self.fail_at_call is not None and call_number == self.fail_at_call:
raise RuntimeError("simulated failure")
payload = {"pair": pair, "side": side, "volume": volume}
self.calls.append(payload)
return {"txid": [f"tx-{call_number}"]}
def _sample_event(cycle: str = "USD->BTC->ETH->USD") -> OpportunityEvent:
return OpportunityEvent(
detected_at=datetime.now(UTC),
cycle=cycle,
updated_pair="BTC/USD",
gross_rate=1.02,
net_rate=1.01,
gross_pct=2.0,
net_pct=1.0,
est_profit=1.0,
allocated_capital=10.0,
)
@pytest.mark.asyncio
async def test_triangular_sequencer_executes_legs_in_order() -> None:
client = _FakeRestClient()
notifier = _FakeAlertNotifier()
sequencer = TriangularExecutionSequencer(
client,
available_pairs=["BTC/USD", "ETH/BTC", "ETH/USD"],
alert_notifier=notifier,
)
result = await sequencer.execute(_sample_event())
assert result.success
assert result.completed_legs == 3
assert [call["pair"] for call in client.calls] == ["BTC/USD", "ETH/BTC", "ETH/USD"]
assert [call["side"] for call in client.calls] == ["buy", "buy", "sell"]
assert len(notifier.events) == 1
assert notifier.events[0]["category"] == "trade"
assert notifier.events[0]["title"] == "Trade execution completed"
@pytest.mark.asyncio
async def test_triangular_sequencer_stops_on_failed_leg() -> None:
client = _FakeRestClient(fail_at_call=2)
notifier = _FakeAlertNotifier()
sequencer = TriangularExecutionSequencer(
client,
available_pairs=["BTC/USD", "ETH/BTC", "ETH/USD"],
alert_notifier=notifier,
)
result = await sequencer.execute(_sample_event())
assert not result.success
assert result.completed_legs == 1
assert result.failure_reason is not None
assert len(client.calls) == 1
assert len(notifier.events) == 1
assert notifier.events[0]["category"] == "error"
assert notifier.events[0]["title"] == "Trade execution failed"
def test_triangular_sequencer_rejects_non_closed_cycle() -> None:
client = _FakeRestClient()
sequencer = TriangularExecutionSequencer(
client,
available_pairs=["BTC/USD", "ETH/BTC", "ETH/USD"],
)
with pytest.raises(ValueError, match="closed triangular path"):
sequencer._build_legs(_sample_event(cycle="USD->BTC->ETH"))
def test_triangular_sequencer_rejects_missing_pair() -> None:
client = _FakeRestClient()
sequencer = TriangularExecutionSequencer(
client,
available_pairs=["BTC/USD", "ETH/BTC"],
)
with pytest.raises(ValueError, match="No tradable pair"):
sequencer._build_legs(_sample_event())
+112
View File
@@ -0,0 +1,112 @@
from __future__ import annotations
from dataclasses import dataclass, field
from datetime import UTC, datetime
from typing import Any
import pytest
from arbitrade.execution.fill_monitor import FillMonitor, OrderFillState
@dataclass(slots=True)
class _FakePollClient:
responses: list[dict[str, Any]]
calls: int = 0
async def query_order(self, *, order_id: str, include_trades: bool = True) -> dict[str, Any]:
self.calls += 1
if self.responses:
return self.responses.pop(0)
return {order_id: {"status": "open", "vol_exec": "0.0", "price": "0.0"}}
@dataclass(slots=True)
class _FakeWsProvider:
states: list[OrderFillState] = field(default_factory=list)
def get(self, _order_id: str) -> OrderFillState | None:
if not self.states:
return None
return self.states.pop(0)
@pytest.mark.asyncio
async def test_fill_monitor_detects_terminal_state_via_polling() -> None:
order_id = "order-1"
client = _FakePollClient(
responses=[
{order_id: {"status": "open", "vol_exec": "0.0", "price": "0.0"}},
{order_id: {"status": "closed", "vol_exec": "1.0", "price": "100.0"}},
]
)
monitor = FillMonitor(client, poll_interval_seconds=0.001, max_wait_seconds=0.1)
result = await monitor.wait_for_terminal_fill(order_id)
assert not result.timed_out
assert result.terminal_state is not None
assert result.terminal_state.status == "closed"
assert result.terminal_state.filled_volume == 1.0
assert result.terminal_state.source == "rest_poll"
@pytest.mark.asyncio
async def test_fill_monitor_times_out_when_no_terminal_state() -> None:
order_id = "order-2"
client = _FakePollClient(
responses=[
{order_id: {"status": "open", "vol_exec": "0.1", "price": "100.0"}},
{order_id: {"status": "partial", "vol_exec": "0.2", "price": "100.0"}},
{order_id: {"status": "open", "vol_exec": "0.2", "price": "100.0"}},
]
)
monitor = FillMonitor(client, poll_interval_seconds=0.001, max_wait_seconds=0.01)
result = await monitor.wait_for_terminal_fill(order_id)
assert result.timed_out
assert result.terminal_state is None
assert result.last_state is not None
assert result.last_state.status in {"open", "partial"}
@pytest.mark.asyncio
async def test_fill_monitor_uses_ws_status_for_fast_terminal_detection() -> None:
order_id = "order-3"
ws_provider = _FakeWsProvider(
states=[
OrderFillState(
order_id=order_id,
status="closed",
filled_volume=0.5,
avg_price=200.0,
updated_at=datetime.now(UTC),
source="ws",
)
]
)
client = _FakePollClient(responses=[])
monitor = FillMonitor(
client,
poll_interval_seconds=0.001,
max_wait_seconds=0.1,
ws_status_provider=ws_provider.get,
)
result = await monitor.wait_for_terminal_fill(order_id)
assert not result.timed_out
assert result.terminal_state is not None
assert result.terminal_state.source == "ws"
assert client.calls == 0
def test_fill_monitor_rejects_invalid_configuration() -> None:
client = _FakePollClient(responses=[])
with pytest.raises(ValueError, match="poll_interval_seconds"):
FillMonitor(client, poll_interval_seconds=0.0)
with pytest.raises(ValueError, match="max_wait_seconds"):
FillMonitor(client, max_wait_seconds=0.0)
+109
View File
@@ -0,0 +1,109 @@
from __future__ import annotations
from dataclasses import dataclass
from datetime import UTC, datetime
from typing import Any
import pytest
from arbitrade.detection.engine import OpportunityEvent
from arbitrade.execution.idempotency import IdempotencyKeyFactory, OrderReconciler
from arbitrade.execution.sequencer import ExecutionLeg
@dataclass(slots=True)
class _FakeHistoryClient:
response: dict[str, Any]
async def query_order(self, *, order_id: str, include_trades: bool = True) -> dict[str, Any]:
return self.response
def _sample_event() -> OpportunityEvent:
return OpportunityEvent(
detected_at=datetime.now(UTC),
cycle="USD->BTC->ETH->USD",
updated_pair="BTC/USD",
gross_rate=1.02,
net_rate=1.01,
gross_pct=2.0,
net_pct=1.0,
est_profit=1.0,
allocated_capital=10.0,
)
def test_idempotency_key_factory_is_deterministic() -> None:
factory = IdempotencyKeyFactory()
event = _sample_event()
leg = ExecutionLeg(
from_currency="USD",
to_currency="BTC",
pair="BTC/USD",
side="buy",
volume=10.0,
)
first = factory.user_ref_for_leg(event, leg, 0)
second = factory.user_ref_for_leg(event, leg, 0)
assert first == second
assert first > 0
@pytest.mark.asyncio
async def test_order_reconciler_maps_query_response_to_report() -> None:
client = _FakeHistoryClient(
response={
"order-1": {
"status": "closed",
"vol_exec": "5.0",
"price": "100.0",
"pair": "BTC/USD",
"type": "buy",
"userref": 12345,
}
}
)
reconciler = OrderReconciler(client)
report = await reconciler.reconcile_order(
order_id="order-1",
user_ref=12345,
expected_pair="BTC/USD",
expected_side="buy",
expected_volume=10.0,
)
assert report.is_terminal
assert report.matches_request
assert report.status == "closed"
assert report.filled_volume == 5.0
assert report.avg_price == 100.0
@pytest.mark.asyncio
async def test_order_reconciler_marks_mismatch() -> None:
client = _FakeHistoryClient(
response={
"order-1": {
"status": "closed",
"vol_exec": "5.0",
"price": "100.0",
"pair": "ETH/USD",
"type": "sell",
"userref": 999,
}
}
)
reconciler = OrderReconciler(client)
report = await reconciler.reconcile_order(
order_id="order-1",
user_ref=12345,
expected_pair="BTC/USD",
expected_side="buy",
expected_volume=10.0,
)
assert not report.matches_request
+306
View File
@@ -0,0 +1,306 @@
from datetime import UTC, datetime, timedelta
import pytest
from arbitrade.detection.engine import IncrementalCycleDetector
from arbitrade.detection.graph import CurrencyGraph, TriangularCycle
from arbitrade.exchange.models import BookLevel
from arbitrade.market_data.order_book import OrderBook
def _make_book(*, bid: float, ask: float) -> OrderBook:
book = OrderBook()
book.apply_bids([BookLevel(price=bid, volume=1.0)])
book.apply_asks([BookLevel(price=ask, volume=1.0)])
return book
def _make_book_levels(
*, bids: list[tuple[float, float]], asks: list[tuple[float, float]]
) -> OrderBook:
book = OrderBook()
book.apply_bids([BookLevel(price=price, volume=volume) for price, volume in bids])
book.apply_asks([BookLevel(price=price, volume=volume) for price, volume in asks])
return book
def test_incremental_detector_scores_only_cycles_touched_by_pair() -> None:
cycle_a = TriangularCycle(
currencies=("USD", "BTC", "ETH"),
pairs=("BTC/USD", "ETH/BTC", "ETH/USD"),
)
cycle_b = TriangularCycle(
currencies=("USD", "BTC", "LTC"),
pairs=("BTC/USD", "LTC/BTC", "LTC/USD"),
)
cycle_c = TriangularCycle(
currencies=("USD", "SOL", "ADA"),
pairs=("SOL/USD", "ADA/SOL", "ADA/USD"),
)
cycles = [cycle_a, cycle_b, cycle_c]
index = CurrencyGraph.index_cycles_by_pair(cycles)
detector = IncrementalCycleDetector(index)
books = {
"BTC/USD": _make_book(bid=100.0, ask=100.0),
"ETH/BTC": _make_book(bid=0.05, ask=0.05),
"ETH/USD": _make_book(bid=5.20, ask=5.21),
"LTC/BTC": _make_book(bid=0.01, ask=0.01),
"LTC/USD": _make_book(bid=1.02, ask=1.03),
"SOL/USD": _make_book(bid=20.0, ask=20.1),
"ADA/SOL": _make_book(bid=0.02, ask=0.021),
"ADA/USD": _make_book(bid=0.42, ask=0.43),
}
scores = detector.score_updated_pair("BTC/USD", books)
assert len(scores) == 2
assert {score.cycle for score in scores} == {cycle_a, cycle_b}
def test_incremental_detector_uses_best_bid_ask_for_gross_rate() -> None:
cycle = TriangularCycle(
currencies=("USD", "BTC", "ETH"),
pairs=("BTC/USD", "ETH/BTC", "ETH/USD"),
)
detector = IncrementalCycleDetector(CurrencyGraph.index_cycles_by_pair([cycle]))
books = {
"BTC/USD": _make_book(bid=99.9, ask=100.0),
"ETH/BTC": _make_book(bid=0.049, ask=0.05),
"ETH/USD": _make_book(bid=5.20, ask=5.21),
}
scores = detector.score_updated_pair("ETH/BTC", books)
assert len(scores) == 1
assert scores[0].gross_rate == 1.04
assert scores[0].net_rate == 1.04
assert scores[0].is_profitable
def test_incremental_detector_applies_fees_to_net_rate() -> None:
cycle = TriangularCycle(
currencies=("USD", "BTC", "ETH"),
pairs=("BTC/USD", "ETH/BTC", "ETH/USD"),
)
detector = IncrementalCycleDetector(
CurrencyGraph.index_cycles_by_pair([cycle]),
fee_rate=0.001,
)
books = {
"BTC/USD": _make_book(bid=99.9, ask=100.0),
"ETH/BTC": _make_book(bid=0.049, ask=0.05),
"ETH/USD": _make_book(bid=5.20, ask=5.21),
}
scores = detector.score_updated_pair("ETH/BTC", books)
assert len(scores) == 1
assert scores[0].gross_rate == 1.04
assert scores[0].net_rate < scores[0].gross_rate
def test_incremental_detector_uses_depth_and_slippage() -> None:
cycle = TriangularCycle(
currencies=("USD", "BTC", "ETH"),
pairs=("BTC/USD", "ETH/BTC", "ETH/USD"),
)
detector = IncrementalCycleDetector(
CurrencyGraph.index_cycles_by_pair([cycle]),
max_depth_levels=2,
)
books = {
"BTC/USD": _make_book_levels(
bids=[(99.9, 5.0)],
asks=[(100.0, 0.002), (101.0, 0.020)],
),
"ETH/BTC": _make_book_levels(
bids=[(0.049, 5.0)],
asks=[(0.05, 0.5)],
),
"ETH/USD": _make_book_levels(
bids=[(5.2, 5.0)],
asks=[(5.21, 5.0)],
),
}
scores = detector.score_updated_pair("BTC/USD", books)
assert len(scores) == 1
assert scores[0].gross_rate < 1.04
def test_incremental_detector_returns_no_score_on_insufficient_depth() -> None:
cycle = TriangularCycle(
currencies=("USD", "BTC", "ETH"),
pairs=("BTC/USD", "ETH/BTC", "ETH/USD"),
)
detector = IncrementalCycleDetector(
CurrencyGraph.index_cycles_by_pair([cycle]),
max_depth_levels=1,
)
books = {
"BTC/USD": _make_book_levels(
bids=[(99.9, 5.0)],
asks=[(100.0, 0.001)],
),
"ETH/BTC": _make_book(bid=0.049, ask=0.05),
"ETH/USD": _make_book(bid=5.2, ask=5.21),
}
scores = detector.score_updated_pair("BTC/USD", books)
assert scores == []
def test_incremental_detector_filters_below_profit_threshold() -> None:
cycle = TriangularCycle(
currencies=("USD", "BTC", "ETH"),
pairs=("BTC/USD", "ETH/BTC", "ETH/USD"),
)
detector = IncrementalCycleDetector(
CurrencyGraph.index_cycles_by_pair([cycle]),
min_profit_threshold=0.05,
)
books = {
"BTC/USD": _make_book(bid=99.9, ask=100.0),
"ETH/BTC": _make_book(bid=0.049, ask=0.05),
"ETH/USD": _make_book(bid=5.20, ask=5.21),
}
scores = detector.score_updated_pair("ETH/BTC", books)
assert scores == []
def test_incremental_detector_enforces_min_order_size_by_pair() -> None:
cycle = TriangularCycle(
currencies=("USD", "BTC", "ETH"),
pairs=("BTC/USD", "ETH/BTC", "ETH/USD"),
)
detector = IncrementalCycleDetector(
CurrencyGraph.index_cycles_by_pair([cycle]),
min_order_size_by_pair={"BTC/USD": 0.02},
)
books = {
"BTC/USD": _make_book_levels(
bids=[(99.9, 5.0)],
asks=[(100.0, 0.005), (101.0, 0.005), (102.0, 0.005)],
),
"ETH/BTC": _make_book(bid=0.049, ask=0.05),
"ETH/USD": _make_book(bid=5.2, ask=5.21),
}
scores = detector.score_updated_pair("BTC/USD", books)
assert scores == []
def test_incremental_detector_rejects_stale_books() -> None:
cycle = TriangularCycle(
currencies=("USD", "BTC", "ETH"),
pairs=("BTC/USD", "ETH/BTC", "ETH/USD"),
)
detector = IncrementalCycleDetector(
CurrencyGraph.index_cycles_by_pair([cycle]),
max_book_age_seconds=1.0,
)
books = {
"BTC/USD": _make_book(bid=99.9, ask=100.0),
"ETH/BTC": _make_book(bid=0.049, ask=0.05),
"ETH/USD": _make_book(bid=5.20, ask=5.21),
}
books["ETH/BTC"]._updated_at = datetime.now(UTC) - timedelta(seconds=5)
scores = detector.score_updated_pair("ETH/BTC", books)
assert scores == []
def test_incremental_detector_accepts_fresh_books_with_staleness_enabled() -> None:
cycle = TriangularCycle(
currencies=("USD", "BTC", "ETH"),
pairs=("BTC/USD", "ETH/BTC", "ETH/USD"),
)
detector = IncrementalCycleDetector(
CurrencyGraph.index_cycles_by_pair([cycle]),
max_book_age_seconds=5.0,
)
books = {
"BTC/USD": _make_book(bid=99.9, ask=100.0),
"ETH/BTC": _make_book(bid=0.049, ask=0.05),
"ETH/USD": _make_book(bid=5.20, ask=5.21),
}
now = datetime.now(UTC)
for book in books.values():
book._updated_at = now - timedelta(seconds=0.2)
scores = detector.score_updated_pair("ETH/BTC", books)
assert len(scores) == 1
def test_incremental_detector_emits_structured_opportunity_event() -> None:
cycle = TriangularCycle(
currencies=("USD", "BTC", "ETH"),
pairs=("BTC/USD", "ETH/BTC", "ETH/USD"),
)
detector = IncrementalCycleDetector(
CurrencyGraph.index_cycles_by_pair([cycle]),
min_profit_threshold=0.01,
)
books = {
"BTC/USD": _make_book(bid=99.9, ask=100.0),
"ETH/BTC": _make_book(bid=0.049, ask=0.05),
"ETH/USD": _make_book(bid=5.20, ask=5.21),
}
opportunities = detector.opportunities_for_updated_pair(
"ETH/BTC",
books,
base_capital=500.0,
)
assert len(opportunities) == 1
event = opportunities[0]
assert event.cycle == "USD->BTC->ETH->USD"
assert event.updated_pair == "ETH/BTC"
assert event.gross_pct == pytest.approx(4.0)
assert event.net_pct == pytest.approx(4.0)
assert event.est_profit == pytest.approx(20.0)
def test_incremental_detector_estimated_profit_scales_with_capital() -> None:
cycle = TriangularCycle(
currencies=("USD", "BTC", "ETH"),
pairs=("BTC/USD", "ETH/BTC", "ETH/USD"),
)
detector = IncrementalCycleDetector(
CurrencyGraph.index_cycles_by_pair([cycle]),
min_profit_threshold=0.0,
)
books = {
"BTC/USD": _make_book(bid=99.9, ask=100.0),
"ETH/BTC": _make_book(bid=0.049, ask=0.05),
"ETH/USD": _make_book(bid=5.20, ask=5.21),
}
opportunities = detector.opportunities_for_updated_pair(
"ETH/BTC",
books,
base_capital=250.0,
)
assert len(opportunities) == 1
assert opportunities[0].est_profit == pytest.approx(10.0)
+27
View File
@@ -0,0 +1,27 @@
from __future__ import annotations
from arbitrade.risk.kill_switch import KillSwitch
def test_kill_switch_can_activate_and_deactivate() -> None:
kill_switch = KillSwitch()
assert not kill_switch.is_active
assert kill_switch.reason is None
kill_switch.activate(reason="manual")
assert kill_switch.is_active
assert kill_switch.reason == "manual"
kill_switch.deactivate()
assert not kill_switch.is_active
assert kill_switch.reason is None
def test_kill_switch_active_on_init_sets_reason() -> None:
kill_switch = KillSwitch(active=True)
assert kill_switch.is_active
assert kill_switch.reason == "manual"
+248
View File
@@ -0,0 +1,248 @@
import httpx
import pytest
import respx
from arbitrade.config.settings import Settings
from arbitrade.exchange.kraken_rest import KrakenRestClient
@pytest.mark.asyncio
async def test_server_time_success() -> None:
settings = Settings(_env_file=None)
client = KrakenRestClient(settings)
with respx.mock(base_url=settings.kraken_rest_url) as mock_router:
mock_router.get("/0/public/Time").respond(
200,
json={"error": [], "result": {"unixtime": 1}},
)
payload = await client.server_time()
await client.close()
assert payload["unixtime"] == 1
@pytest.mark.asyncio
async def test_retry_then_success() -> None:
settings = Settings(
_env_file=None,
kraken_retry_attempts=2,
kraken_retry_base_delay_seconds=0.0,
)
client = KrakenRestClient(settings)
with respx.mock(base_url=settings.kraken_rest_url) as mock_router:
route = mock_router.get("/0/public/Time")
route.side_effect = [
httpx.ConnectError("boom"),
httpx.Response(200, json={"error": [], "result": {"unixtime": 2}}),
]
payload = await client.server_time()
await client.close()
assert payload["unixtime"] == 2
@pytest.mark.asyncio
async def test_balances_private_call_uses_headers() -> None:
settings = Settings(
_env_file=None,
KRAKEN_API_KEY="key",
KRAKEN_API_SECRET="c2VjcmV0", # base64("secret")
kraken_private_rate_limit_seconds=0.0,
)
client = KrakenRestClient(settings)
with respx.mock(base_url=settings.kraken_rest_url) as mock_router:
route = mock_router.post("/0/private/Balance").respond(
200,
json={"error": [], "result": {"ZUSD": "10.0"}},
)
payload = await client.balances()
await client.close()
request = route.calls.last.request
assert request.headers.get("API-Key") == "key"
assert request.headers.get("API-Sign")
assert payload["ZUSD"] == "10.0"
@pytest.mark.asyncio
async def test_balances_requires_credentials() -> None:
settings = Settings(
_env_file=None,
KRAKEN_API_KEY=None,
KRAKEN_API_SECRET=None,
kraken_private_rate_limit_seconds=0.0,
)
client = KrakenRestClient(settings)
with pytest.raises(RuntimeError, match="Missing Kraken API credentials"):
await client.balances()
await client.close()
def test_compliance_default_ok() -> None:
settings = Settings(_env_file=None)
client = KrakenRestClient(settings)
issues = client.validate_compliance()
assert issues == []
def test_compliance_detects_insecure_config() -> None:
settings = Settings(
_env_file=None,
KRAKEN_REST_URL="http://api.kraken.com",
KRAKEN_PRIVATE_RATE_LIMIT_SECONDS=0.0,
KRAKEN_RETRY_ATTEMPTS=0,
KRAKEN_RETRY_BASE_DELAY_SECONDS=-1.0,
)
client = KrakenRestClient(settings)
issues = client.validate_compliance()
assert any("https://" in issue for issue in issues)
assert any("below 1.0" in issue for issue in issues)
assert any("ATTEMPTS" in issue for issue in issues)
assert any("BASE_DELAY" in issue for issue in issues)
@pytest.mark.asyncio
async def test_place_market_order_posts_add_order_payload() -> None:
settings = Settings(
_env_file=None,
KRAKEN_API_KEY="key",
KRAKEN_API_SECRET="c2VjcmV0", # base64("secret")
kraken_private_rate_limit_seconds=0.0,
)
client = KrakenRestClient(settings)
with respx.mock(base_url=settings.kraken_rest_url) as mock_router:
route = mock_router.post("/0/private/AddOrder").respond(
200,
json={"error": [], "result": {"txid": ["m1"]}},
)
payload = await client.place_market_order(
pair="XBTUSD",
side="buy",
volume=0.05,
)
await client.close()
request_body = route.calls.last.request.content.decode()
assert "pair=XBTUSD" in request_body
assert "type=buy" in request_body
assert "ordertype=market" in request_body
assert "volume=0.05" in request_body
assert payload["txid"] == ["m1"]
@pytest.mark.asyncio
async def test_place_limit_order_posts_add_order_payload() -> None:
settings = Settings(
_env_file=None,
KRAKEN_API_KEY="key",
KRAKEN_API_SECRET="c2VjcmV0", # base64("secret")
kraken_private_rate_limit_seconds=0.0,
)
client = KrakenRestClient(settings)
with respx.mock(base_url=settings.kraken_rest_url) as mock_router:
route = mock_router.post("/0/private/AddOrder").respond(
200,
json={"error": [], "result": {"txid": ["l1"]}},
)
payload = await client.place_limit_order(
pair="ETHUSD",
side="sell",
volume=1.5,
price=3500.0,
)
await client.close()
request_body = route.calls.last.request.content.decode()
assert "pair=ETHUSD" in request_body
assert "type=sell" in request_body
assert "ordertype=limit" in request_body
assert "price=3500.0" in request_body
assert "volume=1.5" in request_body
assert payload["txid"] == ["l1"]
@pytest.mark.asyncio
async def test_place_order_validates_inputs() -> None:
settings = Settings(
_env_file=None,
KRAKEN_API_KEY="key",
KRAKEN_API_SECRET="c2VjcmV0", # base64("secret")
kraken_private_rate_limit_seconds=0.0,
)
client = KrakenRestClient(settings)
with pytest.raises(ValueError, match="side"):
await client.place_market_order(pair="XBTUSD", side="hold", volume=0.1)
with pytest.raises(ValueError, match="volume"):
await client.place_market_order(pair="XBTUSD", side="buy", volume=0.0)
with pytest.raises(ValueError, match="price"):
await client.place_limit_order(
pair="XBTUSD",
side="buy",
volume=0.1,
price=0.0,
)
await client.close()
@pytest.mark.asyncio
async def test_query_order_posts_query_orders_payload() -> None:
settings = Settings(
_env_file=None,
KRAKEN_API_KEY="key",
KRAKEN_API_SECRET="c2VjcmV0", # base64("secret")
kraken_private_rate_limit_seconds=0.0,
)
client = KrakenRestClient(settings)
with respx.mock(base_url=settings.kraken_rest_url) as mock_router:
route = mock_router.post("/0/private/QueryOrders").respond(
200,
json={"error": [], "result": {"order-1": {"status": "closed"}}},
)
payload = await client.query_order(order_id="order-1", include_trades=False)
await client.close()
request_body = route.calls.last.request.content.decode()
assert "txid=order-1" in request_body
assert "trades=false" in request_body
assert payload["order-1"]["status"] == "closed"
@pytest.mark.asyncio
async def test_cancel_order_posts_cancel_order_payload() -> None:
settings = Settings(
_env_file=None,
KRAKEN_API_KEY="key",
KRAKEN_API_SECRET="c2VjcmV0", # base64("secret")
kraken_private_rate_limit_seconds=0.0,
)
client = KrakenRestClient(settings)
with respx.mock(base_url=settings.kraken_rest_url) as mock_router:
route = mock_router.post("/0/private/CancelOrder").respond(
200,
json={"error": [], "result": {"count": 1}},
)
payload = await client.cancel_order(order_id="order-1")
await client.close()
request_body = route.calls.last.request.content.decode()
assert "txid=order-1" in request_body
assert payload["count"] == 1
+141
View File
@@ -0,0 +1,141 @@
from __future__ import annotations
import asyncio
from dataclasses import dataclass, field
from typing import Any
import orjson
import pytest
from arbitrade.config.settings import Settings
from arbitrade.exchange.kraken_ws import KrakenWsClient
@dataclass(slots=True)
class _FakeAlertNotifier:
events: list[dict[str, str]] = field(default_factory=list)
async def notify(
self,
*,
category: str,
severity: str,
title: str,
message: str,
details: dict[str, str] | None = None,
) -> bool:
self.events.append(
{
"category": category,
"severity": severity,
"title": title,
"message": message,
**(details or {}),
}
)
return True
class _FakeWebSocket:
def __init__(self, messages: list[Any]) -> None:
self._messages = messages
async def recv(self) -> str:
if not self._messages:
await asyncio.sleep(0)
return orjson.dumps({"channel": "heartbeat"}).decode("utf-8")
next_item = self._messages.pop(0)
if isinstance(next_item, Exception):
raise next_item
return next_item
class _FakeConnectContext:
def __init__(self, ws: _FakeWebSocket) -> None:
self._ws = ws
async def __aenter__(self) -> _FakeWebSocket:
return self._ws
async def __aexit__(self, exc_type: object, exc: object, tb: object) -> bool:
return False
def test_parse_book_delta() -> None:
client = KrakenWsClient(Settings())
message = {
"channel": "book",
"symbol": "BTC/USD",
"data": [
{
"bids": [{"price": "100.0", "qty": "1.2"}],
"asks": [{"price": "100.5", "qty": "0.8"}],
"checksum": 123,
"timestamp": 1717232000000,
}
],
}
delta = client.parse_book_delta(message)
assert delta is not None
assert delta.symbol == "BTC/USD"
assert len(delta.bids) == 1
assert len(delta.asks) == 1
assert delta.checksum == 123
@pytest.mark.asyncio
async def test_connect_stream_emits_disconnect_and_reconnect_alerts(
monkeypatch: pytest.MonkeyPatch,
) -> None:
notifier = _FakeAlertNotifier()
settings = Settings(_env_file=None, WS_HEARTBEAT_TIMEOUT_SECONDS=1.0)
client = KrakenWsClient(settings, alert_notifier=notifier)
first_payload = orjson.dumps(
{"channel": "book", "symbol": "BTC/USD", "data": [{"bids": [], "asks": []}]}
).decode("utf-8")
second_payload = orjson.dumps(
{"channel": "book", "symbol": "ETH/USD", "data": [{"bids": [], "asks": []}]}
).decode("utf-8")
sessions = [
_FakeWebSocket([first_payload, RuntimeError("socket dropped")]),
_FakeWebSocket([second_payload]),
]
def _fake_connect(*_args: object, **_kwargs: object) -> _FakeConnectContext:
return _FakeConnectContext(sessions.pop(0))
monkeypatch.setattr("arbitrade.exchange.kraken_ws.websockets.connect", _fake_connect)
stream = client.connect_stream()
first = await anext(stream)
second = await anext(stream)
await client.stop()
await stream.aclose()
assert first.payload["symbol"] == "BTC/USD"
assert second.payload["symbol"] == "ETH/USD"
titles = [event["title"] for event in notifier.events]
assert "WebSocket disconnected" in titles
assert "WebSocket reconnected" in titles
@pytest.mark.asyncio
async def test_recv_loop_emits_staleness_alert_on_timeout() -> None:
notifier = _FakeAlertNotifier()
settings = Settings(_env_file=None, WS_HEARTBEAT_TIMEOUT_SECONDS=0.001)
client = KrakenWsClient(settings, alert_notifier=notifier)
class _NeverReturnsWebSocket:
async def recv(self) -> str:
await asyncio.sleep(1)
return "{}"
with pytest.raises(TimeoutError):
await anext(client._recv_loop(_NeverReturnsWebSocket()))
assert len(notifier.events) == 1
assert notifier.events[0]["title"] == "WebSocket staleness abort"
+49
View File
@@ -0,0 +1,49 @@
from __future__ import annotations
from arbitrade.perf.guardrails import evaluate_guardrails
from arbitrade.perf.latency import run_latency_profile
def test_run_latency_profile_contains_expected_shape() -> None:
profile = run_latency_profile(iterations=50)
scenarios = profile.get("scenarios")
assert isinstance(scenarios, dict)
assert set(scenarios) == {
"book_update_burst",
"execution_spike",
"reconnect_storm",
}
for payload in scenarios.values():
assert isinstance(payload, dict)
stages = payload.get("stages")
assert isinstance(stages, dict)
assert "end_to_end" in stages
def test_evaluate_guardrails_flags_regression() -> None:
baseline = {
"scenarios": {
"book_update_burst": {
"stages": {
"end_to_end": {"p95_ms": 1.0, "p99_ms": 1.0},
}
}
}
}
current = {
"scenarios": {
"book_update_burst": {
"stages": {
"end_to_end": {"p95_ms": 4.0, "p99_ms": 4.0},
}
}
}
}
thresholds = {"default": {"p95_ms": 2.0, "p99_ms": 2.0}}
failures = evaluate_guardrails(baseline=baseline, current=current, thresholds=thresholds)
assert failures
assert "latency regression" in failures[0]
+90
View File
@@ -0,0 +1,90 @@
from __future__ import annotations
import asyncio
from datetime import UTC, datetime, timedelta
from typing import Any
import pytest
from arbitrade.risk.loss_limits import LossLimitGuard
class _FakeAlertNotifier:
def __init__(self) -> None:
self.events: list[dict[str, Any]] = []
async def notify(
self,
*,
category: str,
severity: str,
title: str,
message: str,
details: dict[str, str] | None = None,
) -> bool:
self.events.append(
{
"category": category,
"severity": severity,
"title": title,
"message": message,
"details": details or {},
}
)
return True
def test_loss_limit_guard_tracks_daily_and_cumulative_pnl() -> None:
guard = LossLimitGuard(daily_loss_limit=100.0, cumulative_loss_limit=200.0)
t0 = datetime.now(UTC)
guard.register_realized_pnl(-40.0, at=t0)
guard.register_realized_pnl(10.0, at=t0)
assert guard.cumulative_pnl == -30.0
assert guard.daily_pnl(t0.date()) == -30.0
assert not guard.is_halted
def test_loss_limit_guard_halts_on_daily_limit() -> None:
guard = LossLimitGuard(daily_loss_limit=50.0)
t0 = datetime.now(UTC)
guard.register_realized_pnl(-30.0, at=t0)
guard.register_realized_pnl(-25.0, at=t0)
assert guard.is_halted
assert guard.halted_reason == "daily_loss_limit_breached"
def test_loss_limit_guard_halts_on_cumulative_limit_across_days() -> None:
guard = LossLimitGuard(cumulative_loss_limit=60.0)
t0 = datetime.now(UTC)
guard.register_realized_pnl(-40.0, at=t0)
guard.register_realized_pnl(-25.0, at=t0 + timedelta(days=1))
assert guard.is_halted
assert guard.halted_reason == "cumulative_loss_limit_breached"
def test_loss_limit_guard_rejects_invalid_limits() -> None:
with pytest.raises(ValueError, match="daily_loss_limit"):
LossLimitGuard(daily_loss_limit=0.0)
with pytest.raises(ValueError, match="cumulative_loss_limit"):
LossLimitGuard(cumulative_loss_limit=-1.0)
@pytest.mark.asyncio
async def test_loss_limit_guard_emits_alert_on_breach() -> None:
notifier = _FakeAlertNotifier()
guard = LossLimitGuard(daily_loss_limit=50.0, alert_notifier=notifier)
guard.register_realized_pnl(-60.0, at=datetime.now(UTC))
await asyncio.sleep(0)
assert guard.is_halted
assert len(notifier.events) == 1
assert notifier.events[0]["category"] == "threshold"
assert notifier.events[0]["title"] == "Daily loss limit breached"
+484
View File
@@ -0,0 +1,484 @@
from __future__ import annotations
import asyncio
from dataclasses import dataclass
from datetime import UTC, datetime
from types import SimpleNamespace
import pytest
from arbitrade.detection.engine import OpportunityEvent
from arbitrade.exchange.models import BookDelta, BookLevel
from arbitrade.market_data.feed import ExecutionOutcome, MarketDataFeed
from arbitrade.risk.kill_switch import KillSwitch
from arbitrade.risk.loss_limits import LossLimitGuard
from arbitrade.risk.pre_trade import PreTradeValidator
from arbitrade.risk.stop_conditions import StopConditionsGuard
from arbitrade.risk.trade_limits import TradeLimitsGuard
@dataclass(slots=True)
class _FakeWsClient:
delta: BookDelta
async def connect_stream(self):
yield SimpleNamespace(payload={"channel": "book"})
def parse_book_delta(self, _payload: dict[str, object]) -> BookDelta:
return self.delta
class _FakeSnapshotWriter:
def __init__(self) -> None:
self.items: list[object] = []
async def enqueue(self, snapshot: object) -> None:
self.items.append(snapshot)
class _FakeOpportunityWriter:
def __init__(self) -> None:
self.items: list[OpportunityEvent] = []
async def enqueue(self, event: OpportunityEvent) -> None:
self.items.append(event)
class _FakeDetector:
def __init__(self, event: OpportunityEvent) -> None:
self._event = event
self.last_base_capital: float | None = None
def opportunities_for_updated_pair(
self,
_updated_pair: str,
_books: dict[str, object],
*,
base_capital: float,
):
self.last_base_capital = base_capital
return [self._event]
class _FakeExecutor:
def __init__(self) -> None:
self.calls: list[OpportunityEvent] = []
self.realized_pnls: list[float | None] = []
self.outcomes: list[ExecutionOutcome] = []
async def execute(self, event: OpportunityEvent) -> ExecutionOutcome | float | None:
self.calls.append(event)
if self.outcomes:
return self.outcomes.pop(0)
if not self.realized_pnls:
return None
return self.realized_pnls.pop(0)
class _FakeFailingExecutor:
def __init__(self) -> None:
self.calls: int = 0
async def execute(self, _event: OpportunityEvent) -> None:
self.calls += 1
raise RuntimeError("executor failure")
class _FakeAlertNotifier:
def __init__(self) -> None:
self.events: list[dict[str, str]] = []
async def notify(
self,
*,
category: str,
severity: str,
title: str,
message: str,
details: dict[str, str] | None = None,
) -> bool:
self.events.append(
{
"category": category,
"severity": severity,
"title": title,
"message": message,
**(details or {}),
}
)
return True
@dataclass(slots=True)
class _FakeWsClientTwoMessages:
delta: BookDelta
async def connect_stream(self):
yield SimpleNamespace(payload={"channel": "book", "seq": 1})
yield SimpleNamespace(payload={"channel": "book", "seq": 2})
def parse_book_delta(self, _payload: dict[str, object]) -> BookDelta:
return self.delta
def _sample_event(*, allocated_capital: float = 1.0) -> OpportunityEvent:
return OpportunityEvent(
detected_at=datetime.now(UTC),
cycle="USD->BTC->ETH->USD",
updated_pair="BTC/USD",
gross_rate=1.04,
net_rate=1.03,
gross_pct=4.0,
net_pct=3.0,
est_profit=0.03,
allocated_capital=allocated_capital,
)
def _sample_delta() -> BookDelta:
return BookDelta(
symbol="BTC/USD",
bids=[BookLevel(price=100.0, volume=1.0)],
asks=[BookLevel(price=100.5, volume=1.0)],
)
@pytest.mark.asyncio
async def test_market_data_feed_dry_run_does_not_execute_orders() -> None:
event = _sample_event()
executor = _FakeExecutor()
feed = MarketDataFeed(
ws_client=_FakeWsClient(_sample_delta()),
snapshot_writer=_FakeSnapshotWriter(),
detector=_FakeDetector(event),
opportunity_writer=_FakeOpportunityWriter(),
paper_trading_mode=True,
opportunity_executor=executor.execute,
)
await feed.run()
assert executor.calls == []
@pytest.mark.asyncio
async def test_market_data_feed_live_mode_executes_orders() -> None:
event = _sample_event()
executor = _FakeExecutor()
feed = MarketDataFeed(
ws_client=_FakeWsClient(_sample_delta()),
snapshot_writer=_FakeSnapshotWriter(),
detector=_FakeDetector(event),
opportunity_writer=_FakeOpportunityWriter(),
paper_trading_mode=False,
opportunity_executor=executor.execute,
)
await feed.run()
assert len(executor.calls) == 1
assert executor.calls[0].cycle == "USD->BTC->ETH->USD"
@pytest.mark.asyncio
async def test_market_data_feed_enforces_per_trade_capital_limit() -> None:
event = _sample_event()
detector = _FakeDetector(event)
feed = MarketDataFeed(
ws_client=_FakeWsClient(_sample_delta()),
snapshot_writer=_FakeSnapshotWriter(),
detector=detector,
opportunity_writer=_FakeOpportunityWriter(),
paper_trading_mode=True,
trade_capital=250.0,
max_trade_capital=100.0,
)
await feed.run()
assert detector.last_base_capital == 100.0
@pytest.mark.asyncio
async def test_market_data_feed_auto_halts_on_daily_loss_limit() -> None:
event = _sample_event()
detector = _FakeDetector(event)
executor = _FakeExecutor()
executor.realized_pnls = [-60.0, -10.0]
loss_guard = LossLimitGuard(daily_loss_limit=50.0)
feed = MarketDataFeed(
ws_client=_FakeWsClientTwoMessages(_sample_delta()),
snapshot_writer=_FakeSnapshotWriter(),
detector=detector,
opportunity_writer=_FakeOpportunityWriter(),
paper_trading_mode=False,
opportunity_executor=executor.execute,
loss_limit_guard=loss_guard,
)
await feed.run()
assert len(executor.calls) == 1
assert loss_guard.is_halted
assert loss_guard.halted_reason == "daily_loss_limit_breached"
@pytest.mark.asyncio
async def test_market_data_feed_auto_halts_on_cumulative_loss_limit() -> None:
event = _sample_event()
detector = _FakeDetector(event)
executor = _FakeExecutor()
executor.realized_pnls = [-40.0, -15.0]
loss_guard = LossLimitGuard(cumulative_loss_limit=50.0)
feed = MarketDataFeed(
ws_client=_FakeWsClientTwoMessages(_sample_delta()),
snapshot_writer=_FakeSnapshotWriter(),
detector=detector,
opportunity_writer=_FakeOpportunityWriter(),
paper_trading_mode=False,
opportunity_executor=executor.execute,
loss_limit_guard=loss_guard,
)
await feed.run()
assert len(executor.calls) == 2
assert loss_guard.is_halted
assert loss_guard.halted_reason == "cumulative_loss_limit_breached"
@pytest.mark.asyncio
async def test_market_data_feed_enforces_max_concurrent_trades() -> None:
event = _sample_event()
detector = _FakeDetector(event)
executor = _FakeExecutor()
executor.outcomes = [ExecutionOutcome(realized_pnl=None, close_trade=False)]
trade_guard = TradeLimitsGuard(max_concurrent_trades=1)
feed = MarketDataFeed(
ws_client=_FakeWsClientTwoMessages(_sample_delta()),
snapshot_writer=_FakeSnapshotWriter(),
detector=detector,
opportunity_writer=_FakeOpportunityWriter(),
paper_trading_mode=False,
opportunity_executor=executor.execute,
trade_limits_guard=trade_guard,
)
await feed.run()
assert len(executor.calls) == 1
assert trade_guard.active_trades == 1
@pytest.mark.asyncio
async def test_market_data_feed_enforces_per_asset_exposure_cap() -> None:
event = _sample_event(allocated_capital=100.0)
detector = _FakeDetector(event)
executor = _FakeExecutor()
trade_guard = TradeLimitsGuard(max_exposure_per_asset=50.0)
feed = MarketDataFeed(
ws_client=_FakeWsClient(_sample_delta()),
snapshot_writer=_FakeSnapshotWriter(),
detector=detector,
opportunity_writer=_FakeOpportunityWriter(),
paper_trading_mode=False,
opportunity_executor=executor.execute,
trade_limits_guard=trade_guard,
)
await feed.run()
assert len(executor.calls) == 0
@pytest.mark.asyncio
async def test_market_data_feed_blocks_when_pre_trade_balance_insufficient() -> None:
event = _sample_event(allocated_capital=100.0)
detector = _FakeDetector(event)
executor = _FakeExecutor()
validator = PreTradeValidator(min_order_size_by_asset={"USD": 50.0})
feed = MarketDataFeed(
ws_client=_FakeWsClient(_sample_delta()),
snapshot_writer=_FakeSnapshotWriter(),
detector=detector,
opportunity_writer=_FakeOpportunityWriter(),
paper_trading_mode=False,
opportunity_executor=executor.execute,
pre_trade_validator=validator,
balance_provider=lambda: {"USD": 25.0},
quote_balance_asset="USD",
)
await feed.run()
assert len(executor.calls) == 0
@pytest.mark.asyncio
async def test_market_data_feed_blocks_when_pre_trade_min_order_not_met() -> None:
event = _sample_event(allocated_capital=25.0)
detector = _FakeDetector(event)
executor = _FakeExecutor()
validator = PreTradeValidator(min_order_size_by_asset={"USD": 50.0})
feed = MarketDataFeed(
ws_client=_FakeWsClient(_sample_delta()),
snapshot_writer=_FakeSnapshotWriter(),
detector=detector,
opportunity_writer=_FakeOpportunityWriter(),
paper_trading_mode=False,
opportunity_executor=executor.execute,
pre_trade_validator=validator,
balance_provider=lambda: {"USD": 500.0},
quote_balance_asset="USD",
)
await feed.run()
assert len(executor.calls) == 0
@pytest.mark.asyncio
async def test_market_data_feed_allows_when_pre_trade_validation_passes() -> None:
event = _sample_event(allocated_capital=75.0)
detector = _FakeDetector(event)
executor = _FakeExecutor()
validator = PreTradeValidator(min_order_size_by_asset={"USD": 50.0})
feed = MarketDataFeed(
ws_client=_FakeWsClient(_sample_delta()),
snapshot_writer=_FakeSnapshotWriter(),
detector=detector,
opportunity_writer=_FakeOpportunityWriter(),
paper_trading_mode=False,
opportunity_executor=executor.execute,
pre_trade_validator=validator,
balance_provider=lambda: {"USD": 500.0},
quote_balance_asset="USD",
)
await feed.run()
assert len(executor.calls) == 1
@pytest.mark.asyncio
async def test_market_data_feed_blocks_when_kill_switch_active() -> None:
event = _sample_event(allocated_capital=75.0)
detector = _FakeDetector(event)
executor = _FakeExecutor()
kill_switch = KillSwitch(active=True, reason="manual")
feed = MarketDataFeed(
ws_client=_FakeWsClient(_sample_delta()),
snapshot_writer=_FakeSnapshotWriter(),
detector=detector,
opportunity_writer=_FakeOpportunityWriter(),
paper_trading_mode=False,
opportunity_executor=executor.execute,
kill_switch=kill_switch,
)
await feed.run()
assert len(executor.calls) == 0
@pytest.mark.asyncio
async def test_market_data_feed_allows_when_kill_switch_inactive() -> None:
event = _sample_event(allocated_capital=75.0)
detector = _FakeDetector(event)
executor = _FakeExecutor()
kill_switch = KillSwitch(active=False)
feed = MarketDataFeed(
ws_client=_FakeWsClient(_sample_delta()),
snapshot_writer=_FakeSnapshotWriter(),
detector=detector,
opportunity_writer=_FakeOpportunityWriter(),
paper_trading_mode=False,
opportunity_executor=executor.execute,
kill_switch=kill_switch,
)
await feed.run()
assert len(executor.calls) == 1
@pytest.mark.asyncio
async def test_market_data_feed_halts_on_abnormal_source_latency() -> None:
event = _sample_event(allocated_capital=75.0)
detector = _FakeDetector(event)
executor = _FakeExecutor()
kill_switch = KillSwitch(active=False)
stop_guard = StopConditionsGuard(max_source_latency_ms=1.0)
delta = _sample_delta()
delta.source_timestamp_ms = 0
feed = MarketDataFeed(
ws_client=_FakeWsClient(delta),
snapshot_writer=_FakeSnapshotWriter(),
detector=detector,
opportunity_writer=_FakeOpportunityWriter(),
paper_trading_mode=False,
opportunity_executor=executor.execute,
kill_switch=kill_switch,
stop_conditions_guard=stop_guard,
)
await feed.run()
assert stop_guard.is_halted
assert stop_guard.halted_reason == "source_latency_limit_breached"
assert kill_switch.is_active
assert kill_switch.reason == "source_latency_limit_breached"
assert len(executor.calls) == 0
@pytest.mark.asyncio
async def test_market_data_feed_halts_on_repeated_execution_failures() -> None:
event = _sample_event(allocated_capital=75.0)
detector = _FakeDetector(event)
executor = _FakeFailingExecutor()
kill_switch = KillSwitch(active=False)
stop_guard = StopConditionsGuard(max_consecutive_failures=2)
feed = MarketDataFeed(
ws_client=_FakeWsClientTwoMessages(_sample_delta()),
snapshot_writer=_FakeSnapshotWriter(),
detector=detector,
opportunity_writer=_FakeOpportunityWriter(),
paper_trading_mode=False,
opportunity_executor=executor.execute,
kill_switch=kill_switch,
stop_conditions_guard=stop_guard,
)
await feed.run()
assert executor.calls == 2
assert stop_guard.is_halted
assert stop_guard.halted_reason == "consecutive_failures_limit_breached"
assert kill_switch.is_active
assert kill_switch.reason == "consecutive_failures_limit_breached"
@pytest.mark.asyncio
async def test_market_data_feed_emits_critical_alert_on_executor_exception() -> None:
event = _sample_event(allocated_capital=75.0)
detector = _FakeDetector(event)
executor = _FakeFailingExecutor()
notifier = _FakeAlertNotifier()
feed = MarketDataFeed(
ws_client=_FakeWsClient(_sample_delta()),
snapshot_writer=_FakeSnapshotWriter(),
detector=detector,
opportunity_writer=_FakeOpportunityWriter(),
paper_trading_mode=False,
opportunity_executor=executor.execute,
alert_notifier=notifier,
)
await feed.run()
await asyncio.sleep(0)
assert executor.calls == 1
assert len(notifier.events) == 1
assert notifier.events[0]["category"] == "system"
assert notifier.events[0]["severity"] == "critical"
assert notifier.events[0]["title"] == "Critical execution exception"
+144
View File
@@ -0,0 +1,144 @@
from __future__ import annotations
from datetime import UTC, datetime, timedelta
import pytest
from arbitrade.config.settings import Settings
from arbitrade.metrics import MetricsCalculator
from arbitrade.storage.db import DuckDBStore
def test_metrics_calculator_summarizes_execution_data(tmp_path) -> None:
settings = Settings(_env_file=None, DUCKDB_PATH=tmp_path / "metrics.duckdb")
store = DuckDBStore(settings)
store.migrate()
started = datetime.now(UTC)
finished = started + timedelta(seconds=30)
started_two = started + timedelta(minutes=1)
finished_two = started_two + timedelta(seconds=90)
with store.connect() as conn:
conn.execute(
"""
INSERT INTO trades (
trade_ref,
started_at,
finished_at,
status,
realized_pnl,
estimated_pnl,
capital_used,
cycle,
leg_count
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?), (?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
[
"trade-1",
started,
finished,
"filled",
12.5,
10.0,
100.0,
"USD->BTC->ETH->USD",
3,
"trade-2",
started_two,
finished_two,
"filled",
-4.5,
-2.0,
200.0,
"USD->ETH->BTC->USD",
3,
],
)
conn.execute(
"""
INSERT INTO opportunities (
detected_at,
cycle,
gross_pct,
net_pct,
est_profit,
executed
) VALUES (?, ?, ?, ?, ?, ?), (?, ?, ?, ?, ?, ?), (?, ?, ?, ?, ?, ?)
""",
[
started,
"USD->BTC->ETH->USD",
4.0,
3.0,
0.03,
True,
started_two,
"USD->ETH->BTC->USD",
2.0,
1.0,
0.01,
False,
started_two + timedelta(seconds=30),
"USD->BTC->ETH->USD",
5.0,
4.0,
0.04,
True,
],
)
conn.execute(
"""
INSERT INTO orders (
trade_ref,
order_ref,
leg_index,
pair,
side,
volume,
user_ref,
status,
filled_volume,
avg_price,
raw_response,
recorded_at
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?), (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
[
"trade-1",
"order-1",
0,
"BTC/USD",
"buy",
2.0,
101,
"closed",
2.0,
100.0,
"{}",
started,
"trade-2",
"order-2",
0,
"ETH/USD",
"sell",
4.0,
202,
"closed",
3.0,
200.0,
"{}",
started_two,
],
)
metrics = MetricsCalculator(store).compute()
assert metrics.realized_pnl_usd == 8.0
assert metrics.win_rate == 0.5
assert metrics.avg_trade_duration_seconds == 60.0
assert metrics.opportunities_per_minute == 2.0
assert metrics.fill_rate == 0.875
assert metrics.latency_p50_seconds == 60.0
assert metrics.latency_p95_seconds == 87.0
assert metrics.latency_p99_seconds == pytest.approx(89.4)
+48
View File
@@ -0,0 +1,48 @@
from __future__ import annotations
from datetime import UTC, datetime
import pytest
from arbitrade.config.settings import Settings
from arbitrade.detection.engine import OpportunityEvent
from arbitrade.storage.db import DuckDBStore
from arbitrade.storage.opportunities import AsyncOpportunityWriter
from arbitrade.storage.repositories import OpportunityRepository
@pytest.mark.asyncio
async def test_async_opportunity_writer_persists_events(tmp_path) -> None:
settings = Settings(_env_file=None, DUCKDB_PATH=tmp_path / "test.duckdb")
store = DuckDBStore(settings)
store.migrate()
repository = OpportunityRepository(store)
writer = AsyncOpportunityWriter(repository, max_queue_size=10)
await writer.start()
event = OpportunityEvent(
detected_at=datetime.now(UTC),
cycle="USD->BTC->ETH->USD",
updated_pair="BTC/USD",
gross_rate=1.04,
net_rate=1.03,
gross_pct=4.0,
net_pct=3.0,
est_profit=0.03,
)
await writer.enqueue(event)
await writer.stop()
with store.connect() as conn:
rows = conn.execute(
"SELECT cycle, gross_pct, net_pct, est_profit, executed FROM opportunities"
).fetchall()
assert len(rows) == 1
assert rows[0][0] == "USD->BTC->ETH->USD"
assert rows[0][1] == 4.0
assert rows[0][2] == 3.0
assert rows[0][3] == 0.03
assert rows[0][4] is False
+27
View File
@@ -0,0 +1,27 @@
from arbitrade.exchange.models import BookLevel
from arbitrade.market_data.order_book import OrderBook
def test_order_book_apply_and_best_levels() -> None:
book = OrderBook()
book.apply_bids([BookLevel(price=100.0, volume=1.0), BookLevel(price=99.5, volume=2.0)])
book.apply_asks([BookLevel(price=100.5, volume=1.1), BookLevel(price=101.0, volume=0.9)])
best_bid = book.best_bid()
best_ask = book.best_ask()
assert best_bid is not None
assert best_ask is not None
assert best_bid.price == 100.0
assert best_ask.price == 100.5
def test_order_book_checksum_matches_self() -> None:
book = OrderBook()
book.apply_bids([BookLevel(price=100.0, volume=1.0)])
book.apply_asks([BookLevel(price=100.5, volume=1.0)])
checksum = book.compute_checksum()
assert isinstance(checksum, int)
assert checksum == book.compute_checksum()
+96
View File
@@ -0,0 +1,96 @@
from __future__ import annotations
from dataclasses import dataclass
from datetime import UTC, datetime
from typing import Any
import pytest
from arbitrade.execution.fill_monitor import FillMonitorResult, OrderFillState
from arbitrade.execution.recovery import PartialFillRecovery
@dataclass(slots=True)
class _FakeRestClient:
cancel_calls: list[str] = None # type: ignore[assignment]
market_calls: list[dict[str, Any]] = None # type: ignore[assignment]
def __post_init__(self) -> None:
self.cancel_calls = []
self.market_calls = []
async def cancel_order(self, *, order_id: str) -> dict[str, Any]:
self.cancel_calls.append(order_id)
return {"result": {"count": 1}}
async def place_market_order(self, *, pair: str, side: str, volume: float) -> dict[str, Any]:
self.market_calls.append({"pair": pair, "side": side, "volume": volume})
return {"txid": ["hedge-1"]}
def _monitor_result(
*, status: str, filled_volume: float | None, timed_out: bool
) -> FillMonitorResult:
state = OrderFillState(
order_id="order-1",
status=status,
filled_volume=filled_volume,
avg_price=100.0,
updated_at=datetime.now(UTC),
source="rest_poll",
)
return FillMonitorResult(
order_id="order-1",
timed_out=timed_out,
terminal_state=None if status in {"open", "partial"} else state,
last_state=state,
elapsed_seconds=1.0,
)
@pytest.mark.asyncio
async def test_partial_fill_recovery_cancels_open_order_and_hedges_residual() -> None:
client = _FakeRestClient()
recovery = PartialFillRecovery(client)
result = await recovery.recover_partial_fill(
order_id="order-1",
pair="BTC/USD",
side="buy",
requested_volume=10.0,
fill_result=_monitor_result(status="partial", filled_volume=4.0, timed_out=True),
)
assert result.canceled
assert result.hedged
assert client.cancel_calls == ["order-1"]
assert client.market_calls == [{"pair": "BTC/USD", "side": "sell", "volume": 6.0}]
assert result.hedge_volume == 6.0
assert result.reason == "canceled_partial_order"
@pytest.mark.asyncio
async def test_partial_fill_recovery_no_hedge_when_no_residual() -> None:
client = _FakeRestClient()
recovery = PartialFillRecovery(client)
result = await recovery.recover_partial_fill(
order_id="order-1",
pair="BTC/USD",
side="sell",
requested_volume=5.0,
fill_result=_monitor_result(status="closed", filled_volume=5.0, timed_out=False),
)
assert not result.canceled
assert not result.hedged
assert client.cancel_calls == []
assert client.market_calls == []
def test_partial_fill_recovery_rejects_invalid_volume() -> None:
client = _FakeRestClient()
recovery = PartialFillRecovery(client)
with pytest.raises(ValueError, match="requested_volume"):
recovery._residual_volume(None, 0.0)
+37
View File
@@ -0,0 +1,37 @@
from __future__ import annotations
import pytest
from arbitrade.risk.pre_trade import PreTradeValidator
def test_pre_trade_validator_accepts_when_balance_and_min_size_pass() -> None:
validator = PreTradeValidator(min_order_size_by_asset={"USD": 50.0})
assert validator.validate(
balances_by_asset={"USD": 100.0},
required_by_asset={"USD": 75.0},
)
def test_pre_trade_validator_rejects_when_balance_insufficient() -> None:
validator = PreTradeValidator(min_order_size_by_asset={"USD": 50.0})
assert not validator.validate(
balances_by_asset={"USD": 40.0},
required_by_asset={"USD": 75.0},
)
def test_pre_trade_validator_rejects_when_below_min_size() -> None:
validator = PreTradeValidator(min_order_size_by_asset={"USD": 50.0})
assert not validator.validate(
balances_by_asset={"USD": 100.0},
required_by_asset={"USD": 30.0},
)
def test_pre_trade_validator_rejects_invalid_min_order_size_config() -> None:
with pytest.raises(ValueError, match="minimum order size"):
PreTradeValidator(min_order_size_by_asset={"USD": 0.0})
+140
View File
@@ -0,0 +1,140 @@
from __future__ import annotations
from dataclasses import dataclass
from datetime import UTC, datetime
import pytest
from arbitrade.api.app import create_app
from arbitrade.config.settings import Settings
from arbitrade.runtime.lifecycle import (
graceful_shutdown,
persist_runtime_snapshot,
restore_runtime_state,
)
from arbitrade.storage.repositories import RuntimeStateRecord
@dataclass(slots=True)
class _FakeWorker:
stopped: bool = False
async def stop(self) -> None:
self.stopped = True
@dataclass(slots=True)
class _FakeStartupReconciler:
called: bool = False
async def reconcile_open_trades(self) -> None:
self.called = True
@pytest.mark.asyncio
async def test_persist_runtime_snapshot_writes_record(tmp_path) -> None:
app = create_app(Settings(_env_file=None, DUCKDB_PATH=tmp_path / "runtime.duckdb"))
app.state.dashboard_controls.is_running = True
app.state.dashboard_controls.kill_switch.deactivate()
snapshot = persist_runtime_snapshot(app, note="unit-test")
assert snapshot is not None
assert snapshot.note == "unit-test"
latest = app.state.runtime_state_repository.latest()
assert latest is not None
assert latest.note == "unit-test"
assert latest.is_running is True
@pytest.mark.asyncio
async def test_restore_runtime_state_applies_snapshot(tmp_path) -> None:
app = create_app(Settings(_env_file=None, DUCKDB_PATH=tmp_path / "restore.duckdb"))
app.state.runtime_state_repository.insert(
RuntimeStateRecord(
snapshot_at=datetime.now(UTC),
is_running=False,
kill_switch_active=True,
kill_switch_reason="manual-stop",
open_trade_count=0,
last_known_balances={"USD": 100.0},
note="seed",
)
)
report = await restore_runtime_state(app)
assert report.restored_from_snapshot is True
assert app.state.dashboard_controls.is_running is False
assert app.state.dashboard_controls.kill_switch.is_active is True
assert app.state.dashboard_controls.kill_switch.reason == "manual-stop"
@pytest.mark.asyncio
async def test_restore_runtime_state_enables_restart_guard_for_open_trades(tmp_path) -> None:
app = create_app(Settings(_env_file=None, DUCKDB_PATH=tmp_path / "open-trades.duckdb"))
with app.state.store.connect() as conn:
conn.execute(
"""
INSERT INTO trades (
trade_ref,
started_at,
finished_at,
status,
realized_pnl,
estimated_pnl,
capital_used,
cycle,
leg_count
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
[
"open-trade-1",
datetime.now(UTC),
None,
"open",
None,
1.0,
100.0,
"USD->BTC->ETH->USD",
3,
],
)
report = await restore_runtime_state(app)
assert report.open_trades_detected == 1
assert report.restart_guard_active is True
assert app.state.dashboard_controls.is_running is False
assert app.state.dashboard_controls.kill_switch.is_active is True
assert app.state.dashboard_controls.kill_switch.reason == "recovery_open_trades_detected"
@pytest.mark.asyncio
async def test_graceful_shutdown_drains_workers_and_persists_snapshot(tmp_path) -> None:
app = create_app(Settings(_env_file=None, DUCKDB_PATH=tmp_path / "shutdown.duckdb"))
worker = _FakeWorker()
app.state.background_workers = [worker]
app.state.dashboard_controls.is_running = True
await graceful_shutdown(app)
assert worker.stopped is True
assert app.state.dashboard_controls.is_running is False
latest = app.state.runtime_state_repository.latest()
assert latest is not None
assert latest.note == "graceful_shutdown"
@pytest.mark.asyncio
async def test_restore_runtime_state_calls_startup_reconciler(tmp_path) -> None:
app = create_app(Settings(_env_file=None, DUCKDB_PATH=tmp_path / "reconciler.duckdb"))
reconciler = _FakeStartupReconciler()
app.state.startup_reconciler = reconciler
await restore_runtime_state(app)
assert reconciler.called is True
+55
View File
@@ -0,0 +1,55 @@
import pytest
from pydantic import ValidationError
from arbitrade.config.settings import Settings
def test_dashboard_auth_requires_both_fields() -> None:
with pytest.raises(ValidationError):
Settings(_env_file=None, DASHBOARD_AUTH_USERNAME="admin")
def test_kraken_api_auth_requires_key_and_secret() -> None:
with pytest.raises(ValidationError):
Settings(
_env_file=None,
KRAKEN_API_KEY="key-only",
KRAKEN_API_SECRET="",
)
def test_kraken_permissions_require_query_and_trade() -> None:
with pytest.raises(ValidationError):
Settings(
_env_file=None,
KRAKEN_API_KEY="k",
KRAKEN_API_SECRET="s",
KRAKEN_API_KEY_PERMISSIONS="query",
)
def test_kraken_permissions_forbid_withdrawal_scope() -> None:
with pytest.raises(ValidationError):
Settings(
_env_file=None,
KRAKEN_API_KEY="k",
KRAKEN_API_SECRET="s",
KRAKEN_API_KEY_PERMISSIONS="query,trade,withdraw",
)
def test_alert_min_severity_is_validated() -> None:
with pytest.raises(ValidationError):
Settings(_env_file=None, ALERT_MIN_SEVERITY="nope")
def test_valid_security_configuration_passes() -> None:
settings = Settings(
_env_file=None,
KRAKEN_API_KEY="k",
KRAKEN_API_SECRET="s",
KRAKEN_API_KEY_PERMISSIONS="query,trade",
ALERT_MIN_SEVERITY="warning",
)
assert settings.kraken_api_key_permissions == "query,trade"
+99
View File
@@ -0,0 +1,99 @@
from __future__ import annotations
import asyncio
from typing import Any
import pytest
from arbitrade.risk.stop_conditions import StopConditionsGuard
class _FakeAlertNotifier:
def __init__(self) -> None:
self.events: list[dict[str, Any]] = []
async def notify(
self,
*,
category: str,
severity: str,
title: str,
message: str,
details: dict[str, str] | None = None,
) -> bool:
self.events.append(
{
"category": category,
"severity": severity,
"title": title,
"message": message,
"details": details or {},
}
)
return True
def test_stop_conditions_guard_halts_on_source_latency_breach() -> None:
guard = StopConditionsGuard(max_source_latency_ms=50.0)
guard.observe_latency(source_latency_ms=75.0, apply_latency_ms=1.0)
assert guard.is_halted
assert guard.halted_reason == "source_latency_limit_breached"
def test_stop_conditions_guard_halts_on_apply_latency_breach() -> None:
guard = StopConditionsGuard(max_apply_latency_ms=2.0)
guard.observe_latency(source_latency_ms=None, apply_latency_ms=3.5)
assert guard.is_halted
assert guard.halted_reason == "apply_latency_limit_breached"
def test_stop_conditions_guard_halts_on_consecutive_failures() -> None:
guard = StopConditionsGuard(max_consecutive_failures=2)
guard.register_failure()
assert not guard.is_halted
guard.register_failure()
assert guard.is_halted
assert guard.halted_reason == "consecutive_failures_limit_breached"
def test_stop_conditions_guard_resets_failures_after_success() -> None:
guard = StopConditionsGuard(max_consecutive_failures=3)
guard.register_failure()
guard.register_success()
guard.register_failure()
assert guard.consecutive_failures == 1
assert not guard.is_halted
def test_stop_conditions_guard_rejects_invalid_configuration() -> None:
with pytest.raises(ValueError, match="max_source_latency_ms"):
StopConditionsGuard(max_source_latency_ms=0.0)
with pytest.raises(ValueError, match="max_apply_latency_ms"):
StopConditionsGuard(max_apply_latency_ms=-1.0)
with pytest.raises(ValueError, match="max_consecutive_failures"):
StopConditionsGuard(max_consecutive_failures=0)
@pytest.mark.asyncio
async def test_stop_conditions_guard_emits_alert_on_failure_threshold() -> None:
notifier = _FakeAlertNotifier()
guard = StopConditionsGuard(max_consecutive_failures=1, alert_notifier=notifier)
guard.register_failure()
await asyncio.sleep(0)
assert guard.is_halted
assert len(notifier.events) == 1
assert notifier.events[0]["category"] == "threshold"
assert notifier.events[0]["title"] == "Consecutive failures limit breached"
+84
View File
@@ -0,0 +1,84 @@
from __future__ import annotations
import asyncio
from typing import Any
import pytest
from arbitrade.risk.trade_limits import TradeLimitsGuard
class _FakeAlertNotifier:
def __init__(self) -> None:
self.events: list[dict[str, Any]] = []
async def notify(
self,
*,
category: str,
severity: str,
title: str,
message: str,
details: dict[str, str] | None = None,
) -> bool:
self.events.append(
{
"category": category,
"severity": severity,
"title": title,
"message": message,
"details": details or {},
}
)
return True
def test_trade_limits_guard_blocks_when_max_concurrent_reached() -> None:
guard = TradeLimitsGuard(max_concurrent_trades=1)
guard.open_trade({"BTC": 10.0})
assert not guard.is_trade_allowed({"BTC": 1.0})
def test_trade_limits_guard_blocks_when_asset_exposure_would_breach_cap() -> None:
guard = TradeLimitsGuard(max_exposure_per_asset=100.0)
guard.open_trade({"BTC": 80.0})
assert not guard.is_trade_allowed({"BTC": 25.0})
assert guard.is_trade_allowed({"ETH": 25.0})
def test_trade_limits_guard_releases_exposure_on_close() -> None:
guard = TradeLimitsGuard(max_concurrent_trades=2, max_exposure_per_asset=100.0)
guard.open_trade({"BTC": 80.0})
guard.close_trade({"BTC": 80.0})
assert guard.active_trades == 0
assert guard.exposure_for_asset("BTC") == 0.0
assert guard.is_trade_allowed({"BTC": 100.0})
def test_trade_limits_guard_rejects_invalid_configuration() -> None:
with pytest.raises(ValueError, match="max_concurrent_trades"):
TradeLimitsGuard(max_concurrent_trades=0)
with pytest.raises(ValueError, match="max_exposure_per_asset"):
TradeLimitsGuard(max_exposure_per_asset=0.0)
@pytest.mark.asyncio
async def test_trade_limits_guard_emits_alert_when_rejecting_trade() -> None:
notifier = _FakeAlertNotifier()
guard = TradeLimitsGuard(max_concurrent_trades=1, alert_notifier=notifier)
guard.open_trade({"BTC": 10.0})
allowed = guard.is_trade_allowed({"BTC": 1.0})
await asyncio.sleep(0)
assert not allowed
assert len(notifier.events) == 1
assert notifier.events[0]["category"] == "threshold"
assert notifier.events[0]["title"] == "Concurrent trade limit reached"
+128 -26
View File
@@ -3,44 +3,146 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>{{ title or "Arbitrade" }}</title>
<title>{% block title %}{{ title or "Arbitrade" }}{% endblock %}</title>
<script src="https://unpkg.com/htmx.org@1.9.12"></script>
{% block head_scripts %}{% endblock %}
<style>
:root {
--bg: #f4f7f5;
--ink: #122118;
--accent: #1f7a4c;
--card: #ffffff;
}
body {
margin: 0;
font-family: "Segoe UI", sans-serif;
color: var(--ink);
background: radial-gradient(circle at top, #e9f7ef, var(--bg));
font-family: Arial, sans-serif;
background: #0b1220;
color: #e5eefb;
}
.wrap {
max-width: 720px;
margin: 4rem auto;
padding: 0 1rem;
.shell {
max-width: 1120px;
margin: 0 auto;
padding: 32px 20px 48px;
}
.hero {
display: flex;
justify-content: space-between;
gap: 24px;
align-items: end;
margin-bottom: 24px;
}
.title {
font-size: 2rem;
margin: 0 0 8px;
}
.subtitle {
margin: 0;
color: #9fb2d0;
}
.panel {
background: rgba(255, 255, 255, 0.04);
border: 1px solid rgba(255, 255, 255, 0.08);
border-radius: 16px;
padding: 20px;
}
.grid {
display: grid;
gap: 16px;
grid-template-columns: repeat(auto-fit, minmax(220px, 1fr));
}
.card {
background: var(--card);
border: 1px solid #d8e7dd;
border-radius: 12px;
padding: 1.25rem;
box-shadow: 0 6px 24px rgba(18, 33, 24, 0.08);
background: rgba(255, 255, 255, 0.04);
border-radius: 14px;
padding: 16px;
border: 1px solid rgba(255, 255, 255, 0.08);
}
.badge {
display: inline-block;
background: #d9f3e5;
color: var(--accent);
border-radius: 999px;
padding: 0.2rem 0.6rem;
.label {
color: #9fb2d0;
font-size: 0.85rem;
margin-bottom: 8px;
}
.value {
font-size: 1.4rem;
font-weight: 700;
}
.meta {
margin-top: 18px;
color: #7f95b7;
font-size: 0.85rem;
}
.toolbar {
display: flex;
gap: 12px;
flex-wrap: wrap;
}
.toolbar form {
margin: 0;
}
.button {
display: inline-flex;
align-items: center;
justify-content: center;
border: 0;
cursor: pointer;
padding: 10px 14px;
border-radius: 999px;
background: #2d6cdf;
color: white;
text-decoration: none;
font: inherit;
}
.button.secondary {
background: transparent;
border: 1px solid rgba(255, 255, 255, 0.14);
}
.button.danger {
background: #ba3d4f;
}
.form-grid {
display: grid;
gap: 12px;
}
.field {
display: grid;
gap: 6px;
color: #9fb2d0;
font-size: 0.9rem;
}
.field input {
width: 100%;
box-sizing: border-box;
padding: 10px 12px;
border-radius: 10px;
border: 1px solid rgba(255, 255, 255, 0.14);
background: rgba(255, 255, 255, 0.04);
color: #e5eefb;
font: inherit;
}
.field.checkbox {
display: flex;
align-items: center;
gap: 8px;
}
.field.checkbox input {
width: auto;
}
.control-actions {
display: flex;
gap: 12px;
flex-wrap: wrap;
}
.chart-head {
display: flex;
justify-content: space-between;
align-items: center;
gap: 12px;
flex-wrap: wrap;
}
.chart-canvas {
width: 100%;
min-height: 260px;
}
</style>
{% block extra_style %}{% endblock %}
</head>
<body>
<main class="wrap">{% block content %}{% endblock %}</main>
<main class="{% block main_class %}shell{% endblock %}">
{% block content %}{% endblock %}
</main>
{% block scripts %}{% endblock %}
</body>
</html>
+179
View File
@@ -0,0 +1,179 @@
{% extends "base.html" %} {% block title %}{{ title }}{% endblock %} {% block
head_scripts %}
<script defer src="https://unpkg.com/alpinejs@3.x.x/dist/cdn.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/chart.js@4.4.3/dist/chart.umd.min.js"></script>
{% endblock %} {% block main_class %}shell{% endblock %} {% block content %}
<section class="hero">
<div>
<h1 class="title">Arbitrade Dashboard</h1>
<p class="subtitle">Live execution, P&amp;L, and system state.</p>
</div>
<div class="toolbar">
<a
class="button"
href="{{ metrics_endpoint }}"
hx-get="{{ metrics_endpoint }}"
hx-target="#metrics-panel"
hx-swap="outerHTML"
>Refresh metrics</a
>
<a class="button secondary" href="/health">Health</a>
</div>
</section>
<section
id="metrics-shell"
hx-get="{{ metrics_endpoint }}"
hx-target="this"
hx-trigger="load, every 15s"
hx-swap="outerHTML"
>
{% include "partials/metrics.html" %}
</section>
<section
id="overview-shell"
hx-get="{{ overview_endpoint }}"
hx-target="this"
hx-trigger="load, every 10s"
hx-swap="outerHTML"
>
{% include "partials/overview.html" %}
</section>
<section
id="controls-shell"
hx-get="{{ controls_endpoint }}"
hx-target="this"
hx-trigger="load, every 20s"
hx-swap="outerHTML"
>
{% include "partials/controls.html" %}
</section>
<section
id="charts-shell"
hx-get="{{ charts_endpoint }}"
hx-target="this"
hx-trigger="load, every 30s"
hx-swap="outerHTML"
>
{% include "partials/charts.html" %}
</section>
<section
id="audit-shell"
hx-get="{{ audit_endpoint }}"
hx-target="this"
hx-trigger="load, every 20s"
hx-swap="outerHTML"
>
{% include "partials/audit.html" %}
</section>
{% endblock %} {% block scripts %}
<script>
window.arbitradeRenderCharts = (payload) => {
const chartHost = document.getElementById("opportunity-chart");
if (!chartHost || typeof Chart === "undefined") {
return;
}
const existing = Chart.getChart(chartHost);
if (existing) {
existing.destroy();
}
const data = JSON.parse(payload);
if (!data.has_chart_data) {
return;
}
new Chart(chartHost, {
type: "line",
data: {
labels: data.labels,
datasets: [
{
label: "Net %",
data: data.net_pct_values,
borderColor: "#2d6cdf",
backgroundColor: "rgba(45, 108, 223, 0.18)",
tension: 0.3,
fill: true,
yAxisID: "y",
},
{
label: "Est profit USD",
data: data.est_profit_values,
borderColor: "#52c41a",
backgroundColor: "rgba(82, 196, 26, 0.12)",
tension: 0.3,
fill: false,
yAxisID: "y1",
},
],
},
options: {
responsive: true,
maintainAspectRatio: false,
interaction: {
mode: "index",
intersect: false,
},
plugins: {
legend: {
labels: {
color: "#e5eefb",
},
},
},
scales: {
x: {
ticks: {
color: "#9fb2d0",
maxRotation: 0,
autoSkip: true,
},
grid: {
color: "rgba(255, 255, 255, 0.06)",
},
},
y: {
position: "left",
ticks: {
color: "#9fb2d0",
},
grid: {
color: "rgba(255, 255, 255, 0.06)",
},
},
y1: {
position: "right",
ticks: {
color: "#9fb2d0",
},
grid: {
drawOnChartArea: false,
},
},
},
},
});
};
const stream = new EventSource("{{ stream_endpoint }}");
stream.addEventListener("metrics", (event) => {
const panel = document.getElementById("metrics-panel");
if (panel) {
panel.outerHTML = JSON.parse(event.data);
}
});
const overviewStream = new EventSource("{{ overview_stream_endpoint }}");
overviewStream.addEventListener("overview", (event) => {
const panel = document.getElementById("overview-panel");
if (panel) {
panel.outerHTML = JSON.parse(event.data);
}
});
</script>
{% endblock %}
+37
View File
@@ -0,0 +1,37 @@
<div id="audit-panel" class="panel" style="margin-top: 16px">
<div class="label">Audit Trail</div>
<div class="meta">Generated {{ generated_at }}</div>
<div style="overflow-x: auto; margin-top: 12px">
<table style="width: 100%; border-collapse: collapse; font-size: 0.9rem">
<thead>
<tr>
<th style="text-align: left; padding: 8px">Time</th>
<th style="text-align: left; padding: 8px">Actor</th>
<th style="text-align: left; padding: 8px">Event</th>
<th style="text-align: left; padding: 8px">Decision</th>
<th style="text-align: left; padding: 8px">Payload</th>
<th style="text-align: left; padding: 8px">Correlation</th>
</tr>
</thead>
<tbody>
{% if entries %}
{% for entry in entries %}
<tr>
<td style="padding: 8px; color: #9fb2d0">{{ entry.occurred_at }}</td>
<td style="padding: 8px">{{ entry.actor }}</td>
<td style="padding: 8px">{{ entry.event_type }}</td>
<td style="padding: 8px">{{ entry.decision }}</td>
<td style="padding: 8px; color: #9fb2d0">{{ entry.payload }}</td>
<td style="padding: 8px; color: #9fb2d0">{{ entry.correlation_id }}</td>
</tr>
{% endfor %}
{% else %}
<tr>
<td colspan="6" style="padding: 8px; color: #9fb2d0">No audit entries yet.</td>
</tr>
{% endif %}
</tbody>
</table>
</div>
</div>
+37
View File
@@ -0,0 +1,37 @@
<div
id="charts-panel"
class="panel"
style="margin-top: 16px"
x-data="{ expanded: true }"
>
<div class="chart-head">
<div>
<div class="label">Opportunity Trend</div>
<div class="meta">Recent opportunities from DuckDB. Updated {{ generated_at }}</div>
</div>
<button type="button" class="button secondary" x-on:click="expanded = !expanded">
<span x-text="expanded ? 'Hide chart' : 'Show chart'"></span>
</button>
</div>
<div x-show="expanded" x-transition style="margin-top: 16px">
<div class="card" style="padding: 12px">
{% if has_chart_data %}
<canvas id="opportunity-chart" class="chart-canvas"></canvas>
<script>
window.arbitradeRenderCharts(
{{ {
"has_chart_data": has_chart_data,
"labels": labels,
"net_pct_values": net_pct_values,
"est_profit_values": est_profit_values,
"cycles": cycles,
} | tojson }}
);
</script>
{% else %}
<div class="meta">No opportunity data yet.</div>
{% endif %}
</div>
</div>
</div>
+121
View File
@@ -0,0 +1,121 @@
<div id="controls-panel" class="panel" style="margin-top: 16px">
<div class="grid">
<article class="card">
<div class="label">Runtime Status</div>
<div class="value">{{ execution_status }}</div>
<div class="meta">Updated {{ updated_at }}</div>
</article>
<article class="card">
<div class="label">Kill Switch</div>
<div class="value">{{ kill_switch_status }}</div>
<div class="meta">Reason {{ kill_switch_reason }}</div>
</article>
<article class="card">
<div class="label">Config Snapshot</div>
<div class="meta">Paper trading: {{ paper_trading_mode }}</div>
<div class="meta">Trade capital: {{ trade_capital_usd }}</div>
<div class="meta">Max trade capital: {{ max_trade_capital_usd }}</div>
<div class="meta">Max concurrent trades: {{ max_concurrent_trades }}</div>
</article>
<article class="card">
<div class="label">Alerting</div>
<div class="meta">Status: {{ alerts_enabled }}</div>
<div class="meta">Channels: {{ alerts_channels }}</div>
<div class="meta">Min severity: {{ alerts_min_severity }}</div>
<div class="meta">Dedup window: {{ alerts_dedup_seconds }}s</div>
<div class="meta">Last result: {{ alerts_last_result }}</div>
<div class="meta">Last attempted: {{ alerts_last_attempted_at }}</div>
<div class="meta">Last success: {{ alerts_last_success_at }}</div>
<div class="meta">Last event: {{ alerts_last_event_title }}</div>
<div class="meta">Last error: {{ alerts_last_error }}</div>
{% if alerts_last_channel_results %} {% for item in
alerts_last_channel_results %}
<div class="meta">{{ item }}</div>
{% endfor %} {% endif %}
</article>
</div>
<div
class="grid"
style="
margin-top: 16px;
grid-template-columns: repeat(auto-fit, minmax(260px, 1fr));
"
>
<article class="card">
<div class="label">Execution Controls</div>
<div class="control-actions">
<form
hx-post="{{ start_endpoint }}"
hx-target="#controls-panel"
hx-swap="outerHTML"
>
<button type="submit" class="button">Start</button>
</form>
<form
hx-post="{{ stop_endpoint }}"
hx-target="#controls-panel"
hx-swap="outerHTML"
>
<button type="submit" class="button secondary">Stop</button>
</form>
<form
hx-post="{{ kill_switch_endpoint }}"
hx-target="#controls-panel"
hx-swap="outerHTML"
>
<input type="hidden" name="reason" value="manual" />
<button type="submit" class="button danger">
Trigger Kill Switch
</button>
</form>
</div>
</article>
<article class="card">
<div class="label">Edit Config</div>
<form
class="form-grid"
hx-post="{{ config_endpoint }}"
hx-target="#controls-panel"
hx-swap="outerHTML"
>
<label class="field">
<span>Trade capital USD</span>
<input
name="trade_capital_usd"
type="number"
min="0"
step="0.01"
value="{{ trade_capital_usd_value }}"
/>
</label>
<label class="field">
<span>Max trade capital USD</span>
<input
name="max_trade_capital_usd"
type="number"
min="0"
step="0.01"
value="{{ max_trade_capital_usd_value }}"
/>
</label>
<label class="field">
<span>Max concurrent trades</span>
<input
name="max_concurrent_trades"
type="number"
min="1"
step="1"
value="{{ max_concurrent_trades_value }}"
/>
</label>
<label class="field checkbox">
{% set check = "checked" if paper_trading_mode == "enabled" else "" %}
<input name="paper_trading_mode" type="checkbox" {{ check }} />
<span>Paper trading mode</span>
</label>
<button type="submit" class="button">Save config</button>
</form>
</article>
</div>
</div>
+31
View File
@@ -0,0 +1,31 @@
<div id="metrics-panel" class="panel">
<div class="grid">
<article class="card">
<div class="label">Realized P&amp;L</div>
<div class="value">{{ realized_pnl }}</div>
</article>
<article class="card">
<div class="label">Win Rate</div>
<div class="value">{{ win_rate }}</div>
</article>
<article class="card">
<div class="label">Avg Trade Duration</div>
<div class="value">{{ avg_trade_duration }}</div>
</article>
<article class="card">
<div class="label">Opportunities / Min</div>
<div class="value">{{ opportunities_per_minute }}</div>
</article>
<article class="card">
<div class="label">Fill Rate</div>
<div class="value">{{ fill_rate }}</div>
</article>
<article class="card">
<div class="label">Latency p50 / p95 / p99</div>
<div class="value">
{{ latency_p50 }} | {{ latency_p95 }} | {{ latency_p99 }}
</div>
</article>
</div>
<div class="meta">Updated {{ generated_at }}</div>
</div>
+67
View File
@@ -0,0 +1,67 @@
<div id="overview-panel" class="panel" style="margin-top: 16px">
<div class="grid">
<article class="card">
<div class="label">Status</div>
<div class="value">{{ status }}</div>
</article>
<article class="card">
<div class="label">Balances</div>
<div class="value">{{ balances }}</div>
</article>
<article class="card">
<div class="label">Open Trades</div>
<div class="value">{{ open_trade_count }}</div>
</article>
<article class="card">
<div class="label">Realized P&amp;L</div>
<div class="value">{{ realized_pnl_total }}</div>
</article>
</div>
<div
class="grid"
style="
margin-top: 16px;
grid-template-columns: repeat(auto-fit, minmax(260px, 1fr));
"
>
<article class="card">
<div class="label">Open Trades</div>
<ul>
{% for trade in open_trades %}
<li>
{{ trade.trade_ref }} - {{ trade.status }} - {{ trade.cycle }} - {{
trade.started_at }}
</li>
{% else %}
<li>No open trades.</li>
{% endfor %}
</ul>
</article>
<article class="card">
<div class="label">Balances Snapshot</div>
<div
class="value"
style="font-size: 1rem; font-weight: 500; word-break: break-word"
>
{{ balances }}
</div>
<div class="meta">Total value {{ total_value }}</div>
</article>
<article class="card">
<div class="label">Opportunity Feed</div>
<ul>
{% for opp in opportunities %}
<li>
{{ opp.cycle }} - {{ opp.net_pct }} - {{ opp.est_profit }} - {{
opp.detected_at }}
</li>
{% else %}
<li>No opportunities.</li>
{% endfor %}
</ul>
</article>
</div>
<div class="meta">Updated {{ generated_at }}</div>
</div>