allucanget/arbitrade
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add backtesting functionality with UI and API endpoints
CI / lint-test-build (push) Successful in 2m31s
Details

Browse Source 
- Introduced backtesting page and fragment in the dashboard for running backtests and viewing recent reports.
- Implemented backtest run logic with configuration options including event path, starting balances, trade capital, and fee profiles.
- Added recent backtest reports storage and retrieval.
- Created a new strategy module for statistical arbitrage experiments with validation on configuration parameters.
- Updated settings to include parameters for the statistical arbitrage strategy.
- Enhanced dashboard controls to support the new strategy mode.
- Added unit tests for backtesting functionality and strategy validation.
- Updated templates for backtesting UI integration.
This commit is contained in:
zwitschi
2026-06-02 09:28:22 +02:00
parent f612c8533a
commit 38e1d64437
17 changed files with 1089 additions and 165 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture/arc42.md
+156
View File
@@ -0,0 +1,156 @@
# Arbitrade Architecture Overview (arc42)
## 1. Introduction and Goals
Arbitrade is a Python 3.12+ cryptocurrency arbitrage bot for Kraken, focused on triangular arbitrage on a single exchange. The system is designed for low-latency detection, configurable risk control, replayable backtesting, and operator-visible dashboard control.
Primary goals:
- Detect and execute triangular opportunities on Kraken with fee/slippage-aware math.
- Keep hot-path latency low with incremental order-book updates and event-driven scoring.
- Persist operational data in DuckDB for dev, test, and prod.
- Provide operator controls, audit trail, and alerting through a server-rendered dashboard.
- Support backtesting, parameter sweeps, and deferred experimental strategy work behind feature flags.
## 2. Constraints
- Python 3.12+ runtime.
- Native Kraken WebSocket on the hot path.
- HTMX + Jinja2 UI, no SPA build step.
- DuckDB everywhere.
- Self-hosted Gitea Actions CI and Gitea registry.
- Windows development support.
- Secrets must stay out of the repository.
## 3. Context and Scope
### 3.1 Business Context
The bot consumes Kraken market data, detects opportunities, and executes trades or paper-trades depending on configuration. Operators monitor and control the system through a dashboard and alerting channels.
### 3.2 Technical Context
- Kraken REST + WebSocket provide market data and execution.
- FastAPI serves HTML fragments, JSON endpoints, and SSE streams.
- DuckDB stores trades, opportunities, snapshots, audit events, and runtime state.
- Coolify can deploy the published image using environment variables and persistent storage.
## 4. Solution Strategy
- Use an incremental currency graph to re-score only cycles touched by a changed pair.
- Use top-of-book plus depth-aware pricing and configurable fee/slippage buffers.
- Use a single-process asyncio model with uvloop where available.
- Keep strategy, risk, execution, and backtesting logic reusable across paper and replay modes.
- Expose configuration through the dashboard and environment variables.
## 5. Building Block View
### 5.1 Runtime Blocks
- `api/` - FastAPI app, routes, dashboard fragments, backtesting endpoints.
- `exchange/` - Kraken REST and WebSocket integration.
- `market_data/` - live order-book state and ingestion.
- `detection/` - triangular graph and incremental detector.
- `risk/` - pre-trade and trade-limit guards.
- `execution/` - multi-leg trade sequencing.
- `backtesting/` - replay engine, parameter sweep, experiment scaffolds.
- `strategy/` - experimental strategy modules such as stat-arb.
- `storage/` - DuckDB schema and repositories.
- `alerting/` - multi-channel notifications.
- `runtime/` - startup recovery and graceful shutdown.
### 5.2 Important Dependencies
- `fastapi`, `uvicorn`, `jinja2`, `htmx`-driven templates.
- `orjson` for low-alloc parsing.
- `sortedcontainers` for book state.
- `duckdb` for persistence and analytics.
- `pydantic` / `pydantic-settings` for typed configuration.
- `cryptography` / keyring for secret handling.
## 6. Runtime View
### 6.1 Live Trading Flow
1. Kraken WS delivers book updates.
2. Order book updates in memory.
3. Incremental detector scores impacted cycles.
4. Risk manager validates the opportunity.
5. Execution sequencer places legs if approved.
6. Trades and snapshots persist to DuckDB.
7. Dashboard and alerts reflect state changes.
### 6.2 Dashboard Control Flow
- `/dashboard/control/*` mutates runtime state.
- `/dashboard/fragment/*` renders HTMX partials.
- `/dashboard/stream/*` provides SSE live updates.
- `/dashboard/backtesting` provides a dedicated replay control page.
### 6.3 Backtesting Flow
1. User selects JSONL replay file and run parameters.
2. Replay engine loads ordered book events.
3. Detector, risk, and execution logic run in simulation mode.
4. Report is stored in memory for recent UI display.
5. Parameter sweeps split data into train/test windows, rank results, and flag overfit.
## 7. Deployment View
### 7.1 Local Development
- `uv venv` for environment creation.
- `uv pip install -e .[dev]` for editable install.
- `docker compose up --build` for local container workflow.
### 7.2 CI/CD
- Gitea Actions runs lint, tests, security checks, latency guards, and image publish.
- CI publishes `git.allucanget.biz/allucanget/arbitrade:latest`.
### 7.3 Coolify
- Deploy from the published image.
- Configure runtime via environment variables.
- Mount persistent storage at `/app/data` for DuckDB.
## 8. Cross-Cutting Concepts
- Staleness checks prevent stale book execution.
- Kill switch halts execution immediately.
- Audit trail records dashboard and runtime decisions.
- Alerting spans Telegram, Discord, and email.
- Feature flags gate experimental strategy code.
- Config is environment-driven and validated at startup.
## 9. Architecture Decisions
- Native Kraken WS instead of a generic exchange abstraction on the hot path.
- DuckDB as the single database engine.
- HTMX + Jinja2 instead of SPA frontend.
- Backtesting reuses production detector/risk/execution logic.
- Experimental stat-arb stays behind a feature flag.
- Published image is the deployment artifact; Coolify owns runtime env vars.
## 10. Quality Requirements
- Low latency on book-update-to-decision path.
- Safe startup and restart behavior.
- Strong operator visibility.
- Reproducible backtests and sweeps.
- Secrets protection and strict validation.
## 11. Risks and Technical Debt
- Exchange API schema changes.
- Spread decay and execution slippage.
- Cross-venue strategy complexity if/when enabled.
- UI and backtesting paths can drift if not kept aligned with production logic.
## 12. Glossary
- WS: WebSocket.
- HTMX: HTML-over-the-wire UI library.
- SSE: Server-Sent Events.
- DUCKDB: Embedded analytical database used for all environments.
- Stat arb: Statistical arbitrage, currently experimental and feature-flagged.
docs/architecture/current-implementation.md
+47
View File
@@ -0,0 +1,47 @@
# Current Implementation Snapshot
This document summarizes the code that exists now, not the original plan.
## Runtime
- FastAPI app starts from [src/arbitrade/api/app.py](../../src/arbitrade/api/app.py).
- Settings come from `pydantic-settings` in [src/arbitrade/config/settings.py](../../src/arbitrade/config/settings.py).
- DuckDB is initialized and migrated on startup.
- Runtime recovery persists and restores control state and snapshots.
## Market Data and Detection
- Kraken market data is handled by native WS and thin REST code.
- Incremental triangular detector is implemented in [src/arbitrade/detection/engine.py](../../src/arbitrade/detection/engine.py).
- Currency graph and cycle indexing live in [src/arbitrade/detection/graph.py](../../src/arbitrade/detection/graph.py).
## Execution and Risk
- Multi-leg execution sequencer exists for triangular cycles.
- Pre-trade validation and trade-limit guards are wired into execution flow.
- Kill switch and stop conditions are supported.
## Dashboard
- Server-rendered dashboard uses FastAPI + Jinja2 + HTMX.
- Live metrics, overview, controls, charts, and audit fragments are exposed as separate endpoints.
- Dedicated backtesting page exists at `/dashboard/backtesting`.
## Backtesting
- Replay engine lives in [src/arbitrade/backtesting/replay.py](../../src/arbitrade/backtesting/replay.py).
- Parameter sweep runner lives in [src/arbitrade/backtesting/sweep.py](../../src/arbitrade/backtesting/sweep.py).
- Backtesting UI runs replay from a JSONL file, stores recent reports in app state, and exposes a recent-reports API.
- Experimental stat-arb scaffold lives in [src/arbitrade/strategy/stat_arb.py](../../src/arbitrade/strategy/stat_arb.py) and is gated by feature flag.
## Deployment
- Dockerfile installs runtime dependencies from `requirements/latest-runtime.in`.
- CI publishes `git.allucanget.biz/allucanget/arbitrade:latest`.
- Coolify deploys the prebuilt image and owns runtime env vars and persistent storage.
## Current Gaps
- Cross-exchange arbitrage remains deferred.
- Stat-arb is experimental, not part of default live strategy.
- Backtesting UI is functional but still a single-run/report workflow, not a full job queue.