feat: update documentation structure and add deployment guide
This commit is contained in:
@@ -0,0 +1,156 @@
|
||||
# Arbitrade Architecture Overview (arc42)
|
||||
|
||||
## 1. Introduction and Goals
|
||||
|
||||
Arbitrade is a Python 3.12+ cryptocurrency arbitrage bot for Kraken, focused on triangular arbitrage on a single exchange. The system is designed for low-latency detection, configurable risk control, replayable backtesting, and operator-visible dashboard control.
|
||||
|
||||
Primary goals:
|
||||
|
||||
- Detect and execute triangular opportunities on Kraken with fee/slippage-aware math.
|
||||
- Keep hot-path latency low with incremental order-book updates and event-driven scoring.
|
||||
- Persist operational data in DuckDB for dev, test, and prod.
|
||||
- Provide operator controls, audit trail, and alerting through a server-rendered dashboard.
|
||||
- Support backtesting, parameter sweeps, and deferred experimental strategy work behind feature flags.
|
||||
|
||||
## 2. Constraints
|
||||
|
||||
- Python 3.12+ runtime.
|
||||
- Native Kraken WebSocket on the hot path.
|
||||
- HTMX + Jinja2 UI, no SPA build step.
|
||||
- DuckDB everywhere.
|
||||
- Self-hosted Gitea Actions CI and Gitea registry.
|
||||
- Windows development support.
|
||||
- Secrets must stay out of the repository.
|
||||
|
||||
## 3. Context and Scope
|
||||
|
||||
### 3.1 Business Context
|
||||
|
||||
The bot consumes Kraken market data, detects opportunities, and executes trades or paper-trades depending on configuration. Operators monitor and control the system through a dashboard and alerting channels.
|
||||
|
||||
### 3.2 Technical Context
|
||||
|
||||
- Kraken REST + WebSocket provide market data and execution.
|
||||
- FastAPI serves HTML fragments, JSON endpoints, and SSE streams.
|
||||
- DuckDB stores trades, opportunities, snapshots, audit events, and runtime state.
|
||||
- Coolify can deploy the published image using environment variables and persistent storage.
|
||||
|
||||
## 4. Solution Strategy
|
||||
|
||||
- Use an incremental currency graph to re-score only cycles touched by a changed pair.
|
||||
- Use top-of-book plus depth-aware pricing and configurable fee/slippage buffers.
|
||||
- Use a single-process asyncio model with uvloop where available.
|
||||
- Keep strategy, risk, execution, and backtesting logic reusable across paper and replay modes.
|
||||
- Expose configuration through the dashboard and environment variables.
|
||||
|
||||
## 5. Building Block View
|
||||
|
||||
### 5.1 Runtime Blocks
|
||||
|
||||
- `api/` - FastAPI app, routes, dashboard fragments, backtesting endpoints.
|
||||
- `exchange/` - Kraken REST and WebSocket integration.
|
||||
- `market_data/` - live order-book state and ingestion.
|
||||
- `detection/` - triangular graph and incremental detector.
|
||||
- `risk/` - pre-trade and trade-limit guards.
|
||||
- `execution/` - multi-leg trade sequencing.
|
||||
- `backtesting/` - replay engine, parameter sweep, experiment scaffolds.
|
||||
- `strategy/` - experimental strategy modules such as stat-arb.
|
||||
- `storage/` - DuckDB schema and repositories.
|
||||
- `alerting/` - multi-channel notifications.
|
||||
- `runtime/` - startup recovery and graceful shutdown.
|
||||
|
||||
### 5.2 Important Dependencies
|
||||
|
||||
- `fastapi`, `uvicorn`, `jinja2`, `htmx`-driven templates.
|
||||
- `orjson` for low-alloc parsing.
|
||||
- `sortedcontainers` for book state.
|
||||
- `duckdb` for persistence and analytics.
|
||||
- `pydantic` / `pydantic-settings` for typed configuration.
|
||||
- `cryptography` / keyring for secret handling.
|
||||
|
||||
## 6. Runtime View
|
||||
|
||||
### 6.1 Live Trading Flow
|
||||
|
||||
1. Kraken WS delivers book updates.
|
||||
2. Order book updates in memory.
|
||||
3. Incremental detector scores impacted cycles.
|
||||
4. Risk manager validates the opportunity.
|
||||
5. Execution sequencer places legs if approved.
|
||||
6. Trades and snapshots persist to DuckDB.
|
||||
7. Dashboard and alerts reflect state changes.
|
||||
|
||||
### 6.2 Dashboard Control Flow
|
||||
|
||||
- `/dashboard/control/*` mutates runtime state.
|
||||
- `/dashboard/fragment/*` renders HTMX partials.
|
||||
- `/dashboard/stream/*` provides SSE live updates.
|
||||
- `/dashboard/backtesting` provides a dedicated replay control page.
|
||||
|
||||
### 6.3 Backtesting Flow
|
||||
|
||||
1. User selects JSONL replay file and run parameters.
|
||||
2. Replay engine loads ordered book events.
|
||||
3. Detector, risk, and execution logic run in simulation mode.
|
||||
4. Report is stored in memory for recent UI display.
|
||||
5. Parameter sweeps split data into train/test windows, rank results, and flag overfit.
|
||||
|
||||
## 7. Deployment View
|
||||
|
||||
### 7.1 Local Development
|
||||
|
||||
- `uv venv` for environment creation.
|
||||
- `uv pip install -e .[dev]` for editable install.
|
||||
- `docker compose up --build` for local container workflow.
|
||||
|
||||
### 7.2 CI/CD
|
||||
|
||||
- Gitea Actions runs lint, tests, security checks, latency guards, and image publish.
|
||||
- CI publishes `git.allucanget.biz/allucanget/arbitrade:latest`.
|
||||
|
||||
### 7.3 Coolify
|
||||
|
||||
- Deploy from the published image.
|
||||
- Configure runtime via environment variables.
|
||||
- Mount persistent storage at `/app/data` for DuckDB.
|
||||
|
||||
## 8. Cross-Cutting Concepts
|
||||
|
||||
- Staleness checks prevent stale book execution.
|
||||
- Kill switch halts execution immediately.
|
||||
- Audit trail records dashboard and runtime decisions.
|
||||
- Alerting spans Telegram, Discord, and email.
|
||||
- Feature flags gate experimental strategy code.
|
||||
- Config is environment-driven and validated at startup.
|
||||
|
||||
## 9. Architecture Decisions
|
||||
|
||||
- Native Kraken WS instead of a generic exchange abstraction on the hot path.
|
||||
- DuckDB as the single database engine.
|
||||
- HTMX + Jinja2 instead of SPA frontend.
|
||||
- Backtesting reuses production detector/risk/execution logic.
|
||||
- Experimental stat-arb stays behind a feature flag.
|
||||
- Published image is the deployment artifact; Coolify owns runtime env vars.
|
||||
|
||||
## 10. Quality Requirements
|
||||
|
||||
- Low latency on book-update-to-decision path.
|
||||
- Safe startup and restart behavior.
|
||||
- Strong operator visibility.
|
||||
- Reproducible backtests and sweeps.
|
||||
- Secrets protection and strict validation.
|
||||
|
||||
## 11. Risks and Technical Debt
|
||||
|
||||
- Exchange API schema changes.
|
||||
- Spread decay and execution slippage.
|
||||
- Cross-venue strategy complexity if/when enabled.
|
||||
- UI and backtesting paths can drift if not kept aligned with production logic.
|
||||
|
||||
## 12. Glossary
|
||||
|
||||
- WS: WebSocket.
|
||||
- HTMX: HTML-over-the-wire UI library.
|
||||
- SSE: Server-Sent Events.
|
||||
- DUCKDB: Embedded analytical database used for all environments.
|
||||
- Stat arb: Statistical arbitrage, currently experimental and feature-flagged.
|
||||
Reference in New Issue
Block a user