arbitrade/docs/architecture/README.md

Arbitrade Architecture Overview (arc42)

1. Introduction and Goals

Arbitrade is a Python 3.12+ cryptocurrency arbitrage bot for Kraken, focused on triangular arbitrage on a single exchange. The system is designed for low-latency detection, configurable risk control, replayable backtesting, and operator-visible dashboard control.

Primary goals:

• Detect and execute triangular opportunities on Kraken with fee/slippage-aware math.
• Keep hot-path latency low with incremental order-book updates and event-driven scoring.
• Persist operational data in DuckDB for dev, test, and prod.
• Provide operator controls, audit trail, and alerting through a server-rendered dashboard.
• Support backtesting, parameter sweeps, and deferred experimental strategy work behind feature flags.

2. Constraints

• Python 3.12+ runtime.
• Native Kraken WebSocket on the hot path.
• HTMX + Jinja2 UI, no SPA build step.
• DuckDB everywhere.
• Self-hosted Gitea Actions CI and Gitea registry.
• Windows development support.
• Secrets must stay out of the repository.

3. Context and Scope

3.1 Business Context

The bot consumes Kraken market data, detects opportunities, and executes trades or paper-trades depending on configuration. Operators monitor and control the system through a dashboard and alerting channels.

3.2 Technical Context

• Kraken REST + WebSocket provide market data and execution.
• FastAPI serves HTML fragments, JSON endpoints, and SSE streams.
• DuckDB stores trades, opportunities, snapshots, audit events, and runtime state.
• Coolify can deploy the published image using environment variables and persistent storage.

4. Solution Strategy

• Use an incremental currency graph to re-score only cycles touched by a changed pair.
• Use top-of-book plus depth-aware pricing and configurable fee/slippage buffers.
• Use a single-process asyncio model with uvloop where available.
• Keep strategy, risk, execution, and backtesting logic reusable across paper and replay modes.
• Expose configuration through the dashboard and environment variables.

5. Building Block View

5.1 Runtime Blocks

• api/ - FastAPI app, routes, dashboard fragments, backtesting endpoints.
• exchange/ - Kraken REST and WebSocket integration.
• market_data/ - live order-book state and ingestion.
• detection/ - triangular graph and incremental detector.
• risk/ - pre-trade and trade-limit guards.
• execution/ - multi-leg trade sequencing.
• backtesting/ - replay engine, parameter sweep, experiment scaffolds.
• strategy/ - experimental strategy modules such as stat-arb.
• storage/ - DuckDB schema and repositories.
• alerting/ - multi-channel notifications.
• runtime/ - startup recovery and graceful shutdown.

5.2 Important Dependencies

• fastapi, uvicorn, jinja2, htmx-driven templates.
• orjson for low-alloc parsing.
• sortedcontainers for book state.
• duckdb for persistence and analytics.
• pydantic / pydantic-settings for typed configuration.
• cryptography / keyring for secret handling.

6. Runtime View

6.1 Live Trading Flow

1. Kraken WS delivers book updates.
2. Order book updates in memory.
3. Incremental detector scores impacted cycles.
4. Risk manager validates the opportunity.
5. Execution sequencer places legs if approved.
6. Trades and snapshots persist to DuckDB.
7. Dashboard and alerts reflect state changes.

6.2 Dashboard Control Flow

• /dashboard/control/* mutates runtime state.
• /dashboard/fragment/* renders HTMX partials.
• /dashboard/stream/* provides SSE live updates.
• /dashboard/backtesting provides a dedicated replay control page.

6.3 Backtesting Flow

1. User selects JSONL replay file and run parameters.
2. Replay engine loads ordered book events.
3. Detector, risk, and execution logic run in simulation mode.
4. Report is stored in memory for recent UI display.
5. Parameter sweeps split data into train/test windows, rank results, and flag overfit.

7. Deployment View

7.1 Local Development

• uv venv for environment creation.
• uv pip install -e .[dev] for editable install.
• docker compose up --build for local container workflow.

7.2 CI/CD

• Gitea Actions runs lint, tests, security checks, latency guards, and image publish.
• CI publishes git.allucanget.biz/allucanget/arbitrade:latest.

7.3 Coolify

• Deploy from the published image.
• Configure runtime via environment variables.
• Mount persistent storage at /app/data for DuckDB.

8. Cross-Cutting Concepts

• Staleness checks prevent stale book execution.
• Kill switch halts execution immediately.
• Audit trail records dashboard and runtime decisions.
• Alerting spans Telegram, Discord, and email.
• Feature flags gate experimental strategy code.
• Config is environment-driven and validated at startup.

9. Architecture Decisions

• Native Kraken WS instead of a generic exchange abstraction on the hot path.
• DuckDB as the single database engine.
• HTMX + Jinja2 instead of SPA frontend.
• Backtesting reuses production detector/risk/execution logic.
• Experimental stat-arb stays behind a feature flag.
• Published image is the deployment artifact; Coolify owns runtime env vars.

10. Quality Requirements

• Low latency on book-update-to-decision path.
• Safe startup and restart behavior.
• Strong operator visibility.
• Reproducible backtests and sweeps.
• Secrets protection and strict validation.

11. Risks and Technical Debt

• Exchange API schema changes.
• Spread decay and execution slippage.
• Cross-venue strategy complexity if/when enabled.
• UI and backtesting paths can drift if not kept aligned with production logic.

12. Glossary

• WS: WebSocket.
• HTMX: HTML-over-the-wire UI library.
• SSE: Server-Sent Events.
• DUCKDB: Embedded analytical database used for all environments.
• Stat arb: Statistical arbitrage, currently experimental and feature-flagged.