Arbitrade

Low-latency cryptocurrency arbitrage bot scaffold for Kraken.

Current stack:

• Python 3.12+
• FastAPI + HTMX/Jinja2
• DuckDB for dev/test/prod
• Native Kraken WebSocket planned for market-data hot path
• Gitea Actions + Gitea container registry

Project plan lives in PLAN.md. Task checklist lives in .github/instructions/TODO.md.

Current Status

Bootstrap complete for foundation layer:

• repo initialized
• typed settings and env loading
• structured logging
• encrypted secret helpers
• DuckDB connection + base schema
• FastAPI app with health endpoint
• Gitea Actions CI scaffold
• Docker / docker-compose scaffold

Not implemented yet:

• Kraken REST client
• Kraken native WebSocket client
• arbitrage detection engine
• trade execution
• dashboard beyond health/bootstrap page

Prerequisites

• Python 3.12+
• uv for env/package management
• Git
• Docker Desktop or Docker Engine
• Gitea account on git.allucanget.biz for push/CI/registry access

Optional:

• PowerShell 7 on Windows

Repository Setup

Clone repo:

git clone https://git.allucanget.biz/allucanget/arbitrade.git
Set-Location arbitrade

If repo already exists locally, confirm remote:

git remote -v

Expected origin:

https://git.allucanget.biz/allucanget/arbitrade.git

Local Development Setup

Create virtualenv with uv:

uv venv

Activate env on Windows:

.\.venv\Scripts\Activate.ps1

Install app + dev dependencies:

uv pip install -e .[dev]

Dependency source of truth:

• Runtime dependencies live in requirements/latest-runtime.in.
• Dev dependencies live in requirements/latest-dev.in.
• pyproject.toml reads both files dynamically during package install.

Create local env file:

Copy-Item .env.example .env

Minimum .env values:

APP_ENV=dev
APP_HOST=0.0.0.0
APP_PORT=9090
LOG_LEVEL=INFO
LOG_JSON=true
DUCKDB_PATH=./data/arbitrade.duckdb
FERNET_KEY=
KRAKEN_API_KEY=
KRAKEN_API_SECRET=
KRAKEN_API_KEY_PERMISSIONS=query,trade

Notes:

• Leave Kraken creds empty until Kraken integration lands.
• If Kraken creds are set, both key and secret are required.
• KRAKEN_API_KEY_PERMISSIONS must include query,trade and must not include withdrawal scope.
• FERNET_KEY optional. If empty, keyring-backed key generation used by secret helper.
• On Windows, app falls back to default asyncio loop. On non-Windows, uvloop installs automatically.

Run App

Start app:

python -m arbitrade.main

Health endpoints:

• HTML: http://localhost:9090/
• JSON: http://localhost:9090/health

Database

DuckDB used everywhere: local dev, tests, production.

Default database file:

./data/arbitrade.duckdb

Schema bootstrap runs automatically on app startup.

Current tables:

• schema_migrations
• opportunities
• trades
• portfolio_snapshots

Audit trail table:

• audit_events (append-only operational decision log)

Audit retention and compaction guidance:

• Keep at least 30 days of audit_events in active DB for incident triage.
• Archive older rows to a timestamped export file before deletion.
• Example monthly archive workflow:

COPY (
  SELECT *
  FROM audit_events
  WHERE occurred_at < NOW() - INTERVAL 30 DAY
) TO 'data/audit_events_archive_YYYYMM.parquet' (FORMAT PARQUET);

DELETE FROM audit_events
WHERE occurred_at < NOW() - INTERVAL 30 DAY;

• Back up archive files and the main DuckDB file together.
• For production, run archive + backup as scheduled maintenance (cron/task scheduler).

Quality Checks

Run tests:

pytest -q

Run Ruff:

ruff check .

Run Black check:

black --check .

Run mypy:

mypy src

Run dependency vulnerability audit:

pip-audit -r requirements/latest-runtime.in

Run secret scan (worktree + git history):

python scripts/security_scan.py

Generate latency profile baseline:

python scripts/profile_latency.py --iterations 600 --output ops/performance/latency_baseline.json

Run latency regression guardrails:

python scripts/check_latency_regression.py --baseline ops/performance/latency_baseline.json --thresholds ops/performance/latency_thresholds.json --iterations 600

Install pre-commit hooks:

pre-commit install

Run hooks manually:

pre-commit run --all-files

Docker

Build locally:

docker build -t arbitrade:local .

Container dependency install flow:

• Docker installs runtime dependencies from requirements/latest-runtime.in.
• Docker then installs the package with --no-deps so dependency resolution is driven by requirements files.

Run with compose:

docker compose up --build

Compose mounts local data/ folder into container at /app/data.

Important:

• docker-compose.yml uses git.allucanget.biz/allucanget/arbitrade:latest as the default image reference.

Coolify Deployment (Prebuilt Image)

Use this when deploying from the image published by CI instead of building from Git inside Coolify.

1) Create application in Coolify

• In Coolify, create a new Application using Docker Image / Public Image / Private Registry Image.
• Image: git.allucanget.biz/allucanget/arbitrade:latest
• Registry: git.allucanget.biz
• If registry auth is required, configure the same registry credentials in Coolify.

2) Configure build and start behavior

Set these in Coolify application settings:

• Build Command: leave empty.
• Install Command: leave empty.
• Start Command: leave empty unless you explicitly want to override the image default.
• Port: 9090 (coolify uses 8000 internally)

3) Configure health check and networking

• Health Check Path: /health
• Exposed Port: 9090
• Use Coolify-generated domain or attach your own domain.

4) Configure persistent storage

Add a persistent volume in Coolify:

• Mount Path: /app/data

This preserves DuckDB and other runtime artifacts across restarts/redeploys.

5) Configure environment variables

Add runtime environment variables in Coolify (UI: Environment Variables):

• APP_ENV=prod
• APP_HOST=0.0.0.0
• APP_PORT=9090
• DUCKDB_PATH=/app/data/arbitrade.duckdb
• LOG_LEVEL=INFO
• LOG_JSON=true
• KRAKEN_API_KEY=...
• KRAKEN_API_SECRET=...
• KRAKEN_API_KEY_PERMISSIONS=query,trade

Recommended:

• Configure FERNET_KEY in Coolify secrets (do not commit it).
• Keep all exchange keys/secrets in Coolify secret variables only.

Coolify should own runtime configuration through environment variables. CI only publishes the image.

6) Deploy and verify

• Trigger deploy in Coolify after CI publishes git.allucanget.biz/allucanget/arbitrade:latest.
• Verify app boot logs show startup completed.
• Verify GET /health returns success on deployed URL.

Gitea CI / Registry Setup

CI file:

• .gitea/workflows/ci.yml

Required Gitea Actions secrets:

• REGISTRY_USERNAME
• REGISTRY_TOKEN

Example registry login:

docker login git.allucanget.biz

Example pushed image tag shape:

git.allucanget.biz/allucanget/arbitrade:latest

Architecture Docs

Implementation detail moved into arc42 docs:

• arc42 overview - system context, building blocks, runtime, deployment, quality goals, risks.
• current implementation snapshot - codebase state, active routes, backtesting, strategy flags, deployment flow.

For navigation from README, use the docs above instead of this file for deep architecture detail.