arbitrade/README.md

Arbitrade

Low-latency cryptocurrency arbitrage bot scaffold for Kraken.

Current stack:

• Python 3.12+
• FastAPI + HTMX/Jinja2
• DuckDB for dev/test/prod
• Native Kraken WebSocket planned for market-data hot path
• Gitea Actions + Gitea container registry

Project plan lives in PLAN.md. Task checklist lives in .github/instructions/TODO.md.

Current Status

Bootstrap complete for foundation layer:

• repo initialized
• typed settings and env loading
• structured logging
• encrypted secret helpers
• DuckDB connection + base schema
• FastAPI app with health endpoint
• Gitea Actions CI scaffold
• Docker / docker-compose scaffold

Not implemented yet:

• Kraken REST client
• Kraken native WebSocket client
• arbitrage detection engine
• trade execution
• dashboard beyond health/bootstrap page

Prerequisites

• Python 3.12+
• uv for env/package management
• Git
• Docker Desktop or Docker Engine
• Gitea account on git.allucanget.biz for push/CI/registry access

Optional:

• PowerShell 7 on Windows

Repository Setup

Clone repo:

git clone https://git.allucanget.biz/allucanget/arbitrade.git
Set-Location arbitrade

If repo already exists locally, confirm remote:

git remote -v

Expected origin:

https://git.allucanget.biz/allucanget/arbitrade.git

Local Development Setup

Create virtualenv with uv:

uv venv

Activate env on Windows:

.\.venv\Scripts\Activate.ps1

Install app + dev dependencies:

uv pip install -e .[dev]

Create local env file:

Copy-Item .env.example .env

Minimum .env values:

APP_ENV=dev
APP_HOST=0.0.0.0
APP_PORT=8000
LOG_LEVEL=INFO
LOG_JSON=true
DUCKDB_PATH=./data/arbitrade.duckdb
FERNET_KEY=
KRAKEN_API_KEY=
KRAKEN_API_SECRET=
KRAKEN_API_KEY_PERMISSIONS=query,trade

Notes:

• Leave Kraken creds empty until Kraken integration lands.
• If Kraken creds are set, both key and secret are required.
• KRAKEN_API_KEY_PERMISSIONS must include query,trade and must not include withdrawal scope.
• FERNET_KEY optional. If empty, keyring-backed key generation used by secret helper.
• On Windows, app falls back to default asyncio loop. On non-Windows, uvloop installs automatically.

Run App

Start app:

python -m arbitrade.main

Health endpoints:

• HTML: http://localhost:8000/
• JSON: http://localhost:8000/health

Database

DuckDB used everywhere: local dev, tests, production.

Default database file:

./data/arbitrade.duckdb

Schema bootstrap runs automatically on app startup.

Current tables:

• schema_migrations
• opportunities
• trades
• portfolio_snapshots

Audit trail table:

• audit_events (append-only operational decision log)

Audit retention and compaction guidance:

• Keep at least 30 days of audit_events in active DB for incident triage.
• Archive older rows to a timestamped export file before deletion.
• Example monthly archive workflow:

COPY (
  SELECT *
  FROM audit_events
  WHERE occurred_at < NOW() - INTERVAL 30 DAY
) TO 'data/audit_events_archive_YYYYMM.parquet' (FORMAT PARQUET);

DELETE FROM audit_events
WHERE occurred_at < NOW() - INTERVAL 30 DAY;

• Back up archive files and the main DuckDB file together.
• For production, run archive + backup as scheduled maintenance (cron/task scheduler).

Quality Checks

Run tests:

pytest -q

Run Ruff:

ruff check .

Run Black check:

black --check .

Run mypy:

mypy src

Run dependency vulnerability audit:

pip-audit --skip-editable

Run secret scan (worktree + git history):

python scripts/security_scan.py

Install pre-commit hooks:

pre-commit install

Run hooks manually:

pre-commit run --all-files

Docker

Build locally:

docker build -t arbitrade:local .

Run with compose:

docker compose up --build

Compose mounts local data/ folder into container at /app/data.

Important:

• docker-compose.yml still uses registry image placeholder format.
• Replace git.allucanget.biz/OWNER/arbitrade:latest with actual namespace image, likely git.allucanget.biz/allucanget/arbitrade:latest.

Gitea CI / Registry Setup

CI file:

• .gitea/workflows/ci.yml

Required Gitea Actions secrets:

• REGISTRY_USERNAME
• REGISTRY_TOKEN
• REGISTRY_NAMESPACE

Expected namespace now likely:

allucanget

Example registry login:

docker login git.allucanget.biz

Example pushed image tag shape:

git.allucanget.biz/allucanget/arbitrade:<tag>

Project Layout

arbitrade/
├── .gitea/workflows/ci.yml
├── .github/instructions/TODO.md
├── PLAN.md
├── pyproject.toml
├── src/arbitrade/
│   ├── api/
│   ├── config/
│   ├── storage/
│   ├── logging_setup.py
│   └── main.py
├── tests/
└── web/templates/

Next Work

Next planned implementation slice:

• Kraken REST client skeleton
• native Kraken WebSocket client
• in-memory order book cache
• latency instrumentation

Troubleshooting

PowerShell blocks activation script:

Set-ExecutionPolicy -Scope Process -ExecutionPolicy RemoteSigned

Then activate again:

.\.venv\Scripts\Activate.ps1

If app import fails, confirm editable install ran:

uv pip install -e .[dev]

If DuckDB file missing, start app once or create data/ directory manually.

Security Hardening

Threat model notes:

• Primary risk surfaces: environment secrets, dashboard auth credentials, exchange API key scope, and dependency supply chain.
• Assumed attacker model: leaked repository content, leaked CI logs/artifacts, or unauthorized dashboard access.
• High-impact outcomes to prevent: credential exfiltration, unauthorized withdrawals, and unsafe live-trading control changes.

Hardening checklist:

• Use least-privilege Kraken API keys: query + trade only; never enable withdrawal.
• Rotate API keys immediately if secret scan flags a potential exposure.
• Keep dashboard auth enabled in non-local environments and avoid default/shared credentials.
• Run pip-audit --skip-editable in CI; treat vulnerability findings as release blockers.
• Run python scripts/security_scan.py before release and after major merges.
• Store secrets in environment/secret manager; never commit .env or key material.