# Arbitrade

Low-latency cryptocurrency arbitrage bot scaffold for Kraken.

Current stack:

- Python 3.12+
- FastAPI + HTMX/Jinja2
- DuckDB for dev/test/prod
- Native Kraken WebSocket planned for market-data hot path
- Gitea Actions + Gitea container registry

Project plan lives in [PLAN.md](PLAN.md).
Task checklist lives in [.github/instructions/TODO.md](.github/instructions/TODO.md).

## Current Status

Bootstrap complete for foundation layer:

- repo initialized
- typed settings and env loading
- structured logging
- encrypted secret helpers
- DuckDB connection + base schema
- FastAPI app with health endpoint
- Gitea Actions CI scaffold
- Docker / docker-compose scaffold

Not implemented yet:

- Kraken REST client
- Kraken native WebSocket client
- arbitrage detection engine
- trade execution
- dashboard beyond health/bootstrap page

## Prerequisites

- Python 3.12+
- `uv` for env/package management
- Git
- Docker Desktop or Docker Engine
- Gitea account on `git.allucanget.biz` for push/CI/registry access

Optional:

- PowerShell 7 on Windows

## Repository Setup

Clone repo:

```powershell
git clone https://git.allucanget.biz/allucanget/arbitrade.git
Set-Location arbitrade
```

If repo already exists locally, confirm remote:

```powershell
git remote -v
```

Expected origin:

```text
https://git.allucanget.biz/allucanget/arbitrade.git
```

## Local Development Setup

Create virtualenv with `uv`:

```powershell
uv venv
```

Activate env on Windows:

```powershell
.\.venv\Scripts\Activate.ps1
```

Install app + dev dependencies:

```powershell
uv pip install -e .[dev]
```

Create local env file:

```powershell
Copy-Item .env.example .env
```

Minimum `.env` values:

```env
APP_ENV=dev
APP_HOST=0.0.0.0
APP_PORT=8000
LOG_LEVEL=INFO
LOG_JSON=true
DUCKDB_PATH=./data/arbitrade.duckdb
FERNET_KEY=
KRAKEN_API_KEY=
KRAKEN_API_SECRET=
KRAKEN_API_KEY_PERMISSIONS=query,trade
```

Notes:

- Leave Kraken creds empty until Kraken integration lands.
- If Kraken creds are set, both key and secret are required.
- `KRAKEN_API_KEY_PERMISSIONS` must include `query,trade` and must not include withdrawal scope.
- `FERNET_KEY` optional. If empty, keyring-backed key generation used by secret helper.
- On Windows, app falls back to default `asyncio` loop. On non-Windows, `uvloop` installs automatically.

## Run App

Start app:

```powershell
python -m arbitrade.main
```

Health endpoints:

- HTML: `http://localhost:8000/`
- JSON: `http://localhost:8000/health`

## Database

DuckDB used everywhere: local dev, tests, production.

Default database file:

```text
./data/arbitrade.duckdb
```

Schema bootstrap runs automatically on app startup.

Current tables:

- `schema_migrations`
- `opportunities`
- `trades`
- `portfolio_snapshots`

Audit trail table:

- `audit_events` (append-only operational decision log)

Audit retention and compaction guidance:

- Keep at least 30 days of `audit_events` in active DB for incident triage.
- Archive older rows to a timestamped export file before deletion.
- Example monthly archive workflow:

```sql
COPY (
  SELECT *
  FROM audit_events
  WHERE occurred_at < NOW() - INTERVAL 30 DAY
) TO 'data/audit_events_archive_YYYYMM.parquet' (FORMAT PARQUET);

DELETE FROM audit_events
WHERE occurred_at < NOW() - INTERVAL 30 DAY;
```

- Back up archive files and the main DuckDB file together.
- For production, run archive + backup as scheduled maintenance (cron/task scheduler).

## Quality Checks

Run tests:

```powershell
pytest -q
```

Run Ruff:

```powershell
ruff check .
```

Run Black check:

```powershell
black --check .
```

Run mypy:

```powershell
mypy src
```

Run dependency vulnerability audit:

```powershell
pip-audit --skip-editable
```

Run secret scan (worktree + git history):

```powershell
python scripts/security_scan.py
```

Install pre-commit hooks:

```powershell
pre-commit install
```

Run hooks manually:

```powershell
pre-commit run --all-files
```

## Docker

Build locally:

```powershell
docker build -t arbitrade:local .
```

Run with compose:

```powershell
docker compose up --build
```

Compose mounts local `data/` folder into container at `/app/data`.

Important:

- [docker-compose.yml](docker-compose.yml) still uses registry image placeholder format.
- Replace `git.allucanget.biz/OWNER/arbitrade:latest` with actual namespace image, likely `git.allucanget.biz/allucanget/arbitrade:latest`.

## Gitea CI / Registry Setup

CI file:

- [.gitea/workflows/ci.yml](.gitea/workflows/ci.yml)

Required Gitea Actions secrets:

- `REGISTRY_USERNAME`
- `REGISTRY_TOKEN`
- `REGISTRY_NAMESPACE`

Expected namespace now likely:

```text
allucanget
```

Example registry login:

```powershell
docker login git.allucanget.biz
```

Example pushed image tag shape:

```text
git.allucanget.biz/allucanget/arbitrade:<tag>
```

## Project Layout

```text
arbitrade/
├── .gitea/workflows/ci.yml
├── .github/instructions/TODO.md
├── PLAN.md
├── pyproject.toml
├── src/arbitrade/
│   ├── api/
│   ├── config/
│   ├── storage/
│   ├── logging_setup.py
│   └── main.py
├── tests/
└── web/templates/
```

## Next Work

Next planned implementation slice:

- Kraken REST client skeleton
- native Kraken WebSocket client
- in-memory order book cache
- latency instrumentation

## Troubleshooting

PowerShell blocks activation script:

```powershell
Set-ExecutionPolicy -Scope Process -ExecutionPolicy RemoteSigned
```

Then activate again:

```powershell
.\.venv\Scripts\Activate.ps1
```

If app import fails, confirm editable install ran:

```powershell
uv pip install -e .[dev]
```

If DuckDB file missing, start app once or create `data/` directory manually.

## Security Hardening

Threat model notes:

- Primary risk surfaces: environment secrets, dashboard auth credentials, exchange API key scope, and dependency supply chain.
- Assumed attacker model: leaked repository content, leaked CI logs/artifacts, or unauthorized dashboard access.
- High-impact outcomes to prevent: credential exfiltration, unauthorized withdrawals, and unsafe live-trading control changes.

Hardening checklist:

- Use least-privilege Kraken API keys: query + trade only; never enable withdrawal.
- Rotate API keys immediately if secret scan flags a potential exposure.
- Keep dashboard auth enabled in non-local environments and avoid default/shared credentials.
- Run `pip-audit --skip-editable` in CI; treat vulnerability findings as release blockers.
- Run `python scripts/security_scan.py` before release and after major merges.
- Store secrets in environment/secret manager; never commit `.env` or key material.