Files

T

CI / lint-test-build (push) Failing after 1m23s

Details

Add integration tests for execution persistence, metrics, and opportunity writing

- Implemented integration tests for the execution writer to ensure trade orders and PnL are persisted correctly.
- Created integration tests for the metrics calculator to summarize execution data accurately.
- Added integration tests for the opportunity writer to verify event persistence.
- Established PostgreSQL schema validation tests to ensure all expected tables, columns, and constraints exist.
- Removed outdated unit tests that relied on DuckDB and replaced them with tests using PgStore.

2026-06-07 14:37:53 +02:00

5.7 KiB

Raw Blame History

Database Layer: Schema & Repositories

Database engine: PostgreSQL 15+ on 192.168.88.35 Driver: asyncpg (async connection pool) Store class: PgStore in src/arbitrade/storage/pg_store.py

Connection Lifecycle

FastAPI lifespan (create_app)
  └─ PgStore.start()         # creates asyncpg connection pool
      └─ PgStore.migrate()   # reads schema_pg.sql, creates tables
          └─ ... application runs ...
  └─ PgStore.stop()          # closes the pool

All repository classes accept a PgStore instance and acquire connections via async with self._store.pool.acquire() as conn:.

Schema

Defined in src/arbitrade/storage/schema_pg.sql. 15 tables:

Table	Purpose	PK	Notes
`schema_migrations`	Version tracking	`version`	Single-row per version
`config_sections`	Config section metadata	`id` (SERIAL)	`name` UNIQUE
`config_settings`	Key-value config store	`key` (VARCHAR)	JSON-serialized values
`config_pairings`	Currency pairs to monitor	`id` (SERIAL)	`(base_asset, quote_asset)` UNIQUE
`config_backtesting_defaults`	Default backtest params	`id` (SERIAL)	Singleton via `ORDER BY id DESC LIMIT 1`
`opportunities`	Detected arb opportunities	`id` (UUID)
`trades`	Executed trades	`id` (UUID)
`orders`	Individual leg orders	`id` (UUID)
`pnl_events`	P&L event stream	`id` (UUID)
`portfolio_snapshots`	Balance snapshots	—	Append-only
`market_snapshots`	Raw order-book snapshots	—	Append-only
`audit_events`	Audit trail	`id` (UUID)
`runtime_state_snapshots`	Runtime state history	—	Append-only
`kraken_account_snapshots`	Fee tier + account data	—	Append-only
`backtest_jobs`	Backtest job records	`id` (UUID)

JSON columns use JSONB for indexability. UUID primary keys use gen_random_uuid() (requires pgcrypto extension).

Repository Classes

All in src/arbitrade/storage/repositories.py. Every method is async def.

Class	Key Methods	Used By
`MarketSnapshotRepository`	`insert()`	`AsyncMarketSnapshotWriter`
`OpportunityRepository`	`insert()`	`AsyncOpportunityWriter`
`TradeRepository`	`insert()`	`AsyncExecutionWriter`
`OrderRepository`	`insert()`	`AsyncExecutionWriter`
`PnLRepository`	`insert()`	`AsyncExecutionWriter`
`AuditRepository`	`insert()`, `list_recent()`	API routes, lifecycle
`RuntimeStateRepository`	`insert()`, `latest()`	Lifecycle, API
`ConfigSectionRepository`	`create_section()`, `get_section()`, `list_sections()`	Config service
`ConfigSettingRepository`	Full CRUD + `get_latest_updated_at()`	Config service
`ConfigPairingRepository`	Full CRUD + `upsert_pairing()`, `list_pairings()`	Feeds, pairing sync
`ConfigBacktestingDefaultsRepository`	`create_defaults()`, `get_defaults()`, `update_defaults()`	Config service
`KrakenAccountSnapshotRepository`	`insert_snapshot()`, `latest_snapshot()`	Fee sync loop
`BacktestJobRepository`	Full CRUD	Backtesting UI + worker

Async Writers

Three background writer tasks buffer high-frequency writes:

AsyncExecutionWriter — trades/orders/P&L queue
AsyncMarketSnapshotWriter — order-book snapshot queue
AsyncOpportunityWriter — opportunity event queue

Each uses an asyncio.Queue and drains it in a background task with await repo.insert(...).

Integration Tests

tests/integration/test_postgresql_schema.py verifies:

Connection to PostgreSQL server
pgcrypto extension availability
All 15 tables exist after migration
Migration is idempotent
Correct columns per table
Primary keys and unique constraints
Tables start empty
Simple INSERT/SELECT round-trip
ON CONFLICT ... DO UPDATE on config_pairings

5.7 KiB Raw Blame History