How it works

Spot and perps run in one exchange process, with mark price, liquidation, and funding beside the book. The web app never writes a fill itself. It asks the OMS, and the OMS asks the engine through Redis.

Three stores stay separate on purpose. Postgres holds users and product orders. Redis moves commands, engine events, and the durable market-data stream. Timescale holds the tape you chart. Shared types live in the monorepo: engine commands and events in exchange-types, Redis stream names in app-contracts, and the Postgres schema in Prisma.

Sign-in is Google. The first session creates the user row in Postgres. From there the product is two markets, SOL-USD and SOL-USD-PERP, plus a paper ledger.

• Home credit adds USD or SOL by posting a command. The balance you see afterward is the engine's available and locked amounts.
• The spot desk shows the book, your orders, and a chart. Place and cancel go to the OMS. The book, balances, and last trade are read from the gateway.
• The perps desk adds leverage, position, mark, liquidation distance, and funding on the same wallet. A perp fill changes the position and USD PnL. It does not deliver SOL.
• Charts and the tape come from the ingester, which serves trades, best bid/offer history, and one-minute candles.

Five processes. The gateway is the sole client of the exchange. Everyone else talks to Redis, Postgres, or Timescale.

Web

Next.js trading UI and BFF. Google sign-in creates the user in Postgres. Place, cancel, and credit go to the OMS. Balances, book, mark, positions, and funding are read from the gateway. Charts come from the ingester.

OMS

Product order state in Postgres, plus a command outbox in the same transaction. It publishes orders:commands and applies orders:events. It does not match, and it does not talk to the exchange event bus.

Engine gateway

The only HTTP and SSE client of the exchange. It consumes commands, calls the engine, and fans events back to Redis. Live BBO and trades go out on pub/sub. A stream gap is closed with GET /v1/markets/:market/reconcile.

Exchange

Single-writer matcher for SOL-USD and SOL-USD-PERP. One process-wide command queue sits inside the shared wallet so spot and perps share one ledger. Authoritative for balances, positions, fills, and risk.

Ingester

Reads only the durable md:events stream and writes Timescale: trades, BBO, and 1-minute candles. It does not publish back to Redis.

Redis

orders:commands (OMS → gateway), orders:events (gateway → OMS), md:events (gateway → ingester), and pub/sub channels md:{market}:bbo and md:{market}:trade.

Postgres

Users from the web app. Orders and the command outbox from the OMS.

TimescaleDB

Market history the desks chart. Independent of the live pub/sub fan-out.

Focus

Place and cancel are commands with a durable handoff, so a restart can finish work that Redis has not acknowledged yet.

1. The BFF checks the session, then sends place or cancel to the OMS. Credit takes the same door.
2. OMS writes order intent and an outbox row in one Postgres transaction, then appends the command to orders:commands.
3. The gateway consumes the stream and posts to the exchange HTTP API. Duplicate and replayed commands are counted, not applied twice.
4. MarketRuntime enqueues onto the shared command queue. OrderPlacementService runs, the WAL is appended, then every market WAL is fsync'd and wallet.snapshot.json is saved once per batch.
5. The in-process event bus feeds SSE. The gateway writes orders:events. The OMS updates product order state. Maker and taker share a trade id and are stored per order.

The engine decides what matched. The OMS is the projection the desk renders and can replay. A cancel is accepted only while the order is still pending, accepted, open, or partially filled, so a fill that lands first cannot be overwritten as a cancel.

Prices and sizes are integers. The book is in memory. Disk is for recovery, not for the match itself. Reads (book, balances, mark, positions) do not enter the command queue.

• GTC rests. IOC fills what it can and cancels the rest. FOK fills the whole size or rejects. FOK_BUDGET is a market buy that must fill the requested quantity inside quoteBudget, or it rejects before any match.
• A spot market buy needs quoteBudget. A perp market order needs it on both sides, as a notional cap for margin.
• Place is idempotent on orderId plus intent. Credit with the same commandId does not apply twice. The gateway writes the outcome to Redis before it publishes, so a crash mid-flight replays that outcome instead of matching again.
• Maker and taker fills share one trade id and are stored as two order rows.

“Add USD” and “Add SOL” on the home page post a credit command. The amount is written into the engine ledger, not into a UI counter.

Available and locked balances are read back from the engine. An open spot order or perp margin keeps size reserved until fill, cancel, or liquidation.

Live market data is pub/sub for best bid/offer and trades. The SSE stream carries order, trade, BBO, credit, position, liquidation, and funding events. That is what the desk shows now. The exchange keeps a 4096-event ring for catch-up.

History is the durable md:events stream. The ingester writes Timescale from it, so charts and the tape remain after the live fan-out is gone.

On reconnect the browser asks the web app for a stream ticket, then opens an EventSource on the gateway. Catch-up uses streamSeq. If the ring was overrun, the gateway reconciles from the exchange HTTP API and the OMS applies the retained state.

• WAL, then one fsync — each command appends a WAL line with no fsync. After the batch, every market WAL is fsync'd and the shared wallet snapshot is saved. Market snapshots are written every 1024 sequences, not every order.
• Boot — the exchange loads the market snapshot and wallet.snapshot.json, then replays the WAL tail. CREDIT, DEBIT, PLACE, CANCEL, LIQUIDATE, and FUNDING are all journaled.
• OMS outbox — the command row exists before Redis is asked to accept it. An OMS restart can publish again from Postgres.
• Engine sequence — events carry a sequence so consumers can detect gaps. The gateway reconciles over HTTP. The event bus does not call the gateway.

Focus

SOL-USD-PERP uses the same engine and the same wallet as spot. Margin, positions, mark, liquidation, and funding are checked with matching.

• Opening a perp locks USD margin, ceil(notional / leverage). Spot and perp share available and locked USD and SOL. The books and positions stay per market.
• Mark is the book mid, or the last trade. It drives unrealized PnL. A maintenance-margin breach force-closes the position at mark.
• Funding runs on the demo schedule, 100 bps of notional every 60 seconds. When the rate is positive, longs pay shorts.

Each service exposes counters and gauges, and writes one JSON log line per event. Prometheus scrapes the numbers. Promtail ships the logs to Loki. Grafana is where both are read, on the PaperDesk Operations dashboard.

Prometheus

Scrapes /metrics on the exchange, gateway, OMS, and ingester, and /api/metrics on the web app. Shared series include cex_service_up, process uptime, and cex_dependency_ok. OMS exports consumer lag on orders:events. The ingester exports it on md:events (cex_stream_lag, cex_stream_pending). The gateway reports command backlog age, command outcomes, SSE connection, reconnects, gap reconciles, and BBO or trade publish counts.

Loki

Services log JSON with service, level, and fields such as requestId, orderId, commandId, and market. Promtail tails those process logs, labels them by service, and pushes batches to Loki. A failed command can be found by order id instead of by reading five terminals.

Grafana

PaperDesk Operations queries both datasources. Fleet status is green when a process is answering /metrics. The same board shows dependency health, stream lag, and a live log panel per service: gateway commands and SSE, OMS lag on orders:events, ingester lag on md:events, then exchange and web. Metrics say something stalled. Logs say which command.