Real-Time Data Ingestion in Option Chains

Real-Time Synchronization

Derivatives pricing moves fast. On NSE index options, the option chain can update hundreds of times per second during the opening and closing bursts, and individual strikes can see meaningful OI shifts in the time it takes a delayed screen to refresh once. Relying on a delayed option chain makes it nearly impossible to trace sudden shifts in institutional positioning — by the time your screen shows the change, the move has happened, and you're trading against the new reality rather than with it.

This is not a theoretical concern. The difference between a 50ms-latency chain and a 5-second-latency chain is the difference between seeing an OI wall being built and seeing where it was built. One of those is actionable information. The other is archaeology.

What "real-time" actually means

Most retail option chain screens poll the broker's REST API on a 1–5 second interval. That's what "real-time" means on most retail platforms. It is not real-time in any meaningful sense for derivatives trading.

Strat Ai's ingestion path uses the Zerodha Kite WebSocket feed, which pushes binary tick frames directly to the client. The path is:

1. Kite WebSocket sends a 184-byte binary tick frame.
2. The frame is parsed (big-endian decode) into a Protobuf Tick message.
3. The tick is published to a Redpanda (Kafka-compatible) topic and written to QuestDB in parallel.
4. The aggregator and the charting client receive the tick via WebSocket.

End-to-end this takes under 50ms. The chart update path bypasses React state reconciliation entirely — we invoke the Lightweight Charts API directly with the new tick — to prevent the browser's layout engine from becoming a bottleneck. This is why the chart stays smooth at 60 FPS even when the chain is updating hundreds of times per second during the opening burst.

Why the latency matters for traders

The concrete trader-facing consequences of low-latency ingestion:

1. OI wall detection — you see the wall being built, not where it was built. A 5-second delay means you're reacting to a wall that's already defended.
2. PCR velocity — the rate of PCR change is a more actionable signal than the level. You can't measure rate of change on a 5-second polling interval — the noise dominates.
3. Slippage estimation — the bracket auditor's slippage estimate uses live bid-ask spread and order-book depth, which both move intraday. A stale spread estimate under-prices your slippage risk during high-volatility windows.
4. Stop-loss execution — if you're running automated bracket exits, a 5-second delay on stop-loss triggers can turn a 1-R loss into a 3-R loss on fast moves. The terminal's VirtualPortfolio evaluates every incoming tick against open positions, so stops fire within 50ms of the trigger price being touched.

Tracking Intraday Shifts

With real-time options data, you can view changes in open interest as they happen, helping you spot resistance walls and support floors before they trigger sudden price moves. More importantly, you can see when existing walls are being defended — a strike where OI is being added rapidly as price approaches is a defended level, and that's a high-confidence rejection signal. A strike where OI is being shed as price approaches is an abandoned level, and that's a low-confidence rejection signal. Both look identical on a delayed screen.

Architecture notes (for the curious)

The ingestion service is written in Rust for predictable tail latency — we need the parsing-and-publish path to be sub-millisecond with no GC pauses. Equity ticks and option ticks are routed to separate async tasks so that option-chain DB stalls can never block the equity execution path. The tick stream is published to Redpanda on a market.ticks topic and to QuestDB via the ILP (InfluxDB Line Protocol) TCP connector on port 9009 — both write paths run concurrently so a Kafka hiccup doesn't lose ticks to the database.

The aggregator consumes both the raw ticks and the technical indicator signals (computed by a separate Rust agent that also consumes from Kafka), fuses them into AggregatedDecision messages, and broadcasts via WebSocket on port 8080. The charting client subscribes to this WebSocket and updates the chart directly. The full architecture is documented in our deep architecture post on the Strat Ai terminal and the docs on consensus inputs.

External context

The pattern of "WebSocket → parse → publish to Kafka → write to TSDB" is the standard modern low-latency market-data pipeline. The published reference architecture is Confluent's real-time market data pipeline writeup, which uses the same shape with Java instead of Rust. QuestDB publishes their own benchmark comparisons against InfluxDB and TimescaleDB for tick-storage write throughput. We chose QuestDB specifically for the ILP write path — it's the fastest TSDB write path we benchmarked for the 100k-ticks/second sustained load we see on NSE expiry days.