Ojaswi Sharma

Why this project. Quant data systems live or die on market data infrastructure, and every tutorial pretends it’s easy. I wanted to see market data the way an exchange does: not in a notebook, but as a living stream with latency budgets, sequence numbers, and failure modes that only show up at 3am. This is the gap between “did a Kafka course” and “ran a pipeline through a live soak and can show you the P99.”

Why this stack. Kafka (Redpanda) for the backbone, Protobuf on the wire, TimescaleDB and ClickHouse at rest, Prometheus + Grafana watching everything. Boring choices, deliberately. The interesting part is what flows through them, and the discipline was proving each phase with evidence before sealing it.

What it achieves. Binance live WebSocket and Databento CME-futures replay running through the same path with parity. Full L2 order-book reconstruction with sequence-gap detection. A derived microstructure topic (microprice, order-book imbalance, effective spread) that other systems can subscribe to. FathomPulse is designed as the first subscriber. P99 of 63.4 ms tick-to-store, measured over a one-hour live soak, gap-free.

Why this project. Because building pipelines teaches you how systems behave, but building a society teaches you how systems surprise you. This one is for fun, and it turned out to be the most serious thing I’ve made: 100 agents with 10 bits of mind each, dropped into one world with no scripts, developing trade, naming, and grudges on their own.

Why this stack. A deterministic simulation core (AVX2-pinned so every run is reproducible bit-for-bit), checkpointed engine state, agents deliberately small enough to reason about completely. When something emerges, I can prove it emerged. Determinism is what separates an experiment from a screensaver.

V1: what shipped. A two-segment cognition loop I call “free mind, hidden hands”: each tick, a villager gets one unconstrained thought, then a non-LLM C++ resolver reads that thought and extracts a valid action from it, with a thin survival reflex underneath so nobody starves to death by accident. Early versions caged the thought in a JSON grammar, not just the output, and the model went quiet the moment it was constrained. Freeing the thought and moving the constraints to the edges is what made it come alive. Shipped public on a Hugging Face Space: sprite-based live viewer, gossip, dreams, word-coinage, a replay tool.

Why V2. A 3-day audit of the live run said something a demo never would: the village had converged to a boring attractor by day one. Gossip drowned out grounded action (0.6% of events), retellings carried no trace of who was telling them, one myth looped on repeat because salience always picked the same “winner,” and most coined words were inflection noise, not real coinage. I retired that run as a control dataset instead of patching it quietly and calling it done.

What changed in V2. A grounded-action floor so villagers aren’t only gossiping, personality-conditioned prose feeding gossip and dreams instead of flavor-only text, a fatigue penalty so the same retelling can’t loop forever, sentence-aware trimming so distortion doesn’t sever mid-word, and a stricter coinage filter. Then a real cutover: archived Run 1’s full 77,926-tick history as a permanent dataset, wiped state, rebooted clean.

Honest read on quality. The writing is fluent and does invent real lore, place names and character names, unprompted: that’s the actual thesis paying off. What hasn’t fully landed: the trait-differentiated “distinct voice per personality” goal. Most villagers still read in a similar register right now. That’s the next thing to chase, not a claim I’m making today.

Why this project. Fathom proves itself in a P99 number and a dashboard nobody but me has seen. Pulse is the part a recruiter can actually watch: Fathom’s internals turned into a screen, built on the same honesty standard as the pipeline underneath it. Most dashboards pretend to be real-time. Pulse would display its own staleness instead of hiding it.

Why this stack. It rides Fathom’s existing Kafka topics read-only, one more consumer off the same derived stream. FastAPI over WebSocket, one frame per 10Hz tick, each frame a complete snapshot of the trailing second so the client never has to stitch deltas together and get it wrong.

Honest status. Architecture and frame semantics locked, and the design system is more than a mood board: real contrast math run against the actual dark palette before a single component was built. Past that, there’s no ingest, no API, no web app. It stays labeled work-in-progress until it isn’t; this site doesn’t do vaporware.

Why this project. I spent 18 months watching production data change shape between Bronze and Gold, and learned the hard way (see: the silent data loss incident) that lineage you can’t query is lineage you don’t have. FAIR data principles say provenance should be machine-actionable; almost nobody’s pipelines actually deliver that.

Why this stack. W3C PROV as the data model because standards outlive side projects. Lineage recorded in PROV today can become a Fair Digital Object tomorrow without rework.

Honest status. The PROV core works; the ambitions around it are still being built. It stays labeled work-in-progress until it isn’t; this site doesn’t do vaporware.