Simor

Data Infrastructure for Production AI

Practical writing on AI data engineering, feature stores, and the infrastructure choices that determine whether AI systems work in production.

Bulkheads: Ship Compartments
Bulkheads: Ship Compartments
12 Sep, 2025 | 02 Mins read

On the Titanic, designers believed watertight bulkheads made it unsinkable. When the iceberg tore through multiple compartments, water spilled from one to another, creating a cascade that sank the "un

AI Observability: Monitoring Drift, Data Quality & Model Performance
AI Observability: Monitoring Drift, Data Quality & Model Performance
12 Sep, 2025 | 02 Mins read

An insurance company's premium pricing model had been quietly going haywire for two weeks. Young drivers in high-risk areas were getting bargain prices while safe drivers faced astronomical quotes. By

Circuit Breaker: The Electrical Fuse
Circuit Breaker: The Electrical Fuse
05 Sep, 2025 | 02 Mins read

Your home's electrical panel has circuit breakers. Plug in too many appliances, the breaker trips, cutting power to prevent fires. You can't use those outlets until you flip it back on. Annoying, but

Data Contracts: Aligning Producers & Consumers
Data Contracts: Aligning Producers & Consumers
05 Sep, 2025 | 03 Mins read

An engineering team renamed a critical field from 'user_signup_date' to 'account_created_at' without warning. It cascaded through dozens of data pipelines, breaking executive dashboards, halting marke

Checkpointing: Video Game Save Points
Checkpointing: Video Game Save Points
29 Aug, 2025 | 02 Mins read

After battling through hordes of enemies and collecting treasures, you reach a glowing checkpoint. If you fail now, you restart from the save, not the beginning. That's checkpointing: periodically sav

Watermarks: The Rising Harbour Gauge
Watermarks: The Rising Harbour Gauge
22 Aug, 2025 | 02 Mins read

The harbormaster watches a gauge showing tide level. Ships can only depart when the tide rises above their draft mark. Some arrive on time, others are delayed by storms, a few drift in days late. Whe

CDC: The Gossip Column
CDC: The Gossip Column
15 Aug, 2025 | 03 Mins read

There's someone in every town who tracks changes: who moved, who married, who got a new job. They don't track static facts (John lives on Oak Street). They track changes (John moved from Oak to Elm).

Metadata-Driven ELT: Designing Declarative Pipelines
Metadata-Driven ELT: Designing Declarative Pipelines
15 Aug, 2025 | 03 Mins read

A data engineer at an e-commerce company stared at a mess of SQL scripts, Python notebooks, and configuration files. What started as a simple ETL job had mutated into a hydra of interdependent transfo

Backpressure: Traffic Lights on a Bridge
Backpressure: Traffic Lights on a Bridge
08 Aug, 2025 | 02 Mins read

A narrow bridge holds 50 cars safely. When car 51 tries to enter, the light turns red. Cars queue on the approach road, then the streets leading to it, then the highways beyond. The bridge is protect