Simor
Data Infrastructure for Production AI
Practical writing on AI data engineering, feature stores, and the infrastructure choices that determine whether AI systems work in production.
Data pipelines break because data producers and data consumers have different assumptions. The producer assumes the consumer can handle null values in a column. The consumer assumes the column is neve
The market for AI engineers is the tightest it has been since the deep learning boom of 2017. Demand has grown 280% year-over-year for the "AI engineer" title, and the supply of experienced practition
Your iPhone prompts you: iOS 18.4 is available. It includes improvements to battery performance, new photo editing tools, and a fix for crashes in third-party apps. You can install it now or wait. If
Basic retrieval-augmented generation works well in demos and poorly in production. The demo shows a clean pipeline: chunk text, embed chunks, retrieve relevant chunks, feed them to the model. The prod
Publishing aggregate statistics about a dataset sounds safe. The average salary in a department. The number of users in a geographic region. The distribution of query types in a search engine. But agg
Five years ago, "data company" described a specific type of organization: a business whose primary product was data or data services — Snowflake, Databricks, Palantir, Bloomberg. Today, the distinctio
An agriculture technology company built a crop yield prediction model that combined satellite imagery, soil sensor data, weather forecasts, and historical yield records. The model predicted per-field
A pharmaceutical company I consulted for was three months into a project to fine-tune a large language model on their internal research corpus. The project had a team of four engineers, a budget of $8
A model registry is the version control system for your trained models. Without one, teams track model versions by filename, store artifacts in ad-hoc cloud storage locations, and discover which model