Simor

Data Infrastructure for Production AI

Practical writing on AI data engineering, feature stores, and the infrastructure choices that determine whether AI systems work in production.

gRPC Postcards: Typed Messages at Light-Speed
gRPC Postcards: Typed Messages at Light-Speed
14 Mar, 2025 | 03 Mins read

A postal service where every postcard has a strict template. The address fields are always in the same spot. The message area has specific sections for specific types of information. Both sender and r

WebSockets: The Persistent Coffee Line
WebSockets: The Persistent Coffee Line
07 Mar, 2025 | 06 Mins read

You walk into your favorite coffee shop and order your usual. But instead of ordering, paying, leaving, and coming back when you want another coffee (like HTTP requests), imagine you could just stay a

Building AI-Ready Data Pipelines: Key Architecture Considerations
Building AI-Ready Data Pipelines: Key Architecture Considerations
04 Mar, 2025 | 02 Mins read

Data pipelines built for business intelligence often fail when supporting AI workloads. The root cause is usually architectural: BI pipelines assume bounded, relatively static datasets, while AI syste

Designing for Data Quality: How to Build Reliable AI Systems
Designing for Data Quality: How to Build Reliable AI Systems
26 Feb, 2025 | 02 Mins read

Most ML projects fail not because of flawed algorithms but because of poor data quality. Data scientists typically spend 80% of their time on data preparation, and even small data quality issues drama

From Data Silos to Data Mesh: The Evolution of Enterprise Data Architecture
From Data Silos to Data Mesh: The Evolution of Enterprise Data Architecture
15 Feb, 2025 | 03 Mins read

Traditional centralized data architectures worked for BI but struggle with AI workloads. Centralized teams become bottlenecks as data volumes grow. Domain experts who understand the data are separated

Real-Time Feature Engineering: The Key to Operational AI Systems
Real-Time Feature Engineering: The Key to Operational AI Systems
05 Feb, 2025 | 02 Mins read

Most AI pilots succeed. Most AI production deployments fail. The gap between proof-of-concept and operational AI often traces to one root cause: the inability to compute and serve features in real-tim

The Modern Data Stack for AI Readiness: Architecture and Implementation
The Modern Data Stack for AI Readiness: Architecture and Implementation
28 Jan, 2025 | 03 Mins read

Existing data infrastructure often cannot support ML workflows. The modern data stack offers a foundation, but it requires adaptation to become AI-ready. This article covers building a data architectu

Vector Databases: The Missing Piece for Building Effective LLM Applications
Vector Databases: The Missing Piece for Building Effective LLM Applications
10 Jan, 2025 | 03 Mins read

LLM applications face four consistent challenges: hallucination, context window limits, knowledge freshness, and cost. Vector databases enable retrieval-augmented generation (RAG), a pattern that addr

Data Virtualization for Hybrid Analytics
Data Virtualization for Hybrid Analytics
12 Dec, 2024 | 03 Mins read

Organizations navigate complex data landscapes spanning on-premises systems, multiple clouds, and SaaS applications. Centralizing all data for analytics has become impractical. Data virtualization cre