Category: GenAI Ops

Performance Engineering for GenAI Inference: Batching, Caching & Quantisation

A startup's GenAI application cost $0.42 per query at 15-second latency. At this rate, their Series A funding would last six months. The problem wasn't the model—it was unoptimized inference. Each req