MatX CEO Says Batch Size Can Boost AI Inference Efficiency by Up to 1,000x

MatX CEO Reiner Pope said batch size is a critical driver of AI training and inference efficiency, with batching, KV cache design, and better GPU utilization playing a major role in lowering per-token costs.