AI coding agents are testing Nvidia’s CUDA moat, but the pressure is building first in inference
Infinity, a one-year-old startup founded by former Google Brain researcher Jeremy Nixon, used its AI research agent Ignition to build CUDA-like software for d-Matrix in 10 hours, according to the source article. The work focused on generating and optimizing inference kernels and related low-level software, not recreating the full CUDA stack that Nvidia has spent nearly two decades building.
That distinction matters. CUDA is not just a programming model but a broad software platform spanning kernels, compilers, runtime, optimized libraries such as cuBLAS and cuDNN, debugging and validation tools, plus higher-level frameworks including PyTorch and TensorFlow. In large-scale training, where stability, cluster coordination and performance tuning are critical, the article argues Nvidia’s software edge remains hard to dislodge.
Inference looks different. Marshall Choy, chief business officer at South Korea’s AI chip company Rebellions, said CUDA is no longer the decisive factor there and called it a contest in open-source software. Companies including d-Matrix, Rebellions, Cerebras, Amazon with Inferentia, Google with TPU and AMD with MI300X are all targeting that opening. Even so, executives and former Nvidia insiders cited in the piece said code generation is only part of the story. Validation, production-grade optimization, migration costs and Nvidia’s own use of AI agents still leave the CUDA moat far from broken.