A one-year-old startup says AI can now compress part of the chip software grind from months or years into hours. Infinity used its AI research agent, Ignition, to build CUDA-like software for d-Matrix in 10 hours, a result that has reignited debate over how defensible Nvidia’s software moat really is.
The article’s central point, though, is narrower than the headline shock. What was produced in those 10 hours was not a full replica of CUDA’s nearly 20-year software empire, but low-level inference software centered on kernel generation and optimization for a specific chip.
What CUDA actually includes
Nvidia is often identified first with its GPUs, from H100 to Blackwell and Rubin. But the company’s deepest advantage, the article argues, sits in software rather than hardware alone.
CUDA, short for Compute Unified Device Architecture, was built over nearly 20 years under Nvidia executive Ian Buck. It is often described as a programming language, yet the more accurate description is a full software platform built around Nvidia GPUs.
The article breaks CUDA into three layers. At the bottom sits the kernel layer, including kernels, compilers and runtime that directly drive the chip. In the middle are optimized libraries such as cuBLAS and cuDNN, along with debugging, validation and performance-analysis tools. At the top are frameworks such as PyTorch and TensorFlow, enterprise codebases and workflows, and the broader developer ecosystem that grew up around CUDA.
That stack is why customers are not just buying a card. They are buying an established software factory that works out of the box.
Infinity is trying to automate part of the most painful layer. Founder Jeremy Nixon, previously a researcher at Google Brain, built Ignition as an AI research agent that writes low-level software for different AI chips. It can generate GPU kernels, run tests, find errors, measure performance and then rewrite code based on the results. Human engineers set the high-level direction; the repetitive loop is left to the agent.
Using that approach, Infinity built CUDA-like software for AI chip startup d-Matrix in 10 hours.
The article says this is not pure hype. AI agents are well matched to programming tasks with clear feedback loops: code either compiles or it does not, outputs are either correct or not, and performance can be measured directly. That gives the agent a closed loop of writing code, compiling, running, testing correctness and performance, then revising.
Still, Infinity’s progress is concentrated in the first layer of CUDA. It is generating and optimizing inference kernels for different chips and building low-level software capabilities around them. The company is also developing a general inference library for multiple chips, but that is a long way from reproducing CUDA’s full ecosystem.
Investors have responded anyway. The source article says Infinity has raised $15 million and reached a $100 million valuation.
DeepSeek is also trying to reduce hand-written CUDA work
Infinity is not alone in trying to cut back on low-level CUDA coding. The article says DeepSeek has open-sourced a GPU operator library called TileKernels, written with TileLang, to reduce the amount of hand-written low-level CUDA code required.
That points to a broader shift. Parts of chip software development that once demanded repeated manual tuning are starting to move toward AI-assisted generation and higher-level abstractions.
Inference is where the opening appears
If AI coding agents are the tool, inference is the breach they are trying to exploit.
Large-model computing is split into training and inference. Training builds the model and can require tens of thousands of chips working together for months. Inference uses the trained model to answer questions and can run on far smaller clusters, sometimes even a single card.
On the training side, the article says CUDA remains extremely difficult to challenge. Large-scale training is highly sensitive to performance, stability and cluster coordination. Interconnect behavior, memory scheduling and recovery after failures all matter. Any small efficiency loss, multiplied across thousands or tens of thousands of chips, turns into real time and cost.
That is why companies stay conservative when selecting a training platform. Even if another chip looks good on paper, immature software or unstable clustering can erase any hardware savings through development and trial-and-error costs.
Inference follows a different set of economics. Marshall Choy, chief business officer at South Korean AI chip company Rebellions, said: “In inference, CUDA is no longer the deciding factor. This will be a competition in open-source software.”
The reason is simple. Training prioritizes maximum performance. Inference prioritizes the cost of each answer. Training demands massive coordination; inference can be distributed across smaller systems. Training stacks are hard to swap out; inference gives customers more room to experiment. If a system runs and costs less, users are willing to test it.
Another key difference is that inference software can run across chips. If an inference framework supports multiple chips, users can switch hardware without rewriting their code. That weakens one of CUDA’s strongest lock-in effects.
This creates room for dedicated inference chips. Many inference workloads do not need the full set of capabilities CUDA provides from training through cluster coordination. A chip redesigned around a particular model or scenario has a shot if it is faster, cheaper or more power-efficient.
d-Matrix fits that profile. Founded in 2019, the company focuses on generative AI inference chips. Co-founder and CEO Sid Sheth recalled that the company concluded early on that the opportunity in AI inference would ultimately exceed training.
The CUDA-like software produced in 10 hours by Infinity was built for d-Matrix’s inference chip. That ties the story together: a new chip company wants to enter the inference market but lacks mature software, and an AI agent generates and optimizes the low-level kernels quickly enough to shrink what may once have taken months or years into hours or days.
Sid Sheth put it directly: “Although Nvidia maintains its dominance in training, its moat in inference is weaker.”
d-Matrix is not the only company chasing that gap. The article lists inference-focused Rebellions and d-Matrix, wafer-scale chip player Cerebras, Amazon’s Inferentia, Google’s TPU and AMD’s MI300X. Chipmakers and cloud giants have already positioned themselves for the fight over inference.
They do not need to replace Nvidia across the board. They only need to show that, for some inference workloads, they can run faster or cheaper without relying on Nvidia’s full hardware and CUDA stack.
Why the CUDA moat still looks intact for now
That is also where the limits become clear. What was rewritten in 10 hours was adaptation code for one chip. CUDA also includes nearly two decades of accumulated libraries, debugging tools, community support and millions of developers. None of that disappears just because an agent can produce code quickly.
Bing Xu, founder of INT21, previously built AI chip software company HippoML, which was later acquired by Nvidia. He left Nvidia in April this year. His view is that agents can generate large amounts of code quickly, but validation remains the biggest bottleneck.
An AI system may produce 10,000 lines of code quickly. Proving that those lines compute correctly and run stably across edge cases is much slower. Xu argues that CUDA’s deepest asset is its validation toolchain, and in the age of agents, the validation ecosystem may become Nvidia’s next moat.
Chris Lattner, co-founder and CEO of Modular, also pushed back on the excitement. He said the gains from coding agents are incremental and that “the hype is severely overblown.”
His reasoning has three parts. First, writing code is only a small part of software engineering; production-grade optimization is what determines chip performance and the cost of running AI. Second, chip software is a narrow, elite field with far less public code than application development, leaving AI with less training data to learn from. Third, the migration cost tied to millions of lines of existing code is still there, and that is not purely a technical problem but an organizational one.
Nvidia is using AI too
One detail in the article cuts against the idea that only challengers benefit from AI coding agents: Nvidia is using them as well.
Ankit Patel, Nvidia’s vice president of developer ecosystem, said: “We are also using AI agents to develop CUDA faster and validate it at larger scale.”
That changes the competitive calculus. The same tools that help rivals close the gap can also help Nvidia iterate faster. Xu summarized the contest this way: the outcome depends on whether competitors can improve faster than Nvidia can improve itself.
At this stage, the article says, the world’s largest chipmaker is clearly not standing still.
The first real shifts may show up in inference
Viewed over a longer timeline, the article’s conclusion is restrained. In the near term, Nvidia’s CUDA moat remains intact, especially in training, where its edge in software, validation and cluster-scale reliability has not been seriously displaced.
But change may begin quietly on the inference side. As inference grows and multi-chip portability becomes more valuable, AI agents that shorten low-level adaptation work could make market entry easier for specialized chip vendors.
In that sense, the deeper question is not whether CUDA has already been replaced. It is whether the center of gravity in the moat is shifting away from accumulated code and toward validation, optimization and the surrounding ecosystem.
Whoever secures that position first may shape the next stage of the market.
Reference: https://www.businessinsider.com/nvidia-cuda-new-threats-ai-coding-agents-2026-8
Original Chinese source credited in the input: WeChat account “量子位,” by 听雨.