Former Google employee says AI compute bottleneck is data center capacity and power, not just GPUs

Former Google employee says AI compute bottleneck is data center capacity and power, not just GPUs

N
News Editor
2026-10-21 06:47:40
A former Google employee said in a recent interview that the main constraint on AI compute in the United States is not simply GPU production, but the lack of data center capacity and power needed to bring those chips online. The person argued that low-latency GPU capacity inside the U.S. remains especially tight, even though companies can technically source GPUs from places such as Singapore, Malaysia, or Australia if they are willing to accept higher latency. The interview also broke the GPU cloud market into three groups: hyperscalers such as AWS, Azure, and Oracle, which are reliable but expensive and often require multi-year commitments; Neocloud providers including Nebius, Vultr, and CoreWeave, which offer substantial capacity but can be less reliable; and smaller new entrants operating scattered U.S. data centers with around 4,000 GPUs per site. The former Google employee said their company places roughly 80% to 90% of its capacity with Neocloud providers, despite the operational burden. The person also said open-source models have been catching up quickly and that, since around the second quarter of this year, customers have started replacing some Anthropic and OpenAI models in multi-agent setups. Elon Musk later reposted the discussion and said describing the issue as a simple power shortage was an oversimplification, adding that every step of bringing large-scale compute online at high speed is difficult.

A former Google employee said in a recent interview that the real bottleneck in AI compute may be data center capacity and power rather than GPU output alone. According to the interview, GPU supply in the United States is failing to keep up with demand mainly because there is not enough facility space and electricity, while reliability issues at some Neocloud providers have also created problems for customers.

Low-latency GPU capacity in the U.S. is still tight

The interviewee said the biggest constraint right now is capacity. Some companies have seen traffic grow 5x, but usable GPU capacity in the market remains limited, especially for customers with the strictest latency requirements, where deployment has to stay inside the United States.

From a technical standpoint, providers can move workloads to Singapore, Malaysia, or Australia to access GPUs. Many customers, though, do not accept the added latency. The former Google employee described the market this way: there is a lot of available capacity, but not much of it is good.

The person gave the example of deploying large models that need to load quickly. A model of about 3 TB, they said, should ideally be loaded from the file system into GPUs within 10 minutes. That makes network performance and file system quality critical alongside the GPUs themselves.

Three layers of GPU cloud supply, each with its own trade-offs

The former Google employee divided GPU cloud providers into three groups.

  • First are hyperscalers. That group includes AWS, Azure, and, in the speaker’s view, Oracle as well. They are highly reliable, but expensive. Contracts often require at least three years, while customers are now chasing B300. Even B200 has recently started to be treated as outdated, which makes buyers reluctant to lock themselves into a three-year deal for an older generation of GPUs.
  • Second are Neocloud providers. Examples named in the interview were Nebius, Vultr, and CoreWeave, with roughly 20 to 30 companies in this category. The speaker said about 80% to 90% of their company’s capacity sits with this layer, but reliability is weaker and more engineering effort is needed to coordinate operations.
  • Third are smaller new entrants. These are often investors and traditional infrastructure operators building their own data centers to capture demand. They are spread across the U.S., with around 4,000 GPUs at a single site, and they often lack name recognition. The interviewee said some face a basic commercial problem: whether they can sell the capacity at all.

Reliability and payment terms remain pain points for Neocloud customers

On reliability, the former Google employee gave one example involving a customer that deployed 90 model replicas, each requiring 8 GPUs, for a total of 720 GPUs. One night, 20 of those GPUs went offline without warning. The suspected cause was either a network issue or overheating in Blackwell GPUs.

On payment terms, the interviewee said most U.S. Neocloud providers ask for 20% to 30% upfront, with the rest paid monthly or quarterly. After the contract ends, customers then negotiate discounts tied to losses from incidents, such as a long-term 15% traffic hit. By contrast, providers in the Asia-Pacific region usually do not require prepayment.

Open-source models are catching up, and customers are replacing some Anthropic and OpenAI usage

The interview also pointed to changes on the model side. The former Google employee said open-source models are catching up with frontier systems, and that the pace has accelerated over the past month. Since around the second quarter of this year, customers have started replacing some Anthropic and OpenAI models in multi-agent architectures with open-source alternatives.

The replacement path, in the speaker’s view, will move from simpler tasks to more complex ones:

  • starting with tool calling and MCP-style tasks,
  • then moving into low-level orchestration,
  • and eventually reaching day-to-day development and debugging.

Cursor was cited as an example. The top orchestration layer still uses Anthropic and OpenAI models, while lower-level parallel code search can switch to Composer 2.5, which the speaker described as a post-trained model based on Qwen. The interviewee also said GLM 5.2 is the next model to watch, while Kimi K3 is already being discussed in the market.

GPU availability may improve in one to two years, but power remains a constraint

Looking ahead, the former Google employee said GPU availability should improve significantly over the next one to two years as NVIDIA keeps expanding shipments. Even so, the total number of GPUs actually running inside data centers may not rise by much because the U.S. does not have enough power. In the speaker’s words, what is in extreme shortage is data centers, not GPUs.

The interviewee also said Meta and xAI already have some idle GPU clusters and have begun building cloud businesses to rent that capacity out. More companies are expected to follow.

Elon Musk later reposted the discussion and responded: 「‘Power shortage’ is an oversimplification. Every step required to bring massive compute online at very high speed is difficult.」 He added that the task would be relatively easier if there were twice as much time, but in reality there is nowhere near that much time.

This article was originally published by Bit.Fan. For more cryptocurrency news and market insights, visit www.bit.fan.
11200

Disclaimer:

The market information, project data, and third-party content displayed on this platform are for industry information sharing only and do not constitute any form of investment advice or return commitment.

Cryptocurrency trading carries high risks. Users should fully assess their risk tolerance and make independent decisions. All profits, losses, and legal responsibilities are borne by the users themselves.

Former Google employee says AI compute bottleneck is data center capacity and power, not just GPUs | Bit.Fan