OpenAI has published new details about Habitat, its large-scale online storage platform, saying the system now handles more than 70 million requests per second, serves more than 1 billion users each week, spans close to 40 geographic regions, and manages more than 500PB of data.
The company also said that in the second quarter of 2026, Habitat service was rewritten from Python to Rust by two engineers working with Codex and GPT-5.5. Based on OpenAI’s internal data, the Rust version improved CPU efficiency by 6x and memory efficiency by 15x, while average latency and tail latency both fell. OpenAI said the Rust implementation now handles about 95% of production traffic, and the Python version is expected to be fully retired in the next few weeks.
One user action can lead to hundreds of database calls
Habitat is the online storage platform OpenAI built for its products. Logging into ChatGPT, checking Codex settings, or simply opening a new ChatGPT conversation can require access to a wide range of data.
OpenAI said an ordinary user request can sometimes trigger hundreds of database calls. In practice, the speed users feel is often shaped less by average database latency than by the slowest request in the chain. If one data read slows down, the full product experience slows with it. If a critical request fails, the product may stop working.
Habitat started as a Python client-side library for Azure Cosmos DB
OpenAI said Habitat was not originally a large distributed system. When GPTs launched at OpenAI DevDay in 2023, Habitat began as a simple Python client-side library connected to Azure Cosmos DB. Its purpose was to let product engineers work without having to understand the underlying complexity of the database stack.
Habitat handled data schema, routing, permission checks, encryption, serialization, traffic control, and connection pooling through a single layer.
The client library model hit its limits by mid-2025
As ChatGPT and other OpenAI products grew quickly, the client library approach started to break down by mid-2025. Every change to Habitat had to be rolled out across dozens of different services.
OpenAI said it wanted to reduce the effect of failures in any single region, and planned to split important data across different Azure Cosmos DB accounts in different regions. But even adding new routing logic meant changing the Habitat client first, adding a feature flag, and then waiting for each service to update. A deployment could take days.
If the team later wanted to add new shadow testing, it had to redeploy again. If a bug appeared, the process had to be repeated once more. OpenAI said a service could also roll back for unrelated reasons just as a new feature was about to be enabled, returning to an older Habitat client with a bug and creating the kind of outage the company had been trying to avoid.
OpenAI moved Habitat into a centralized service
OpenAI ultimately decided to separate Habitat from the client library model and turn it into an independent centralized service. With that change, all products accessed data through Habitat service, allowing the company to manage deployment, monitoring, permission controls, audit logs, data security, and database connections in one place.
OpenAI said it accepted technical debt by using Python first
OpenAI said it already knew Python was not the ideal language for a high-throughput backend service, but chose to build Habitat service in Python anyway.
The company said the priority at that stage was not to minimize CPU or memory use, but to remove development bottlenecks for product teams as quickly as possible and establish a stable API and infrastructure layer. For that reason, the team deliberately accepted the technical debt.
OpenAI also said it understood that if the system scaled another 100x, Python’s CPU, memory, and latency costs would become unacceptable, making a rewrite almost unavoidable. The team’s bet was that by the time a rewrite became necessary, AI coding models would be strong enough to cut migration costs sharply.
By Q2 2026, Habitat had become one of OpenAI’s biggest services
According to OpenAI, Habitat had become the company’s second-largest service by CPU core count by the second quarter of 2026, and the Python architecture was nearing its limit. At peak, the Python version of Habitat had supported more than 20 million requests per second.
That was when OpenAI decided to proceed with the migration. The full rewrite was handled by only two engineers. OpenAI said those two engineers, working with Codex and GPT-5.5, rewrote Habitat service from Python to Rust in the second quarter of 2026.
The company added that the Rust version now carries about 95% of production traffic, while the Python version is expected to be retired fully in the coming weeks.
Internal figures show 6x CPU efficiency and 15x memory efficiency
OpenAI said its internal data shows the Rust version improved CPU efficiency by 6x and memory efficiency by 15x. It also reduced average latency and tail latency.
The company presented the migration as one of its clearest internal examples so far of AI-assisted coding replacing engineering labor hours. A core infrastructure migration that would once have required a much larger engineering effort was completed by a very small team working with a coding agent.