Background: Balancing Token Growth and AI Costs
On June 27, Coinbase CEO Brian Armstrong shared the company's practical approach to controlling AI costs. Armstrong noted that to keep AI spending stable while token usage grows exponentially, the key is not to introduce friction or spending alerts, but to adopt better default models, routing mechanisms, and caching strategies. Based on this philosophy, Coinbase implemented a series of system optimizations that ultimately reduced AI spending by nearly half while maintaining token usage growth.
Three Core Strategies: Default Models, Routing, and Caching
Switching Default Models to Open-Weight
Coinbase's LLM gateway now defaults to open-weight models such as GLM 5.2 and Kimi 2.7. The company found that 91% of employees never hit usage limits, so instead of lowering quotas or adding reminders, they simply moved to lower-cost default models. Engineers are still encouraged to select the most suitable model for each task.
Intelligent Model Routing: Frontier for Planning, Lightweight for Execution
In the model routing layer, Coinbase preprocesses prompts in a custom pipeline, then routes tasks based on cache hit rate and model pricing. For example, planning may require a frontier model, but using one for execution would be wasteful. Armstrong believes that in the future, AI should automatically select models rather than humans.
Cache Optimization: Hit Rate Jumps from 5% to 60%
Cache misses are the easiest way to drive up costs. All Coinbase requests are cache-aware to maximize reuse of hot cache. After proper implementation, LibreChat's cache hit rate soared from 5% to 60%, significantly reducing duplicate computation. Additionally, Coinbase requires engineers to keep context lean—starting new sessions when switching tasks, narrowing file context, and disconnecting unused tools.
Results and Outlook: Cost Reduction Without Suppression
Through these practices, Coinbase has cut its total AI spending by nearly half, while token usage continues to climb. Armstrong stressed that the goal is not to curb AI usage, but to build infrastructure that can support exponential growth. In the future, automatic optimization of model selection, routing, and caching will become the norm, with humans only setting overall strategy.