Core of Cost Control: From Quotas to Model and Routing Optimization
Coinbase CEO Brian Armstrong recently outlined the company's latest progress in AI cost optimization. As AI usage and token consumption continue to rise, Armstrong emphasized that the key to controlling costs is not limiting employee usage or sending frequent budget reminders, but optimizing default model selection, task routing mechanisms, and caching strategies. He revealed that 91% of employees have never reached their AI usage limits, so Coinbase chose not to tighten quotas but to improve overall efficiency through lower-cost model solutions.
Default Model Switch: GLM 5.2 and Kimi 2.7 Become New Workhorses
Coinbase is using an internal LLM gateway to experiment with open-weight models like GLM 5.2 and Kimi 2.7 as default options, while still allowing engineers to choose other models based on specific task requirements. In terms of model routing, Coinbase preprocesses prompts and automatically assigns tasks to the most suitable model based on cache hit rates and pricing across different models. Armstrong believes that complex tasks such as planning and reasoning may require frontier models, but execution-oriented tasks don't necessarily need costlier models. In the future, model selection should be more automated by AI rather than relying on human decisions.
Cache and Context Optimization: Hit Rate Jumps from 5% to 60%
Armstrong noted that cache hit rate is a key factor affecting AI costs. Coinbase has integrated a cache-aware mechanism into its request pipeline to increase reuse of historical results. For example, with LibreChat, after optimizing the caching scheme, the cache hit rate rose from 5% to 60%. Additionally, the company requires engineers to keep contexts lean—starting new sessions when switching tasks, narrowing file contexts, and closing unused tools—to reduce unnecessary token consumption. He revealed that through these measures, Coinbase has successfully reduced AI spending by nearly 50% while token usage continues to grow.