Background: Surging AI Usage and the Need for Cost Control
As AI adoption and token consumption accelerate, companies worldwide face mounting budget pressure. Coinbase CEO Brian Armstrong recently published a detailed breakdown of the company's AI cost optimization strategy. He emphasized that the key to controlling costs is not to restrict employee access or send frequent budget reminders, but to optimize model selection, task routing, and caching mechanisms from a technical perspective.
Model Selection and Task Routing: Open-Weight Models Take Center Stage
Coinbase has deployed an internal LLM gateway that defaults to open-weight models such as GLM 5.2 and Kimi 2.7, while still allowing engineers to choose other models based on specific task requirements. Data shows that 91% of employees never hit their AI usage quotas, so Coinbase did not tighten quotas but instead improved efficiency through lower-cost model solutions. For task routing, the system pre-processes prompts and automatically assigns tasks to the most appropriate model based on cache hit rates and pricing across different models. Armstrong noted that complex tasks like planning and reasoning may require cutting-edge models, but execution-oriented tasks do not necessarily need expensive models. In the future, model selection should be increasingly automated by AI rather than relying on human decisions.
Cache Optimization and Context Management: Details Drive Savings
Cache hit rate is a major factor affecting AI costs. Coinbase integrated a cache-aware mechanism into the request flow to improve reuse of historical results. Using LibreChat as an example, after optimizing the caching scheme, the cache hit rate jumped from 5% to 60%. Additionally, the company requires engineers to keep context concise — by starting new sessions when switching tasks, narrowing file context scope, and closing unused tools — to reduce unnecessary token consumption. Armstrong revealed that these combined measures successfully cut Coinbase's AI spending by nearly 50% while token usage continued to grow.