EAGLE-4 boosts Qwen inference 3.6x and speeds up phone CPU generation

EAGLE-4 boosts Qwen inference 3.6x and speeds up phone CPU generation

N
News Editor
2026-10-21 11:04:18
Inference acceleration company Eagle has released EAGLE-4, an updated speculative decoding system aimed at cutting draft-model overhead and scaling prediction size based on hardware capacity and request concurrency. The goal is to avoid cases where extra prediction work slows inference instead of improving it. On a setup with four NVIDIA B300 GPUs, Qwen3.8-27B saw single-request generation speed rise from 240.5 tokens per second to 863.8 tokens per second, or 3.6x standard inference. Under a 512-request concurrent workload, total throughput increased from 21,200 tokens per second to 28,300 tokens per second, a 34% gain. Eagle also reported gains on mobile devices. On a Motorola Razr 70 Ultra, Gemma-4-E2B-it achieved a 3.2x improvement in long-text generation using only the CPU. Runtime for the same task fell from 43 seconds to 14 seconds, while energy use per token dropped 53%. The company has open-sourced the code for EAGLE-4 with support for SGLang, vLLM, and experimental TokenSpeed integration. But unlike the first three generations, which used the Apache 2.0 license, the new release adopts a restrictive commercial license. Companies with more than $20 million in aggregate revenue over a rolling 12-month period, as well as firms offering competing inference services, need separate authorization for commercial use.

Inference acceleration company Eagle has released EAGLE-4, updating its speculative decoding approach. The new version is designed to reduce the compute cost of the draft model and adjust prediction scale based on hardware performance and concurrency, so added prediction work does not end up slowing generation.

Higher speed on GPU workloads

On four NVIDIA B300 GPUs, Qwen3.8-27B increased single-request generation speed from 240.5 tokens per second to 863.8 tokens per second, reaching 3.6x the speed of standard inference.

With 512 requests running at the same time, total throughput rose from 21,200 tokens per second to 28,300 tokens per second, a 34% increase.

Mobile CPU performance also improved

Eagle said the gains also showed up on phones. On a Motorola Razr 70 Ultra, Gemma-4-E2B-it delivered a 3.2x improvement in long-text generation using only the CPU. Time to complete the same task dropped from 43 seconds to 14 seconds, and energy consumed per token fell 53%.

Code is public, but licensing has changed

EAGLE-4 has been released with code support for SGLang, vLLM, and experimental TokenSpeed.

Unlike the first three generations, which used the Apache 2.0 open-source license, the new version switches to a restrictive commercial license. Enterprises with aggregate revenue above $20 million over a consecutive 12-month period, along with companies that provide competing inference services, must obtain separate authorization for commercial use.

This article was originally published by Bit.Fan. For more cryptocurrency news and market insights, visit www.bit.fan.
9500

Disclaimer:

The market information, project data, and third-party content displayed on this platform are for industry information sharing only and do not constitute any form of investment advice or return commitment.

Cryptocurrency trading carries high risks. Users should fully assess their risk tolerance and make independent decisions. All profits, losses, and legal responsibilities are borne by the users themselves.