Prime Intellect launches Prime Mega MoE, reporting a 3.11x gain in MoE training forward speed

Prime Intellect launches Prime Mega MoE, reporting a 3.11x gain in MoE training forward speed

N
News Editor
2026-10-29 19:24:58
Prime Intellect, an AI infrastructure company, has introduced Prime Mega MoE, a trainable extension built on DeepSeek’s open-source Mega MoE kernel. The release adds fast, overlapped backward propagation to a kernel that previously supported only forward propagation, allowing it to be used for training rather than inference alone. According to Prime Intellect, the full training step for the Qwen3 MoE model runs up to 34% faster inside its prime-rl trainer, and the gain reaches as high as 55% when compared with TorchTitan. The company also said the kernel supports NVIDIA Hopper and Blackwell GPUs, along with BF16, MXFP8, and blockwise FP8 data types. Prime Intellect described Prime Mega MoE as a fused and overlapped MoE training kernel designed to address GPU idle time caused by communication bottlenecks during MoE training. It said the result is a direct reduction in required GPU hours for teams training mixture-of-experts models. The item was cited by MarkTechPost and published by Techub.

Prime Intellect has released Prime Mega MoE, a trainable extension built on DeepSeek’s open-source Mega MoE kernel.

According to the company, DeepSeek’s original kernel supported forward propagation only. Prime Mega MoE adds fast, overlapped backward propagation, which means the kernel can be used for training instead of inference alone.

Training speed figures

Prime Intellect said that inside its prime-rl trainer, the full training step for the Qwen3 MoE model is up to 34% faster. Against TorchTitan, the improvement reaches as high as 55%.

The headline figure states that forward speed in MoE training improved by 3.11x.

Hardware and data type support

The kernel supports NVIDIA Hopper and Blackwell GPUs. It also works with BF16, MXFP8, and blockwise FP8 data types.

Focus on cutting GPU idle time

Prime Intellect said the release directly reduces the number of GPU hours needed for teams training mixture-of-experts, or MoE, models. The company described Prime Mega MoE as a fused, overlapped MoE training kernel aimed at addressing GPU idle time caused by communication bottlenecks during MoE training.

The item was cited by MarkTechPost and published by Techub.

This article was originally published by Bit.Fan. For more cryptocurrency news and market insights, visit www.bit.fan.
6300

Disclaimer:

The market information, project data, and third-party content displayed on this platform are for industry information sharing only and do not constitute any form of investment advice or return commitment.

Cryptocurrency trading carries high risks. Users should fully assess their risk tolerance and make independent decisions. All profits, losses, and legal responsibilities are borne by the users themselves.