NVIDIA said in a technical blog post that it tested three optimizations for Mixture-of-Experts inference on DGX Spark using Ling-3.0-flash, a model from Ant Group-backed inclusionAI. The model is described as a 124B-parameter MoE system with 5.1B parameters activated per token. The optimizations listed were the Humming MoE kernel, an FP8 LM head, and MTP speculative decoding. According to the figures cited by MarkTechPost, results from a 1,319-question GSM8K test showed decoding throughput rising from 37.81 tokens per second to 53.89 tokens per second, a 42.5% increase. Average latency fell 24.5%, while accuracy stayed nearly unchanged at 96.89% and 96.82%, respectively. The post focused on inference efficiency gains rather than changes to model quality, with the reported accuracy figures showing little difference between the compared setups.
NVIDIA said in a technical blog post that it demonstrated three optimizations for accelerating Mixture-of-Experts inference on DGX Spark, using Ling-3.0-flash as the test model.
Ling-3.0-flash is a 124B-parameter MoE model from inclusionAI, under Ant Group, with 5.1B parameters activated per token. The three optimizations named by NVIDIA were the Humming MoE kernel, an FP8 LM head, and MTP speculative decoding.
According to figures cited by MarkTechPost, a GSM8K test covering 1,319 questions showed decoding throughput increasing from 37.81 tokens/s to 53.89 tokens/s, up 42.5%. Average latency dropped 24.5%, while accuracy was largely unchanged at 96.89% and 96.82%, respectively.
This article was originally published by Bit.Fan. For more cryptocurrency news and market insights, visit www.bit.fan. Disclaimer:
The market information, project data, and third-party content displayed on this platform are for industry information sharing only and do not constitute any form of investment advice or return commitment.
Cryptocurrency trading carries high risks. Users should fully assess their risk tolerance and make independent decisions. All profits, losses, and legal responsibilities are borne by the users themselves.
