NVIDIA details three inference optimizations for Ling, lifting throughput 42.5% on DGX Spark
NVIDIA said in a technical blog post that it tested three optimizations for Mixture-of-Experts inference on DGX Spark using Ling-3.0-flash, a model from Ant Group-backed inclusionAI. The model is described as a 124B-parameter MoE system with 5.1B parameters activated per token. The optimizations listed were the Humming MoE kernel, an FP8 LM head, and MTP speculative decoding. According to the figures cited by MarkTechPost, results from a 1,319-question GSM8K test showed decoding throughput rising from 37.81 tokens per second to 53.89 tokens per second, a 42.5% increase. Average latency fell 24.5%, while accuracy stayed nearly unchanged at 96.89% and 96.82%, respectively. The post focused on inference efficiency gains rather than changes to model quality, with the reported accuracy figures showing little difference between the compared setups.
