Stanford and Together AI introduce NCM, a text generation approach reporting up to 31x faster inference

Researchers from Stanford University and Together AI have proposed Next-Concept Modeling, or NCM, as a text generation framework that differs from both autoregressive and diffusion models. Instead of producing text token by token or denoising a full sequence over many rounds, NCM first generates a coarse conceptual outline in parallel and then fills in details layer by layer until the final text is reconstructed. The method borrows the idea of next-scale prediction from the NeurIPS 2024 best paper VAR in image generation and adapts it to language with a multi-scale residual VQ-VAE plus a Transformer trained to predict the next layer of representation. In experiments reported in the paper, NCM achieved similar or better quality while delivering substantial speed gains against diffusion language model baselines: about 21x faster on LM1B unconditional generation, about 10x on OpenWebText, and about 31x on XSum summarization at comparable ROUGE-L. The paper also reports better results on hard Sudoku tasks with fewer generator calls. The authors note that key questions remain, including whether the approach keeps its advantages at larger scales and how well it handles precise or long-form text generation.