Z.ai founder Jie Tang says bigger parameter counts no longer tell the full story of model strength
Jie Tang, founder of Z.ai and a professor at Tsinghua University, argues that asking only how many parameters a model has no longer says much about how strong it is. In his review of the evolution of scaling laws—from GPT-3 to Chinchilla and then Mixture of Experts (MoE)—he says model capability depends on more than parameter count. Training data volume, where compute is spent, and how a model is actually used all matter. Tang’s point is that the old training-first view of scaling is less useful once commercial AI systems are deployed and called billions of times a day. Under that setup, inference cost changes the optimization target. A smaller model trained for longer may make more sense than a larger one trained less efficiently. He cited Llama-2-7B and Gemma-2-9B as examples of models trained far beyond the classic Chinchilla ratio. He also said MoE makes headline parameter numbers even less informative, because total parameters and activated parameters describe different things. For reasoning-heavy workloads, Tang argued that effective depth in a single inference pass and post-training may now be more important scaling dimensions. He described GLM-5.3 as a controlled test of that idea, keeping the base model and parameter counts unchanged from GLM-5.2 while expanding long-horizon environments and reinforcement learning over a month.








