In a flash update on Aug 28, BlockBeats detailed the architectural changes behind Tencent's Hunyuan Hy4 preview. The model is bigger, but the bigger story is under the hood. Hy3 used full attention; Hy4 moves to Gated DSA, a gated form of DeepSeek Sparse Attention. On long inputs, instead of recomputing the entire context, the model selects the most relevant parts and focuses compute there. In Hy3 every layer had to decide what mattered; Hy4 layers Zhipu's IndexCache on top of DSA, so some layers can reuse earlier selections and cut repeated work. Tencent credits DeepSeek and GLM as inspirations. Residual structure also gets a rethink: a standard Transformer is often described as having a single information trunk, while Hy4's iHC widens that to four trunks so the network can retain more information at depth. MoE configuration moves from Hy3's 192 experts to 256 routing experts plus one shared expert, with each token choosing 8 routing experts while always passing through the shared expert.
Tencent's Hunyuan Hy4 preview is not just a larger model. Behind the scenes, the base architecture has been substantially reworked. The previous Hy3 generation used conventional full attention. Hy4 switches to Gated DSA, a gated version of DeepSeek Sparse Attention. When facing very long text, the model stops recomputing everything that came before; instead it picks out the most relevant parts and concentrates computation there.
Hy4 also adds Zhipu's IndexCache to DSA. Each layer used to re-evaluate which content mattered most. Some layers can now reuse results already selected earlier, cutting down on repeated computation. Tencent explicitly says the design was inspired by DeepSeek and GLM.
The residual structure is another area of change. A conventional Transformer can be roughly understood as having a single information trunk. Hy4 uses iHC to expand that into four trunks, allowing the network to retain more information at depth. MoE expert counts rise from 192 in Hy3 to 256 routing experts plus one shared expert. Every token picks 8 routing experts and always passes through the shared expert as well.
This article was originally published by Bit.Fan. For more cryptocurrency news and market insights, visit www.bit.fan. Disclaimer:
The market information, project data, and third-party content displayed on this platform are for industry information sharing only and do not constitute any form of investment advice or return commitment.
Cryptocurrency trading carries high risks. Users should fully assess their risk tolerance and make independent decisions. All profits, losses, and legal responsibilities are borne by the users themselves.
