Yanshan Technology’s Yanxin Matrix, working with the Hong Kong University of Science and Technology and Carnegie Mellon University, has released and open-sourced the EdgeRock-4B model, according to Techub News. The model uses a new memory mechanism that keeps KV cache usage fixed at 72 MiB under a 256K context window, a 99.1% reduction from the source model.
The release also includes performance figures for long-context inference. On a single 80GB GPU, EdgeRock-4B can handle 512 concurrent long inference streams and deliver output throughput of 12,124 tokens per second. That is 6.42 times the source model’s level, while the report says inference capability is largely preserved.
Model weights and code have been made available on both GitHub and Hugging Face. The brief was attributed to Techub News, citing Quantum Bit.
Yanshan Technology’s Yanxin Matrix, together with the Hong Kong University of Science and Technology and Carnegie Mellon University, has released and open-sourced the EdgeRock-4B model, according to Techub News.
The model uses a new memory mechanism. Under a 256K context length, KV cache usage is fixed at 72 MiB, down 99.1% from the source model.
On a single 80GB GPU, EdgeRock-4B can process 512 concurrent long inference streams, with output throughput reaching 12,124 tokens per second. That is 6.42 times the source model’s level, while inference capability is largely retained.
The model weights and code have been open-sourced on GitHub and Hugging Face.
This article was originally published by Bit.Fan. For more cryptocurrency news and market insights, visit www.bit.fan. Disclaimer:
The market information, project data, and third-party content displayed on this platform are for industry information sharing only and do not constitute any form of investment advice or return commitment.
Cryptocurrency trading carries high risks. Users should fully assess their risk tolerance and make independent decisions. All profits, losses, and legal responsibilities are borne by the users themselves.
