Researchers from Tsinghua University and the Chinese University of Hong Kong have open-sourced AKV, an inference serving system built for key-value cache engineering in large language model agents. Instead of summarizing older interaction history, AKV lets agents discard and relocate KV cache entries. In tests covering three models and four agent benchmarks, the system reduced average cost by 27.7% and improved average task scores by 1.7 points. The code is released under the Apache-2.0 license, built on top of SGLang, and runs on NVIDIA GPUs. The research paper used eight A800-80GB GPUs. According to the reported results, AKV delivered its best performance with the GPT-OSS-120B model on the SWE-bench Verified benchmark, where per-task cost fell from $0.20 to $0.097. The development was reported by MarkTechPost.
Researchers from Tsinghua University and the Chinese University of Hong Kong have open-sourced AKV, an inference serving system designed for key-value cache engineering in large language model agents, according to Techub News.
AKV allows agents to discard and relocate KV cache entries instead of summarizing older history. Across three models and four agent benchmarks, the system reduced average cost by 27.7% and lifted average task scores by 1.7 points.
The code is released under the Apache-2.0 license, built on SGLang, and runs on NVIDIA GPUs. The research paper used eight A800-80GB GPUs.
Its strongest reported result came with the GPT-OSS-120B model on the SWE-bench Verified benchmark, where per-task cost dropped from $0.20 to $0.097.
The report cited MarkTechPost.
This article was originally published by Bit.Fan. For more cryptocurrency news and market insights, visit www.bit.fan. Disclaimer:
The market information, project data, and third-party content displayed on this platform are for industry information sharing only and do not constitute any form of investment advice or return commitment.
Cryptocurrency trading carries high risks. Users should fully assess their risk tolerance and make independent decisions. All profits, losses, and legal responsibilities are borne by the users themselves.
