Zhipu AI has successfully identified and fixed two critical bugs in its GLM-5 series models used for Coding Agent tasks. The issues, reported by users since March, resulted in garbled text and repetitive outputs, particularly under high concurrency and long context lengths. These defects severely impacted developer workflows and model reliability.
Bug #1: Race Condition in PD-Separation Architecture
The first bug originated from the Prefill-Decode (PD) separation architecture adopted by Zhipu AI. Under high concurrency, a race condition caused memory to be prematurely reclaimed, leading to data overwrites during generation. The engineering team addressed this by introducing proper synchronization in memory deallocation, ensuring all pending read operations complete before memory is released.
Bug #2: Missing Synchronization in HiCache
The second bug resided in Zhipu's self-developed HiCache system, which manages KV cache offloading asynchronously to reduce latency. However, the original implementation lacked proper synchronization, causing premature data reads from cache during concurrent access. This resulted in garbled output and repetition. The fix added memory barriers and reordered read-write operations to guarantee atomicity and consistency of cache operations.
New Anomaly Detection via Speculative Sampling Acceptance Rate
During debugging, Zhipu AI discovered that the acceptance rate metric in speculative sampling serves as a reliable early indicator of generation anomalies. By monitoring this metric in real time, the system can trigger automatic retries before anomalies manifest in output. This provides a low-cost, proactive approach to maintaining stability in large-scale AI inference.
LayerSplit KV Cache Delivers Up to 132% Throughput Boost
To fundamentally enhance performance in long-context scenarios, Zhipu AI optimized the LayerSplit KV Cache mechanism. For requests with token lengths between 40K and 120K, throughput improved by up to 132%. The gain scales linearly with context length, making the optimization especially valuable for tasks involving extensive codebases or lengthy documents.
All fixes have been deployed to production. Zhipu AI plans to integrate further stability and performance improvements in upcoming GLM-5 releases. This incident highlights the engineering challenges faced when deploying large language models in enterprise coding agent applications, where subtle timing defects can be magnified under complex conditions.