Debate over Agent security has largely centered on model misalignment, prompt injection, and outside attackers. The ZCode controversy has shifted attention to a different question: whether the maker of an Agent can itself become the source of data exfiltration.
On Sept. 18, tech blogger ferstar published a reverse-engineering analysis of Zhipu’s ZCode. He said that when a user is logged in, ZCode packages an entire working project together with its full edit history, encrypts the bundle, and uploads it to cloud servers in the background. The software interface, he said, does not provide a switch that actually stops the process, and the key needed to decrypt the uploaded package is stored only on Zhipu’s side.
Zhipu apologized shortly after the findings spread through the community. The company said the issue came from a feature that had been enabled by default, said uploaded data was destroyed immediately after use and not retained, and promised to open-source the ZCode codebase, bring in third-party reviewers, and give all users an extra weekly quota reset.
The case is not isolated. Independent security researcher cereblab previously said, based on packet capture analysis, that xAI’s coding Agent tool Grok Build uploaded entire user projects to Google Cloud Storage, including files users had explicitly told the AI not to read and passwords that had not been redacted. Earlier still, Claude Code was found to be sending location and identity-related information back without users’ knowledge, and an Anthropic engineer later confirmed that it had been an intentional experiment.
What stands out is how these incidents came to light. They were not uncovered by regulators or formal audits. They were found by individuals in the community. At the same time, the security frameworks built for Agents still do not contain a rule aimed squarely at constraining the vendor’s own data behavior.
It started with unusual disk usage
Ferstar said he first noticed something was wrong while checking ZCode’s local directories on Sept. 18. He found an encrypted file of about 313 MB, far larger than expected. The file itself could not be opened, but an attached file list showed that the package contained about 42,000 files, with more than 80% of them belonging to the project’s historical revision records.
Those records did not just include project files. They also covered cached large files that had been downloaded before and local operation logs. In other words, what was being packaged was not only the project in its current form, but much of the project’s full history.
Later code analysis, according to the report, showed that this historical record was exempted from all security filtering rules during packaging. ZCode’s file filtering logic was said to run in sequence, with the allow rule for the history directory placed ahead of key filtering and file-size limits. That meant filters for password-related files such as pem and key, as well as the 1 MB size cap, did not apply to anything under the history directory.
The result was that a package measuring hundreds of megabytes could be taken as a whole, while passwords and keys that had once been committed into history and later deleted could still be uploaded in their original form.
Ferstar also reconstructed the upload path. He said the client first requested an upload credential and a public key for encryption from Zhipu’s server, then compressed and encrypted the data locally, then uploaded it directly to Alibaba Cloud object storage rather than through Zhipu’s own business server, and finally triggered a callback from cloud storage to Zhipu’s backend.
Under that setup, the public key acts like a lock that anyone can use to seal the box, while the private key acts like the only key that can open it. Ferstar said ZCode’s design used a lock temporarily issued by the server, while the key remained only on the server side. The Agent generated an encrypted package on the user’s machine, but the user could not inspect what was inside.
He said the mechanism was triggered at two points: before a user sent a request to the AI, and after a task ended. During one active usage session, he observed as many as 62 snapshot records.
Ferstar also examined two interface options that appeared related. After comparing them against the code logic, he concluded that one option, labeled as an experience optimization setting, only controlled whether data could be used for model training. The other, labeled repository snapshot indexing, only controlled whether the server would build a retrieval index after receiving the data. Even with both turned off, local packaging and upload still ran. The component responsible for snapshots and uploads, he said, was loaded unconditionally when the software started, as long as the user was logged in.
Each snapshot also included ZCode’s own global configuration, not just project files, which meant data could be carried across projects. Ferstar said he manually deleted the 313 MB file waiting to be sent, only to see ZCode generate a new package about half an hour later. In his account, the client kept retrying after failed uploads.
Zhipu’s explanation did not answer the same questions the community was asking
Zhipu responded the same night. The company said the issue stemmed from ZCode’s repository indexing feature, which was intended to generate a local repository index to support session checkpoint recovery, historical version rollback, and Repo Wiki. It said Repo Wiki could trigger repository data uploads when generating a Wiki page, and that the uploaded data would be destroyed immediately after the page was generated in the cloud and would not be stored. Because the feature had been enabled by default in the early stage of rollout, some users were affected, the company said, adding that the issue had been fixed.
Zhipu also said it would open-source the ZCode codebase soon, invite third-party evaluators to review system operations, and issue an extra weekly quota reset to all users on the same day.
The article notes that the functions themselves are not unusual. Repository indexing is used to build a directory and retrieval system for project files. Repo Wiki generates project documentation. Session checkpoint recovery and historical rollback let users return to an earlier step in an AI-assisted workflow. The problem, however, was that the company’s explanation and the community’s questions were not aligned.
Zhipu said the indexing system was meant to help users generate an index locally. That immediately raised a basic question: if the work is local, why does the entire project need to be sent to the cloud? Local indexing, local snapshots, and local rollback are technically possible, and the article says there are already tools on the market that work that way.
The timing was another point of dispute. Zhipu tied the issue to Repo Wiki, saying repository data uploads could be triggered while generating a Wiki page. Ferstar’s reverse-engineering record, however, showed that one upload trigger occurred before each user prompt, which is not the same thing as generating documentation.
ZCode’s current official documentation also says that when generating documentation, the system does not read the project’s historical revision records and instead reads filtered code context as needed. On that description alone, the feature would not appear to require historical records. Yet ferstar said 86.6% of the uploaded package consisted of history. The company’s statement did not explain why the upload scope was so broad, whether that scope was by design or the result of a bug, or from which version the behavior had been corrected.
The wording in Zhipu’s statement also drew criticism. By saying the feature had been enabled by default in the early stage of rollout, the company appeared to frame the matter as a switch-setting issue. Ferstar’s code analysis suggested something more fundamental: the visible controls in the interface did not govern the packaging and upload behavior itself.
The community then surfaced another piece of evidence: release notes for ZCode v3.12.2 dated Sept. 16, 2026, two days before ferstar’s post. One line said the update had optimized memory usage for repository snapshot uploads. In the community’s view, engineering teams do not usually optimize memory use for an accidental behavior. That suggested repository snapshot upload had been treated internally as a normal feature under active iteration. The release note was later deleted after the controversy spread, and the removal of a public record became a separate issue in its own right.
Zhipu also said uploaded data would be destroyed immediately and not retained. That addresses retention time, but not the full set of questions users were asking: whether the data had already left the machine, who could access it during server-side processing, who held the ability to decrypt the package, and what deletion policy had been applied to data uploaded earlier.
Given the encryption design described in the report, the private key was held on the server side. That means encrypted upload can show that data is protected in transit from third parties, but it does not show that Zhipu itself is unable to read the contents.
The article also points to wording in ZCode’s English privacy policy, which describes collected material as text, files, and code submitted by users “through conversation.” A background snapshot is not something a user submits through conversation. The difference between “submitted to us in conversation” and “automatically packaged in the background” became another focus of criticism.
The same privacy policy says that when a new feature involves collecting information that is not directly or reasonably related to the original purpose, users should be separately informed and consent should be obtained through page prompts or interaction flows.
Another developer, Feng Ruohang, carried out his own code analysis and said the client requested upload credentials from the server every time a user sent a prompt. If the server issued the credential, collection proceeded. If not, it did not. The 313 MB file found by ferstar was in a pending state at the time and had already failed 564 times. Feng said he independently reproduced ferstar’s forensic process on his own Mac and, across snapshot records from four workspaces, confirmed that at least one snapshot status file had been written with a server acceptance marker.
Under the code logic he described, that marker would only be generated after the server acknowledged receipt of the upload. That means data from at least one machine did in fact leave the local device. Zhipu’s apology, promises, and compensation all came within hours of the controversy spreading, which the article says does not look like a company preparing to hide the issue over the long term. Even so, statements such as “destroyed immediately” and “not stored” cannot be independently verified from the outside. What users can see is that the data left their computer. What happened after that depends on the vendor’s own restraint.
Whether Zhipu’s promised open-sourcing and third-party review can change that depends on which version is opened and whether the review covers the client, the server, or both. For now, those questions remain unanswered.
Why the community focused on history and usage traces
The article argues that if some vendors really wanted to collect user data, the most valuable target might not be code text by itself. Public repositories already provide abundant training material for models. Private code certainly contains commercial secrets, but from the standpoint of model improvement, simply obtaining another batch of code text may not offer the highest marginal value.
What is scarcer, the article says, are three other things.
- First is the causal chain of edits. A project’s revision history does not just store snapshots. It stores the sequence of what the code looked like before, why it changed, and what it became after the change. For coding models, that kind of before-and-after sequence with intent is highly valuable training material.
- Second is usage traces with outcome labels. ZCode’s mechanism takes a full snapshot before each user prompt, and the product also supports rollback. Put together, those actions naturally record a loop of prompt, pre-action state, post-action state, and whether the user accepted the result or rolled it back. In AI training, that kind of data is expensive because it often requires dedicated human labeling.
- Third is real projects that no model has seen before. Public coding benchmarks have largely already been seen during training. Private projects that have not entered training sets are more useful for internal capability evaluation.
The article says those three categories line up closely with the composition of ZCode’s uploaded package, which is why the community has not fully accepted the explanation that the upload was only for generating documentation.
At the same time, the report stops short of claiming intent as a settled fact. If the goal had really been systematic collection of training data, a more precise approach would have been to extract only prompts and code changes, rather than package hundreds of megabytes of large-file cache and full operation logs. Based on the rough scope of the upload package, the article says the evidence fits an aggressive product decision combined with engineering shortcuts more closely than a definitive claim that Zhipu deliberately set out to do something else.
That does not reduce the seriousness of the issue. Whatever the motive, the security risk to users is real.
Grok Build and Claude Code faced similar disputes
The article places ZCode in a broader pattern.
In July this year, independent security researcher cereblab published a full packet-capture analysis of xAI’s Grok Build, together with evidence and reproduction steps. He said Grok Build packaged entire user projects and uploaded them to Google Cloud Storage, covering all files, including files users had explicitly told the AI not to read. In a 12 GB test project, more than 5 GB of files had been confirmed as uploaded before the packet capture was interrupted.
The test also found that password and key files in the project were uploaded in their original form without redaction. Turning off the Improve Model setting did not stop the upload. It only changed training authorization, not whether code left the machine. After the incident became public, Elon Musk said all uploaded data would be deleted, and xAI disabled the upload function on the server side.
Earlier, on March 31, Claude Code accidentally included about 60 MB of source map files in a public installation package because of a configuration mistake in a release. That allowed outside developers to inspect the tool’s architecture.
The community also found that Claude Code polled Anthropic’s servers once an hour for remote configuration. Those configuration items included switches that could force the program to exit or bypass user permission prompts, all taking effect in the background without a manual update from the user.
Claude Code had also been found reading environmental signals such as a user’s proxy settings, gateway address, and China time zone, then sending classification results back to the server through hidden characters in system prompts. An Anthropic engineer later confirmed that this was an intentional experiment aimed at anti-abuse and anti-distillation.
Measured by intent, the article says Claude Code involved an intentional experiment acknowledged by the vendor, Grok Build has not denied the existence of its upload mechanism, and ZCode’s intent remains unresolved. Measured by collection scope, Grok Build uploaded files users had explicitly said should not be read, ZCode packaged 86.6% of project history, and Claude Code sent behavioral metadata. The scale differed, but all three cases raised questions about informed consent.
They also shared another feature: none of them were uncovered by vendor-led debugging, industry audits, or regulatory inspection. One came from a release configuration mistake, one from active packet capture by a security researcher, and one from a blogger noticing abnormal disk usage.
When ZCode launched in July, Zhipu’s marketing positioned it directly against Claude Code. That came only weeks after Claude Code’s telemetry dispute, and ZCode was presented as an alternative that could avoid remote control by the vendor. Grok Build’s upload controversy also happened in July, almost the same month ZCode launched. Three months later, a similar issue surfaced in ZCode itself, with an even broader data scope. The article describes that reversal as one of the more cautionary moments in this year’s AI tool competition.
Current Agent rules are aimed outward
Over the past year, Agents have gained a level of access that exceeds what most consumer software on personal computers had before. They can read all files in the current project directory, execute command-line operations on their own, stay connected to vendor servers continuously, and receive remote configuration updates in the background.
Security rules around those new powers have moved quickly. At the end of 2025, OWASP released its first top-10 risk list for autonomous AI Agents. In January 2026, Singapore introduced the first governance framework aimed at autonomous AI Agents, requiring each Agent to carry a verifiable digital identity. In February, the U.S. National Institute of Standards and Technology launched an AI Agent standards initiative. On Aug. 2, the high-risk obligations under the European Union’s AI Act formally took effect. Industry-specific certification standards for coding Agents have also begun to appear.
But those rules are mainly designed to stop outside misuse, such as malicious instructions hijacking a tool or pushing it to exceed its permissions and call other systems. The whole defensive model assumes the vendor is on the user’s side and the threat comes from outside.
The exfiltration channels described in the ZCode and Grok Build cases sit in the blind spot of that assumption. They are not part of the AI tool’s visible capability list, they are not governed by permission approval flows, and they operate outside the main tool loop. The assistant itself may not even be aware they exist. Run those behaviors through existing security frameworks line by line, and they may not trigger any alarm at all.
Audit, open source, and more practical constraints
One proposal raised in the article is to audit Agent data behavior in the way listed companies’ financial statements are audited. The comparison works only in part. A periodic, standardized review by an independent third party, presented in a form buyers can understand, is a sensible structure.
Still, financial audits examine books that companies are legally required to keep. There is no comparable legal requirement today for vendors to preserve a full record of what data left a user’s computer. That means the evidence base is weak from the start. On top of that, Agent clients may update weekly, and some poll remote configuration every hour to change their own behavior. An annual audit report can be outdated the moment it is issued. Financial audits also sit on top of securities law and auditor liability. Agent audits do not yet have an equivalent backbone.
Open source is another route under discussion. Zhipu has promised to open-source the ZCode codebase after this incident. OpenAI’s Codex CLI and the earlier Gemini CLI both use open-source licenses. Open source can let the community inspect whether a client contains a similar exfiltration mechanism, and that transparency has value on its own.
But open source has hard limits. It can illuminate the client, not the server. Once data reaches the vendor’s server, outsiders still cannot see what happens next. If only the fixed version is opened and the code that was running during the incident is not published, the release says little about past behavior. Users also receive compiled binaries from app stores or installers. Unless reproducible verification is done, there is no direct way to know whether those binaries match the published source code.
The article points to several more practical paths.
- Vendors could be required to disclose which server addresses an Agent connects to and what categories of data it transmits, making abnormal traffic easier to compare with independent tools. Cereblab’s Grok Build analysis was done with standard packet-capture tools, and a vendor-provided egress declaration would lower verification costs.
- A readable, exportable exfiltration log could be kept on the user’s own machine, showing the size, destination, and data category of each transmission. That would directly address the design problem in which an encrypted package is generated on the user’s device but the user cannot see what is inside.
- Liability insurance could be introduced so that underwriters, rather than certification bodies, assess vendor data behavior. Because underwriters have to pay for bad judgments, the article says this is the only mechanism on the table that clearly turns serious review into an economic incentive.
None of those ideas are technically difficult, the article says. The hard part is motivation. Right now, the forces pushing them are mainly enterprise procurement reviews and occasional community exposure. The first usually covers enterprise editions only. The second depends on chance.
The person who clicks consent is not the one who bears the risk
The article closes with a structural problem that may be harder than any technical fix. In cases like ZCode and Grok Build, the person clicking “agree” is the individual developer. The party that bears the consequences of a data leak is often the developer’s employer or client. That employer or client never appears in the consent flow and may have no channel to learn that its code was packaged and uploaded at all.
Because the risk bearer and the authorizing party are not the same, individual informed consent cannot fully solve the problem. No matter how clear the pop-up is or how visible the switch appears, that mismatch remains.
The article says the likely outcome is a layered market. Large enterprises will add data-behavior clauses and audit rights to procurement contracts, and the cost will show up in pricing. Consumer versions used by individual developers may remain in a state where no one audits them and no one is clearly accountable. Yet that is exactly where many people continue writing code after work, and where they are most likely to open company projects with personal accounts.