The 2026 World Artificial Intelligence Conference (WAIC) opened in Shanghai on Thursday, where a panel on AI agent governance drew a clear line around what machines should not be allowed to decide.
Xue Lan, dean of Tsinghua University’s Institute for AI International Governance, Nicholas B. Dirks, president and CEO of the New York Academy of Sciences, and Mark Nitzberg, executive director of the Center for Human-Compatible AI (CHAI) at the University of California, Berkeley, shared a common view: AI is moving from assistive judgment to autonomous action, but decisions involving life and death, irreversible outcomes, and ethical or value-based judgment must not be led by AI.
From recommendation tool to acting agent
The panel framed the shift as a digital-age version of the classic principal-agent problem. AI systems are no longer limited to searching for information or offering suggestions. They are increasingly able to call tools, place orders, and carry out tasks directly, taking on the role of agents that act on behalf of people.
That creates a harder governance problem than traditional human delegation, the speakers said. A human agent at least shares common sense and exists within a recognizable accountability structure. AI, by contrast, presents two key difficulties in this account: its goals may not align with what a person actually wants, and its internal operation can remain opaque, making it a black box that cannot itself assume legal responsibility.
Humans may authorize action, not responsibility
Because AI cannot bear responsibility, accountability must run through the full execution chain rather than stop at the model. The panelists said responsibility should rest with each part of that chain, including developers, deployers, and regulators.
Their formulation was direct: humans can authorize AI to act, but they cannot authorize AI to be responsible. Every authorization should be revocable, and every action should be traceable to someone who can be held accountable. Treating AI as an object that can absorb blame, they argued, starts from the wrong premise.
Three categories of decisions were set out as red lines
The speakers also identified three areas where AI should not take the lead:
- decisions involving life-and-death consequences,
- scenarios where mistakes cannot be repaired, and
- questions that involve ethical and value judgment.
The point, as framed in the discussion, is not whether AI should be used at all. It is that some matters should never be handed over to AI as the deciding authority in the first place.
What trustworthy AI requires
Beyond broad principles, the panel outlined three engineering characteristics for trustworthy AI: strong foundations, transparent operation, and controllability during use.
At the institutional level, the discussion called for wider coordination as well. The speakers pointed to the need for globally unified AI safety evaluation standards, mutually recognized testing systems, incident data-sharing mechanisms, clearly defined red lines for AI development, and early-warning monitoring, with AI safety treated as a global public good.
A governance debate that returns to human responsibility
The panel closed by pushing the discussion beyond technical boundaries. In their view, the limits placed on AI governance are not only about where technology should stop. They also mark a starting point for rethinking human values, responsibility, and the direction of civilization.
The source article said the discussion was compiled from the AI agent governance roundtable at the 2026 WAIC. It also noted that the exchange focused on ethics and may still sit at some distance from the current path of real-world AI development.
Key takeaways from the roundtable
- AI is shifting from assistive judgment to autonomous action.
- AI agent governance can be understood as a digital version of the principal-agent problem.
- AI must not lead decisions involving life and death, irreversible scenarios, or ethical and value judgment.
- Humans may authorize AI to act, but not to bear responsibility.
- Trustworthy AI requires strong foundations, transparent operation, and controllability during use.