MemoBench Exposes a Core Weakness in World Models as 10 Video Generators Score Below 0.6 on Object Reappearance

Researchers from Harvard, MIT, IBM, Boston University, Google, Johns Hopkins, CMU, and the Kempner Institute have introduced MemoBench, a new benchmark designed to test whether video generation models can preserve object permanence in dynamically changing environments. The benchmark addresses a major blind spot in current world-model evaluation: most existing tests focus on frame consistency while objects remain visible, but rarely measure whether a model can maintain identity, update state, and restore an object correctly after it leaves the camera view and continues changing off-screen. Built on 360 high-quality ground-truth videos spanning both synthetic and real-world scenes, MemoBench evaluates 10 leading video and world-generation models using automated metrics and VQA-based semantic scoring. The headline result is that no model achieved an object reappearance score above 0.6 out of 1. The findings suggest that visually coherent generation still falls far short of genuine world understanding, especially when memory continuity and state evolution under occlusion are required.