Princeton AI Lab’s SSoN benchmark shows humans at 83.3% and top VLM at 39.7% on spatial reasoning

Princeton University’s AI Lab has introduced Same Space or Not, or SSoN, a benchmark designed to test a basic but difficult question for multimodal AI systems: can they tell whether images taken from different viewpoints still describe the same 3D scene. The benchmark uses 1,000 questions and 4,000 indoor images. In each question, three images come from an unchanged scene and one comes from an edited version, and the model must identify the inconsistent image and explain why. The gap between humans and models was wide. Human participants reached 83.3% accuracy across the full benchmark. Among six tested vision-language models, GPT-6 Astra Pro posted the best score at 39.7%, while the other five fell below the 25% random baseline for a four-choice task. The study also found that 17 visual encoders, including CLIP, ViT, and DINOv2 variants, scored only 16.9% to 21.7% when asked to pick the outlier by feature similarity. Researchers also evaluated interventions such as prompting, 3D reconstruction, camera metadata, and fine-tuning. Real camera parameters produced the largest gain on a 100-question intervention set, lifting answer accuracy to 61% and True solve to 48%. Fine-tuning Qwen2.5-VL-7B on separated room splits raised accuracy on unseen-room test questions from 21.89% to 41.79% when rationale supervision was added.