Princeton University’s AI Lab has released a new benchmark called Same Space or Not, or SSoN, aimed at testing whether AI systems can reliably understand a 3D scene across different viewpoints. Instead of focusing on dynamic interaction or action planning, the benchmark strips the problem down to a static setting: a model sees four indoor images, identifies the one that does not belong to the same scene, and explains why.
On the full set of 1,000 questions, humans reached 83.3% accuracy. Across six evaluated vision-language models, GPT-6 Astra Pro led with 39.7%. The other five all scored below the 25% random baseline for a four-choice task. The result suggests that the gap between humans and current models already appears at a more basic level of cross-view spatial understanding, even before real-time control enters the picture.
How the SSoN task works
The setup is simple. Among four images, three come from an unchanged 3D scene and one comes from an edited version. There is no designated reference image and no object is marked in advance. The model has to decide which observations can be reconciled and which image cannot coexist with the others.
The paper uses a basketball example to show the difference between appearance mismatch and spatial inconsistency. Images B and D jointly confirm the ball’s location. Image C does not show that spot and reveals objects not visible elsewhere, but that can still be explained by viewpoint change. Image A, by contrast, shows the place where the basketball should appear, yet the ball is missing. The key question is whether the spatial relations remain self-consistent, not whether one image merely looks different.
The team has published the project homepage, paper, dataset, code, and fine-tuned weights through the SSoN website, arXiv, Hugging Face, and GitHub.
Where models struggle across six edit types
SSoN spans seven indoor scene sources, six categories of 3D edits, 1,000 questions, and 4,000 images. The six edit types are swap, detail change, mirror, removal, movement, and rotation. Breaking results down by category shows that current models do not fail uniformly.
Swap and detail change were the easiest categories for the tested systems. When two objects exchange positions, or when one object changes in shape, color, or texture, models can still use object correspondence and appearance cues. All six tested models scored above their own overall averages on these two categories. Astra Pro exceeded 80% on both.
Mirror and removal sat in the middle, but they were still difficult for most models. Mirror questions require the model to decide whether the room structure has actually been flipped left to right, rather than just viewed from another camera position. Removal questions require it to separate true disappearance from occlusion or an object moving out of frame. On mirror questions, every model except Astra Pro fell below the 25% random baseline. On removal, the best score among those five was 42.57% from Sol Pro.
Movement and rotation exposed the broadest geometric weakness. The highest accuracy on movement questions was only 27.20%, and rotation was worse at 9.54%. Humans scored above 70% on both categories. The researchers said that even when models were explicitly told what the six edit types were, scores on movement and rotation did not improve in a meaningful way.
Why some systems fell below random guessing
The paper stresses that 25% is the expected score for blind guessing in a four-way choice, not a floor on capability. SSoN was built with strong distractor views that look different while remaining spatially valid. If a model notices appearance changes but cannot judge whether those changes come from viewpoint shifts, it can be pushed in the wrong direction often enough to perform below random.
That does not mean the model has no visual ability. It points to a gap between detecting differences and understanding space. A system may react when something looks changed, yet still fail to integrate those clues into a coherent 3D judgment once the viewpoint changes.
All 17 visual encoders also scored below the random line
The researchers also tested a simpler strategy: if only one image is anomalous, perhaps the task can be solved by selecting the image least similar to the other three.
They evaluated 17 visual encoders from the CLIP, ViT, and DINOv2 families and used image-feature similarity to pick the outlier view. Accuracy ranged from 16.9% to 21.7%, with every encoder below the 25% random baseline.
This result makes the role of hard distractors more concrete. A correct new viewpoint can change a large portion of visible content, while a real edit may leave only local evidence. A rule based on global appearance difference cannot reliably separate the two. In diagnostic experiments during development, legally valid but misleading views attracted more wrong answers. Replacing them with clearer frontal references improved performance across several categories.
Correct choices often failed the evidence check
SSoN does not stop at answer accuracy. It also checks whether the explanation actually supports the choice. The benchmark defines a metric called True solve, which requires both a correct selection and an explanation that matches the image evidence and is sufficient to justify the cross-view contradiction.
Across 1,368 correct selections made by the six models, 29.5% failed that evidence review. The top model’s score dropped from 39.7% answer accuracy to 34.9% on True solve. In other words, close to one-third of the correct picks did not come with valid support.
The paper gives examples of this failure mode. A model may choose A correctly but claim that an unchanged object was moved. It may also mention a true clue that is still not enough to rule out the other views. If only the final option is scored, those responses count as correct. Under evidence review, they do not hold up.
To reduce ambiguity in the benchmark itself, the team validated both question construction and grading. During dataset creation, each item had to include usable evidence in the anomalous image and in at least two normal views, while multi-solution cases and obvious rendering shortcuts were filtered out. During evaluation, every model response with a correct choice was manually checked for evidence. The final 1,000 questions came from iterative refinement over more than 10,000 development candidates.
Prompting, reconstruction, and camera metadata
To see whether the weakness could be reduced with extra help, the researchers built a 100-question intervention set. On this subset, baseline answer accuracy was 24% and True solve was 12%. All intervention comparisons were run on the same fixed set.
Prompting helped, but only to a point. The team guided models to look for environmental references, match objects, reason about occlusion, or first describe the room layout. They also told the model the six possible edit types without revealing which one applied to the current question. Under the combination of comparative guidance and edit-category knowledge, answer accuracy reached 48% and True solve reached 34%.
The researchers also had models perform 3D reconstruction and inspect camera viewpoints. That execution-verified pipeline produced 47% answer accuracy and 34% True solve. Removal and detail-change questions improved more than movement and rotation, which remained difficult. Adding reconstruction to the workflow was not enough to fix the geometric reasoning problem.
The strongest gain came from giving the model real camera information. Starting from the comparative-guidance setup, the addition of ground-truth camera intrinsics, extrinsics, and image size lifted answer accuracy to 61% and True solve to 48%.
Movement questions showed the clearest jump. Under comparative guidance alone, both answer accuracy and True solve were 20%. With camera information added, both rose to 70%. The result suggests that camera geometry helps models separate an object moving in the image from an object moving in the room.
Even so, the gap between 61% answer accuracy and 48% True solve remained. On this 100-question set, fewer than half of the responses with full support were correct even when real camera parameters were available.
Fine-tuning nearly doubled accuracy on unseen rooms
The team also tested whether targeted training could improve spatial consistency judgments. They fine-tuned Qwen2.5-VL-7B and separated train, validation, and test splits by related rooms and scene families. The final evaluation used 201 test questions from rooms the model had not seen during training.
Without fine-tuning, the model scored 21.89% accuracy. Supervising only the final answer raised that to 33.33%. Adding rationale supervision alongside answer supervision pushed accuracy to 41.79%, nearly doubling the original result.
According to the paper, the difference lies in whether training tells the model why the evidence supports the answer. Rationale supervision adds the edit type, relevant objects, and supporting views, teaching the model to connect cross-view evidence with the final choice. At test time, the model still has to inspect the images and answer on its own.
Rotation questions saw the largest jump. Answer-only supervision brought rotation accuracy to 32.84%. With rationale supervision, that figure rose to 58.21%. When the order of the four options was shuffled, both average accuracy and consistency across orderings also improved.
Team and public resources
SSoN was developed by the Princeton University AI Lab team. First author Huizi Cao is an undergraduate at Carleton College and has previously published as first author in Nature Medicine. The team also includes researchers from the University of Pennsylvania and Adobe Research. The project was jointly advised by Professor Shilong Liu and Professor Mengdi Wang of Princeton AI Lab, and Professor Manling Li of Northwestern University.
The project homepage, paper, dataset, code, and fine-tuned weights are publicly available at https://sson.aisexam.com/, https://arxiv.org/abs/2610.23959, https://huggingface.co/datasets/HuiziCao/SSoN, https://github.com/sson-benchmark/SSoN, and https://huggingface.co/HuiziCao/SSoN-Qwen2.5-VL-7B-LoRA.
The benchmark asks for something basic: the camera can move while the scene stays the same, and the image can change while spatial relations remain consistent. By removing action control and real-time change, SSoN isolates whether a model can integrate multiple viewpoints, tolerate legitimate visual differences, and identify the real contradiction in a scene. On that test, current systems still trail humans by a wide margin.
