A research team from Tsinghua University has argued that object hallucinations in multimodal large language models are not explained by language priors alone. In a paper accepted to ACM MM 2026, the authors said that when model outputs are reduced to short Yes/No answers, inference relies more heavily on the visual modality, exposing a separate failure mode rooted in visual feature extraction and image-text embedding misalignment. The team calls this mechanism “visual-origin hallucination.” To address it, the paper introduces AHAF, short for Adversarial Hallucination Attribute Flipping, and ACFT, or Adversarial Contrastive Fine-Tuning. AHAF uses PGD-based perturbations within a small ℓ∞ ball to flip a non-hallucinating image into one that triggers hallucination, creating tightly aligned positive and negative pairs. ACFT then optimizes those pairs in embedding space by increasing similarity between a text anchor and the positive image while decreasing similarity to the negative image. According to the paper, ACFT uses just 0.9% of the COCO dataset and adds no inference overhead. The reported experiments cover LLaVA, MiniGPT-4, and Qwen2.5-VL, with evaluation on POPE, MME, and four description-level hallucination benchmarks. The authors also report quantitative evidence from embedding similarity, attention entropy, and encoder intervention tests, arguing that visual feature quality plays a causal role in short-answer hallucinations.