Abstract
As current Multimodal Large Language Models rapidly saturate canonical visual reasoning benchmarks, a key question emerges: do these strong scores genuinely reflect robust visual understanding? We identify a pervasive vulnerability, the Cartesian Shortcut: models frequently discretize the orthogonal grid-based layouts prevalent in visual reasoning benchmarks into explicit textual coordinates, offloading reasoning from visual perception to text-based deduction. To re-evaluate visual reasoning when this shortcut is unavailable, we introduce Polaris-Bench, which re-formulates 53 visual reasoning tasks in Polar coordinate space with paired Cartesian counterparts that preserve task semantics, disrupting the orthogonal structure that models exploit. Comprehensive evaluation across 14 state-of-the-art MLLMs reveals that frontier models achieving 69–83% on Cartesian layouts collapse to 31–39% on Polar equivalents. Moreover, thinking gains largely vanish on Polar layouts, prompting interventions fail to close the gap, and comparable drops arise on other non-orthogonal layouts. These findings reveal that current MLLMs’ visual reasoning performance is strongly coupled to orthogonal grid structure, a fragility that is consistent across model families but far smaller in humans, who maintain 88.8% accuracy on Polar layouts.
The Cartesian Shortcut
When solving grid-based visual tasks such as Sudoku, maze navigation, or graph coloring, state-of-the-art MLLMs frequently discretize the orthogonal grid layout into explicit textual coordinates (e.g., "row 2", "(x,y)"), offloading reasoning from visual perception to text-based deduction. We call this behavior the Cartesian Shortcut.
We analyze over 3,800 questions across 9 prominent visual reasoning benchmarks. Frontier models explicitly invoke textual coordinates in over 56% of their intermediate Chain-of-Thought reasoning; on Cartesian layouts, Gemini-3.1-Pro and GPT-5.2 invoke spatial discretization in 79.6% and 87.7% of their reasoning, respectively. By offloading reasoning from visual perception to text-based deduction, this shortcut confounds the evaluation of visual reasoning on grid-based benchmarks.
Polaris-Bench
To rigorously test whether models can reason about visual structure independently of grid coordinates, we introduce Polaris-Bench, a benchmark that re-formulates 53 visual reasoning tasks in Polar coordinate space, each paired with a Cartesian counterpart that preserves task semantics and serves as a controlled reference under consistent logical constraints.
The Cartesian-to-Polar transformation disrupts the orthogonal structure underlying the Cartesian Shortcut, making it much harder to discretize the layout into textual coordinates. For most tasks the transformation is the only variable (fully aligned, with identical rules and ground truths); for a deliberately preserved subset, Polar angular wrapping alters the solution space (partially aligned). If model performance depends on orthogonal grid structure, accuracy should drop on the Polar counterparts.
5 categories, 53 tasks.
Representative examples for each of the five core taxonomies in Polaris-Bench.
Key Findings
We evaluate 14 state-of-the-art MLLMs on Polaris-Bench, including Gemini-3.1 Pro, GPT-5.2, Claude Sonnet 4.6, and open-weight models like Qwen3.5-397B and Gemma-4. We compare each model's accuracy on the Cartesian and Polar versions of every task, alongside a human reference.
As shown below (left), every evaluated model falls below the equal-accuracy diagonal: frontier models achieving 69–83% on Cartesian layouts collapse to 31–39% on Polar equivalents, and while Cartesian accuracy spans 19–83% across models, Polar accuracy compresses into a narrow 19–39% band, suggesting a shared performance floor. Humans, by contrast, drop only 5.7 points (94.5% → 88.8%). The collapse persists on fully aligned tasks, where the coordinate transformation is the only variable. Furthermore (right), thinking gains that are substantial on Cartesian layouts (up to +50.8 pts for GPT-5.2) are systematically compressed on Polar layouts, with most models improving by fewer than 10 points, showing that thinking gains are strongly coupled to orthogonal grid structure.
Is the Gap Specific to Polar Layouts?
No. On Word Search, we compare Cartesian and Polar layouts with Octagonal and Hexagonal tilings. Octagonal tilings, which preserve local orthogonal adjacency, stay close to Cartesian accuracy, whereas Hexagonal and Polar layouts show large drops for most models. We further apply progressive geometric deformations to Cartesian images: for models with substantial Cartesian accuracy, a sine-wave deformation, which introduces neither wrap-around nor radial compression, already causes a large drop (e.g., Gemini-3.1-Pro: 89.0% → 44.0%), and accuracy approaches the Polar level by a 90° arc. Comparable drops therefore arise without Polar-specific properties, on layouts that depart from the standard orthogonal grid.
Word Search in Cartesian, Octagonal, Hexagonal, and Polar coordinates.
Word Search accuracy across 4 coordinate systems. The drop tracks deviation from orthogonal grid structure.
Can Prompting Close the Gap?
No. On six tasks with large drops, we evaluate two test-time interventions across six models: (1) a conversion hint instructing the model to re-map the Polar layout onto a Cartesian grid before solving, and (2) 5-shot in-context learning with solved Polar examples. Conversion hints change accuracy by less than 4 points relative to the Polar baseline, and 5-shot examples lower accuracy for five of the six models (e.g., GPT-5.2: 38.3% → 26.0%). Test-time instruction and demonstrations therefore do not close the gap between familiar orthogonal layouts and Polar layouts, which humans solve with a modest drop.
Leaderboard
High reasoning mode. C = Cartesian, P = Polar accuracy (%). Δ = C→P drop. Click column headers to sort.
Task Examples
Paired Cartesian and Polar evaluation instances across 20 representative benchmark tasks.
Getting Started
Load via HuggingFace datasets. Evaluation scripts on GitHub.
from datasets import load_dataset ds = load_dataset("google/polaris-bench", split="test") s = ds[0] print(s["task"], s["question_type"]) print(s["question"]) s["image"].show()
Citation
@misc{hu2026cartesianshortcutreevaluatevision,
title = {The Cartesian Shortcut:
Re-evaluate Vision Reasoning
in Polar Coordinate Space},
author = {Xia Hu and Zhenrui Yue
and Brian Potetz and Howard Zhou
and Leonidas Guibas and Chun-Ta Lu
and Zhicheng Wang},
year = {2026},
eprint = {2605.09883},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2605.09883}
}