One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Grounding

• URL: http://arxiv.org/abs/2606.30084v1
• Date: Mon, 29 Jun 2026 10:20:39 GMT
• Title: One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Grounding
• Abstract summary: We propose InnerZoom, a single-forward framework for cross-layer evidence bridging.<n>Results suggest InnerZoom achieves 64.7 on OSWorld-G, 40.2 on UI-Vision, 73.1 on OSWorld-GR, and 87.6 on MMBench-GUI.
• Score: 23.58705585303581
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: MLLM-based GUI grounding methods commonly formulate target localization as autoregressive coordinate generation, enabling models to leverage the strong instruction-following and semantic understanding capabilities of MLLMs. However, this formulation requires the model to retain region-level target evidence while decoding coordinate tokens with the spatial precision demanded by GUI clicking. Our diagnostic analysis reveals that target-region awareness emerges in intermediate decoder layers but is neither retained nor translated into the final coordinate prediction. Existing ZoomIn-style methods address this issue through an external crop-and-rerun pass, which improves localization but increases end-to-end latency and computational cost. To retain the accuracy benefits of two-pass zooming without this extra cost, we propose InnerZoom, a single-forward framework for cross-layer evidence bridging. InnerZoom transforms target-related cues from the original forward pass into a compact cross-layer evidence state, then preserves, refines, and reinjects this state throughout later decoding layers to guide coordinate prediction. Extensive experimental results suggest that InnerZoom-4B achieves state-of-the-art performance on all six GUI grounding benchmarks, obtaining 64.7 on OSWorld-G, 40.2 on UI-Vision, 73.1 on OSWorld-GR, and 87.6 on MMBench-GUI, surpassing the previous best results by 4.1, 3.2, 2.9, and 2.3 points, respectively. Under a controlled 4B setting, InnerZoom improves the same SFT+RL baseline by 5.3 points on average and outperforms two-pass ZoomIn by 1.3 points on average, while reducing end-to-end latency by up to 31.8% and TFLOPs by about 29%. Code and models will be publicly available.

Related papers

• MANGO-Grasp: Mahalanobis Fields over Geometry-Oriented 3D Gaussians for Cross-Embodiment Dexterous Grasping [82.63833405368159]
  Cross-embodiment dexterous grasping aims to synthesize stable grasps across heterogeneous multi-fingered hands with little or no embodiment-specific tuning.<n>We propose MANGO-Grasp, an anisotropic interaction framework that represents objects as geometry-oriented 3D Gaussian primitives and robot hands as surface keypoints encoded into morpho-kinematic descriptors.
  arXiv  Detail & Related papers  (2026-08-03T10:12:15Z)
• FoundationGeo: Learning Spatial Pixel-Wise Fields for Monocular Metric Geometry [52.232826445242644]
  FoundationGeo bridges relative and metric prediction via spatial calibration and principled data design.<n>Stage 1 learns a high-fidelity, affine-invariant geometry model by initializing with DINOv3.<n>Stage 2 moves beyond global scaling by introducing pixel-wise calibration fields for metric estimation.
  arXiv  Detail & Related papers  (2026-07-13T14:10:01Z)
• KPGrasp: Scalable Keypoint Flow Matching for Dexterous Grasp Generation [23.028638559051597]
  We present KPGrasp, a flow-matching framework that learns dexterous grasp priors from large-scale data.<n>Experiments on 20 diverse objects demonstrate that the pipeline can be deployed in a real-world setup.
  arXiv  Detail & Related papers  (2026-06-08T10:19:29Z)
• LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence [107.63317552620231]
  We present LLaVA-OneVision-2 (LLaVA-OV-2), the most capable vision-OneVision model to date.<n>It achieves superior performance across a broad range of multimodal benchmarks.<n>A standout capability is its unified perception across video understanding, temporal grounding, spatial grounding, and manipulation-language reasoning.
  arXiv  Detail & Related papers  (2026-05-25T15:54:04Z)
• Efficient Hybrid CNN-GNN Architecture for Monocular Depth Estimation [0.0]
  We present GraphDepth, a monocular depth estimation architecture that integrates Graph Neural Networks (GNNs) within a convolutional encoder-decoder framework.<n>Our approach embeds efficient GraphSAGE layers at multiple scales of a ResNet-101 U-Net backbone.<n> Experiments on NYU Depth V2, WHU Aerial, ETH3D, and Mid-Air benchmarks demonstrate competitive accuracy within 4.6% of state-of-the-art transformers.
  arXiv  Detail & Related papers  (2026-05-11T09:21:04Z)
• Global and Local Topology-Aware Attention with Persistent Homology and Euler Biases for Time-Series Forecasting [0.0]
  Time series often encode predictive geometric structure, including connectivity, cycles, shell-like geometry, directional changes, and nonlinear neighborhoods.<n>We introduce a topology-aware attention framework that adds such structure to attention logits using persistent homology (H0-H2)<n>We evaluate guarded topology-aware variants across three architecture families: lightweight attention/Ridge, PatchTSTForRegression, and TimeSeriesTransformerForPrediction.
  arXiv  Detail & Related papers  (2026-05-04T21:19:12Z)
• GUI-G$^2$: Gaussian Reward Modeling for GUI Grounding [51.497245303008015]
  Graphical User Interface (GUI) grounding maps natural language instructions to precise interface locations for autonomous interaction.<n>Motivated by human clicking behavior that naturally forms Gaussian distributions centered on target elements, we introduce GUI Gaussian Grounding Rewards.<n>We show that GUI-G$2$, substantially outperforms state-of-the-art method UI-TARS-72B, with the most significant improvement of 24.7% on ScreenSpot-Pro.
  arXiv  Detail & Related papers  (2025-07-21T17:53:42Z)
• G-MSGINet: A Grouped Multi-Scale Graph-Involution Network for Contactless Fingerprint Recognition [20.458766184257147]
  G-MSGINet is a unified framework for robust contactless fingerprint recognition.<n>It jointly performs minutiae localization and identity embedding directly from raw input images.<n>Extensive experiments on three benchmark datasets show G-MSGINet consistently achieves minutiae F1-scores in the range of $0.83pm0.02$ and Rank-1 identification accuracies between 97.0% and 99.1%.
  arXiv  Detail & Related papers  (2025-05-13T05:24:24Z)
• Coordinate Transformer: Achieving Single-stage Multi-person Mesh Recovery from Videos [91.44553585470688]
  Multi-person 3D mesh recovery from videos is a critical first step towards automatic perception of group behavior in virtual reality, physical therapy and beyond. We propose the Coordinate transFormer (CoordFormer) that directly models multi-person spatial-temporal relations and simultaneously performs multi-mesh recovery in an end-to-end manner. Experiments on the 3DPW dataset demonstrate that CoordFormer significantly improves the state-of-the-art, outperforming the previously best results by 4.2%, 8.8% and 4.7% according to the MPJPE, PAMPJPE, and PVE metrics, respectively.
  arXiv  Detail & Related papers  (2023-08-20T18:23:07Z)
• PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation [111.7241018610573]
  We present PointGroup, a new end-to-end bottom-up architecture for instance segmentation. We design a two-branch network to extract point features and predict semantic labels and offsets, for shifting each point towards its respective instance centroid. A clustering component is followed to utilize both the original and offset-shifted point coordinate sets, taking advantage of their complementary strength. We conduct extensive experiments on two challenging datasets, ScanNet v2 and S3DIS, on which our method achieves the highest performance, 63.6% and 64.0%, compared to 54.9% and 54.4% achieved by former best
  arXiv  Detail & Related papers  (2020-04-03T16:26:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.