presto: Efficient, Training-free, and Open-world Object Placement via Imaginary Search

• URL: http://arxiv.org/abs/2608.21543v1
• Date: Fri, 21 Aug 2026 18:32:28 GMT
• Title: presto: Efficient, Training-free, and Open-world Object Placement via Imaginary Search
• Abstract summary: We reformulate open-world object placement as a search task guided by reasoning from a Multimodal Large Language Model (MLLM)<n>We introduce textsfpresto, a zero-shot, training-free framework that operates within an imaginary action space to iteratively refine object position and scale.<n> Experiments show that textsfprestoachieves state-of-the-art performance, particularly in previously unseen, open-world settings.
• Score: 7.245565016902194
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Object placement is critical in image composition, requiring spatially and semantically coherent positioning of objects within diverse scenes. Existing approaches typically rely on hand-crafted rules or supervised learning on limited datasets, which restricts their generalization and interpretability, especially in open-world scenarios involving novel objects and scenes. In this work, we reformulate open-world object placement as a heuristic search task guided by reasoning from a Multimodal Large Language Model (MLLM). We introduce \textsf{presto}, a zero-shot, training-free framework that operates within an imaginary action space to iteratively refine object position and scale. Our coarse-to-fine search strategy ensures fast convergence, and we evaluate two decision-making variants: Metric-guided Selection and MLLM-as-a-judge. Experiments across multiple benchmarks show that \textsf{presto}~achieves state-of-the-art performance, particularly in previously unseen, open-world settings. Human studies further reveal that the MLLM-as-a-judge variant produces more perceptually coherent placements than metric-driven approaches, highlighting a gap between standard evaluation metrics and human visual judgment.

Related papers

• Unleashing Multimodal Large Language Models for Training-free HOI Detection in the Wild [49.77540427384148]
  We present AgentHOI, a training-free, agentic framework that transfers the generalist multimodal reasoning capabilities of foundation models to HOI detection in the wild.<n>To address the challenges of incomplete interaction discovery and ambiguous localization in complex scenes, we introduce two key mechanisms.<n>AgentHOI achieves superior performance over state-of-the-art supervised and weakly supervised methods in real-world settings.
  arXiv  Detail & Related papers  (2026-07-15T14:30:20Z)
• Compact Object-Level Representations with Open-Vocabulary Understanding for Indoor Visual Relocalization [93.79141221821858]
  OpenReLoc is a camera relocalization system designed to provide scene understanding and accurate pose estimation capabilities.<n>We propose a multi-modal mechanism to integrate open-vocabulary semantic knowledge for effective 2D-3D object matching.<n> Experimental results demonstrate that OpenReLoc achieves superior relocalization recall and accuracy across various datasets.
  arXiv  Detail & Related papers  (2026-06-23T16:27:04Z)
• FindIt: A Format-Informed Visual Detection Benchmark for Generalist Multimodal LLMs [37.64883536754805]
  We introduce the first comprehensive benchmark specifically designed to assess the promptable localization abilities of generalist MLLMs.<n>Our benchmark spans four core task categories: object detection, referring expression detection, instance-level detection, and video-based detection.<n>We evaluate a diverse set of open-source and proprietary MLLMs, providing an in-depth analysis of their performance and limitations.
  arXiv  Detail & Related papers  (2026-06-02T23:14:46Z)
• CLUE: Adaptively Prioritized Contextual Cues by Leveraging a Unified Semantic Map for Effective Zero-Shot Object-Goal Navigation [8.756342403405204]
  Contextual cues from rooms and objects are critical, but their relative importance depends on the target.<n>We present CLUE, a novel navigation framework that adaptively balances the use of contextual rooms and objects.<n>Our framework constructs a unified semantic value map that integrates both types of contextual information.
  arXiv  Detail & Related papers  (2026-05-19T00:15:26Z)
• Relational Semantic Reasoning on 3D Scene Graphs for Open World Interactive Object Search [14.297888615008041]
  We introduce SCOUT: Scene Graph-Based Exploration with Learned Utility for Open-World Interactive Object Search.<n>SCOUT searches directly over 3D scene graphs by assigning utility scores to rooms, frontiers, and objects.<n>We present SymSearch, a scalable symbolic benchmark for evaluating semantic reasoning in interactive object search tasks.
  arXiv  Detail & Related papers  (2026-03-05T19:52:26Z)
• LEGS-POMDP: Language and Gesture-Guided Object Search in Partially Observable Environments [7.458277072199384]
  LanguagE and Gesture-Guided Object Search in Partially Observable Environments (LEGS-POMDP)<n>We introduce LanguagE and Gesture-Guided Object Search in Partially Observable Environments (LEGS-POMDP), a modular POMDP system that integrates language, gesture, and visual observations for open-world object search.<n>In simulation, multimodal fusion significantly outperforms unimodal baselines, achieving an average success rate of 89% across challenging environments and object categories.
  arXiv  Detail & Related papers  (2026-03-05T01:04:23Z)
• Bootstrapping MLLM for Weakly-Supervised Class-Agnostic Object Counting [59.37613121962146]
  We propose WS-COC, the first MLLM-driven weakly-supervised framework for class-agnostic object counting.<n> WS-COC matches or even surpasses many state-of-art fully-supervised methods while significantly reducing annotation costs.
  arXiv  Detail & Related papers  (2026-02-13T09:58:35Z)
• A Multimodal Depth-Aware Method For Embodied Reference Understanding [56.30142869506262]
  Embodied Reference Understanding requires identifying a target object in a visual scene based on both language instructions and pointing cues.<n>We propose a novel ERU framework that jointly leverages data augmentation, depth-map modality, and a depth-aware decision module.
  arXiv  Detail & Related papers  (2025-10-09T14:32:21Z)
• Teaching VLMs to Localize Specific Objects from In-context Examples [56.797110842152]
  We find that present-day Vision-Language Models (VLMs) lack a fundamental cognitive ability: learning to localize specific objects in a scene by taking into account the context.<n>This work is the first to explore and benchmark personalized few-shot localization for VLMs.
  arXiv  Detail & Related papers  (2024-11-20T13:34:22Z)
• Zero-Shot Object-Centric Representation Learning [72.43369950684057]
  We study current object-centric methods through the lens of zero-shot generalization. We introduce a benchmark comprising eight different synthetic and real-world datasets. We find that training on diverse real-world images improves transferability to unseen scenarios.
  arXiv  Detail & Related papers  (2024-08-17T10:37:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.