PersONAL: Towards a Comprehensive Benchmark for Personalized Embodied Agents

• URL: http://arxiv.org/abs/2509.19843v1
• Date: Wed, 24 Sep 2025 07:39:16 GMT
• Title: PersONAL: Towards a Comprehensive Benchmark for Personalized Embodied Agents
• Authors: Filippo Ziliotto, Jelin Raphael Akkara, Alessandro Daniele, Lamberto Ballan, Luciano Serafini, Tommaso Campari,
• Abstract summary: PersONAL is a benchmark to study personalization in Embodied AI.<n>It comprises over 2,000 high-quality episodes across 30+ photorealistic homes from the HM3D dataset.<n>The benchmark supports two evaluation modes: (1) active navigation in unseen environments, and (2) object grounding in previously mapped scenes.
• Score: 47.44972258523047
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in Embodied AI have enabled agents to perform increasingly complex tasks and adapt to diverse environments. However, deploying such agents in realistic human-centered scenarios, such as domestic households, remains challenging, particularly due to the difficulty of modeling individual human preferences and behaviors. In this work, we introduce PersONAL (PERSonalized Object Navigation And Localization, a comprehensive benchmark designed to study personalization in Embodied AI. Agents must identify, retrieve, and navigate to objects associated with specific users, responding to natural-language queries such as "find Lily's backpack". PersONAL comprises over 2,000 high-quality episodes across 30+ photorealistic homes from the HM3D dataset. Each episode includes a natural-language scene description with explicit associations between objects and their owners, requiring agents to reason over user-specific semantics. The benchmark supports two evaluation modes: (1) active navigation in unseen environments, and (2) object grounding in previously mapped scenes. Experiments with state-of-the-art baselines reveal a substantial gap to human performance, highlighting the need for embodied agents capable of perceiving, reasoning, and memorizing over personalized information; paving the way towards real-world assistive robot.

Related papers

• EmbodMocap: In-the-Wild 4D Human-Scene Reconstruction for Embodied Agents [85.77432303199176]
  We propose EmbodMocap, a portable and affordable data collection pipeline using two moving iPhones.<n>Our key idea is to jointly calibrate dual RGB-D sequences to reconstruct both humans and scenes.<n>Based on the collected data, we empower three embodied AI tasks: monocular human-scene-reconstruction, where we fine-tune feedforward models that output metric-scale, world-space aligned humans and scenes; physics-based character animation, where we prove our data could be used to scale human-object interaction skills and scene-aware motion tracking; and robot motion control, where we train a humanoid robot via
  arXiv  Detail & Related papers  (2026-02-26T16:53:41Z)
• Break Out the Silverware -- Semantic Understanding of Stored Household Items [5.413873477820601]
  Stored Household Item Challenge is a benchmark task for evaluating service robots' cognitive capabilities.<n>We introduce NOAM, a hybrid agent pipeline that combines structured scene understanding with large language model inference.
  arXiv  Detail & Related papers  (2025-12-25T15:21:49Z)
• HOSIG: Full-Body Human-Object-Scene Interaction Generation with Hierarchical Scene Perception [57.37135310143126]
  HO SIG is a novel framework for synthesizing full-body interactions through hierarchical scene perception.<n>Our framework supports unlimited motion length through autoregressive generation and requires minimal manual intervention.<n>This work bridges the critical gap between scene-aware navigation and dexterous object manipulation.
  arXiv  Detail & Related papers  (2025-06-02T12:08:08Z)
• Personalized Instance-based Navigation Toward User-Specific Objects in Realistic Environments [44.6372390798904]
  We propose a new task denominated Personalized Instance-based Navigation (PIN), in which an embodied agent is tasked with locating and reaching a specific personal object.<n>In each episode, the target object is presented to the agent using two modalities: a set of visual reference images on a neutral background and manually annotated textual descriptions.
  arXiv  Detail & Related papers  (2024-10-23T18:01:09Z)
• Promptable Behaviors: Personalizing Multi-Objective Rewards from Human Preferences [53.353022588751585]
  We present Promptable Behaviors, a novel framework that facilitates efficient personalization of robotic agents to diverse human preferences. We introduce three distinct methods to infer human preferences by leveraging different types of interactions. We evaluate the proposed method in personalized object-goal navigation and flee navigation tasks in ProcTHOR and RoboTHOR.
  arXiv  Detail & Related papers  (2023-12-14T21:00:56Z)
• Physical Reasoning and Object Planning for Household Embodied Agents [19.88210708022216]
  We introduce the CommonSense Object Affordance Task (COAT), a novel framework designed to analyze reasoning capabilities in commonsense scenarios. COAT offers insights into the complexities of practical decision-making in real-world environments. Our contributions include insightful human preference mappings for all three factors and four extensive QA datasets.
  arXiv  Detail & Related papers  (2023-11-22T18:32:03Z)
• Find What You Want: Learning Demand-conditioned Object Attribute Space for Demand-driven Navigation [5.106884746419666]
  The task of Visual Object Navigation (VON) involves an agent's ability to locate a particular object within a given scene. In real-world scenarios, it is often challenging to ensure that these conditions are always met. We propose Demand-driven Navigation (DDN), which leverages the user's demand as the task instruction.
  arXiv  Detail & Related papers  (2023-09-15T04:07:57Z)
• CorNav: Autonomous Agent with Self-Corrected Planning for Zero-Shot Vision-and-Language Navigation [73.78984332354636]
  CorNav is a novel zero-shot framework for vision-and-language navigation. It incorporates environmental feedback for refining future plans and adjusting its actions. It consistently outperforms all baselines in a zero-shot multi-task setting.
  arXiv  Detail & Related papers  (2023-06-17T11:44:04Z)
• Embodied Scene-aware Human Pose Estimation [25.094152307452]
  We propose embodied scene-aware human pose estimation. Our method is one stage, causal, and recovers global 3D human poses in a simulated environment.
  arXiv  Detail & Related papers  (2022-06-18T03:50:19Z)
• Object Manipulation via Visual Target Localization [64.05939029132394]
  Training agents to manipulate objects, poses many challenges. We propose an approach that explores the environment in search for target objects, computes their 3D coordinates once they are located, and then continues to estimate their 3D locations even when the objects are not visible. Our evaluations show a massive 3x improvement in success rate over a model that has access to the same sensory suite.
  arXiv  Detail & Related papers  (2022-03-15T17:59:01Z)
• SOON: Scenario Oriented Object Navigation with Graph-based Exploration [102.74649829684617]
  The ability to navigate like a human towards a language-guided target from anywhere in a 3D embodied environment is one of the 'holy grail' goals of intelligent robots. Most visual navigation benchmarks focus on navigating toward a target from a fixed starting point, guided by an elaborate set of instructions that depicts step-by-step. This approach deviates from real-world problems in which human-only describes what the object and its surrounding look like and asks the robot to start navigation from anywhere.
  arXiv  Detail & Related papers  (2021-03-31T15:01:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.