A Glimpse into Long-term Physical Coexistence with Intelligent Robots

• URL: http://arxiv.org/abs/2607.11377v1
• Date: Mon, 13 Jul 2026 10:40:32 GMT
• Title: A Glimpse into Long-term Physical Coexistence with Intelligent Robots
• Abstract summary: PHILIA is a multi-robot agent built around a robot gateway abstraction.<n>It decouples low-frequency, high-semantic agent reasoning from high-frequency, low-level robot execution.<n>We present representative use cases in which agent memory and scene understanding are grounded in robot actions.
• Score: 12.275897522668858
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Long-term physical coexistence with intelligent robots requires more than capable robot policies. A persistent robotic assistant must support diverse user-facing interfaces, maintain long-horizon memory of people and preferences, coordinate across robot embodiments, and translate human intent into safe physical execution. We introduce PHILIA, a multi-robot agent built around a robot gateway abstraction. PHILIA retains the rich interaction and tool ecosystem of OpenClaw while exposing robot-local runtimes, onboard perception, navigation, speaker, and robot policies through a unified capability interface. This design decouples low-frequency, high-semantic agent reasoning from high-frequency, low-level robot execution, enabling plug-and-play integration of user interfaces, robot embodiments, and policy backends. As a result, the user experience becomes compositional: advances in user interfaces, robot embodiments, robot policies, navigation, or interaction algorithms can improve the overall experience without redesigning the system. We validate the architecture on Astribot S1 robots while designing the robot gateway contract to support future heterogeneous robot platforms through a shared capability interface for observation, task execution, navigation, speech playback, status monitoring, and task cancellation. We present representative use cases in which agent memory and scene understanding are grounded in robot actions. These span interactive household scenarios, ranging from simple organization to challenging long-horizon and dexterous service tasks, such as packing a backpack and lifting a garbage bag. We highlight the human-robot interaction flow, where contextual understanding of user intent and preferences, together with human-in-the-loop confirmation or adjustment during execution, is essential for effective assistance.

Related papers

• Ludi${}_{\scriptscriptstyle 0.1}$: An Agentic System for Socially Intelligent Robots [22.26258894927344]
  Ludi$_scriptscriptstyle 0.1$ is an agentic system for socially intelligent robots.<n>It integrates interactive speech, multimodal reasoning, memory, navigation, and learned manipulation.<n>Ludi$_scriptstyle 0.1$ demonstrates a practical path toward fluid human-robot collaboration.
  arXiv  Detail & Related papers  (2026-08-22T16:38:59Z)
• Initiation of Interaction Detection Framework using a Nonverbal Cue for Human-Robot Interaction [15.228284553815593]
  This paper describes an initiation of interaction(IoI) detection framework based on audio and vision sensor fusion.<n>In the proposed framework, the robot has its own audio and vision sensors, and can employ external vision sensor for stable human detection and tracking.
  arXiv  Detail & Related papers  (2026-05-11T07:07:01Z)
• PuppetAI: A Customizable Platform for Designing Tactile-Rich Affective Robot Interaction [4.426747427265242]
  We introduce PuppetAI, a modular soft robot interaction platform.<n>It offers a scalable cable-driven actuation system and a customizable, puppet-inspired robot gesture framework.<n>For our own designs, we have worked with nuanced gestures enacted by "soft robots" with enhanced dexterity and "pleasant-to-touch" plush exteriors.
  arXiv  Detail & Related papers  (2026-02-04T17:37:31Z)
• RobotSeg: A Model and Dataset for Segmenting Robots in Image and Video [56.9581053843815]
  We introduce RobotSeg, a foundation model for robot segmentation in image and video.<n>It addresses the lack of adaptation to articulated robots, reliance on manual prompts, and the need for per-frame training mask annotations.<n>It achieves state-of-the-art performance on both images and videos.
  arXiv  Detail & Related papers  (2025-11-28T07:51:02Z)
• Interpretable Robot Control via Structured Behavior Trees and Large Language Models [0.14990005092937678]
  This paper presents a novel framework that bridges natural language understanding and robotic execution.<n>The proposed approach is practical in real-world scenarios, with an average cognition-to-execution accuracy of approximately 94%.
  arXiv  Detail & Related papers  (2025-08-13T08:53:13Z)
• A roadmap for AI in robotics [55.87087746398059]
  We are witnessing growing excitement in robotics at the prospect of leveraging the potential of AI to tackle some of the outstanding barriers to the full deployment of robots in our daily lives.<n>This article offers an assessment of what AI for robotics has achieved since the 1990s and proposes a short- and medium-term research roadmap listing challenges and promises.
  arXiv  Detail & Related papers  (2025-07-26T15:18:28Z)
• Project Report: Requirements for a Social Robot as an Information Provider in the Public Sector [0.0]
  We have devised an application scenario for integrating a humanoid social robot into an official environment.<n>We developed a corresponding robot application and carried out initial tests and evaluations in a project together with the Kiel City Council.<n>One of the most important insights gained in the project was that a humanoid robot with natural language processing capabilities proved to be much more preferred by users.<n>We propose a connection of the ACT-R cognitive architecture with the robot, where an ACT-R model is used in interaction with the robot application to cognitively process and enhance a dialogue between human and robot.
  arXiv  Detail & Related papers  (2024-12-06T13:07:06Z)
• Controlling diverse robots by inferring Jacobian fields with deep networks [48.279199537720714]
  Mirroring the complex structures and diverse functions of natural organisms is a long-standing challenge in robotics.<n>We introduce a method that uses deep neural networks to map a video stream of a robot to its visuomotor Jacobian field.<n>Our approach achieves accurate closed-loop control and recovers the causal dynamic structure of each robot.
  arXiv  Detail & Related papers  (2024-07-11T17:55:49Z)
• Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Models [81.55156507635286]
  Legged robots are physically capable of navigating a diverse variety of environments and overcoming a wide range of obstructions. Current learning methods often struggle with generalization to the long tail of unexpected situations without heavy human supervision. We propose a system, VLM-Predictive Control (VLM-PC), combining two key components that we find to be crucial for eliciting on-the-fly, adaptive behavior selection.
  arXiv  Detail & Related papers  (2024-07-02T21:00:30Z)
• RoboScript: Code Generation for Free-Form Manipulation Tasks across Real and Simulation [77.41969287400977]
  This paper presents textbfRobotScript, a platform for a deployable robot manipulation pipeline powered by code generation. We also present a benchmark for a code generation benchmark for robot manipulation tasks in free-form natural language. We demonstrate the adaptability of our code generation framework across multiple robot embodiments, including the Franka and UR5 robot arms.
  arXiv  Detail & Related papers  (2024-02-22T15:12:00Z)
• A ROS Architecture for Personalised HRI with a Bartender Social Robot [61.843727637976045]
  BRILLO project has the overall goal of creating an autonomous robotic bartender that can interact with customers while accomplishing its bartending tasks. We present the developed three-layers ROS architecture integrating a perception layer managing the processing of different social signals, a decision-making layer for handling multi-party interactions, and an execution layer controlling the behaviour of a complex robot composed of arms and a face.
  arXiv  Detail & Related papers  (2022-03-13T11:33:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.