Gaze-contingent decoding of human navigation intention on an autonomous wheelchair platform

• URL: http://arxiv.org/abs/2103.03072v1
• Date: Thu, 4 Mar 2021 14:52:06 GMT
• Title: Gaze-contingent decoding of human navigation intention on an autonomous wheelchair platform
• Authors: Mahendran Subramanian, Suhyung Park, Pavel Orlov, Ali Shafti, A. Aldo Faisal
• Abstract summary: We have pioneered the Where-You-Look-Is Where-You-Go approach to controlling mobility platforms. We present a new solution, consisting of 1. deep computer vision to understand what object a user is looking at in their field of view. Our decoding system ultimately determines whether the user wants to drive to e.g., a door or just looks at it.
• Score: 6.646253877148766
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We have pioneered the Where-You-Look-Is Where-You-Go approach to controlling mobility platforms by decoding how the user looks at the environment to understand where they want to navigate their mobility device. However, many natural eye-movements are not relevant for action intention decoding, only some are, which places a challenge on decoding, the so-called Midas Touch Problem. Here, we present a new solution, consisting of 1. deep computer vision to understand what object a user is looking at in their field of view, with 2. an analysis of where on the object's bounding box the user is looking, to 3. use a simple machine learning classifier to determine whether the overt visual attention on the object is predictive of a navigation intention to that object. Our decoding system ultimately determines whether the user wants to drive to e.g., a door or just looks at it. Crucially, we find that when users look at an object and imagine they were moving towards it, the resulting eye-movements from this motor imagery (akin to neural interfaces) remain decodable. Once a driving intention and thus also the location is detected our system instructs our autonomous wheelchair platform, the A.Eye-Drive, to navigate to the desired object while avoiding static and moving obstacles. Thus, for navigation purposes, we have realised a cognitive-level human interface, as it requires the user only to cognitively interact with the desired goal, not to continuously steer their wheelchair to the target (low-level human interfacing).

Related papers

• CoNav: A Benchmark for Human-Centered Collaborative Navigation [66.6268966718022]
  We propose a collaborative navigation (CoNav) benchmark. Our CoNav tackles the critical challenge of constructing a 3D navigation environment with realistic and diverse human activities. We propose an intention-aware agent for reasoning both long-term and short-term human intention.
  arXiv  Detail & Related papers  (2024-06-04T15:44:25Z)
• Latent Object Characteristics Recognition with Visual to Haptic-Audio Cross-modal Transfer Learning [9.178588671620963]
  This work aims to recognise the latent unobservable object characteristics. vision is commonly used for object recognition by robots, but it is ineffective for detecting hidden objects. We propose a cross-modal transfer learning approach from vision to haptic-audio.
  arXiv  Detail & Related papers  (2024-03-15T21:18:14Z)
• Floor extraction and door detection for visually impaired guidance [78.94595951597344]
  Finding obstacle-free paths in unknown environments is a big navigation issue for visually impaired people and autonomous robots. New devices based on computer vision systems can help impaired people to overcome the difficulties of navigating in unknown environments in safe conditions. In this work it is proposed a combination of sensors and algorithms that can lead to the building of a navigation system for visually impaired people.
  arXiv  Detail & Related papers  (2024-01-30T14:38:43Z)
• Object Recognition System on a Tactile Device for Visually Impaired [1.2891210250935146]
  The device will convert visual information into auditory feedback, enabling users to understand their environment in a way that suits their sensory needs. When the device is touched at a specific position, it provides an audio signal that communicates the identification of the object present in the scene at that corresponding position to the visually impaired individual.
  arXiv  Detail & Related papers  (2023-07-05T11:37:17Z)
• DetGPT: Detect What You Need via Reasoning [33.00345609506097]
  We introduce a new paradigm for object detection that we call reasoning-based object detection. Unlike conventional object detection methods that rely on specific object names, our approach enables users to interact with the system using natural language instructions. Our proposed method, called DetGPT, leverages state-of-the-art multi-modal models and open-vocabulary object detectors.
  arXiv  Detail & Related papers  (2023-05-23T15:37:28Z)
• Gesture2Path: Imitation Learning for Gesture-aware Navigation [54.570943577423094]
  We present Gesture2Path, a novel social navigation approach that combines image-based imitation learning with model-predictive control. We deploy our method on real robots and showcase the effectiveness of our approach for the four gestures-navigation scenarios.
  arXiv  Detail & Related papers  (2022-09-19T23:05:36Z)
• Detect and Approach: Close-Range Navigation Support for People with Blindness and Low Vision [13.478275180547925]
  People with blindness and low vision (pBLV) experience significant challenges when locating final destinations or targeting specific objects in unfamiliar environments. We develop a novel wearable navigation solution to provide real-time guidance for a user to approach a target object of interest efficiently and effectively in unfamiliar environments.
  arXiv  Detail & Related papers  (2022-08-17T18:38:20Z)
• Augmented reality navigation system for visual prosthesis [67.09251544230744]
  We propose an augmented reality navigation system for visual prosthesis that incorporates a software of reactive navigation and path planning. It consists on four steps: locating the subject on a map, planning the subject trajectory, showing it to the subject and re-planning without obstacles. Results show how our augmented navigation system help navigation performance by reducing the time and distance to reach the goals, even significantly reducing the number of obstacles collisions.
  arXiv  Detail & Related papers  (2021-09-30T09:41:40Z)
• Pushing it out of the Way: Interactive Visual Navigation [62.296686176988125]
  We study the problem of interactive navigation where agents learn to change the environment to navigate more efficiently to their goals. We introduce the Neural Interaction Engine (NIE) to explicitly predict the change in the environment caused by the agent's actions. By modeling the changes while planning, we find that agents exhibit significant improvements in their navigational capabilities.
  arXiv  Detail & Related papers  (2021-04-28T22:46:41Z)
• Careful with That! Observation of Human Movements to Estimate Objects Properties [106.925705883949]
  We focus on the features of human motor actions that communicate insights on the weight of an object. Our final goal is to enable a robot to autonomously infer the degree of care required in object handling.
  arXiv  Detail & Related papers  (2021-03-02T08:14:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.