EclipseTouch: Touch Segmentation on Ad Hoc Surfaces using Worn Infrared Shadow Casting

• URL: http://arxiv.org/abs/2509.03430v1
• Date: Wed, 03 Sep 2025 15:59:28 GMT
• Title: EclipseTouch: Touch Segmentation on Ad Hoc Surfaces using Worn Infrared Shadow Casting
• Authors: Vimal Mollyn, Nathan DeVrio, Chris Harrison,
• Abstract summary: We propose a new headset-integrated technique called systemname to detect touch events on uninstrumented surfaces.<n>We use a combination of a computer-triggered camera and one or more infrared emitters to create structured shadows, from which we can accurately estimate hover distance.<n>We discuss how our technique works across a range of conditions, including surface material, interaction orientation, and environmental lighting.
• Score: 14.237453119638516
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The ability to detect touch events on uninstrumented, everyday surfaces has been a long-standing goal for mixed reality systems. Prior work has shown that virtual interfaces bound to physical surfaces offer performance and ergonomic benefits over tapping at interfaces floating in the air. A wide variety of approaches have been previously developed, to which we contribute a new headset-integrated technique called \systemname. We use a combination of a computer-triggered camera and one or more infrared emitters to create structured shadows, from which we can accurately estimate hover distance (mean error of 6.9~mm) and touch contact (98.0\% accuracy). We discuss how our technique works across a range of conditions, including surface material, interaction orientation, and environmental lighting.

Related papers

• OPENTOUCH: Bringing Full-Hand Touch to Real-World Interaction [93.88239833545623]
  We present OpenTouch, the first in-the-wild egocentric full-hand tactile dataset.<n>We show that tactile signals provide a compact yet powerful cue for grasp understanding.<n>We aim to advance multimodal egocentric perception, embodied learning, and contact-rich robotic manipulation.
  arXiv  Detail & Related papers  (2025-12-18T18:18:17Z)
• EgoTouch: On-Body Touch Input Using AR/VR Headset Cameras [13.852935460131896]
  We demonstrate high accuracy, bare hands (i.e., no special instrumentation of the user) skin input using just an RGB camera.<n>Our results show this approach can be accurate, and robust across diverse lighting conditions.<n>We believe these are the requisite technical ingredients to unlock more fully on-skin interfaces.
  arXiv  Detail & Related papers  (2025-09-01T21:32:30Z)
• SpikeGen: Decoupled "Rods and Cones" Visual Representation Processing with Latent Generative Framework [53.27177454390712]
  This study seeks to emulate the human visual system by integrating multi-modal visual inputs with modern latent-space generative frameworks.<n>We name it SpikeGen. We evaluate its performance across various spike-RGB tasks, including conditional image and video deblurring, dense frame reconstruction from spike streams, and high-speed scene novel-view synthesis.
  arXiv  Detail & Related papers  (2025-05-23T15:54:11Z)
• Capturing complex hand movements and object interactions using machine learning-powered stretchable smart textile gloves [9.838013581109681]
  Real-time tracking of dexterous hand movements has numerous applications in human-computer interaction, metaverse, robotics, and tele-health. Here, we report accurate and dynamic tracking of articulated hand and finger movements using stretchable, washable smart gloves with embedded helical sensor yarns and inertial measurement units. The sensor yarns have a high dynamic range, responding to low 0.005 % to high 155 % strains, and show stability during extensive use and washing cycles.
  arXiv  Detail & Related papers  (2024-10-03T05:32:16Z)
• Snap-it, Tap-it, Splat-it: Tactile-Informed 3D Gaussian Splatting for Reconstructing Challenging Surfaces [34.831730064258494]
  We propose Tactile-Informed 3DGS, a novel approach that incorporates touch data (local depth maps) with multi-view vision data to achieve surface reconstruction and novel view synthesis. By creating a framework that decreases the transmittance at touch locations, we achieve a refined surface reconstruction, ensuring a uniformly smooth depth map. We conduct evaluation on objects with glossy and reflective surfaces and demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-03-29T16:30:17Z)
• PseudoTouch: Efficiently Imaging the Surface Feel of Objects for Robotic Manipulation [8.997347199266592]
  We introduce PseudoTouch which links high-dimensional structural information to low-dimensional sensor signals.<n>It does so by learning a low-dimensional visual-tactile embedding, wherein we encode a depth patch from which we decode the tactile signal.<n>We demonstrate the utility of our trained PseudoTouch model in two downstream tasks: object recognition and grasp stability prediction.
  arXiv  Detail & Related papers  (2024-03-22T10:51:31Z)
• Controllable Visual-Tactile Synthesis [28.03469909285511]
  We develop a conditional generative model that synthesizes both visual and tactile outputs from a single sketch. We then introduce a pipeline to render high-quality visual and tactile outputs on an electroadhesion-based haptic device.
  arXiv  Detail & Related papers  (2023-05-04T17:59:51Z)
• Recognizing Complex Gestures on Minimalistic Knitted Sensors: Toward Real-World Interactive Systems [0.13048920509133805]
  Our digitally-knitted capacitive active sensors can be manufactured at scale with little human intervention. This work advances the capabilities of such sensors by creating the foundation for an interactive gesture recognition system.
  arXiv  Detail & Related papers  (2023-03-18T04:57:46Z)
• Tactile-Filter: Interactive Tactile Perception for Part Mating [54.46221808805662]
  Humans rely on touch and tactile sensing for a lot of dexterous manipulation tasks. vision-based tactile sensors are being widely used for various robotic perception and control tasks. We present a method for interactive perception using vision-based tactile sensors for a part mating task.
  arXiv  Detail & Related papers  (2023-03-10T16:27:37Z)
• The Gesture Authoring Space: Authoring Customised Hand Gestures for Grasping Virtual Objects in Immersive Virtual Environments [81.5101473684021]
  This work proposes a hand gesture authoring tool for object specific grab gestures allowing virtual objects to be grabbed as in the real world. The presented solution uses template matching for gesture recognition and requires no technical knowledge to design and create custom tailored hand gestures. The study showed that gestures created with the proposed approach are perceived by users as a more natural input modality than the others.
  arXiv  Detail & Related papers  (2022-07-03T18:33:33Z)
• Elastic Tactile Simulation Towards Tactile-Visual Perception [58.44106915440858]
  We propose Elastic Interaction of Particles (EIP) for tactile simulation. EIP models the tactile sensor as a group of coordinated particles, and the elastic property is applied to regulate the deformation of particles during contact. We further propose a tactile-visual perception network that enables information fusion between tactile data and visual images.
  arXiv  Detail & Related papers  (2021-08-11T03:49:59Z)
• RGB2Hands: Real-Time Tracking of 3D Hand Interactions from Monocular RGB Video [76.86512780916827]
  We present the first real-time method for motion capture of skeletal pose and 3D surface geometry of hands from a single RGB camera. In order to address the inherent depth ambiguities in RGB data, we propose a novel multi-task CNN. We experimentally verify the individual components of our RGB two-hand tracking and 3D reconstruction pipeline.
  arXiv  Detail & Related papers  (2021-06-22T12:53:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.