Practical and Rich User Digitization

• URL: http://arxiv.org/abs/2403.00153v1
• Date: Thu, 29 Feb 2024 22:09:27 GMT
• Title: Practical and Rich User Digitization
• Authors: Karan Ahuja
• Abstract summary: User digitization allows computers to intimately understand their users, capturing activity, pose, routine, and behavior. Today's consumer devices offer coarse digital representations of users with metrics such as step count, heart rate, and a handful of human activities like running and biking. My research aims to break this trend, developing sensing systems that increase user digitization fidelity to create new and powerful computing experiences.
• Score: 7.021516368759671
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A long-standing vision in computer science has been to evolve computing devices into proactive assistants that enhance our productivity, health and wellness, and many other facets of our lives. User digitization is crucial in achieving this vision as it allows computers to intimately understand their users, capturing activity, pose, routine, and behavior. Today's consumer devices - like smartphones and smartwatches provide a glimpse of this potential, offering coarse digital representations of users with metrics such as step count, heart rate, and a handful of human activities like running and biking. Even these very low-dimensional representations are already bringing value to millions of people's lives, but there is significant potential for improvement. On the other end, professional, high-fidelity comprehensive user digitization systems exist. For example, motion capture suits and multi-camera rigs that digitize our full body and appearance, and scanning machines such as MRI capture our detailed anatomy. However, these carry significant user practicality burdens, such as financial, privacy, ergonomic, aesthetic, and instrumentation considerations, that preclude consumer use. In general, the higher the fidelity of capture, the lower the user's practicality. Most conventional approaches strike a balance between user practicality and digitization fidelity. My research aims to break this trend, developing sensing systems that increase user digitization fidelity to create new and powerful computing experiences while retaining or even improving user practicality and accessibility, allowing such technologies to have a societal impact. Armed with such knowledge, our future devices could offer longitudinal health tracking, more productive work environments, full body avatars in extended reality, and embodied telepresence experiences, to name just a few domains.

Related papers

• Digitizing Touch with an Artificial Multimodal Fingertip [51.7029315337739]
  Humans and robots both benefit from using touch to perceive and interact with the surrounding environment. Here, we describe several conceptual and technological innovations to improve the digitization of touch. These advances are embodied in an artificial finger-shaped sensor with advanced sensing capabilities.
  arXiv  Detail & Related papers  (2024-11-04T18:38:50Z)
• Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation [57.60490773016364]
  We combine vision and touch sensing on a multi-fingered hand to estimate an object's pose and shape during in-hand manipulation. Our method, NeuralFeels, encodes object geometry by learning a neural field online and jointly tracks it by optimizing a pose graph problem. Our results demonstrate that touch, at the very least, refines and, at the very best, disambiguates visual estimates during in-hand manipulation.
  arXiv  Detail & Related papers  (2023-12-20T22:36:37Z)
• See, Hear, and Feel: Smart Sensory Fusion for Robotic Manipulation [49.925499720323806]
  We study how visual, auditory, and tactile perception can jointly help robots to solve complex manipulation tasks. We build a robot system that can see with a camera, hear with a contact microphone, and feel with a vision-based tactile sensor.
  arXiv  Detail & Related papers  (2022-12-07T18:55:53Z)
• Cross-Reality Re-Rendering: Manipulating between Digital and Physical Realities [2.538209532048867]
  We investigate the design of a system that enables users to manipulate the perception of both their physical realities and digital realities. Users can inspect their view history from either reality, and generate interventions that can be interoperably rendered cross-reality in real-time.
  arXiv  Detail & Related papers  (2022-11-15T09:31:52Z)
• Force-Aware Interface via Electromyography for Natural VR/AR Interaction [69.1332992637271]
  We design a learning-based neural interface for natural and intuitive force inputs in VR/AR. We show that our interface can decode finger-wise forces in real-time with 3.3% mean error, and generalize to new users with little calibration. We envision our findings to push forward research towards more realistic physicality in future VR/AR.
  arXiv  Detail & Related papers  (2022-10-03T20:51:25Z)
• Towards Everyday Virtual Reality through Eye Tracking [1.2691047660244335]
  Eye tracking is an emerging technology that helps to assess human behaviors in a real time and non-intrusive way. A significant scientific push towards everyday virtual reality has been completed with three main research contributions.
  arXiv  Detail & Related papers  (2022-03-29T16:09:37Z)
• Human Activity Recognition models using Limited Consumer Device Sensors and Machine Learning [0.0]
  Human activity recognition has grown in popularity with its increase of applications within daily lifestyles and medical environments. This paper presents the findings of different models that are limited to train using sensor data from smartphones and smartwatches. Results show promise for models trained strictly using limited sensor data collected from only smartphones and smartwatches coupled with traditional machine learning concepts and algorithms.
  arXiv  Detail & Related papers  (2022-01-21T06:54:05Z)
• Classifying Human Activities with Inertial Sensors: A Machine Learning Approach [0.0]
  Human Activity Recognition (HAR) is an ongoing research topic. It has applications in medical support, sports, fitness, social networking, human-computer interfaces, senior care, entertainment, surveillance, and the list goes on. We examined and analyzed different Machine Learning and Deep Learning approaches for Human Activity Recognition using inertial sensor data of smartphones.
  arXiv  Detail & Related papers  (2021-11-09T08:17:33Z)
• Facial Masks and Soft-Biometrics: Leveraging Face Recognition CNNs for Age and Gender Prediction on Mobile Ocular Images [53.913598771836924]
  We address the use of selfie ocular images captured with smartphones to estimate age and gender. We adapt two existing lightweight CNNs proposed in the context of the ImageNet Challenge. Some networks are further pre-trained for face recognition, for which very large training databases are available.
  arXiv  Detail & Related papers  (2021-03-31T01:48:29Z)
• Hand tracking for immersive virtual reality: opportunities and challenges [0.0]
  Hand tracking has become an integral feature of recent generations of immersive virtual reality head-mounted displays. I outline what I see as the main possibilities for hand tracking to add value to immersive virtual reality. It is hoped that this paper serves as a roadmap for the development of best practices in the field for the development of subsequent generations of hand tracking and virtual reality technologies.
  arXiv  Detail & Related papers  (2021-03-27T09:28:47Z)
• AEGIS: A real-time multimodal augmented reality computer vision based system to assist facial expression recognition for individuals with autism spectrum disorder [93.0013343535411]
  This paper presents the development of a multimodal augmented reality (AR) system which combines the use of computer vision and deep convolutional neural networks (CNN) The proposed system, which we call AEGIS, is an assistive technology deployable on a variety of user devices including tablets, smartphones, video conference systems, or smartglasses. We leverage both spatial and temporal information in order to provide an accurate expression prediction, which is then converted into its corresponding visualization and drawn on top of the original video frame.
  arXiv  Detail & Related papers  (2020-10-22T17:20:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.