Improve accessibility for Low Vision and Blind people using Machine Learning and Computer Vision

• URL: http://arxiv.org/abs/2404.00043v1
• Date: Sun, 24 Mar 2024 21:19:17 GMT
• Title: Improve accessibility for Low Vision and Blind people using Machine Learning and Computer Vision
• Authors: Jasur Shukurov,
• Abstract summary: This project explores how machine learning and computer vision could be utilized to improve accessibility for people with visual impairments. This project will concentrate on building a mobile application that helps blind people to orient in space by receiving audio and haptic feedback.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the ever-growing expansion of mobile technology worldwide, there is an increasing need for accommodation for those who are disabled. This project explores how machine learning and computer vision could be utilized to improve accessibility for people with visual impairments. There have been many attempts to develop various software that would improve accessibility in the day-to-day lives of blind people. However, applications on the market have low accuracy and only provide audio feedback. This project will concentrate on building a mobile application that helps blind people to orient in space by receiving audio and haptic feedback, e.g. vibrations, about their surroundings in real-time. The mobile application will have 3 main features. The initial feature is scanning text from the camera and reading it to a user. This feature can be used on paper with text, in the environment, and on road signs. The second feature is detecting objects around the user, and providing audio feedback about those objects. It also includes providing the description of the objects and their location, and giving haptic feedback if the user is too close to an object. The last feature is currency detection which provides a total amount of currency value to the user via the camera.

Related papers

• AIris: An AI-powered Wearable Assistive Device for the Visually Impaired [0.0]
  We introduce AIris, an AI-powered wearable device that provides environmental awareness and interaction capabilities to visually impaired users. We have created a functional prototype system that operates effectively in real-world conditions.
  arXiv  Detail & Related papers  (2024-05-13T10:09:37Z)
• Floor extraction and door detection for visually impaired guidance [78.94595951597344]
  Finding obstacle-free paths in unknown environments is a big navigation issue for visually impaired people and autonomous robots. New devices based on computer vision systems can help impaired people to overcome the difficulties of navigating in unknown environments in safe conditions. In this work it is proposed a combination of sensors and algorithms that can lead to the building of a navigation system for visually impaired people.
  arXiv  Detail & Related papers  (2024-01-30T14:38:43Z)
• Newvision: application for helping blind people using deep learning [0.0]
  We are developing proprietary headgear that will help visually impaired people navigate their surroundings. The headgear will use a combination of computer vision, distance estimation with ultrasonic sensors, voice recognition, and voice assistants. Users will be able to interact with the headgear through voice commands, such as ''What is that?'' to identify an object.
  arXiv  Detail & Related papers  (2023-11-05T06:23:10Z)
• MagicEye: An Intelligent Wearable Towards Independent Living of Visually Impaired [0.17499351967216337]
  Vision impairment can severely impair a person's ability to work, navigate, and retain independence. We present MagicEye, a state-of-the-art intelligent wearable device designed to assist visually impaired individuals. With a total of 35 classes, the neural network employed by MagicEye has been specifically designed to achieve high levels of efficiency and precision in object detection.
  arXiv  Detail & Related papers  (2023-03-24T08:59:35Z)
• Play it by Ear: Learning Skills amidst Occlusion through Audio-Visual Imitation Learning [62.83590925557013]
  We learn a set of challenging partially-observed manipulation tasks from visual and audio inputs. Our proposed system learns these tasks by combining offline imitation learning from tele-operated demonstrations and online finetuning. In a set of simulated tasks, we find that our system benefits from using audio, and that by using online interventions we are able to improve the success rate of offline imitation learning by 20%.
  arXiv  Detail & Related papers  (2022-05-30T04:52:58Z)
• ASHA: Assistive Teleoperation via Human-in-the-Loop Reinforcement Learning [91.58711082348293]
  Reinforcement learning from online user feedback on the system's performance presents a natural solution to this problem. This approach tends to require a large amount of human-in-the-loop training data, especially when feedback is sparse. We propose a hierarchical solution that learns efficiently from sparse user feedback.
  arXiv  Detail & Related papers  (2022-02-05T02:01:19Z)
• Can machines learn to see without visual databases? [93.73109506642112]
  This paper focuses on developing machines that learn to see without needing to handle visual databases. This might open the doors to a truly competitive track concerning deep learning technologies for vision.
  arXiv  Detail & Related papers  (2021-10-12T13:03:54Z)
• Learning Perceptual Locomotion on Uneven Terrains using Sparse Visual Observations [75.60524561611008]
  This work aims to exploit the use of sparse visual observations to achieve perceptual locomotion over a range of commonly seen bumps, ramps, and stairs in human-centred environments. We first formulate the selection of minimal visual input that can represent the uneven surfaces of interest, and propose a learning framework that integrates such exteroceptive and proprioceptive data. We validate the learned policy in tasks that require omnidirectional walking over flat ground and forward locomotion over terrains with obstacles, showing a high success rate.
  arXiv  Detail & Related papers  (2021-09-28T20:25:10Z)
• VisBuddy -- A Smart Wearable Assistant for the Visually Challenged [0.0]
  VisBuddy is a voice-based assistant, where the user can give voice commands to perform specific tasks. It uses the techniques of image captioning for describing the user's surroundings, optical character recognition (OCR) for reading the text in the user's view, object detection to search and find the objects in a room and web scraping to give the user the latest news.
  arXiv  Detail & Related papers  (2021-08-17T17:15:23Z)
• Gaze-contingent decoding of human navigation intention on an autonomous wheelchair platform [6.646253877148766]
  We have pioneered the Where-You-Look-Is Where-You-Go approach to controlling mobility platforms. We present a new solution, consisting of 1. deep computer vision to understand what object a user is looking at in their field of view. Our decoding system ultimately determines whether the user wants to drive to e.g., a door or just looks at it.
  arXiv  Detail & Related papers  (2021-03-04T14:52:06Z)
• Assisted Perception: Optimizing Observations to Communicate State [112.40598205054994]
  We aim to help users estimate the state of the world in tasks like robotic teleoperation and navigation with visual impairments. We synthesize new observations that lead to more accurate internal state estimates when processed by the user.
  arXiv  Detail & Related papers  (2020-08-06T19:08:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.