VisBuddy -- A Smart Wearable Assistant for the Visually Challenged
- URL: http://arxiv.org/abs/2108.07761v1
- Date: Tue, 17 Aug 2021 17:15:23 GMT
- Title: VisBuddy -- A Smart Wearable Assistant for the Visually Challenged
- Authors: Ishwarya Sivakumar, Nishaali Meenakshisundaram, Ishwarya Ramesh,
Shiloah Elizabeth D, Sunil Retmin Raj C
- Abstract summary: VisBuddy is a voice-based assistant, where the user can give voice commands to perform specific tasks.
It uses the techniques of image captioning for describing the user's surroundings, optical character recognition (OCR) for reading the text in the user's view, object detection to search and find the objects in a room and web scraping to give the user the latest news.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Vision plays a crucial role to comprehend the world around us as more than
85% of the external information is obtained through the vision system. It
largely influences our mobility, cognition, information access, and interaction
with the environment as well as with other people. Blindness prevents a person
from gaining knowledge of the surrounding environment and makes unassisted
navigation, object recognition, obstacle avoidance, and reading tasks major
challenges. Many existing systems are often limited by cost and complexity. To
help the visually challenged overcome these difficulties faced in everyday
life, we propose the idea of VisBuddy, a smart assistant which will help the
visually challenged with their day-to-day activities. VisBuddy is a voice-based
assistant, where the user can give voice commands to perform specific tasks.
VisBuddy uses the techniques of image captioning for describing the user's
surroundings, optical character recognition (OCR) for reading the text in the
user's view, object detection to search and find the objects in a room and web
scraping to give the user the latest news. VisBuddy has been built by combining
the concepts from Deep Learning and the Internet of Things. Thus, VisBuddy
serves as a cost-efficient, powerful and all-in-one assistant for the visually
challenged by helping them with their day-to-day activities.
Related papers
- TEXT2TASTE: A Versatile Egocentric Vision System for Intelligent Reading Assistance Using Large Language Model [2.2469442203227863]
We propose an intelligent reading assistant based on smart glasses with embedded RGB cameras and a Large Language Model (LLM)
The video recorded from the egocentric perspective of a person wearing the glasses is processed to localise text information using object detection and optical character recognition methods.
The LLM processes the data and allows the user to interact with the text and responds to a given query, thus extending the functionality of corrective lenses.
arXiv Detail & Related papers (2024-04-14T13:39:02Z) - Improve accessibility for Low Vision and Blind people using Machine Learning and Computer Vision [0.0]
This project explores how machine learning and computer vision could be utilized to improve accessibility for people with visual impairments.
This project will concentrate on building a mobile application that helps blind people to orient in space by receiving audio and haptic feedback.
arXiv Detail & Related papers (2024-03-24T21:19:17Z) - Floor extraction and door detection for visually impaired guidance [78.94595951597344]
Finding obstacle-free paths in unknown environments is a big navigation issue for visually impaired people and autonomous robots.
New devices based on computer vision systems can help impaired people to overcome the difficulties of navigating in unknown environments in safe conditions.
In this work it is proposed a combination of sensors and algorithms that can lead to the building of a navigation system for visually impaired people.
arXiv Detail & Related papers (2024-01-30T14:38:43Z) - Newvision: application for helping blind people using deep learning [0.0]
We are developing proprietary headgear that will help visually impaired people navigate their surroundings.
The headgear will use a combination of computer vision, distance estimation with ultrasonic sensors, voice recognition, and voice assistants.
Users will be able to interact with the headgear through voice commands, such as ''What is that?'' to identify an object.
arXiv Detail & Related papers (2023-11-05T06:23:10Z) - MagicEye: An Intelligent Wearable Towards Independent Living of Visually
Impaired [0.17499351967216337]
Vision impairment can severely impair a person's ability to work, navigate, and retain independence.
We present MagicEye, a state-of-the-art intelligent wearable device designed to assist visually impaired individuals.
With a total of 35 classes, the neural network employed by MagicEye has been specifically designed to achieve high levels of efficiency and precision in object detection.
arXiv Detail & Related papers (2023-03-24T08:59:35Z) - Play it by Ear: Learning Skills amidst Occlusion through Audio-Visual
Imitation Learning [62.83590925557013]
We learn a set of challenging partially-observed manipulation tasks from visual and audio inputs.
Our proposed system learns these tasks by combining offline imitation learning from tele-operated demonstrations and online finetuning.
In a set of simulated tasks, we find that our system benefits from using audio, and that by using online interventions we are able to improve the success rate of offline imitation learning by 20%.
arXiv Detail & Related papers (2022-05-30T04:52:58Z) - ASHA: Assistive Teleoperation via Human-in-the-Loop Reinforcement
Learning [91.58711082348293]
Reinforcement learning from online user feedback on the system's performance presents a natural solution to this problem.
This approach tends to require a large amount of human-in-the-loop training data, especially when feedback is sparse.
We propose a hierarchical solution that learns efficiently from sparse user feedback.
arXiv Detail & Related papers (2022-02-05T02:01:19Z) - Augmented reality navigation system for visual prosthesis [67.09251544230744]
We propose an augmented reality navigation system for visual prosthesis that incorporates a software of reactive navigation and path planning.
It consists on four steps: locating the subject on a map, planning the subject trajectory, showing it to the subject and re-planning without obstacles.
Results show how our augmented navigation system help navigation performance by reducing the time and distance to reach the goals, even significantly reducing the number of obstacles collisions.
arXiv Detail & Related papers (2021-09-30T09:41:40Z) - Deep Learning for Embodied Vision Navigation: A Survey [108.13766213265069]
"Embodied visual navigation" problem requires an agent to navigate in a 3D environment mainly rely on its first-person observation.
This paper attempts to establish an outline of the current works in the field of embodied visual navigation by providing a comprehensive literature survey.
arXiv Detail & Related papers (2021-07-07T12:09:04Z) - I-ViSE: Interactive Video Surveillance as an Edge Service using
Unsupervised Feature Queries [70.69741666849046]
This paper proposes an Interactive Video Surveillance as an Edge service (I-ViSE) based on unsupervised feature queries.
An I-ViSE prototype is built following the edge-fog computing paradigm and the experimental results verified the I-ViSE scheme meets the design goal of scene recognition in less than two seconds.
arXiv Detail & Related papers (2020-03-09T14:26:45Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.