TactileNet: Bridging the Accessibility Gap with AI-Generated Tactile Graphics for Individuals with Vision Impairment

• URL: http://arxiv.org/abs/2504.04722v2
• Date: Thu, 15 May 2025 15:09:00 GMT
• Title: TactileNet: Bridging the Accessibility Gap with AI-Generated Tactile Graphics for Individuals with Vision Impairment
• Abstract summary: Tactile graphics are essential for providing access to visual information for 43 million people globally living with vision loss.<n>Traditional methods for creating these graphics are labor-intensive and cannot meet growing demand.<n>We introduce TactileNet, the first comprehensive dataset and AI-driven framework for generating embossing-ready 2D tactile templates.
• Score: 5.627346969563955
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Tactile graphics are essential for providing access to visual information for the 43 million people globally living with vision loss. Traditional methods for creating these graphics are labor-intensive and cannot meet growing demand. We introduce TactileNet, the first comprehensive dataset and AI-driven framework for generating embossing-ready 2D tactile templates using text-to-image Stable Diffusion (SD) models. By integrating Low-Rank Adaptation (LoRA) and DreamBooth, our method fine-tunes SD models to produce high-fidelity, guideline-compliant graphics while reducing computational costs. Quantitative evaluations with tactile experts show 92.86% adherence to accessibility standards. Structural fidelity analysis revealed near-human design similarity, with an SSIM of 0.538 between generated graphics and expert-designed tactile images. Notably, our method preserves object silhouettes better than human designs (SSIM = 0.259 vs. 0.215 for binary masks), addressing a key limitation of manual tactile abstraction. The framework scales to 32,000 images (7,050 high-quality) across 66 classes, with prompt editing enabling customizable outputs (e.g., adding or removing details). By automating the 2D template generation step-compatible with standard embossing workflows-TactileNet accelerates production while preserving design flexibility. This work demonstrates how AI can augment (not replace) human expertise to bridge the accessibility gap in education and beyond. Code, data, and models will be publicly released to foster further research.

Related papers

• Learning to Detect UI Principle Violations via Reinforcement Learning [8.915333011185671]
  Small language models and coding agents increasingly generate web front-end code, yet their outputs are typically evaluated for functional correctness.<n>A generated interface may compile, render, and pass unit tests while still violating established interface quality principles.<n>We investigate whether a lightweight vision-language model can serve as an effective critic for generated interfaces.
  arXiv  Detail & Related papers  (2026-07-22T19:42:59Z)
• HyMobileAgent: Data-Environment Co-Scaling for Efficient GUI Agents [57.53434680014667]
  HyMobileAgent is a vision-native foundation model built on Hy3.0-VL-A3B.<n>We develop a joint data and environment centric scaling framework to address the key bottlenecks of mobile interaction.
  arXiv  Detail & Related papers  (2026-07-16T04:12:42Z)
• Text-based Tactile Graphics Generation for the Visually Impaired [51.53004839882403]
  Tactile graphics are a primary medium for blind and low-vision (BLV) individuals to access non-textual information.<n>We present the first integrated generative system that produces fabrication-ready 2.5D tactile graphics directly from natural language prompts.
  arXiv  Detail & Related papers  (2026-07-10T03:04:30Z)
• HT-Bench: Benchmarking and Learning Dexterous Full-Hand Tactile Representations with Egocentric Vision [7.104046531938793]
  HT-Bench is a large-scale benchmark for dexterous full-hand tactile sensing.<n>HandTouch is a vector-quantized vision--tactile encoder that learns tactile representations through progressive spatial, cross-modal, and temporal training.
  arXiv  Detail & Related papers  (2026-06-17T15:01:30Z)
• T-Rex: Tactile-Reactive Dexterous Manipulation [139.71263755530654]
  We introduce a variable-rate Mixture-of-Transformers (MoT) architecture equipped with a novel temporal tactile VQ-VAE encoder.<n>We demonstrate the effectiveness of tactile-reactive policies on 12 manipulation tasks requiring delicate force control and deformable object manipulation.
  arXiv  Detail & Related papers  (2026-06-15T17:59:55Z)
• HY-Embodied-0.5: Embodied Foundation Models for Real-World Agents [71.09493646932046]
  We introduce HY-Embodied-0.5, a family of foundation models specifically designed for real-world embodied agents.<n>The HY-Embodied-0.5 suite comprises two primary variants: an efficient model with 2B activated parameters designed for edge deployment, and a powerful model with 32B activated parameters targeted for complex reasoning.<n>Our MoT-2B model outperforms similarly sized state-of-the-art models on 16 benchmarks, while the 32B variant achieves performance comparable to frontier models such as Gemini 3.0 Pro.
  arXiv  Detail & Related papers  (2026-04-08T17:59:48Z)
• POINTS-GUI-G: GUI-Grounding Journey [22.35782799756431]
  We introduce POINTS-GUIG-8B, which achieves state-of-the-art performance with scores of 59.9 on ScreenSpotPro, 66.0 on OSWorld-G, 95.7 on ScreenSpot-v2, and 49.9 on UIVision.<n>Our model's success is driven by three key factors: (1) Refined Data Engineering; (2) Improved Training Strategies; and (3) Reinforcement Learning with Verifiable Rewards.
  arXiv  Detail & Related papers  (2026-02-06T05:14:11Z)
• ShowUI-$π$: Flow-based Generative Models as GUI Dexterous Hands [59.222064425122795]
  We develop ShowUI-$$, the first flow-based generative model as GUI dexterous hand.<n>ShowUI-$$ achieves 26.98 with only 450M parameters, underscoring both the difficulty of the task and the effectiveness of our approach.
  arXiv  Detail & Related papers  (2025-12-31T16:51:14Z)
• DINOv3 [62.31809406012177]
  Self-supervised learning holds the promise of eliminating the need for manual data annotation, enabling models to scale effortlessly to massive datasets and larger architectures.<n>This technical report introduces DINOv3, a major milestone toward realizing this vision by leveraging simple yet effective strategies.<n>DINOv3 produces high-quality dense features that achieve outstanding performance on various vision tasks.
  arXiv  Detail & Related papers  (2025-08-13T18:00:55Z)
• Towards Generalization of Tactile Image Generation: Reference-Free Evaluation in a Leakage-Free Setting [25.355424080824996]
  Tactile sensing is critical for human perception and underpins applications in computer vision, robotics, and multimodal learning.<n>Because tactile data is often scarce and costly to acquire, generating synthetic tactile images provides a scalable solution to augment real-world measurements.<n>We demonstrate that overlapping training and test samples in commonly used datasets inflate performance metrics, obscuring the true generalizability of tactile models.
  arXiv  Detail & Related papers  (2025-03-10T02:37:22Z)
• FlexVAR: Flexible Visual Autoregressive Modeling without Residual Prediction [91.09318592542509]
  This work challenges the residual prediction paradigm in visual autoregressive modeling.<n>It presents a new Flexible Visual AutoRegressive image generation paradigm.<n>This simple, intuitive approach swiftly learns visual distributions and makes the generation process more flexible and adaptable.
  arXiv  Detail & Related papers  (2025-02-27T17:39:17Z)
• Aguvis: Unified Pure Vision Agents for Autonomous GUI Interaction [69.57190742976091]
  We introduce Aguvis, a unified vision-based framework for autonomous GUI agents.<n>Our approach leverages image-based observations, and grounding instructions in natural language to visual elements.<n>To address the limitations of previous work, we integrate explicit planning and reasoning within the model.
  arXiv  Detail & Related papers  (2024-12-05T18:58:26Z)
• EDGE: Enhanced Grounded GUI Understanding with Enriched Multi-Granularity Synthetic Data [15.801018643716437]
  This paper aims to enhance the GUI understanding and interacting capabilities of large vision-language models (LVLMs) through a data-driven approach. We propose EDGE, a general data synthesis framework that automatically generates large-scale, multi-granularity training data from webpages across the Web. Our approach significantly reduces the dependence on manual annotations, empowering researchers to harness the vast public resources available on the Web to advance their work.
  arXiv  Detail & Related papers  (2024-10-25T10:46:17Z)
• OSPC: Artificial VLM Features for Hateful Meme Detection [0.0]
  This paper introduces a solution developed by team 'Baseline' for the AI Singapore Online Safety Prize Challenge. Focusing on computational efficiency and feature engineering, the solution achieved an AUROC of 0.76 and an accuracy of 0.69 on the test dataset.
  arXiv  Detail & Related papers  (2024-07-03T21:35:52Z)
• DICE: End-to-end Deformation Capture of Hand-Face Interactions from a Single Image [98.29284902879652]
  We present DICE, the first end-to-end method for Deformation-aware hand-face Interaction reCovEry from a single image.<n>It features disentangling the regression of local deformation fields and global mesh locations into two network branches.<n>It achieves state-of-the-art performance on a standard benchmark and in-the-wild data in terms of accuracy and physical plausibility.
  arXiv  Detail & Related papers  (2024-06-26T00:08:29Z)
• Enhancing Large Vision Language Models with Self-Training on Image Comprehension [131.14381425260706]
  We introduce Self-Training on Image (STIC), which emphasizes a self-training approach specifically for image comprehension. First, the model self-constructs a preference for image descriptions using unlabeled images. To further self-improve reasoning on the extracted visual information, we let the model reuse a small portion of existing instruction-tuning data.
  arXiv  Detail & Related papers  (2024-05-30T05:53:49Z)
• Cross-Modal Self-Training: Aligning Images and Pointclouds to Learn Classification without Labels [69.55622471172941]
  Large-scale vision 2D vision language models, such as CLIP can be aligned with a 3D encoder to learn generalizable (open-vocabulary) 3D vision models. We propose an optimization framework: Cross-MoST: Cross-Modal Self-Training, to improve the label-free classification performance of a zero-shot 3D vision model.
  arXiv  Detail & Related papers  (2024-04-15T21:30:50Z)
• Diffusion Models are Efficient Data Generators for Human Mesh Recovery [55.37787289869703]
  We show that synthetic data created by generative models is complementary to CG-rendered data.<n>We propose an effective data generation pipeline based on recent diffusion models, termed HumanWild.<n>Our work could pave the way for scaling up 3D human recovery to in-the-wild scenes.
  arXiv  Detail & Related papers  (2024-03-17T06:31:16Z)
• A survey of synthetic data augmentation methods in computer vision [0.0]
  This paper presents an extensive review of synthetic data augmentation techniques. We focus on the important data generation and augmentation techniques, general scope of application and specific use-cases. We provide a summary of common synthetic datasets for training computer vision models.
  arXiv  Detail & Related papers  (2024-03-15T07:34:08Z)
• GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embedding Fine-tuning [0.0]
  GISTEmbed is a novel strategy that enhances in-batch negative selection during contrastive training through a guide model. Benchmarked against the Massive Text Embedding Benchmark (MTEB), GISTEmbed showcases consistent performance improvements across various model sizes.
  arXiv  Detail & Related papers  (2024-02-26T18:55:15Z)
• Data-efficient Large Vision Models through Sequential Autoregression [58.26179273091461]
  We develop an efficient, autoregression-based vision model on a limited dataset. We demonstrate how this model achieves proficiency in a spectrum of visual tasks spanning both high-level and low-level semantic understanding. Our empirical evaluations underscore the model's agility in adapting to various tasks, heralding a significant reduction in the parameter footprint.
  arXiv  Detail & Related papers  (2024-02-07T13:41:53Z)
• Contrastive Transformer Learning with Proximity Data Generation for Text-Based Person Search [60.626459715780605]
  Given a descriptive text query, text-based person search aims to retrieve the best-matched target person from an image gallery. Such a cross-modal retrieval task is quite challenging due to significant modality gap, fine-grained differences and insufficiency of annotated data. In this paper, we propose a simple yet effective dual Transformer model for text-based person search.
  arXiv  Detail & Related papers  (2023-11-15T16:26:49Z)
• DAT++: Spatially Dynamic Vision Transformer with Deformable Attention [87.41016963608067]
  We present Deformable Attention Transformer ( DAT++), a vision backbone efficient and effective for visual recognition. DAT++ achieves state-of-the-art results on various visual recognition benchmarks, with 85.9% ImageNet accuracy, 54.5 and 47.0 MS-COCO instance segmentation mAP, and 51.5 ADE20K semantic segmentation mIoU.
  arXiv  Detail & Related papers  (2023-09-04T08:26:47Z)
• Controllable Visual-Tactile Synthesis [28.03469909285511]
  We develop a conditional generative model that synthesizes both visual and tactile outputs from a single sketch. We then introduce a pipeline to render high-quality visual and tactile outputs on an electroadhesion-based haptic device.
  arXiv  Detail & Related papers  (2023-05-04T17:59:51Z)
• Guiding AI-Generated Digital Content with Wireless Perception [69.51950037942518]
  We introduce an integration of wireless perception with AI-generated content (AIGC) to improve the quality of digital content production. The framework employs a novel multi-scale perception technology to read user's posture, which is difficult to describe accurately in words, and transmits it to the AIGC model as skeleton images. Since the production process imposes the user's posture as a constraint on the AIGC model, it makes the generated content more aligned with the user's requirements.
  arXiv  Detail & Related papers  (2023-03-26T04:39:03Z)
• VisTaNet: Attention Guided Deep Fusion for Surface Roughness Classification [0.0]
  This paper presents a visual dataset that augments an existing tactile dataset. We propose a novel deep fusion architecture that fuses visual and tactile data using four types of fusion strategies. Our model shows significant performance improvements (97.22%) in surface roughness classification accuracy over tactile only.
  arXiv  Detail & Related papers  (2022-09-18T09:37:06Z)
• Simple and Effective Synthesis of Indoor 3D Scenes [78.95697556834536]
  We study the problem of immersive 3D indoor scenes from one or more images. Our aim is to generate high-resolution images and videos from novel viewpoints. We propose an image-to-image GAN that maps directly from reprojections of incomplete point clouds to full high-resolution RGB-D images.
  arXiv  Detail & Related papers  (2022-04-06T17:54:46Z)
• Florence: A New Foundation Model for Computer Vision [97.26333007250142]
  We introduce a new computer vision foundation model, Florence, to expand the representations from coarse (scene) to fine (object) By incorporating universal visual-language representations from Web-scale image-text data, our Florence model can be easily adapted for various computer vision tasks. Florence achieves new state-of-the-art results in majority of 44 representative benchmarks.
  arXiv  Detail & Related papers  (2021-11-22T18:59:55Z)
• KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D [67.50776195828242]
  KITTI-360 is a suburban driving dataset which comprises richer input modalities, comprehensive semantic instance annotations and accurate localization. For efficient annotation, we created a tool to label 3D scenes with bounding primitives, resulting in over 150k semantic and instance annotated images and 1B annotated 3D points. We established benchmarks and baselines for several tasks relevant to mobile perception, encompassing problems from computer vision, graphics, and robotics on the same dataset.
  arXiv  Detail & Related papers  (2021-09-28T00:41:29Z)
• Spatio-temporal Attention Model for Tactile Texture Recognition [25.06942319117782]
  We propose a novel Spatio-Temporal Attention Model (STAM) for tactile texture recognition. The proposed STAM pays attention to both spatial focus of each single tactile texture and the temporal correlation of a tactile sequence. In the experiments to discriminate 100 different fabric textures, the spatially and temporally selective attention has resulted in a significant improvement of the recognition accuracy.
  arXiv  Detail & Related papers  (2020-08-10T22:32:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.