Visual Fixation-Based Retinal Prosthetic Simulation

• URL: http://arxiv.org/abs/2410.11688v1
• Date: Tue, 15 Oct 2024 15:24:08 GMT
• Title: Visual Fixation-Based Retinal Prosthetic Simulation
• Authors: Yuli Wu, Do Dinh Tan Nguyen, Henning Konermann, Rüveyda Yilmaz, Peter Walter, Johannes Stegmaier,
• Abstract summary: The fixation-based framework achieves a classification accuracy of 87.72%, using computational parameters based on a real subject's physiological data. Our approach shows promising potential for producing more semantically understandable percepts with the limited resolution available in retinal prosthetics.
• Score: 1.0075717342698087
• License:
• Abstract: This study proposes a retinal prosthetic simulation framework driven by visual fixations, inspired by the saccade mechanism, and assesses performance improvements through end-to-end optimization in a classification task. Salient patches are predicted from input images using the self-attention map of a vision transformer to mimic visual fixations. These patches are then encoded by a trainable U-Net and simulated using the pulse2percept framework to predict visual percepts. By incorporating a learnable encoder, we aim to optimize the visual information transmitted to the retinal implant, addressing both the limited resolution of the electrode array and the distortion between the input stimuli and resulting phosphenes. The predicted percepts are evaluated using the self-supervised DINOv2 foundation model, with an optional learnable linear layer for classification accuracy. On a subset of the ImageNet validation set, the fixation-based framework achieves a classification accuracy of 87.72%, using computational parameters based on a real subject's physiological data, significantly outperforming the downsampling-based accuracy of 40.59% and approaching the healthy upper bound of 92.76%. Our approach shows promising potential for producing more semantically understandable percepts with the limited resolution available in retinal prosthetics.

Related papers

• PREMAP: A Unifying PREiMage APproximation Framework for Neural Networks [30.701422594374456]
  We present a framework for preimage abstraction that produces under- and over-approximations of any polyhedral output set. We evaluate our method on a range of tasks, demonstrating significant improvement in efficiency and scalability to high-input-dimensional image classification tasks.
  arXiv  Detail & Related papers  (2024-08-17T17:24:47Z)
• Deep Domain Adaptation: A Sim2Real Neural Approach for Improving Eye-Tracking Systems [80.62854148838359]
  Eye image segmentation is a critical step in eye tracking that has great influence over the final gaze estimate. We use dimensionality-reduction techniques to measure the overlap between the target eye images and synthetic training data. Our methods result in robust, improved performance when tackling the discrepancy between simulation and real-world data samples.
  arXiv  Detail & Related papers  (2024-03-23T22:32:06Z)
• ViTaL: An Advanced Framework for Automated Plant Disease Identification in Leaf Images Using Vision Transformers and Linear Projection For Feature Reduction [0.0]
  This paper introduces a robust framework for the automated identification of diseases in plant leaf images. The framework incorporates several key stages to enhance disease recognition accuracy. We propose a novel hardware design specifically tailored for scanning diseased leaves in an omnidirectional fashion.
  arXiv  Detail & Related papers  (2024-02-27T11:32:37Z)
• Slicer Networks [8.43960865813102]
  We propose the Slicer Network, a novel architecture for medical image analysis. The Slicer Network strategically refines and upsamples feature maps via a splatting-blurring-slicing process. Experiments across different medical imaging applications have verified the Slicer Network's improved accuracy and efficiency.
  arXiv  Detail & Related papers  (2024-01-18T09:50:26Z)
• Forgery-aware Adaptive Transformer for Generalizable Synthetic Image Detection [106.39544368711427]
  We study the problem of generalizable synthetic image detection, aiming to detect forgery images from diverse generative methods. We present a novel forgery-aware adaptive transformer approach, namely FatFormer. Our approach tuned on 4-class ProGAN data attains an average of 98% accuracy to unseen GANs, and surprisingly generalizes to unseen diffusion models with 95% accuracy.
  arXiv  Detail & Related papers  (2023-12-27T17:36:32Z)
• A Deep Learning-based in silico Framework for Optimization on Retinal Prosthetic Stimulation [3.870538485112487]
  We propose a neural network-based framework to optimize the perceptions simulated by the in silico retinal implant model pulse2percept. The pipeline consists of a trainable encoder, a pre-trained retinal implant model and a pre-trained evaluator.
  arXiv  Detail & Related papers  (2023-02-07T16:32:05Z)
• Hybrid Predictive Coding: Inferring, Fast and Slow [62.997667081978825]
  We propose a hybrid predictive coding network that combines both iterative and amortized inference in a principled manner. We demonstrate that our model is inherently sensitive to its uncertainty and adaptively balances balances to obtain accurate beliefs using minimum computational expense.
  arXiv  Detail & Related papers  (2022-04-05T12:52:45Z)
• Prediction of progressive lens performance from neural network simulations [62.997667081978825]
  The purpose of this study is to present a framework to predict visual acuity (VA) based on a convolutional neural network (CNN) The proposed holistic simulation tool was shown to act as an accurate model for subjective visual performance.
  arXiv  Detail & Related papers  (2021-03-19T14:51:02Z)
• Learning Ultrasound Rendering from Cross-Sectional Model Slices for Simulated Training [13.640630434743837]
  Computational simulations can facilitate the training of such skills in virtual reality. We propose herein to bypass any rendering and simulation process at interactive time. We use a generative adversarial framework with a dedicated generator architecture and input feeding scheme.
  arXiv  Detail & Related papers  (2021-01-20T21:58:19Z)
• RetiNerveNet: Using Recursive Deep Learning to Estimate Pointwise 24-2 Visual Field Data based on Retinal Structure [109.33721060718392]
  glaucoma is the leading cause of irreversible blindness in the world, affecting over 70 million people. Due to the Standard Automated Perimetry (SAP) test's innate difficulty and its high test-retest variability, we propose the RetiNerveNet.
  arXiv  Detail & Related papers  (2020-10-15T03:09:08Z)
• Improved Slice-wise Tumour Detection in Brain MRIs by Computing Dissimilarities between Latent Representations [68.8204255655161]
  Anomaly detection for Magnetic Resonance Images (MRIs) can be solved with unsupervised methods. We have proposed a slice-wise semi-supervised method for tumour detection based on the computation of a dissimilarity function in the latent space of a Variational AutoEncoder. We show that by training the models on higher resolution images and by improving the quality of the reconstructions, we obtain results which are comparable with different baselines.
  arXiv  Detail & Related papers  (2020-07-24T14:02:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.