Human-in-the-Loop Optimization for Deep Stimulus Encoding in Visual Prostheses

• URL: http://arxiv.org/abs/2306.13104v2
• Date: Fri, 27 Oct 2023 22:56:28 GMT
• Title: Human-in-the-Loop Optimization for Deep Stimulus Encoding in Visual Prostheses
• Abstract summary: Neuroprostheses show potential in restoring lost sensory function and enhancing human capabilities, but the sensations produced by current devices often seem unnatural or distorted. We propose a novel, practically feasible approach that overcomes both of these fundamental limitations. We show that our approach quickly learns a personalized stimulus encoder, leads to dramatic improvements in the quality of restored vision, and is robust to noisy patient feedback.
• Score: 2.7309692684728617
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neuroprostheses show potential in restoring lost sensory function and enhancing human capabilities, but the sensations produced by current devices often seem unnatural or distorted. Exact placement of implants and differences in individual perception lead to significant variations in stimulus response, making personalized stimulus optimization a key challenge. Bayesian optimization could be used to optimize patient-specific stimulation parameters with limited noisy observations, but is not feasible for high-dimensional stimuli. Alternatively, deep learning models can optimize stimulus encoding strategies, but typically assume perfect knowledge of patient-specific variations. Here we propose a novel, practically feasible approach that overcomes both of these fundamental limitations. First, a deep encoder network is trained to produce optimal stimuli for any individual patient by inverting a forward model mapping electrical stimuli to visual percepts. Second, a preferential Bayesian optimization strategy utilizes this encoder to optimize patient-specific parameters for a new patient, using a minimal number of pairwise comparisons between candidate stimuli. We demonstrate the viability of this approach on a novel, state-of-the-art visual prosthesis model. We show that our approach quickly learns a personalized stimulus encoder, leads to dramatic improvements in the quality of restored vision, and is robust to noisy patient feedback and misspecifications in the underlying forward model. Overall, our results suggest that combining the strengths of deep learning and Bayesian optimization could significantly improve the perceptual experience of patients fitted with visual prostheses and may prove a viable solution for a range of neuroprosthetic technologies.

Related papers

• Uncertainty-Aware Exploratory Direct Preference Optimization for Multimodal Large Language Models [53.15468578562038]
  We propose an Uncertainty-aware Exploratory Direct Preference Optimization (UE-DPO) method for Multimodal Large Language Models (MLLMs)<n>We first quantify the uncertainty from the model's failure to ground token predictions in the given image.<n>We then encourage more learning pressure on visually deficient tokens in preferred samples, and alleviate the over-penalization of beneficial knowledge in dispreferred samples.
  arXiv  Detail & Related papers  (2026-05-06T13:08:12Z)
• Percept-Aware Surgical Planning for Visual Cortical Prostheses with Vascular Avoidance [6.649753747542209]
  Cortical visual prostheses aim to restore sight by electrically stimulating neurons in early visual cortex (V1)<n>Existing strategies emphasize visual field coverage and anatomicals but do not directly optimize predicted perceptual outcomes under safety constraints.<n>We present a percept-aware framework for surgical planning of cortical visual prostheses that formulates electrode placement as a constrained optimization problem in anatomical space.
  arXiv  Detail & Related papers  (2026-02-27T22:59:18Z)
• Leveraging Metaheuristic Approaches to Improve Deep Learning Systems for Anxiety Disorder Detection [0.0]
  This study introduces a comprehensive model that integrates deep learning architectures with optimization strategies inspired by swarm intelligence.<n>Using multimodal and wearable-sensor datasets, the framework analyzes physiological, emotional, and behavioral signals.<n>Our evaluation shows that the fusion of these two computational paradigms significantly enhances detection performance.
  arXiv  Detail & Related papers  (2025-11-24T07:03:15Z)
• Personalized Transcranial Electrical Stimulation: A Review of Computational Modeling and Optimization [17.51951846648584]
  This review presents a comprehensive overview of advances in computational techniques supporting personalized tES.<n>Recent advances have substantially accelerated the construction of subject-specific head conductor models.<n>These advances allow for dynamic and individualized stimulation planning, moving beyond empirical trial-and-error approaches.
  arXiv  Detail & Related papers  (2025-09-01T07:18:04Z)
• Evaluating Deep Human-in-the-Loop Optimization for Retinal Implants Using Sighted Participants [2.3301643766310374]
  Human-in-the-loop optimization (HILO) is a promising approach for personalizing visual prostheses by iteratively refining stimulus parameters based on user feedback. Previous work demonstrated HILO's efficacy in simulation, but its performance with human participants remains untested. Here we evaluate HILO using sighted participants viewing simulated prosthetic vision to assess its ability to optimize stimulation strategies under realistic conditions.
  arXiv  Detail & Related papers  (2025-01-31T21:41:44Z)
• Adaptive Preference Scaling for Reinforcement Learning with Human Feedback [103.36048042664768]
  Reinforcement learning from human feedback (RLHF) is a prevalent approach to align AI systems with human values. We propose a novel adaptive preference loss, underpinned by distributionally robust optimization (DRO) Our method is versatile and can be readily adapted to various preference optimization frameworks.
  arXiv  Detail & Related papers  (2024-06-04T20:33:22Z)
• Deep Kernel and Image Quality Estimators for Optimizing Robotic Ultrasound Controller using Bayesian Optimization [2.0971479389679337]
  Autonomous Robotic Ultrasound (A-RUS) is an appealing alternative to manual procedure in order to reduce sonographers' workload. The key challenge to A-RUS is optimizing the ultrasound image quality for the region of interest across different patients. This requires knowledge of anatomy, recognition of error sources and precise probe position, orientation and pressure.
  arXiv  Detail & Related papers  (2023-10-11T11:20:35Z)
• Adapting Brain-Like Neural Networks for Modeling Cortical Visual Prostheses [68.96380145211093]
  Cortical prostheses are devices implanted in the visual cortex that attempt to restore lost vision by electrically stimulating neurons. Currently, the vision provided by these devices is limited, and accurately predicting the visual percepts resulting from stimulation is an open challenge. We propose to address this challenge by utilizing 'brain-like' convolutional neural networks (CNNs), which have emerged as promising models of the visual system.
  arXiv  Detail & Related papers  (2022-09-27T17:33:19Z)
• A Hybrid Neural Autoencoder for Sensory Neuroprostheses and Its Applications in Bionic Vision [0.0]
  Sensory neuroprostheses are emerging as a promising technology to restore lost sensory function or augment human capacities. In this paper we show how a deep neural network encoder is trained to invert a known, fixed forward model that approximates the underlying biological system. As a proof of concept, we demonstrate the effectiveness of our hybrid neural autoencoder (HNA) on the use case of visual neuroprostheses.
  arXiv  Detail & Related papers  (2022-05-26T20:52:00Z)
• Position-wise optimizer: A nature-inspired optimization algorithm [0.0]
  A novel nature-inspired optimization algorithm is introduced that imitates biological neural plasticity. The model is tested on three datasets and the results are compared with gradient descent optimization.
  arXiv  Detail & Related papers  (2022-04-11T15:30:52Z)
• Hybrid Predictive Coding: Inferring, Fast and Slow [62.997667081978825]
  We propose a hybrid predictive coding network that combines both iterative and amortized inference in a principled manner. We demonstrate that our model is inherently sensitive to its uncertainty and adaptively balances balances to obtain accurate beliefs using minimum computational expense.
  arXiv  Detail & Related papers  (2022-04-05T12:52:45Z)
• No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models [132.90062129639705]
  We propose a novel training strategy that encourages all parameters to be trained sufficiently. A parameter with low sensitivity is redundant, and we improve its fitting by increasing its learning rate. In contrast, a parameter with high sensitivity is well-trained and we regularize it by decreasing its learning rate to prevent further overfitting.
  arXiv  Detail & Related papers  (2022-02-06T00:22:28Z)
• Factorized Neural Processes for Neural Processes: $K$-Shot Prediction of Neural Responses [9.792408261365043]
  We develop a Factorized Neural Process to infer a neuron's tuning function from a small set of stimulus-response pairs. We show on simulated responses that the predictions and reconstructed receptive fields from the Neural Process approach ground truth with increasing number of trials. We believe this novel deep learning systems identification framework will facilitate better real-time integration of artificial neural network modeling into neuroscience experiments.
  arXiv  Detail & Related papers  (2020-10-22T15:43:59Z)
• Bayesian optimization for automatic design of face stimuli [2.572404739180802]
  We propose a novel framework which combines generative networks (GANs) with Bayesian optimization to identify individual response patterns to many different faces. Formally, we employ Bayesian optimization to efficiently search the latent space of state-of-the-art GAN models, with the aim to automatically generate novel faces. We show how the algorithm can efficiently locate an individual's optimal face while mapping out their response across different semantic transformations of a face.
  arXiv  Detail & Related papers  (2020-07-20T10:27:18Z)
• Human Preference-Based Learning for High-dimensional Optimization of Exoskeleton Walking Gaits [55.59198568303196]
  This work presents LineCoSpar, a human-in-the-loop preference-based framework to learn user preferences in high dimensions. In simulations and human trials, we empirically verify that LineCoSpar is a sample-efficient approach for high-dimensional preference optimization. This result has implications for exoskeleton gait synthesis, an active field with applications to clinical use and patient rehabilitation.
  arXiv  Detail & Related papers  (2020-03-13T22:02:15Z)
• End-to-End Facial Deep Learning Feature Compression with Teacher-Student Enhancement [57.18801093608717]
  We propose a novel end-to-end feature compression scheme by leveraging the representation and learning capability of deep neural networks. In particular, the extracted features are compactly coded in an end-to-end manner by optimizing the rate-distortion cost. We verify the effectiveness of the proposed model with the facial feature, and experimental results reveal better compression performance in terms of rate-accuracy.
  arXiv  Detail & Related papers  (2020-02-10T10:08:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.