Personalized visual encoding model construction with small data
- URL: http://arxiv.org/abs/2202.02245v1
- Date: Fri, 4 Feb 2022 17:24:50 GMT
- Title: Personalized visual encoding model construction with small data
- Authors: Zijin Gu, Keith Jamison, Mert Sabuncu, and Amy Kuceyeski
- Abstract summary: We propose and test an alternative personalized ensemble encoding model approach to utilize existing encoding models.
We show that these personalized ensemble encoding models trained with small amounts of data for a specific individual.
Importantly, the personalized ensemble encoding models preserve patterns of inter-individual variability in the image-response relationship.
- Score: 1.6799377888527687
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Encoding models that predict brain response patterns to stimuli are one way
to capture this relationship between variability in bottom-up neural systems
and individual's behavior or pathological state. However, they generally need a
large amount of training data to achieve optimal accuracy. Here, we propose and
test an alternative personalized ensemble encoding model approach to utilize
existing encoding models, to create encoding models for novel individuals with
relatively little stimuli-response data. We show that these personalized
ensemble encoding models trained with small amounts of data for a specific
individual, i.e. ~400 image-response pairs, achieve accuracy not different from
models trained on ~24,000 image-response pairs for the same individual.
Importantly, the personalized ensemble encoding models preserve patterns of
inter-individual variability in the image-response relationship. Additionally,
we use our personalized ensemble encoding model within the recently developed
NeuroGen framework to generate optimal stimuli designed to maximize specific
regions' activations for a specific individual. We show that the
inter-individual differences in face area responses to images of dog vs human
faces observed previously is replicated using NeuroGen with the ensemble
encoding model. Finally, and most importantly, we show the proposed approach is
robust against domain shift by validating on a prospectively collected set of
image-response data in novel individuals with a different scanner and
experimental setup. Our approach shows the potential to use previously
collected, deeply sampled data to efficiently create accurate, personalized
encoding models and, subsequently, personalized optimal synthetic images for
new individuals scanned under different experimental conditions.
Related papers
- A Simple Approach to Unifying Diffusion-based Conditional Generation [63.389616350290595]
We introduce a simple, unified framework to handle diverse conditional generation tasks.
Our approach enables versatile capabilities via different inference-time sampling schemes.
Our model supports additional capabilities like non-spatially aligned and coarse conditioning.
arXiv Detail & Related papers (2024-10-15T09:41:43Z) - Variational autoencoder-based neural network model compression [4.992476489874941]
Variational Autoencoders (VAEs), as a form of deep generative model, have been widely used in recent years.
This paper aims to explore neural network model compression method based on VAE.
arXiv Detail & Related papers (2024-08-25T09:06:22Z) - JeDi: Joint-Image Diffusion Models for Finetuning-Free Personalized Text-to-Image Generation [49.997839600988875]
Existing personalization methods rely on finetuning a text-to-image foundation model on a user's custom dataset.
We propose Joint-Image Diffusion (jedi), an effective technique for learning a finetuning-free personalization model.
Our model achieves state-of-the-art generation quality, both quantitatively and qualitatively, significantly outperforming both the prior finetuning-based and finetuning-free personalization baselines.
arXiv Detail & Related papers (2024-07-08T17:59:02Z) - Multimodal Large Language Model is a Human-Aligned Annotator for Text-to-Image Generation [87.50120181861362]
VisionPrefer is a high-quality and fine-grained preference dataset that captures multiple preference aspects.
We train a reward model VP-Score over VisionPrefer to guide the training of text-to-image generative models and the preference prediction accuracy of VP-Score is comparable to human annotators.
arXiv Detail & Related papers (2024-04-23T14:53:15Z) - Lafite2: Few-shot Text-to-Image Generation [132.14211027057766]
We propose a novel method for pre-training text-to-image generation model on image-only datasets.
It considers a retrieval-then-optimization procedure to synthesize pseudo text features.
It can be beneficial to a wide range of settings, including the few-shot, semi-supervised and fully-supervised learning.
arXiv Detail & Related papers (2022-10-25T16:22:23Z) - Few-shot Generation of Personalized Neural Surrogates for Cardiac
Simulation via Bayesian Meta-Learning [6.978382728087236]
We present a new concept to achieve personalized neural surrogates in a single coherent framework of meta-learning.
As test time, metaPNS delivers a personalized neural surrogate by fast feed-forward embedding of a small and flexible number of data available from an individual.
metaPNS was able to improve personalization and predictive accuracy in comparison to conventionally-optimized cardiac simulation models.
arXiv Detail & Related papers (2022-10-06T14:59:27Z) - Variational Model Inversion Attacks [26.613251410498755]
In model inversion attacks, a malicious user attempts to recover the private dataset used to train a supervised neural network.
A successful model inversion attack should generate realistic and diverse samples that accurately describe each of the classes in the private dataset.
In this work, we provide a probabilistic interpretation of model inversion attacks, and formulate a variational objective that accounts for both diversity and accuracy.
arXiv Detail & Related papers (2022-01-26T07:39:13Z) - Meta Internal Learning [88.68276505511922]
Internal learning for single-image generation is a framework, where a generator is trained to produce novel images based on a single image.
We propose a meta-learning approach that enables training over a collection of images, in order to model the internal statistics of the sample image more effectively.
Our results show that the models obtained are as suitable as single-image GANs for many common image applications.
arXiv Detail & Related papers (2021-10-06T16:27:38Z) - Deep Co-Attention Network for Multi-View Subspace Learning [73.3450258002607]
We propose a deep co-attention network for multi-view subspace learning.
It aims to extract both the common information and the complementary information in an adversarial setting.
In particular, it uses a novel cross reconstruction loss and leverages the label information to guide the construction of the latent representation.
arXiv Detail & Related papers (2021-02-15T18:46:44Z) - A shared neural encoding model for the prediction of subject-specific
fMRI response [17.020869686284165]
We propose a shared convolutional neural encoding method that accounts for individual-level differences.
Our method leverages multi-subject data to improve the prediction of subject-specific responses evoked by visual or auditory stimuli.
arXiv Detail & Related papers (2020-06-29T04:10:14Z) - Pose Manipulation with Identity Preservation [0.0]
We introduce Character Adaptive Identity Normalization GAN (CainGAN) which uses spatial characteristic features extracted by an embedder and combined across source images.
CainGAN receives figures of faces from a certain individual and produces new ones while preserving the person's identity.
Experimental results show that the quality of generated images scales with the size of the input set used during inference.
arXiv Detail & Related papers (2020-04-20T09:51:31Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.