Cortex Inspired Learning to Recover Damaged Signal Modality with ReD-SOM Model

• URL: http://arxiv.org/abs/2307.15095v1
• Date: Thu, 27 Jul 2023 09:44:12 GMT
• Title: Cortex Inspired Learning to Recover Damaged Signal Modality with ReD-SOM Model
• Authors: Artem Muliukov, Laurent Rodriguez, Benoit Miramond
• Abstract summary: Recent progress in AI and cognitive sciences opens up new challenges that were previously inaccessible to study. One of such modern tasks is recovering lost data of one modality by using the data from another one. We propose a way to simulate such an effect and use it to reconstruct lost data modalities by combining Variational Auto-Encoders, Self-Organizing Maps, and Hebb connections.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent progress in the fields of AI and cognitive sciences opens up new challenges that were previously inaccessible to study. One of such modern tasks is recovering lost data of one modality by using the data from another one. A similar effect (called the McGurk Effect) has been found in the functioning of the human brain. Observing this effect, one modality of information interferes with another, changing its perception. In this paper, we propose a way to simulate such an effect and use it to reconstruct lost data modalities by combining Variational Auto-Encoders, Self-Organizing Maps, and Hebb connections in a unified ReD-SOM (Reentering Deep Self-organizing Map) model. We are inspired by human's capability to use different zones of the brain in different modalities, in case of having a lack of information in one of the modalities. This new approach not only improves the analysis of ambiguous data but also restores the intended signal! The results obtained on the multimodal dataset demonstrate an increase of quality of the signal reconstruction. The effect is remarkable both visually and quantitatively, specifically in presence of a significant degree of signal's distortion.

Related papers

• Detecting and Mitigating Adversarial Attacks on Deep Learning-Based MRI Reconstruction Without Any Retraining [2.5943586090617377]
  We propose a novel approach for detecting and mitigating adversarial attacks on MRI reconstruction models without any retraining. Our detection strategy is based on the idea of cyclic measurement consistency. We show that our method substantially reduces the impact of adversarial perturbations across different datasets.
  arXiv  Detail & Related papers  (2025-01-03T17:23:52Z)
• Representation Learning for Wearable-Based Applications in the Case of Missing Data [20.37256375888501]
  multimodal sensor data in real-world environments is still challenging due to low data quality and limited data annotations. We investigate representation learning for imputing missing wearable data and compare it with state-of-the-art statistical approaches. Our study provides insights for the design and development of masking-based self-supervised learning tasks.
  arXiv  Detail & Related papers  (2024-01-08T08:21:37Z)
• Causal disentanglement of multimodal data [1.589226862328831]
  We introduce a causal representation learning algorithm (causalPIMA) that can use multimodal data and known physics to discover important features with causal relationships. Our results demonstrate the capability of learning an interpretable causal structure while simultaneously discovering key features in a fully unsupervised setting.
  arXiv  Detail & Related papers  (2023-10-27T20:30:11Z)
• A Discrepancy Aware Framework for Robust Anomaly Detection [51.710249807397695]
  We present a Discrepancy Aware Framework (DAF), which demonstrates robust performance consistently with simple and cheap strategies. Our method leverages an appearance-agnostic cue to guide the decoder in identifying defects, thereby alleviating its reliance on synthetic appearance. Under the simple synthesis strategies, it outperforms existing methods by a large margin. Furthermore, it also achieves the state-of-the-art localization performance.
  arXiv  Detail & Related papers  (2023-10-11T15:21:40Z)
• PartMix: Regularization Strategy to Learn Part Discovery for Visible-Infrared Person Re-identification [76.40417061480564]
  We present a novel data augmentation technique, dubbed PartMix, for part-based Visible-Infrared person Re-IDentification (VI-ReID) models. We synthesize the augmented samples by mixing the part descriptors across the modalities to improve the performance of part-based VI-ReID models.
  arXiv  Detail & Related papers  (2023-04-04T05:21:23Z)
• CausalDialogue: Modeling Utterance-level Causality in Conversations [83.03604651485327]
  We have compiled and expanded upon a new dataset called CausalDialogue through crowd-sourcing. This dataset includes multiple cause-effect pairs within a directed acyclic graph (DAG) structure. We propose a causality-enhanced method called Exponential Average Treatment Effect (ExMATE) to enhance the impact of causality at the utterance level in training neural conversation models.
  arXiv  Detail & Related papers  (2022-12-20T18:31:50Z)
• Improving the Level of Autism Discrimination through GraphRNN Link Prediction [8.103074928419527]
  This paper is based on the latter technique, which learns the edge distribution of real brain network through GraphRNN. The experimental results show that the combination of original and synthetic data greatly improves the discrimination of the neural network.
  arXiv  Detail & Related papers  (2022-02-19T06:50:32Z)
• Reducing Catastrophic Forgetting in Self Organizing Maps with Internally-Induced Generative Replay [67.50637511633212]
  A lifelong learning agent is able to continually learn from potentially infinite streams of pattern sensory data. One major historic difficulty in building agents that adapt is that neural systems struggle to retain previously-acquired knowledge when learning from new samples. This problem is known as catastrophic forgetting (interference) and remains an unsolved problem in the domain of machine learning to this day.
  arXiv  Detail & Related papers  (2021-12-09T07:11:14Z)
• Learning Neural Causal Models with Active Interventions [83.44636110899742]
  We introduce an active intervention-targeting mechanism which enables a quick identification of the underlying causal structure of the data-generating process. Our method significantly reduces the required number of interactions compared with random intervention targeting. We demonstrate superior performance on multiple benchmarks from simulated to real-world data.
  arXiv  Detail & Related papers  (2021-09-06T13:10:37Z)
• Low to High Dimensional Modality Hallucination using Aggregated Fields of View [48.32515709424962]
  We argue modality hallucination as one effective way to ensure consistent modality availability. We present a novel hallucination architecture that aggregates information from multiple fields of view of the local neighborhood. We also conduct extensive classification and segmentation experiments on UWRGBD and NYUD datasets and demonstrate that hallucination allays the negative effects of the modality loss.
  arXiv  Detail & Related papers  (2020-07-13T03:13:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.