Unsupervised physics-informed disentanglement of multimodal data for high-throughput scientific discovery

• URL: http://arxiv.org/abs/2202.03242v1
• Date: Mon, 7 Feb 2022 14:47:00 GMT
• Title: Unsupervised physics-informed disentanglement of multimodal data for high-throughput scientific discovery
• Authors: Nathaniel Trask, Carianne Martinez, Kookjin Lee, Brad Boyce
• Abstract summary: We introduce physics-informed multimodal autoencoders (PIMA) PIMA is a variational inference framework for discovering shared information in multimodal scientific datasets. A dataset of lattice metamaterials from metal additive manufacturing demonstrates accurate cross modal inference.
• Score: 4.923937591056569
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: We introduce physics-informed multimodal autoencoders (PIMA) - a variational inference framework for discovering shared information in multimodal scientific datasets representative of high-throughput testing. Individual modalities are embedded into a shared latent space and fused through a product of experts formulation, enabling a Gaussian mixture prior to identify shared features. Sampling from clusters allows cross-modal generative modeling, with a mixture of expert decoder imposing inductive biases encoding prior scientific knowledge and imparting structured disentanglement of the latent space. This approach enables discovery of fingerprints which may be detected in high-dimensional heterogeneous datasets, avoiding traditional bottlenecks related to high-fidelity measurement and characterization. Motivated by accelerated co-design and optimization of materials manufacturing processes, a dataset of lattice metamaterials from metal additive manufacturing demonstrates accurate cross modal inference between images of mesoscale topology and mechanical stress-strain response.

Related papers

• Sparse mixed linear modeling with anchor-based guidance for high-entropy alloy discovery [0.12499537119440242]
  In this study, we focus on local data structures that emerge from the greedy search behavior inherent to experimental data acquisition. We develop an algorithm that simultaneously performs prediction and feature selection. Through a case study on high-entropy alloys, this study introduces a method that combines anchor-guided clustering and sparse linear modeling.
  arXiv  Detail & Related papers  (2025-04-29T01:44:15Z)
• Towards Robust Multimodal Physiological Foundation Models: Handling Arbitrary Missing Modalities [9.785262633953794]
  Physio Omni is a foundation model for multimodal physiological signal analysis. It trains a decoupled multimodal tokenizer, enabling masked signal pre-training. It achieves state-of-the-art performance while maintaining strong robustness to missing modalities.
  arXiv  Detail & Related papers  (2025-04-28T09:00:04Z)
• Causal Discovery from Data Assisted by Large Language Models [50.193740129296245]
  It is essential to integrate experimental data with prior domain knowledge for knowledge driven discovery. Here we demonstrate this approach by combining high-resolution scanning transmission electron microscopy (STEM) data with insights derived from large language models (LLMs) By fine-tuning ChatGPT on domain-specific literature, we construct adjacency matrices for Directed Acyclic Graphs (DAGs) that map the causal relationships between structural, chemical, and polarization degrees of freedom in Sm-doped BiFeO3 (SmBFO)
  arXiv  Detail & Related papers  (2025-03-18T02:14:49Z)
• DiffMS: Diffusion Generation of Molecules Conditioned on Mass Spectra [60.39311767532607]
  DiffMS is a formula-restricted encoder-decoder generative network. We develop a robust decoder that bridges latent embeddings and molecular structures. Experiments show DiffMS outperforms existing models on $textitde novo$ molecule generation.
  arXiv  Detail & Related papers  (2025-02-13T18:29:48Z)
• Causal Representation Learning from Multimodal Biomedical Observations [57.00712157758845]
  We develop flexible identification conditions for multimodal data and principled methods to facilitate the understanding of biomedical datasets. Key theoretical contribution is the structural sparsity of causal connections between modalities. Results on a real-world human phenotype dataset are consistent with established biomedical research.
  arXiv  Detail & Related papers  (2024-11-10T16:40:27Z)
• Back to Bayesics: Uncovering Human Mobility Distributions and Anomalies with an Integrated Statistical and Neural Framework [14.899157568336731]
  DeepBayesic is a novel framework that integrates Bayesian principles with deep neural networks to model the underlying distributions. We evaluate our approach on several mobility datasets, demonstrating significant improvements over state-of-the-art anomaly detection methods.
  arXiv  Detail & Related papers  (2024-10-01T19:02:06Z)
• Leveraging Mixture of Experts for Improved Speech Deepfake Detection [53.69740463004446]
  Speech deepfakes pose a significant threat to personal security and content authenticity. We introduce a novel approach for enhancing speech deepfake detection performance using a Mixture of Experts architecture.
  arXiv  Detail & Related papers  (2024-09-24T13:24:03Z)
• Unlocking Potential Binders: Multimodal Pretraining DEL-Fusion for Denoising DNA-Encoded Libraries [51.72836644350993]
  Multimodal Pretraining DEL-Fusion model (MPDF) We develop pretraining tasks applying contrastive objectives between different compound representations and their text descriptions. We propose a novel DEL-fusion framework that amalgamates compound information at the atomic, submolecular, and molecular levels.
  arXiv  Detail & Related papers  (2024-09-07T17:32:21Z)
• Joint Multimodal Transformer for Emotion Recognition in the Wild [49.735299182004404]
  Multimodal emotion recognition (MMER) systems typically outperform unimodal systems. This paper proposes an MMER method that relies on a joint multimodal transformer (JMT) for fusion with key-based cross-attention.
  arXiv  Detail & Related papers  (2024-03-15T17:23:38Z)
• Coupled generator decomposition for fusion of electro- and magnetoencephalography data [1.7102695043811291]
  Data fusion modeling can identify common features across diverse data sources while accounting for source-specific variability. We introduce the concept of a textitcoupled generator decomposition and demonstrate how it generalizes sparse principal component analysis for data fusion.
  arXiv  Detail & Related papers  (2024-03-02T12:09:16Z)
• AI-driven emergence of frequency information non-uniform distribution via THz metasurface spectrum prediction [36.84046475101662]
  We unveil previously unreported information characteristics pertaining to different frequencies emerged during our work on predicting the terahertz spectral modulation effects of metasurfaces based on AI-prediction. This approach effectively optimize the utilization of existing datasets and paves the way for interdisciplinary research and applications in artificial intelligence, chemistry, composite material design, biomedicine, and other fields.
  arXiv  Detail & Related papers  (2023-12-05T01:48:58Z)
• HEALNet: Multimodal Fusion for Heterogeneous Biomedical Data [10.774128925670183]
  This paper presents the Hybrid Early-fusion Attention Learning Network (HEALNet), a flexible multimodal fusion architecture. We conduct multimodal survival analysis on Whole Slide Images and Multi-omic data on four cancer datasets from The Cancer Genome Atlas (TCGA) HEALNet achieves state-of-the-art performance compared to other end-to-end trained fusion models.
  arXiv  Detail & Related papers  (2023-11-15T17:06:26Z)
• Exploiting Modality-Specific Features For Multi-Modal Manipulation Detection And Grounding [54.49214267905562]
  We construct a transformer-based framework for multi-modal manipulation detection and grounding tasks. Our framework simultaneously explores modality-specific features while preserving the capability for multi-modal alignment. We propose an implicit manipulation query (IMQ) that adaptively aggregates global contextual cues within each modality.
  arXiv  Detail & Related papers  (2023-09-22T06:55:41Z)
• Differentiable Agent-based Epidemiology [71.81552021144589]
  We introduce GradABM: a scalable, differentiable design for agent-based modeling that is amenable to gradient-based learning with automatic differentiation. GradABM can quickly simulate million-size populations in few seconds on commodity hardware, integrate with deep neural networks and ingest heterogeneous data sources.
  arXiv  Detail & Related papers  (2022-07-20T07:32:02Z)
• Scalable Gaussian Processes for Data-Driven Design using Big Data with Categorical Factors [14.337297795182181]
  Gaussian processes (GP) have difficulties in accommodating big datasets, categorical inputs, and multiple responses. We propose a GP model that utilizes latent variables and functions obtained through variational inference to address the aforementioned challenges simultaneously. Our approach is demonstrated for machine learning of ternary oxide materials and topology optimization of a multiscale compliant mechanism.
  arXiv  Detail & Related papers  (2021-06-26T02:17:23Z)
• Multilinear Compressive Learning with Prior Knowledge [106.12874293597754]
  Multilinear Compressive Learning (MCL) framework combines Multilinear Compressive Sensing and Machine Learning into an end-to-end system. Key idea behind MCL is the assumption of the existence of a tensor subspace which can capture the essential features from the signal for the downstream learning task. In this paper, we propose a novel solution to address both of the aforementioned requirements, i.e., How to find those tensor subspaces in which the signals of interest are highly separable?
  arXiv  Detail & Related papers  (2020-02-17T19:06:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.