Augmenting representations with scientific papers

• URL: http://arxiv.org/abs/2603.04516v1
• Date: Wed, 04 Mar 2026 19:04:45 GMT
• Title: Augmenting representations with scientific papers
• Authors: Nicolò Oreste Pinciroli Vago, Rocco Di Tella, Carolina Cuesta-Lázaro, Michael J. Smith, Cecilia Garraffo, Rafael Martínez-Galarza,
• Abstract summary: Astronomers have acquired vast repositories of multimodal data, including images, spectra, and time series.<n>These data sources are rarely systematically integrated.<n>This work introduces a contrastive learning framework designed to align X-ray spectra with domain knowledge extracted from scientific literature.
• Score: 0.820984376071696
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Astronomers have acquired vast repositories of multimodal data, including images, spectra, and time series, complemented by decades of literature that analyzes astrophysical sources. Still, these data sources are rarely systematically integrated. This work introduces a contrastive learning framework designed to align X-ray spectra with domain knowledge extracted from scientific literature, facilitating the development of shared multimodal representations. Establishing this connection is inherently complex, as scientific texts encompass a broader and more diverse physical context than spectra. We propose a contrastive pipeline that achieves a 20% Recall@1% when retrieving texts from spectra, proving that a meaningful alignment between these modalities is not only possible but capable of accelerating the interpretation of rare or poorly understood sources. Furthermore, the resulting shared latent space effectively encodes physically significant information. By fusing spectral and textual data, we improve the estimation of 20 physical variables by 16-18% over unimodal spectral baselines. Our results indicate that a Mixture of Experts (MoE) strategy, which leverages both unimodal and shared representations, yields superior performance. Finally, outlier analysis within the multimodal latent space identifies high-priority targets for follow-up investigation, including a candidate pulsating ULX (PULX) and a gravitational lens system. Importantly, this framework can be extended to other scientific domains where aligning observational data with existing literature is possible.

Related papers

• Universal Spectral Tokenization via Self-Supervised Panchromatic Representation Learning [39.14992490784682]
  Sequential scientific data span many resolutions and domains, and unifying them into a common representation is a key step toward developing foundation models for the sciences.<n>We present a deep learning model that jointly learns from heterogeneous spectra in a self-supervised manner.<n>For the first time, we demonstrate that a single model can unify spectral data across resolutions and domains.
  arXiv  Detail & Related papers  (2025-10-20T18:00:00Z)
• Extracting latent representations from X-ray spectra. Classification, regression, and accretion signatures of Chandra sources [0.0]
  This work aims to develop a compact and physically meaningful representation of Chandra X-ray spectra using deep learning.<n>We use a transformer-based autoencoder to compress X-ray spectra.<n>We evaluate the learned representation in terms of spectral reconstruction accuracy, clustering performance, and correlation with physical quantities.
  arXiv  Detail & Related papers  (2025-10-15T21:20:32Z)
• How Do LLM-Generated Texts Impact Term-Based Retrieval Models? [76.92519309816008]
  This paper investigates the influence of large language models (LLMs) on term-based retrieval models.<n>Our linguistic analysis reveals that LLM-generated texts exhibit smoother high-frequency and steeper low-frequency Zipf slopes.<n>Our study further explores whether term-based retrieval models demonstrate source bias, concluding that these models prioritize documents whose term distributions closely correspond to those of the queries.
  arXiv  Detail & Related papers  (2025-08-25T06:43:27Z)
• LUMIR: an LLM-Driven Unified Agent Framework for Multi-task Infrared Spectroscopy Reasoning [12.138903544219724]
  This study introduces LUMIR, a framework designed to achieve accurate infrared spectral analysis under low data conditions.<n> LUMIR integrates a structured literature knowledge base, automated preprocessing, feature extraction, and predictive modeling into a unified pipeline.<n>It was validated on diverse datasets, including the publicly available Milk near-infrared dataset, Chinese medicinal herbs, Citri Reticulatae Pericarpium(CRP) with different storage durations, an industrial wastewater COD dataset, Tecator and Corn.
  arXiv  Detail & Related papers  (2025-07-29T03:20:51Z)
• Towards a Unified Textual Graph Framework for Spectral Reasoning via Physical and Chemical Information Fusion [44.90118820073463]
  We propose a novel multi-modal spectral analysis framework that integrates prior knowledge graphs with Large Language Models.<n>Our method bridges physical spectral measurements and chemical structural semantics by representing them in a unified Textual Graph format.<n>Our framework achieves consistently high performance across multiple spectral analysis tasks, including node-level, edge-level, and graph-level classification.
  arXiv  Detail & Related papers  (2025-06-21T16:58:30Z)
• Spatial Knowledge Graph-Guided Multimodal Synthesis [78.11669780958657]
  We introduce a novel multimodal synthesis approach guided by spatial knowledge graphs, grounded in the concept of knowledge-to-data generation.<n>In experiments, data synthesized from diverse types of spatial knowledge, including direction and distance, enhance the spatial perception and reasoning abilities of MLLMs markedly.<n>We hope that the idea of knowledge-based data synthesis can advance the development of spatial intelligence.
  arXiv  Detail & Related papers  (2025-05-28T17:50:21Z)
• Universal Spectral Transfer with Physical Prior-Informed Deep Generative Learning [9.603403541272746]
  We introduce SpectroGen, a novel physical prior-informed deep generative model for generating relevant spectral signatures. Results show generating with 99% correlation and 0.01 root mean square error with superior resolution than experimentally acquired ground truth spectra.
  arXiv  Detail & Related papers  (2024-07-22T23:31:10Z)
• SciMMIR: Benchmarking Scientific Multi-modal Information Retrieval [64.03631654052445]
  Current benchmarks for evaluating MMIR performance in image-text pairing within the scientific domain show a notable gap. We develop a specialised scientific MMIR benchmark by leveraging open-access paper collections. This benchmark comprises 530K meticulously curated image-text pairs, extracted from figures and tables with detailed captions in scientific documents.
  arXiv  Detail & Related papers  (2024-01-24T14:23:12Z)
• Unsupervised Machine Learning for Exploratory Data Analysis of Exoplanet Transmission Spectra [68.8204255655161]
  We focus on unsupervised techniques for analyzing spectral data from transiting exoplanets. We show that there is a high degree of correlation in the spectral data, which calls for appropriate low-dimensional representations. We uncover interesting structures in the principal component basis, namely, well-defined branches corresponding to different chemical regimes.
  arXiv  Detail & Related papers  (2022-01-07T22:26:33Z)
• Spectral Analysis Network for Deep Representation Learning and Image Clustering [53.415803942270685]
  This paper proposes a new network structure for unsupervised deep representation learning based on spectral analysis. It can identify the local similarities among images in patch level and thus more robust against occlusion. It can learn more clustering-friendly representations and is capable to reveal the deep correlations among data samples.
  arXiv  Detail & Related papers  (2020-09-11T05:07:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.