Encoder Fine-tuning with Stochastic Sampling Outperforms Open-weight GPT in Astronomy Knowledge Extraction

• URL: http://arxiv.org/abs/2511.08204v1
• Date: Wed, 12 Nov 2025 01:46:36 GMT
• Title: Encoder Fine-tuning with Stochastic Sampling Outperforms Open-weight GPT in Astronomy Knowledge Extraction
• Authors: Shivam Rawat, Lucie Flek, Akbar Karimi,
• Abstract summary: We present an encoder-based system for extracting knowledge from astronomy articles.<n>Our system, despite its simplicity and low-cost implementation, significantly outperforms the open-weight GPT baseline.
• Score: 11.478263835391433
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scientific literature in astronomy is rapidly expanding, making it increasingly important to automate the extraction of key entities and contextual information from research papers. In this paper, we present an encoder-based system for extracting knowledge from astronomy articles. Our objective is to develop models capable of classifying telescope references, detecting auxiliary semantic attributes, and recognizing instrument mentions from textual content. To this end, we implement a multi-task transformer-based system built upon the SciBERT model and fine-tuned for astronomy corpora classification. To carry out the fine-tuning, we stochastically sample segments from the training data and use majority voting over the test segments at inference time. Our system, despite its simplicity and low-cost implementation, significantly outperforms the open-weight GPT baseline.

Related papers

• The Prism Hypothesis: Harmonizing Semantic and Pixel Representations via Unified Autoencoding [82.53463660564933]
  semantic encoders primarily capture low-frequency components that encode abstract meaning, whereas pixel encoders retain high-frequency information that conveys fine-grained detail.<n>We propose Unified Autoencoding (UAE), a model that harmonizes semantic structure and pixel details via an innovative frequency-band modulator.
  arXiv  Detail & Related papers  (2025-12-22T18:59:57Z)
• Connecting Giants: Synergistic Knowledge Transfer of Large Multimodal Models for Few-Shot Learning [61.73934102302588]
  Few-shot learning addresses the challenge of classifying novel classes with limited training samples.<n>We propose a novel framework, Synergistic Knowledge Transfer, which effectively transfers diverse and complementary knowledge from large multimodal models.<n>We show that SynTrans, even when paired with a simple few-shot vision encoder, significantly outperforms current state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-10-13T08:06:23Z)
• AstroVisBench: A Code Benchmark for Scientific Computing and Visualization in Astronomy [39.94582666929051]
  We introduce AstroVisBench, the first benchmark for both scientific computing and visualization in the astronomy domain.<n>We present an evaluation of state-of-the-art language models, showing a significant gap in their ability to engage in astronomy research as useful assistants.
  arXiv  Detail & Related papers  (2025-05-26T21:49:18Z)
• A method based on Generative Adversarial Networks for disentangling physical and chemical properties of stars in astronomical spectra [0.16385815610837165]
  In this work, an encoder-decoder architecture has been designed, where adversarial training is used in the context of astrophysical spectral analysis. A scheme of deep learning is used with the aim of unraveling in the latent space the desired parameters of the rest of the information contained in the data. To test the effectiveness of the method, synthetic astronomical data are used from the APOGEE and Gaia surveys.
  arXiv  Detail & Related papers  (2024-11-08T20:45:09Z)
• Learning to Extract Structured Entities Using Language Models [52.281701191329]
  Recent advances in machine learning have significantly impacted the field of information extraction. We reformulate the task to be entity-centric, enabling the use of diverse metrics. We contribute to the field by introducing Structured Entity Extraction and proposing the Approximate Entity Set OverlaP metric.
  arXiv  Detail & Related papers  (2024-02-06T22:15:09Z)
• ARFA: An Asymmetric Receptive Field Autoencoder Model for Spatiotemporal Prediction [55.30913411696375]
  We propose an Asymmetric Receptive Field Autoencoder (ARFA) model, which introduces corresponding sizes of receptive field modules. In the encoder, we present large kernel module for globaltemporal feature extraction. In the decoder, we develop a small kernel module for localtemporal reconstruction. We construct the RainBench, a large-scale radar echo dataset for precipitation prediction, to address the scarcity of meteorological data in the domain.
  arXiv  Detail & Related papers  (2023-09-01T07:55:53Z)
• A brief review of contrastive learning applied to astrophysics [0.0]
  Contrastive Learning is a self-supervised machine learning algorithm that extracts informative measurements from multi-dimensional datasets. This paper briefly summarizes the main concepts behind contrastive learning and reviews the first promising applications to astronomy.
  arXiv  Detail & Related papers  (2023-06-08T19:56:32Z)
• Advances on the classification of radio image cubes [4.443085464476228]
  Modern radio telescopes will daily generate data sets on the scale of exabytes for systems like the Square Kilometre Array (SKA) Massive data sets are a source of unknown and rare astrophysical phenomena that lead to discoveries. Recently, there has been a surge in scientific publications focusing on the use of artificial intelligence in radio astronomy.
  arXiv  Detail & Related papers  (2023-05-05T11:15:37Z)
• Radio astronomical images object detection and segmentation: A benchmark on deep learning methods [5.058069142315917]
  In this work, we explore the performance of the most affirmed deep learning approaches, applied to astronomical images obtained by radio interferometric instrumentation, to solve the task of automatic source detection. The goal is to provide an overview of existing techniques, in terms of prediction performance and computational efficiency, to scientists in the astrophysics community who would like to employ machine learning in their research.
  arXiv  Detail & Related papers  (2023-03-08T10:55:24Z)
• Improving Astronomical Time-series Classification via Data Augmentation with Generative Adversarial Networks [1.2891210250935146]
  We propose a data augmentation methodology based on Generative Adrial Networks (GANs) to generate a variety of synthetic light curves from variable stars. The classification accuracy of variable stars is improved significantly when training with synthetic data and testing with real data.
  arXiv  Detail & Related papers  (2022-05-13T16:39:54Z)
• Unsupervised Machine Learning for Exploratory Data Analysis of Exoplanet Transmission Spectra [68.8204255655161]
  We focus on unsupervised techniques for analyzing spectral data from transiting exoplanets. We show that there is a high degree of correlation in the spectral data, which calls for appropriate low-dimensional representations. We uncover interesting structures in the principal component basis, namely, well-defined branches corresponding to different chemical regimes.
  arXiv  Detail & Related papers  (2022-01-07T22:26:33Z)
• UniT: Unified Knowledge Transfer for Any-shot Object Detection and Segmentation [52.487469544343305]
  Methods for object detection and segmentation rely on large scale instance-level annotations for training. We propose an intuitive and unified semi-supervised model that is applicable to a range of supervision.
  arXiv  Detail & Related papers  (2020-06-12T22:45:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.