Efficient Context-Limited Telescope Bibliography Classification for the WASP-2025 Shared Task Using SciBERT

• URL: http://arxiv.org/abs/2609.01647v1
• Date: Sat, 29 Aug 2026 17:21:54 GMT
• Title: Efficient Context-Limited Telescope Bibliography Classification for the WASP-2025 Shared Task Using SciBERT
• Abstract summary: We present an efficient SciBERT-based approach for automatic classification of scientific papers into four categories - science, instrumentation, mention, and not telescope.<n>Despite strict context-length constraints, our approach achieved a macro F1 score of 0.89, ranking at the top of the WASP-2025 leaderboard.<n>We discuss trade-offs between truncation, chunking, and long-context models, providing insights into the efficiency frontier for scientific text curation.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The creation of telescope bibliographies is a crucial part of assessing the scientific impact of observatories and ensuring reproducibility in astronomy. This task involves identifying, categorizing, and linking scientific publications that reference or use specific telescopes. However, this process remains largely manual and resource intensive. In this work, we present an efficient SciBERT-based approach for automatic classification of scientific papers into four categories - science, instrumentation, mention, and not telescope. Despite strict context-length constraints (maximum 512 tokens) and limited compute resources, our approach achieved a macro F1 score of 0.89, ranking at the top of the WASP-2025 leaderboard. We analyze the effect of truncation and show that even with half the samples exceeding the token limit, SciBERT's domain alignment enables robust classification. We discuss trade-offs between truncation, chunking, and long-context models, providing insights into the efficiency frontier for scientific text curation.

Related papers

• Large-scale dataset of automatically classified rhetorical sections in scientific papers [0.09558392439655013]
  We present a dataset of section-level annotations for millions of scientific papers from the Semantic Scholar Open Research Corpus (S2ORC)<n>Using a rule-based classification algorithm, we identified and labeled major sections across 15.6 million papers after quality filtering.<n>The dataset covers primarily STEM disciplines, with strong representation in medicine and biology.
  arXiv  Detail & Related papers  (2026-07-03T14:37:04Z)
• Compass: Navigating Global Marine Lead Data Integration through Expert-Guided LLM Agent [55.77954024801206]
  Marine lead (Pb) and its isotopes are critical tracers for ocean circulation and anthropogenic pollution.<n>Manual extraction is unscalable, while general-purpose Large Language Models (LLMs) lack the necessary domain-specific knowledge.<n>We introduce an expert-guided adaptation approach that enables LLMs to perform rigorous scientific data extraction without fine-tuning.
  arXiv  Detail & Related papers  (2026-05-28T14:06:25Z)
• SciCustom: A Framework for Custom Evaluation of Scientific Capabilities in Large Language Models [63.357710580101894]
  Large language models (LLMs) are increasingly applied to scientific research, yet existing evaluations often fail to reflect the fine-grained capabilities required in practice.<n>We propose a new framework named SciCustom to address the problem.<n>It enables the custom construction of benchmarks from large-scale scientific data to evaluate application-specific scientific capabilities in LLMs.
  arXiv  Detail & Related papers  (2026-05-19T04:41:43Z)
• AstroConcepts: A Large-Scale Multi-Label Classification Corpus for Astrophysics [0.19804102882400357]
  AstroConcepts is a corpus of 21,702 astrophysics papers labeled with 2,367 concepts from the Unified Astronomy Thesaurus.<n>The corpus exhibits severe label imbalance, with 76% of concepts having fewer than 50 training examples.<n>Our evaluation reveals three key patterns that provide new insights into scientific text classification.
  arXiv  Detail & Related papers  (2026-04-02T15:27:00Z)
• RPC-Bench: A Fine-grained Benchmark for Research Paper Comprehension [65.81339691942757]
  RPC-Bench is a large-scale question-answering benchmark built from review-rebuttal exchanges of high-quality computer science papers.<n>We design a fine-grained taxonomy aligned with the scientific research flow to assess models' ability to understand and answer why, what, and how questions in scholarly contexts.
  arXiv  Detail & Related papers  (2026-01-14T11:37:00Z)
• amc: The Automated Mission Classifier for Telescope Bibliographies [0.10262304700896198]
  A modified version of amc performs well on the TRACS Kaggle challenge.<n>Amc can also be used to interrogate historical datasets and surface potential label errors.
  arXiv  Detail & Related papers  (2025-12-12T01:24:42Z)
• Encoder Fine-tuning with Stochastic Sampling Outperforms Open-weight GPT in Astronomy Knowledge Extraction [11.478263835391433]
  We present an encoder-based system for extracting knowledge from astronomy articles.<n>Our system, despite its simplicity and low-cost implementation, significantly outperforms the open-weight GPT baseline.
  arXiv  Detail & Related papers  (2025-11-11T13:08:30Z)
• AstroVisBench: A Code Benchmark for Scientific Computing and Visualization in Astronomy [39.94582666929051]
  We introduce AstroVisBench, the first benchmark for both scientific computing and visualization in the astronomy domain.<n>We present an evaluation of state-of-the-art language models, showing a significant gap in their ability to engage in astronomy research as useful assistants.
  arXiv  Detail & Related papers  (2025-05-26T21:49:18Z)
• SciDMT: A Large-Scale Corpus for Detecting Scientific Mentions [52.35520385083425]
  We present SciDMT, an enhanced and expanded corpus for scientific mention detection. The corpus consists of two components: 1) the SciDMT main corpus, which includes 48 thousand scientific articles with over 1.8 million weakly annotated mention annotations in the format of in-text span, and 2) an evaluation set, which comprises 100 scientific articles manually annotated for evaluation purposes.
  arXiv  Detail & Related papers  (2024-06-20T22:03:21Z)
• SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literature [97.31347312130119]
  SciRIFF (Scientific Resource for Instruction-Following and Finetuning) is a dataset of 137K instruction-following instances for training and evaluation, covering 54 tasks.<n>These tasks span five core scientific literature understanding capabilities: information extraction, summarization, question answering, claim verification, and classification.<n> SciRIFF is unique in being entirely expert-written, high-quality instruction-following dataset for extracting and synthesizing information from research literature across diverse scientific fields.
  arXiv  Detail & Related papers  (2024-06-10T21:22:08Z)
• ACL-Fig: A Dataset for Scientific Figure Classification [15.241086410108512]
  We develop a pipeline that extracts figures and tables from the scientific literature and a deep-learning-based framework that classifies scientific figures using visual features. We build the first large-scale automatically annotated corpus, ACL-Fig, consisting of 112,052 scientific figures extracted from 56K research papers in the ACL Anthology. The ACL-Fig-Pilot dataset contains 1,671 manually labeled scientific figures belonging to 19 categories.
  arXiv  Detail & Related papers  (2023-01-28T20:27:35Z)
• Informational Space of Meaning for Scientific Texts [68.8204255655161]
  We introduce the Meaning Space, in which the meaning of a word is represented by a vector of Relative Information Gain (RIG) about the subject categories that the text belongs to. This new approach is applied to construct the Meaning Space based on Leicester Scientific Corpus (LSC) and Leicester Scientific Dictionary-Core (LScDC) The most informative words are presented for 252 categories. The proposed model based on RIG is shown to have ability to stand out topic-specific words in categories.
  arXiv  Detail & Related papers  (2020-04-28T14:26:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.