AstroConcepts: A Large-Scale Multi-Label Classification Corpus for Astrophysics

• URL: http://arxiv.org/abs/2604.02156v1
• Date: Thu, 02 Apr 2026 15:27:00 GMT
• Title: AstroConcepts: A Large-Scale Multi-Label Classification Corpus for Astrophysics
• Abstract summary: AstroConcepts is a corpus of 21,702 astrophysics papers labeled with 2,367 concepts from the Unified Astronomy Thesaurus.<n>The corpus exhibits severe label imbalance, with 76% of concepts having fewer than 50 training examples.<n>Our evaluation reveals three key patterns that provide new insights into scientific text classification.
• Score: 0.19804102882400357
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scientific multi-label text classification suffers from extreme class imbalance, where specialized terminology exhibits severe power-law distributions that challenge standard classification approaches. Existing scientific corpora lack comprehensive controlled vocabularies, focusing instead on broad categories and limiting systematic study of extreme imbalance. We introduce AstroConcepts, a corpus of English abstracts from 21,702 published astrophysics papers, labeled with 2,367 concepts from the Unified Astronomy Thesaurus. The corpus exhibits severe label imbalance, with 76% of concepts having fewer than 50 training examples. By releasing this resource, we enable systematic study of extreme class imbalance in scientific domains and establish strong baselines across traditional, neural, and vocabulary-constrained LLM methods. Our evaluation reveals three key patterns that provide new insights into scientific text classification. First, vocabulary-constrained LLMs achieve competitive performance relative to domain-adapted models in astrophysics classification, suggesting a potential for parameter-efficient approaches. Second, domain adaptation yields relatively larger improvements for rare, specialized terminology, although absolute performance remains limited across all methods. Third, we propose frequency-stratified evaluation to reveal performance patterns that are hidden by aggregate scores, thereby making robustness assessment central to scientific multi-label evaluation. These results offer actionable insights for scientific NLP and establish benchmarks for research on extreme imbalance.

Related papers

• SCALE: Scientific Concept Aggregation via LLMs and Embeddings for Fine-Grained Taxonomy Extension [0.815557531820863]
  Author keywords offer greater specificity, but their fragmentation, redundancy, and terminological variability limit their use as stable units of knowledge organization.<n>We introduce SCALE, a framework that extends the OpenAlex taxonomy with a new level of scientific Concepts below Topics.<n>The resulting taxonomy enables scientific literature to be read through an intermediate conceptual level between broad research topics and individual documents.
  arXiv  Detail & Related papers  (2026-08-07T14:12:15Z)
• SciIF: Benchmarking Scientific Instruction Following Towards Rigorous Scientific Intelligence [60.202862987441684]
  We introduce scientific instruction following: the capability to solve problems while strictly adhering to the constraints that establish scientific validity.<n>Specifically, we introduce SciIF, a multi-discipline benchmark that evaluates this capability by pairing university-level problems with a fixed catalog of constraints.<n>By measuring both solution correctness and multi-constraint adherence, SciIF enables finegrained diagnosis of compositional reasoning failures.
  arXiv  Detail & Related papers  (2026-01-08T09:45:58Z)
• ATLAS: A High-Difficulty, Multidisciplinary Benchmark for Frontier Scientific Reasoning [118.46980291324148]
  ATLAS is a large-scale, high-difficulty, and cross-disciplinary evaluation suite composed of approximately 800 original problems.<n>Its key features include: High Originality and Contamination Resistance, with all questions newly created or substantially adapted to prevent test data leakage.<n>Preliminary results on leading models demonstrate ATLAS's effectiveness in differentiating their advanced scientific reasoning capabilities.
  arXiv  Detail & Related papers  (2025-11-18T11:13:06Z)
• Publication Trend Analysis and Synthesis via Large Language Model: A Case Study of Engineering in PNAS [0.0]
  This study introduces an adaptable large language model (LLM)-driven framework to quantify thematic trends.<n>The approach is demonstrated over a 20-year collection of more than 1,500 engineering articles published by the Proceedings of the National Academy of Sciences.
  arXiv  Detail & Related papers  (2025-10-17T18:57:11Z)
• TaxoAdapt: Aligning LLM-Based Multidimensional Taxonomy Construction to Evolving Research Corpora [34.103517830260365]
  TaxoAdapt is a framework that adapts an LLM-generated taxonomy to a given corpus across multiple dimensions.<n>We demonstrate its state-of-the-art performance across a diverse set of computer science conferences.
  arXiv  Detail & Related papers  (2025-06-12T14:26:28Z)
• Handling Imbalanced Pseudolabels for Vision-Language Models with Concept Alignment and Confusion-Aware Calibrated Margin [56.37346003683629]
  Adapting vision-language models (VLMs) to downstream tasks with pseudolabels has gained increasing attention.<n>A major obstacle is that the pseudolabels generated by VLMs tend to be imbalanced, leading to inferior performance.<n>We propose a novel framework incorporating concept alignment and confusion-aware margin mechanisms.
  arXiv  Detail & Related papers  (2025-05-04T10:24:34Z)
• Rethinking Scientific Summarization Evaluation: Grounding Explainable Metrics on Facet-aware Benchmark [42.131133762827375]
  This paper presents conceptual and experimental analyses of scientific summarization.<n>We introduce the Facet-aware Metric (FM), employing LLMs for advanced semantic matching to evaluate summaries.<n>Our findings confirm that FM offers a more logical approach to evaluating scientific summaries.
  arXiv  Detail & Related papers  (2024-02-22T07:58:29Z)
• A Saliency-based Clustering Framework for Identifying Aberrant Predictions [49.1574468325115]
  We introduce the concept of aberrant predictions, emphasizing that the nature of classification errors is as critical as their frequency. We propose a novel, efficient training methodology aimed at both reducing the misclassification rate and discerning aberrant predictions. We apply this methodology to the less-explored domain of veterinary radiology, where the stakes are high but have not been as extensively studied compared to human medicine.
  arXiv  Detail & Related papers  (2023-11-11T01:53:59Z)
• Bias and Fairness in Large Language Models: A Survey [73.87651986156006]
  We present a comprehensive survey of bias evaluation and mitigation techniques for large language models (LLMs) We first consolidate, formalize, and expand notions of social bias and fairness in natural language processing. We then unify the literature by proposing three intuitive, two for bias evaluation, and one for mitigation.
  arXiv  Detail & Related papers  (2023-09-02T00:32:55Z)
• Class-Imbalanced Graph Learning without Class Rebalancing [62.1368829847041]
  Class imbalance is prevalent in real-world node classification tasks and poses great challenges for graph learning models. In this work, we approach the root cause of class-imbalance bias from an topological paradigm. We devise a lightweight topological augmentation framework BAT to mitigate the class-imbalance bias without class rebalancing.
  arXiv  Detail & Related papers  (2023-08-27T19:01:29Z)
• Biomedical Named Entity Recognition via Dictionary-based Synonym Generalization [51.89486520806639]
  We propose a novel Synonym Generalization (SynGen) framework that recognizes the biomedical concepts contained in the input text using span-based predictions. We extensively evaluate our approach on a wide range of benchmarks and the results verify that SynGen outperforms previous dictionary-based models by notable margins.
  arXiv  Detail & Related papers  (2023-05-22T14:36:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.