BAGEL: Benchmarking Animal Knowledge Expertise in Language Models

• URL: http://arxiv.org/abs/2604.16241v1
• Date: Fri, 17 Apr 2026 17:00:37 GMT
• Title: BAGEL: Benchmarking Animal Knowledge Expertise in Language Models
• Abstract summary: BAGEL is a benchmark for evaluating animal knowledge expertise in language models.<n>By focusing on closed-book evaluation, BAGEL measures animal-related knowledge of models without external retrieval at inference time.
• Score: 35.401726501331275
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Large language models have shown strong performance on broad-domain knowledge and reasoning benchmarks, but it remains unclear how well language models handle specialized animal-related knowledge under a unified closed-book evaluation protocol. We introduce BAGEL, a benchmark for evaluating animal knowledge expertise in language models. BAGEL is constructed from diverse scientific and reference sources, including bioRxiv, Global Biotic Interactions, Xeno-canto, and Wikipedia, using a combination of curated examples and automatically generated closed-book question-answer pairs. The benchmark covers multiple aspects of animal knowledge, including taxonomy, morphology, habitat, behavior, vocalization, geographic distribution, and species interactions. By focusing on closed-book evaluation, BAGEL measures animal-related knowledge of models without external retrieval at inference time. BAGEL further supports fine-grained analysis across source domains, taxonomic groups, and knowledge categories, enabling a more precise characterization of model strengths and systematic failure modes. Our benchmark provides a new testbed for studying domain-specific knowledge generalization in language models and for improving their reliability in biodiversity-related applications.

Related papers

• RoBiologyDataChoiceQA: A Romanian Dataset for improving Biology understanding of Large Language Models [0.15293427903448023]
  Large language models (LLMs) have demonstrated significant potential across various natural language processing (NLP) tasks.<n>This study introduces a novel Romanian-language dataset for multiple-choice biology questions.
  arXiv  Detail & Related papers  (2025-09-30T05:41:50Z)
• Animer une base de connaissance: des ontologies aux mod{è}les d'I.A. g{é}n{é}rative [0.0]
  Article proposes a reading of the hybridization between symbolic AI and neural (or sub-symbolic) AI based on a field of application.<n>We describe the LaCAS ecosystem -- Open Archives in Linguistic and Cultural Studies.<n>We illustrate this approach using the knowledge domain ''Languages of the world'' (540 languages) and the knowledge object ''Quechua (language)''
  arXiv  Detail & Related papers  (2025-09-01T09:40:55Z)
• Taxonomic Reasoning for Rare Arthropods: Combining Dense Image Captioning and RAG for Interpretable Classification [12.923336716880506]
  We integrate image captioning and retrieval-augmented generation (RAG) with large language models (LLMs) to enhance biodiversity monitoring.<n>Our findings highlight the potential for modern vision-language AI pipelines to support biodiversity conservation initiatives.
  arXiv  Detail & Related papers  (2025-03-13T21:18:10Z)
• Advancing bioinformatics with large language models: components, applications and perspectives [12.728981464533918]
  Large language models (LLMs) are a class of artificial intelligence models based on deep learning.<n>We will provide a comprehensive overview of the essential components of large language models (LLMs) in bioinformatics.<n>Key aspects covered include tokenization methods for diverse data types, the architecture of transformer models, and the core attention mechanism.
  arXiv  Detail & Related papers  (2024-01-08T17:26:59Z)
• Diversifying Knowledge Enhancement of Biomedical Language Models using Adapter Modules and Knowledge Graphs [54.223394825528665]
  We develop an approach that uses lightweight adapter modules to inject structured biomedical knowledge into pre-trained language models. We use two large KGs, the biomedical knowledge system UMLS and the novel biochemical OntoChem, with two prominent biomedical PLMs, PubMedBERT and BioLinkBERT. We show that our methodology leads to performance improvements in several instances while keeping requirements in computing power low.
  arXiv  Detail & Related papers  (2023-12-21T14:26:57Z)
• The KITMUS Test: Evaluating Knowledge Integration from Multiple Sources in Natural Language Understanding Systems [87.3207729953778]
  We evaluate state-of-the-art coreference resolution models on our dataset. Several models struggle to reason on-the-fly over knowledge observed both at pretrain time and at inference time. Still, even the best performing models seem to have difficulties with reliably integrating knowledge presented only at inference time.
  arXiv  Detail & Related papers  (2022-12-15T23:26:54Z)
• Cetacean Translation Initiative: a roadmap to deciphering the communication of sperm whales [97.41394631426678]
  Recent research showed the promise of machine learning tools for analyzing acoustic communication in nonhuman species. We outline the key elements required for the collection and processing of massive bioacoustic data of sperm whales. The technological capabilities developed are likely to yield cross-applications and advancements in broader communities investigating non-human communication and animal behavioral research.
  arXiv  Detail & Related papers  (2021-04-17T18:39:22Z)
• Linguistic Typology Features from Text: Inferring the Sparse Features of World Atlas of Language Structures [73.06435180872293]
  We construct a recurrent neural network predictor based on byte embeddings and convolutional layers. We show that some features from various linguistic types can be predicted reliably.
  arXiv  Detail & Related papers  (2020-04-30T21:00:53Z)
• Unsupervised Commonsense Question Answering with Self-Talk [71.63983121558843]
  We propose an unsupervised framework based on self-talk as a novel alternative to commonsense tasks. Inspired by inquiry-based discovery learning, our approach inquires language models with a number of information seeking questions. Empirical results demonstrate that the self-talk procedure substantially improves the performance of zero-shot language model baselines.
  arXiv  Detail & Related papers  (2020-04-11T20:43:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.