Automated pipeline for herbarium label digitization

• URL: http://arxiv.org/abs/2608.28676v1
• Date: Tue, 25 Aug 2026 14:01:32 GMT
• Title: Automated pipeline for herbarium label digitization
• Abstract summary: HERBIOME is an end-to-end pipeline for automated herbarium label digitization.<n>It integrates YOLOv8-based component detection, CRAFT Hezar word-level text localization, fine-tuned TrOCR for recognition of handwritten and printed text, and GPT-4o Mini for semantic metadata structuring into standardized fields.
• Score: 2.3991879782778853
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Digitized herbarium collections, now comprising over 100 million freely accessible specimen images, have become a critical resource for addressing fundamental questions in ecology and evolutionary biology. Yet the rich metadata encoded in herbarium labels (collector identities, geographic localities, collection dates, and ecological observations) remains largely inaccessible at scale, constraining both biodiversity informatics and the construction of specimen-specific image-text corpora for multimodal AI. We present HERBIOME, a modular end-to-end pipeline for automated herbarium label digitization, integrating YOLOv8-based component detection, CRAFT Hezar word-level text localization, fine-tuned TrOCR for recognition of mixed handwritten and printed text, and GPT-4o Mini for semantic metadata structuring into standardized fields. TrOCR was trained on a multi-source dataset combining general transcription corpora (CREMMA-AN, PictoCatalogs) with herbarium-specific data (RéColNat), achieving a Character Error Rate of 4.05-4.10%. End-to-end evaluation on 450 French herbarium specimens, using a dual-metric framework of Maximum Window Similarity (MWS: 0.614-0.618) and Semantic Metadata Accuracy (SMA: 0.440-0.445), reveals that hybrid training strategies improve semantic fidelity while random sampling maximizes surface similarity, with taxonomic fields remaining the principal bottleneck. By automating the extraction of structured metadata from complex, heterogeneous labels, HERBIOME reduces transcription burden, enables the construction of paired image-text datasets that faithfully capture specimen individuality, which is a prerequisite for next-generation multimodal biodiversity AI systems.

Related papers

• Million-scale multimodal pollen microscopy with expert-guided foundation models [0.0]
  Pollen AI Atlas is a million-scale multimodal pollen microscopy resource assembled from pure-slide bright-field images.<n>1,511,390 released grain detections with 99.6% proposal precision in expert-curated test regions.<n>Gemma4 provided the most controlled primary caption set, combining tight length control, no leakage and the strongest text-retrieval performance.
  arXiv  Detail & Related papers  (2026-06-16T11:35:27Z)
• EmbGen: Teaching with Reassembled Corpora [0.300988853836121]
  We introduce EmbGen, a synthetic data generation pipeline that decomposes a corpus into entity-description pairs.<n>We evaluate EmbGen against EntiGraph, InstructLab and Knowledge-Instruct on three datasets of varied semantic heterogeneity.
  arXiv  Detail & Related papers  (2026-05-19T05:40:12Z)
• RANGER: Sparsely-Gated Mixture-of-Experts with Adaptive Retrieval Re-ranking for Pathology Report Generation [5.081261104622565]
  RANGER is a sparsely-gated Mixture-of-Experts (MoE) framework with adaptive retrieval re-ranking for pathology report generation.<n>We demonstrate consistent improvements over existing approaches across standard natural language generation metrics.<n>Our full RANGER model achieves optimal performance on PathText dataset, reaching BLEU-1 to BLEU-4 scores of 0.4598, 0.3044, 0.2036, and 0.1435, respectively.
  arXiv  Detail & Related papers  (2026-03-04T18:12:31Z)
• MergeDNA: Context-aware Genome Modeling with Dynamic Tokenization through Token Merging [65.07273789940116]
  This paper introduces a hierarchical architecture that jointly optimize a dynamic genomic tokenizer and latent Transformers with context-aware pre-training tasks.<n> MergeDNA achieves superior performance on three popular DNA benchmarks and several multi-omics tasks with fine-tuning or zero-shot evaluation.
  arXiv  Detail & Related papers  (2025-11-17T19:27:41Z)
• AutoMalDesc: Large-Scale Script Analysis for Cyber Threat Research [81.04845910798387]
  Generating natural language explanations for threat detections remains an open problem in cybersecurity research.<n>We present AutoMalDesc, an automated static analysis summarization framework that operates independently at scale.<n>We publish our complete dataset of more than 100K script samples, including annotated seed (0.9K) datasets, along with our methodology and evaluation framework.
  arXiv  Detail & Related papers  (2025-11-17T13:05:25Z)
• HySemRAG: A Hybrid Semantic Retrieval-Augmented Generation Framework for Automated Literature Synthesis and Methodological Gap Analysis [55.2480439325792]
  HySemRAG is a framework that combines Extract, Transform, Load (ETL) pipelines with Retrieval-Augmented Generation (RAG)<n>System addresses limitations in existing RAG architectures through a multi-layered approach.
  arXiv  Detail & Related papers  (2025-08-01T20:30:42Z)
• Hespi: A pipeline for automatically detecting information from hebarium specimen sheets [6.749750044497731]
  Hespi extracts pre-catalogue data from primary specimen labels on herbarium specimens.<n>It classifies labels as printed, typed, handwritten, or mixed.<n>Hespi accurately detects and extracts text from specimen sheets across international herbaria.
  arXiv  Detail & Related papers  (2024-10-11T11:59:40Z)
• Improving Text Embeddings with Large Language Models [59.930513259982725]
  We introduce a novel and simple method for obtaining high-quality text embeddings using only synthetic data and less than 1k training steps. We leverage proprietary LLMs to generate diverse synthetic data for hundreds of thousands of text embedding tasks across 93 languages. Experiments demonstrate that our method achieves strong performance on highly competitive text embedding benchmarks without using any labeled data.
  arXiv  Detail & Related papers  (2023-12-31T02:13:18Z)
• Relation Extraction in underexplored biomedical domains: A diversity-optimised sampling and synthetic data generation approach [0.0]
  sparsity of labelled data is an obstacle to the development of Relation Extraction models. We create the first curated evaluation dataset and extracted literature items from the LOTUS database to build training sets. We evaluate the performance of standard fine-tuning as a generative task and few-shot learning with open Large Language Models.
  arXiv  Detail & Related papers  (2023-11-10T19:36:00Z)
• Generating More Pertinent Captions by Leveraging Semantics and Style on Multi-Source Datasets [56.018551958004814]
  This paper addresses the task of generating fluent descriptions by training on a non-uniform combination of data sources. Large-scale datasets with noisy image-text pairs provide a sub-optimal source of supervision. We propose to leverage and separate semantics and descriptive style through the incorporation of a style token and keywords extracted through a retrieval component.
  arXiv  Detail & Related papers  (2021-11-24T19:00:05Z)
• G-MIND: An End-to-End Multimodal Imaging-Genetics Framework for Biomarker Identification and Disease Classification [49.53651166356737]
  We propose a novel deep neural network architecture to integrate imaging and genetics data, as guided by diagnosis, that provides interpretable biomarkers. We have evaluated our model on a population study of schizophrenia that includes two functional MRI (fMRI) paradigms and Single Nucleotide Polymorphism (SNP) data.
  arXiv  Detail & Related papers  (2021-01-27T19:28:04Z)
• Towards an Automatic Analysis of CHO-K1 Suspension Growth in Microfluidic Single-cell Cultivation [63.94623495501023]
  We propose a novel Machine Learning architecture, which allows us to infuse a neural deep network with human-powered abstraction on the level of data. Specifically, we train a generative model simultaneously on natural and synthetic data, so that it learns a shared representation, from which a target variable, such as the cell count, can be reliably estimated.
  arXiv  Detail & Related papers  (2020-10-20T08:36:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.