Hespi: A pipeline for automatically detecting information from hebarium specimen sheets

• URL: http://arxiv.org/abs/2410.08740v2
• Date: Wed, 09 Jul 2025 03:54:32 GMT
• Title: Hespi: A pipeline for automatically detecting information from hebarium specimen sheets
• Abstract summary: Hespi extracts pre-catalogue data from primary specimen labels on herbarium specimens.<n>It classifies labels as printed, typed, handwritten, or mixed.<n>Hespi accurately detects and extracts text from specimen sheets across international herbaria.
• Score: 6.749750044497731
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Specimen-associated biodiversity data are crucial for biological, environmental, and conservation sciences. A rate shift is needed to extract data from specimen images efficiently, moving beyond human-mediated transcription. We developed `Hespi' (HErbarium Specimen sheet PIpeline) using advanced computer vision techniques to extract pre-catalogue data from primary specimen labels on herbarium specimens. Hespi integrates two object detection models: one for detecting the components of the sheet and another for fields on the primary primary specimen label. It classifies labels as printed, typed, handwritten, or mixed and uses Optical Character Recognition (OCR) and Handwritten Text Recognition (HTR) for extraction. The text is then corrected against authoritative taxon databases and refined using a multimodal Large Language Model (LLM). Hespi accurately detects and extracts text from specimen sheets across international herbaria, and its modular design allows users to train and integrate custom models.

Related papers

• Automated pipeline for herbarium label digitization [2.3991879782778853]
  HERBIOME is an end-to-end pipeline for automated herbarium label digitization.<n>It integrates YOLOv8-based component detection, CRAFT Hezar word-level text localization, fine-tuned TrOCR for recognition of handwritten and printed text, and GPT-4o Mini for semantic metadata structuring into standardized fields.
  arXiv  Detail & Related papers  (2026-08-25T14:01:32Z)
• Automated Histopathology Report Generation via Pyramidal Feature Extraction and the UNI Foundation Model [0.0]
  We propose a hierarchical vision language framework that combines a frozen pathology foundation model with a Transformer decoder for report generation.<n>To make WSI processing tractable, we perform multi resolution pyramidal patch selection and remove background and artifacts.<n>We tokenize the output using BioGPT to better represent biomedical terminology.
  arXiv  Detail & Related papers  (2026-02-18T12:55:20Z)
• RxnCaption: Reformulating Reaction Diagram Parsing as Visual Prompt Guided Captioning [51.393018266721576]
  We propose the RxnCaption framework for the task of chemical Reaction Diagram Parsing (RxnDP)<n>Our framework reformulates the traditional coordinate prediction driven parsing process into an image captioning problem.<n>We introduce a strategy termed "BBox and Index as Visual Prompt" (BIVP), which uses our state-of-the-art molecular detector, MolYOLO, to pre-draw molecular bounding boxes and indices directly onto the input image.
  arXiv  Detail & Related papers  (2025-11-04T09:08:44Z)
• Vision-Language Models are Strong Noisy Label Detectors [76.07846780815794]
  This paper presents a Denoising Fine-Tuning framework, called DeFT, for adapting vision-language models. DeFT utilizes the robust alignment of textual and visual features pre-trained on millions of auxiliary image-text pairs to sieve out noisy labels. Experimental results on seven synthetic and real-world noisy datasets validate the effectiveness of DeFT in both noisy label detection and image classification.
  arXiv  Detail & Related papers  (2024-09-29T12:55:17Z)
• Downstream-Pretext Domain Knowledge Traceback for Active Learning [138.02530777915362]
  We propose a downstream-pretext domain knowledge traceback (DOKT) method that traces the data interactions of downstream knowledge and pre-training guidance. DOKT consists of a traceback diversity indicator and a domain-based uncertainty estimator. Experiments conducted on ten datasets show that our model outperforms other state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-07-20T01:34:13Z)
• Human-in-the-Loop Synthetic Text Data Inspection with Provenance Tracking [11.022295941449919]
  We develop INSPECTOR, a human-in-the-loop data inspection technique. In a user study, INSPECTOR increases the number of texts with correct labels identified by 3X on a sentiment analysis task and by 4X on a hate speech detection task.
  arXiv  Detail & Related papers  (2024-04-29T17:16:27Z)
• Fact Checking Beyond Training Set [64.88575826304024]
  We show that the retriever-reader suffers from performance deterioration when it is trained on labeled data from one domain and used in another domain. We propose an adversarial algorithm to make the retriever component robust against distribution shift. We then construct eight fact checking scenarios from these datasets, and compare our model to a set of strong baseline models.
  arXiv  Detail & Related papers  (2024-03-27T15:15:14Z)
• PEaCE: A Chemistry-Oriented Dataset for Optical Character Recognition on Scientific Documents [4.191058827240492]
  We present the Printed English and Chemical Equations (PEaCE) dataset, containing both synthetic and real-world records. We evaluate the efficacy of transformer-based OCR models when trained on this resource.
  arXiv  Detail & Related papers  (2024-03-23T05:20:36Z)
• Named Entity Recognition for Address Extraction in Speech-to-Text Transcriptions Using Synthetic Data [0.0]
  This paper introduces an approach for building a Named Entity Recognition (NER) model built upon a Bidirectional Representations from Transformers (BERT) architecture. This NER model extracts address parts from data acquired from speech-to-text transcriptions. The performance of our NER model, trained solely on synthetic data, is evaluated using small real test dataset.
  arXiv  Detail & Related papers  (2024-02-08T10:29:11Z)
• Enhancing Scene Text Detectors with Realistic Text Image Synthesis Using Diffusion Models [63.99110667987318]
  We present DiffText, a pipeline that seamlessly blends foreground text with the background's intrinsic features. With fewer text instances, our produced text images consistently surpass other synthetic data in aiding text detectors.
  arXiv  Detail & Related papers  (2023-11-28T06:51:28Z)
• Automated Annotation of Scientific Texts for ML-based Keyphrase Extraction and Validation [0.0]
  We present two novel automated text labeling approaches for the validation of ML-generated metadata for unlabeled texts. Our techniques show the potential of two new ways to leverage existing information about the unlabeled texts and the scientific domain.
  arXiv  Detail & Related papers  (2023-11-08T22:09:31Z)
• Self-Supervised Representation Learning for Online Handwriting Text Classification [0.8594140167290099]
  We propose the novel Part of Stroke Masking (POSM) as a pretext task for pretraining models to extract informative representations from the online handwriting of individuals in English and Chinese languages. To evaluate the quality of the extracted representations, we use both intrinsic and extrinsic evaluation methods. The pretrained models are fine-tuned to achieve state-of-the-art results in tasks such as writer identification, gender classification, and handedness classification.
  arXiv  Detail & Related papers  (2023-10-10T14:07:49Z)
• Domain Adaptive Synapse Detection with Weak Point Annotations [63.97144211520869]
  We present AdaSyn, a framework for domain adaptive synapse detection with weak point annotations. In the WASPSYN challenge at I SBI 2023, our method ranks the 1st place.
  arXiv  Detail & Related papers  (2023-08-31T05:05:53Z)
• DPIC: Decoupling Prompt and Intrinsic Characteristics for LLM Generated Text Detection [56.513637720967566]
  Large language models (LLMs) can generate texts that pose risks of misuse, such as plagiarism, planting fake reviews on e-commerce platforms, or creating inflammatory false tweets. Existing high-quality detection methods usually require access to the interior of the model to extract the intrinsic characteristics. We propose to extract deep intrinsic characteristics of the black-box model generated texts.
  arXiv  Detail & Related papers  (2023-05-21T17:26:16Z)
• TextDiffuser: Diffusion Models as Text Painters [118.30923824681642]
  We introduce TextDiffuser, focusing on generating images with visually appealing text that is coherent with backgrounds. We contribute the first large-scale text images dataset with OCR annotations, MARIO-10M, containing 10 million image-text pairs. We show that TextDiffuser is flexible and controllable to create high-quality text images using text prompts alone or together with text template images, and conduct text inpainting to reconstruct incomplete images with text.
  arXiv  Detail & Related papers  (2023-05-18T10:16:19Z)
• BeCAPTCHA-Type: Biometric Keystroke Data Generation for Improved Bot Detection [63.447493500066045]
  This work proposes a data driven learning model for the synthesis of keystroke biometric data. The proposed method is compared with two statistical approaches based on Universal and User-dependent models. Our experimental framework considers a dataset with 136 million keystroke events from 168 thousand subjects.
  arXiv  Detail & Related papers  (2022-07-27T09:26:15Z)
• Detecting Text Formality: A Study of Text Classification Approaches [78.11745751651708]
  This work proposes the first to our knowledge systematic study of formality detection methods based on statistical, neural-based, and Transformer-based machine learning methods. We conducted three types of experiments -- monolingual, multilingual, and cross-lingual. The study shows the overcome of Char BiLSTM model over Transformer-based ones for the monolingual and multilingual formality classification task.
  arXiv  Detail & Related papers  (2022-04-19T16:23:07Z)
• Slot Filling for Biomedical Information Extraction [0.5330240017302619]
  We present a slot filling approach to the task of biomedical IE. We follow the proposed paradigm of coupling a Tranformer-based bi-encoder, Dense Passage Retrieval, with a Transformer-based reader model.
  arXiv  Detail & Related papers  (2021-09-17T14:16:00Z)
• Detecting Handwritten Mathematical Terms with Sensor Based Data [71.84852429039881]
  We propose a solution to the UbiComp 2021 Challenge by Stabilo in which handwritten mathematical terms are supposed to be automatically classified. The input data set contains data of different writers, with label strings constructed from a total of 15 different possible characters.
  arXiv  Detail & Related papers  (2021-09-12T19:33:34Z)
• Object Detection Based Handwriting Localization [2.6641834518599308]
  We present an object detection based approach to localize handwritten regions from documents. The proposed approach is also expected to facilitate other tasks such as handwriting recognition and signature verification.
  arXiv  Detail & Related papers  (2021-06-28T21:25:20Z)
• DUET: Detection Utilizing Enhancement for Text in Scanned or Captured Documents [1.4866448722906016]
  Our proposed model is designed to perform noise reduction and text region enhancement as well as text detection. We enrich the training data for the model with synthesized document images that are fully labeled for text detection and enhancement. Our methods are demonstrated in a real document dataset with performances exceeding those of other text detection methods.
  arXiv  Detail & Related papers  (2021-06-10T07:08:31Z)
• PharmKE: Knowledge Extraction Platform for Pharmaceutical Texts using Transfer Learning [0.0]
  PharmKE is a text analysis platform that applies deep learning through several stages for thorough semantic analysis of pharmaceutical articles. The methodology is used to create accurately labeled training and test datasets, which are then used to train models for custom entity labeling tasks. The obtained results are compared to the fine-tuned BERT and BioBERT models trained on the same dataset.
  arXiv  Detail & Related papers  (2021-02-25T19:36:35Z)
• Text Mining to Identify and Extract Novel Disease Treatments From Unstructured Datasets [56.38623317907416]
  We use Google Cloud to transcribe podcast episodes of an NPR radio show. We then build a pipeline for systematically pre-processing the text. Our model successfully identified that Omeprazole can help treat heartburn.
  arXiv  Detail & Related papers  (2020-10-22T19:52:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.