Automatic Image-Level Morphological Trait Annotation for Organismal Images

• URL: http://arxiv.org/abs/2604.01619v2
• Date: Tue, 07 Apr 2026 01:00:41 GMT
• Title: Automatic Image-Level Morphological Trait Annotation for Organismal Images
• Abstract summary: We introduce a trait annotation pipeline that localizes salient regions and uses vision-language prompting to generate interpretable trait descriptions.<n>We construct Bioscan-Traits, a dataset of 80K trait annotations spanning 19K insect images from BIOSCAN-5M.
• Score: 11.844505008466884
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Morphological traits are physical characteristics of biological organisms that provide vital clues on how organisms interact with their environment. Yet extracting these traits remains a slow, expert-driven process, limiting their use in large-scale ecological studies. A major bottleneck is the absence of high-quality datasets linking biological images to trait-level annotations. In this work, we demonstrate that sparse autoencoders trained on foundation-model features yield monosemantic, spatially grounded neurons that consistently activate on meaningful morphological parts. Leveraging this property, we introduce a trait annotation pipeline that localizes salient regions and uses vision-language prompting to generate interpretable trait descriptions. Using this approach, we construct Bioscan-Traits, a dataset of 80K trait annotations spanning 19K insect images from BIOSCAN-5M. Human evaluation confirms the biological plausibility of the generated morphological descriptions. We assess design sensitivity through a comprehensive ablation study, systematically varying key design choices and measuring their impact on the quality of the resulting trait descriptions. By annotating traits with a modular pipeline rather than prohibitively expensive manual efforts, we offer a scalable way to inject biologically meaningful supervision into foundation models, enable large-scale morphological analyses, and bridge the gap between ecological relevance and machine-learning practicality.

Related papers

• Deep-learning-based pan-phenomic data reveals the explosive evolution of avian visual disparity [8.871465190908003]
  This study uses deep learning techniques to explore avian morphological evolution.<n>We show that the high-dimensional embedding space encodes phenotypic convergence.<n>We also demonstrate that hierarchical semantic structures emerged in the high-dimensional embedding space despite being trained on flat labels.
  arXiv  Detail & Related papers  (2026-02-03T18:32:15Z)
• Classification of Histopathology Slides with Persistence Homology Convolutions [0.0]
  We present a novel method that generates local persistent homology-based data using a modified version of the convolution operator called Persistent Homology Convolutions.<n>These results indicate that persistent homology convolutions extract meaningful geometric information from the histopathology slides.
  arXiv  Detail & Related papers  (2025-07-18T21:56:53Z)
• BioCLIP 2: Emergent Properties from Scaling Hierarchical Contrastive Learning [60.80381372245902]
  We find emergent behaviors in biological vision models via large-scale contrastive vision-language training.<n>We train BioCLIP 2 on TreeOfLife-200M to distinguish different species.<n>We identify emergent properties in the learned embedding space of BioCLIP 2.
  arXiv  Detail & Related papers  (2025-05-29T17:48:20Z)
• Learning biologically relevant features in a pathology foundation model using sparse autoencoders [2.5919097694815365]
  We trained a Sparse Autoencoder on the embeddings of a pathology pretrained foundation model.<n>We found that Sparse Autoencoder features represent interpretable and monosemantic biological concepts.
  arXiv  Detail & Related papers  (2024-07-15T15:03:01Z)
• Neural Echos: Depthwise Convolutional Filters Replicate Biological Receptive Fields [56.69755544814834]
  We present evidence suggesting that depthwise convolutional kernels are effectively replicating the biological receptive fields observed in the mammalian retina. We propose a scheme that draws inspiration from the biological receptive fields.
  arXiv  Detail & Related papers  (2024-01-18T18:06:22Z)
• Tertiary Lymphoid Structures Generation through Graph-based Diffusion [54.37503714313661]
  In this work, we leverage state-of-the-art graph-based diffusion models to generate biologically meaningful cell-graphs. We show that the adopted graph diffusion model is able to accurately learn the distribution of cells in terms of their tertiary lymphoid structures (TLS) content.
  arXiv  Detail & Related papers  (2023-10-10T14:37:17Z)
• Learning multi-scale functional representations of proteins from single-cell microscopy data [77.34726150561087]
  We show that simple convolutional networks trained on localization classification can learn protein representations that encapsulate diverse functional information. We also propose a robust evaluation strategy to assess quality of protein representations across different scales of biological function.
  arXiv  Detail & Related papers  (2022-05-24T00:00:07Z)
• Self-Supervised Vision Transformers Learn Visual Concepts in Histopathology [5.164102666113966]
  We conduct a search for good representations in pathology by training a variety of self-supervised models with validation on a variety of weakly-supervised and patch-level tasks. Our key finding is in discovering that Vision Transformers using DINO-based knowledge distillation are able to learn data-efficient and interpretable features in histology images.
  arXiv  Detail & Related papers  (2022-03-01T16:14:41Z)
• Deep Low-Shot Learning for Biological Image Classification and Visualization from Limited Training Samples [52.549928980694695]
  In situ hybridization (ISH) gene expression pattern images from the same developmental stage are compared. labeling training data with precise stages is very time-consuming even for biologists. We propose a deep two-step low-shot learning framework to accurately classify ISH images using limited training images.
  arXiv  Detail & Related papers  (2020-10-20T06:06:06Z)
• Automatic image-based identification and biomass estimation of invertebrates [70.08255822611812]
  Time-consuming sorting and identification of taxa pose strong limitations on how many insect samples can be processed. We propose to replace the standard manual approach of human expert-based sorting and identification with an automatic image-based technology. We use state-of-the-art Resnet-50 and InceptionV3 CNNs for the classification task.
  arXiv  Detail & Related papers  (2020-02-05T21:38:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.