Exploring Clustering Capability of Inpainting Model Embeddings for Pattern-based Individual Identification

• URL: http://arxiv.org/abs/2605.04904v1
• Date: Wed, 06 May 2026 13:33:14 GMT
• Title: Exploring Clustering Capability of Inpainting Model Embeddings for Pattern-based Individual Identification
• Abstract summary: We focus on techniques that will make machine learning models more responsive to skin pattern structure.<n>We propose a comparative analysis among four models as an encoder backbone for the individual identification task.<n>We focus on the case study of zebrafish, which is a widely recognized biological model organism.
• Score: 2.227417514684251
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In this paper, we explore deep learning techniques for individual identification of animals based on their skin patterns. Individual identification is crucial in biodiversity monitoring, since it enables analysis of decline or growth of populations, or intra-species interactions within populations. Models trained for the task of individual identification often do not focus on the skin pattern of animals, but on background details or body shape details. These characteristics are not individually specific, or can change drastically through time. We focus on techniques that will make machine learning models more responsive to skin pattern structure when extracting individual visual embeddings from images. For this, we explore image inpainting of task-specific masks as an auxiliary task to enhance ML-based individual identification from animal skin patterns. We propose a comparative analysis among four models as an encoder backbone for the individual identification task. We focus on the case study of zebrafish, which is a widely recognized biological model organism, and which exhibits individually identifying skin patterns. To evaluate encoder backbone performance, we present standard metrics for classification accuracy, embedding clustering metrics, and GradCAM visualizations.

Related papers

• What Does Animal Re-Identification Learn? Linear Biological Concepts and Their Origins in Visual Representations [52.11506532642883]
  We show that sex and age emerge as linear directions that generalize to held-out individuals.<n>We also show that sex direction is causally used by the model, flipping a significant fraction of predictions to the opposite sex.
  arXiv  Detail & Related papers  (2026-09-05T10:46:27Z)
• A non-invasive video-based method for individual identification of wildlife using gait dynamics [21.468912995256698]
  This paper presents a fully automated video-based pipeline for wildlife gait analysis and individual identification.<n>The proposed pipeline uses Segment Anything Model 3 (SAM3) to generate high-quality RGB and binary silhouette masks.<n>Cosine is then used to compare gait signatures, enabling similarity-based clustering of individuals without reliance on physical or invasive markings.
  arXiv  Detail & Related papers  (2026-07-05T21:47:21Z)
• CHIRP dataset: towards long-term, individual-level, behavioral monitoring of bird populations in the wild [2.172490512050822]
  Long-term behavioral monitoring of individual animals is crucial for studying behavioral changes that occur over different time scales.<n>Computer vision methods have proven to benefit biodiversity monitoring, but automated behavior monitoring in wild populations remains challenging.<n>Here, we introduce such a dataset (CHIRP) with a new method (CORVID) for individual re-identification of wild birds.
  arXiv  Detail & Related papers  (2026-03-26T14:57:31Z)
• A neuromorphic model of the insect visual system for natural image processing [0.8602553195689513]
  Insect vision supports complex behaviors including associative learning, navigation, and object detection.<n>Here, we introduce a bio-inspired vision model that transforms dense visual input into sparse, discriminative codes.<n>We evaluate the resulting representations on flower recognition tasks and natural image benchmarks.
  arXiv  Detail & Related papers  (2026-02-06T05:54:28Z)
• Evaluating Multiview Object Consistency in Humans and Image Models [68.36073530804296]
  We leverage an experimental design from the cognitive sciences which requires zero-shot visual inferences about object shape. We collect 35K trials of behavioral data from over 500 participants. We then evaluate the performance of common vision models.
  arXiv  Detail & Related papers  (2024-09-09T17:59:13Z)
• Towards Intrinsic Common Discriminative Features Learning for Face Forgery Detection using Adversarial Learning [59.548960057358435]
  We propose a novel method which utilizes adversarial learning to eliminate the negative effect of different forgery methods and facial identities. Our face forgery detection model learns to extract common discriminative features through eliminating the effect of forgery methods and facial identities.
  arXiv  Detail & Related papers  (2022-07-08T09:23:59Z)
• Revisiting the Shape-Bias of Deep Learning for Dermoscopic Skin Lesion Classification [4.414962444402826]
  It is generally believed that the human visual system is biased towards the recognition of shapes rather than textures. In this paper, we revisit the significance of shape-biases for the classification of skin lesion images.
  arXiv  Detail & Related papers  (2022-06-13T20:59:06Z)
• Persistent Animal Identification Leveraging Non-Visual Markers [71.14999745312626]
  We aim to locate and provide a unique identifier for each mouse in a cluttered home-cage environment through time. This is a very challenging problem due to (i) the lack of distinguishing visual features for each mouse, and (ii) the close confines of the scene with constant occlusion. Our approach achieves 77% accuracy on this animal identification problem, and is able to reject spurious detections when the animals are hidden.
  arXiv  Detail & Related papers  (2021-12-13T17:11:32Z)
• Beyond Tracking: Using Deep Learning to Discover Novel Interactions in Biological Swarms [3.441021278275805]
  We propose training deep network models to predict system-level states directly from generic graphical features from the entire view. Because the resulting predictive models are not based on human-understood predictors, we use explanatory modules. This represents an example of augmented intelligence in behavioral ecology -- knowledge co-creation in a human-AI team.
  arXiv  Detail & Related papers  (2021-08-20T22:50:41Z)
• G-MIND: An End-to-End Multimodal Imaging-Genetics Framework for Biomarker Identification and Disease Classification [49.53651166356737]
  We propose a novel deep neural network architecture to integrate imaging and genetics data, as guided by diagnosis, that provides interpretable biomarkers. We have evaluated our model on a population study of schizophrenia that includes two functional MRI (fMRI) paradigms and Single Nucleotide Polymorphism (SNP) data.
  arXiv  Detail & Related papers  (2021-01-27T19:28:04Z)
• Automatic image-based identification and biomass estimation of invertebrates [70.08255822611812]
  Time-consuming sorting and identification of taxa pose strong limitations on how many insect samples can be processed. We propose to replace the standard manual approach of human expert-based sorting and identification with an automatic image-based technology. We use state-of-the-art Resnet-50 and InceptionV3 CNNs for the classification task.
  arXiv  Detail & Related papers  (2020-02-05T21:38:57Z)
• Learning landmark guided embeddings for animal re-identification [15.356786390476591]
  We propose to improve embedding learning by exploiting body landmarks information explicitly. Body landmarks are provided to the input of a CNN as confidence heatmaps. We evaluate the proposed method on a large synthetic dataset and a small real dataset.
  arXiv  Detail & Related papers  (2020-01-09T01:31:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.