Deep Visual-Genetic Biometrics for Taxonomic Classification of Rare Species

• URL: http://arxiv.org/abs/2305.06695v3
• Date: Wed, 13 Sep 2023 14:39:32 GMT
• Title: Deep Visual-Genetic Biometrics for Taxonomic Classification of Rare Species
• Authors: Tayfun Karaderi, Tilo Burghardt, Raphael Morard, Daniela Schmidt
• Abstract summary: We propose aligned visual-genetic inference spaces with the aim to implicitly encode cross-domain associations for improved performance. We experimentally demonstrate the efficacy of the concept via application to microscopic imagery of 30k+ planktic foraminifer shells. Visual-genetic alignment can significantly benefit visual-only recognition of the rarest species.
• Score: 1.9819034119774483
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Visual as well as genetic biometrics are routinely employed to identify species and individuals in biological applications. However, no attempts have been made in this domain to computationally enhance visual classification of rare classes with little image data via genetics. In this paper, we thus propose aligned visual-genetic inference spaces with the aim to implicitly encode cross-domain associations for improved performance. We demonstrate for the first time that such alignment can be achieved via deep embedding models and that the approach is directly applicable to boosting long-tailed recognition (LTR) particularly for rare species. We experimentally demonstrate the efficacy of the concept via application to microscopic imagery of 30k+ planktic foraminifer shells across 32 species when used together with independent genetic data samples. Most importantly for practitioners, we show that visual-genetic alignment can significantly benefit visual-only recognition of the rarest species. Technically, we pre-train a visual ResNet50 deep learning model using triplet loss formulations to create an initial embedding space. We re-structure this space based on genetic anchors embedded via a Sequence Graph Transform (SGT) and linked to visual data by cross-domain cosine alignment. We show that an LTR approach improves the state-of-the-art across all benchmarks and that adding our visual-genetic alignment improves per-class and particularly rare tail class benchmarks significantly further. We conclude that visual-genetic alignment can be a highly effective tool for complementing visual biological data containing rare classes. The concept proposed may serve as an important future tool for integrating genetics and imageomics towards a more complete scientific representation of taxonomic spaces and life itself. Code, weights, and data splits are published for full reproducibility.

Related papers

• TEPI: Taxonomy-aware Embedding and Pseudo-Imaging for Scarcely-labeled Zero-shot Genome Classification [0.0]
  A species' genetic code or genome encodes valuable evolutionary, biological, and phylogenetic information. Traditional bioinformatics tools have made notable progress but lack scalability and are computationally expensive. We propose addressing this problem through zero-shot learning using TEPI, taxonomy-aware Embedding and Pseudo-Imaging.
  arXiv  Detail & Related papers  (2024-01-24T04:16:28Z)
• Genetic InfoMax: Exploring Mutual Information Maximization in High-Dimensional Imaging Genetics Studies [50.11449968854487]
  Genome-wide association studies (GWAS) are used to identify relationships between genetic variations and specific traits. Representation learning for imaging genetics is largely under-explored due to the unique challenges posed by GWAS. We introduce a trans-modal learning framework Genetic InfoMax (GIM) to address the specific challenges of GWAS.
  arXiv  Detail & Related papers  (2023-09-26T03:59:21Z)
• Tackling the dimensions in imaging genetics with CLUB-PLS [4.829285448503734]
  Cluster-Bootstrap PLS (CLUB-PLS) provides robust statistics for single input features in both domains. CLUB-PLS investigated the genetic basis of surface area and cortical thickness in a sample of 33,000 subjects from the UK Biobank.
  arXiv  Detail & Related papers  (2023-09-13T23:27:45Z)
• Rethinking Semi-Supervised Medical Image Segmentation: A Variance-Reduction Perspective [51.70661197256033]
  We propose ARCO, a semi-supervised contrastive learning framework with stratified group theory for medical image segmentation. We first propose building ARCO through the concept of variance-reduced estimation and show that certain variance-reduction techniques are particularly beneficial in pixel/voxel-level segmentation tasks. We experimentally validate our approaches on eight benchmarks, i.e., five 2D/3D medical and three semantic segmentation datasets, with different label settings.
  arXiv  Detail & Related papers  (2023-02-03T13:50:25Z)
• Unsupervised ensemble-based phenotyping helps enhance the discoverability of genes related to heart morphology [57.25098075813054]
  We propose a new framework for gene discovery entitled Un Phenotype Ensembles. It builds a redundant yet highly expressive representation by pooling a set of phenotypes learned in an unsupervised manner. These phenotypes are then analyzed via (GWAS), retaining only highly confident and stable associations.
  arXiv  Detail & Related papers  (2023-01-07T18:36:44Z)
• Few-Shot Meta Learning for Recognizing Facial Phenotypes of Genetic Disorders [55.41644538483948]
  Automated classification and similarity retrieval aid physicians in decision-making to diagnose possible genetic conditions as early as possible. Previous work has addressed the problem as a classification problem and used deep learning methods. In this study, we used a facial recognition model trained on a large corpus of healthy individuals as a pre-task and transferred it to facial phenotype recognition.
  arXiv  Detail & Related papers  (2022-10-23T11:52:57Z)
• Taxonomy and evolution predicting using deep learning in images [9.98733710208427]
  This study creates a novel recognition framework by systematically studying the mushroom image recognition problem. We present the first method to map images to DNA, namely used an encoder mapping image to genetic distances, and then decoded DNA through a pre-trained decoder.
  arXiv  Detail & Related papers  (2022-06-28T13:54:14Z)
• Self-Supervised Graph Representation Learning for Neuronal Morphologies [75.38832711445421]
  We present GraphDINO, a data-driven approach to learn low-dimensional representations of 3D neuronal morphologies from unlabeled datasets. We show, in two different species and across multiple brain areas, that this method yields morphological cell type clusterings on par with manual feature-based classification by experts. Our method could potentially enable data-driven discovery of novel morphological features and cell types in large-scale datasets.
  arXiv  Detail & Related papers  (2021-12-23T12:17:47Z)
• Fine-Grained Zero-Shot Learning with DNA as Side Information [31.82132159867097]
  We use DNA as side information for fine-grained zero-shot classification of species. We implement a simple hierarchical Bayesian model that uses DNA information to establish the hierarchy in the image space. We show that DNA can be equally promising yet in general a more accessible alternative than word vectors.
  arXiv  Detail & Related papers  (2021-09-29T01:45:22Z)
• Epigenetic evolution of deep convolutional models [81.21462458089142]
  We build upon a previously proposed neuroevolution framework to evolve deep convolutional models. We propose a convolutional layer layout which allows kernels of different shapes and sizes to coexist within the same layer. The proposed layout enables the size and shape of individual kernels within a convolutional layer to be evolved with a corresponding new mutation operator.
  arXiv  Detail & Related papers  (2021-04-12T12:45:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.