Representing Part-Whole Hierarchies in Foundation Models by Learning Localizability, Composability, and Decomposability from Anatomy via Self-Supervision

• URL: http://arxiv.org/abs/2404.15672v1
• Date: Wed, 24 Apr 2024 06:02:59 GMT
• Title: Representing Part-Whole Hierarchies in Foundation Models by Learning Localizability, Composability, and Decomposability from Anatomy via Self-Supervision
• Authors: Mohammad Reza Hosseinzadeh Taher, Michael B. Gotway, Jianming Liang,
• Abstract summary: We introduce Adam-v2, a new self-supervised learning framework extending Adam [79]. Adam-v2 explicitly incorporates part-whole hierarchies into its learning objectives through three key branches. Experimental results across 10 tasks, compared to 11 baselines in zero-shot, few-shot transfer, and full fine-tuning settings, showcase Adam-v2's superior performance.
• Score: 7.869873154804936
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Humans effortlessly interpret images by parsing them into part-whole hierarchies; deep learning excels in learning multi-level feature spaces, but they often lack explicit coding of part-whole relations, a prominent property of medical imaging. To overcome this limitation, we introduce Adam-v2, a new self-supervised learning framework extending Adam [79] by explicitly incorporating part-whole hierarchies into its learning objectives through three key branches: (1) Localizability, acquiring discriminative representations to distinguish different anatomical patterns; (2) Composability, learning each anatomical structure in a parts-to-whole manner; and (3) Decomposability, comprehending each anatomical structure in a whole-to-parts manner. Experimental results across 10 tasks, compared to 11 baselines in zero-shot, few-shot transfer, and full fine-tuning settings, showcase Adam-v2's superior performance over large-scale medical models and existing SSL methods across diverse downstream tasks. The higher generality and robustness of Adam-v2's representations originate from its explicit construction of hierarchies for distinct anatomical structures from unlabeled medical images. Adam-v2 preserves a semantic balance of anatomical diversity and harmony in its embedding, yielding representations that are both generic and semantically meaningful, yet overlooked in existing SSL methods. All code and pretrained models are available at https://github.com/JLiangLab/Eden.

Related papers

• ACE: Anatomically Consistent Embeddings in Composition and Decomposition [5.939793479232325]
  This paper introduces a novel self-supervised learning (SSL) approach called ACE to learn anatomically consistent embedding via composition and decomposition. Experimental results across 6 datasets 2 backbones, evaluated in few-shot learning, fine-tuning, and property analysis, show ACE's superior robustness, transferability, and clinical potential.
  arXiv  Detail & Related papers  (2025-01-17T11:39:47Z)
• Hierarchical Text-to-Vision Self Supervised Alignment for Improved Histopathology Representation Learning [64.1316997189396]
  We present a novel language-tied self-supervised learning framework, Hierarchical Language-tied Self-Supervision (HLSS) for histopathology images. Our resulting model achieves state-of-the-art performance on two medical imaging benchmarks, OpenSRH and TCGA datasets.
  arXiv  Detail & Related papers  (2024-03-21T17:58:56Z)
• Towards Foundation Models Learned from Anatomy in Medical Imaging via Self-Supervision [8.84494874768244]
  We envision a foundation model for medical imaging that is consciously and purposefully developed upon human anatomy. We devise a novel self-supervised learning (SSL) strategy that exploits the hierarchical nature of human anatomy.
  arXiv  Detail & Related papers  (2023-09-27T01:53:45Z)
• Anatomical Invariance Modeling and Semantic Alignment for Self-supervised Learning in 3D Medical Image Analysis [6.87667643104543]
  Self-supervised learning (SSL) has recently achieved promising performance for 3D medical image analysis tasks. Most current methods follow existing SSL paradigm originally designed for photographic or natural images. We propose a new self-supervised learning framework, namely Alice, that explicitly fulfills Anatomical invariance modeling and semantic alignment.
  arXiv  Detail & Related papers  (2023-02-11T06:36:20Z)
• Mine yOur owN Anatomy: Revisiting Medical Image Segmentation with Extremely Limited Labels [54.58539616385138]
  We introduce a novel semi-supervised 2D medical image segmentation framework termed Mine yOur owN Anatomy (MONA) First, prior work argues that every pixel equally matters to the model training; we observe empirically that this alone is unlikely to define meaningful anatomical features. Second, we construct a set of objectives that encourage the model to be capable of decomposing medical images into a collection of anatomical features.
  arXiv  Detail & Related papers  (2022-09-27T15:50:31Z)
• LEAP: Learning Articulated Occupancy of People [56.35797895609303]
  We introduce LEAP (LEarning Articulated occupancy of People), a novel neural occupancy representation of the human body. Given a set of bone transformations and a query point in space, LEAP first maps the query point to a canonical space via learned linear blend skinning (LBS) functions. LEAP efficiently queries the occupancy value via an occupancy network that models accurate identity- and pose-dependent deformations in the canonical space.
  arXiv  Detail & Related papers  (2021-04-14T13:41:56Z)
• Generalized Organ Segmentation by Imitating One-shot Reasoning using Anatomical Correlation [55.1248480381153]
  We propose OrganNet which learns a generalized organ concept from a set of annotated organ classes and then transfer this concept to unseen classes. We show that OrganNet can effectively resist the wide variations in organ morphology and produce state-of-the-art results in one-shot segmentation task.
  arXiv  Detail & Related papers  (2021-03-30T13:41:12Z)
• Symbiotic Adversarial Learning for Attribute-based Person Search [86.7506832053208]
  We present a symbiotic adversarial learning framework, called SAL.Two GANs sit at the base of the framework in a symbiotic learning scheme. Specifically, two different types of generative adversarial networks learn collaboratively throughout the training process.
  arXiv  Detail & Related papers  (2020-07-19T07:24:45Z)
• Learning Semantics-enriched Representation via Self-discovery, Self-classification, and Self-restoration [12.609383051645887]
  We train deep models to learn semantically enriched visual representation by self-discovery, self-classification, and self-restoration of the anatomy underneath medical images. We examine our Semantic Genesis with all the publicly-available pre-trained models, by either self-supervision or fully supervision, on the six distinct target tasks. Our experiments demonstrate that Semantic Genesis significantly exceeds all of its 3D counterparts as well as the de facto ImageNet-based transfer learning in 2D.
  arXiv  Detail & Related papers  (2020-07-14T10:36:10Z)
• Global-Local Bidirectional Reasoning for Unsupervised Representation Learning of 3D Point Clouds [109.0016923028653]
  We learn point cloud representation by bidirectional reasoning between the local structures and the global shape without human supervision. We show that our unsupervised model surpasses the state-of-the-art supervised methods on both synthetic and real-world 3D object classification datasets.
  arXiv  Detail & Related papers  (2020-03-29T08:26:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.