Representing Part-Whole Hierarchies in Foundation Models by Learning Localizability, Composability, and Decomposability from Anatomy via Self-Supervision

• URL: http://arxiv.org/abs/2404.15672v1
• Date: Wed, 24 Apr 2024 06:02:59 GMT
• Title: Representing Part-Whole Hierarchies in Foundation Models by Learning Localizability, Composability, and Decomposability from Anatomy via Self-Supervision
• Authors: Mohammad Reza Hosseinzadeh Taher, Michael B. Gotway, Jianming Liang,
• Abstract summary: We introduce Adam-v2, a new self-supervised learning framework extending Adam [79]. Adam-v2 explicitly incorporates part-whole hierarchies into its learning objectives through three key branches. Experimental results across 10 tasks, compared to 11 baselines in zero-shot, few-shot transfer, and full fine-tuning settings, showcase Adam-v2's superior performance.
• Score: 7.869873154804936
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Humans effortlessly interpret images by parsing them into part-whole hierarchies; deep learning excels in learning multi-level feature spaces, but they often lack explicit coding of part-whole relations, a prominent property of medical imaging. To overcome this limitation, we introduce Adam-v2, a new self-supervised learning framework extending Adam [79] by explicitly incorporating part-whole hierarchies into its learning objectives through three key branches: (1) Localizability, acquiring discriminative representations to distinguish different anatomical patterns; (2) Composability, learning each anatomical structure in a parts-to-whole manner; and (3) Decomposability, comprehending each anatomical structure in a whole-to-parts manner. Experimental results across 10 tasks, compared to 11 baselines in zero-shot, few-shot transfer, and full fine-tuning settings, showcase Adam-v2's superior performance over large-scale medical models and existing SSL methods across diverse downstream tasks. The higher generality and robustness of Adam-v2's representations originate from its explicit construction of hierarchies for distinct anatomical structures from unlabeled medical images. Adam-v2 preserves a semantic balance of anatomical diversity and harmony in its embedding, yielding representations that are both generic and semantically meaningful, yet overlooked in existing SSL methods. All code and pretrained models are available at https://github.com/JLiangLab/Eden.

Related papers

• Hierarchical Text-to-Vision Self Supervised Alignment for Improved Histopathology Representation Learning [64.1316997189396]
  We present a novel language-tied self-supervised learning framework, Hierarchical Language-tied Self-Supervision (HLSS) for histopathology images. Our resulting model achieves state-of-the-art performance on two medical imaging benchmarks, OpenSRH and TCGA datasets.
  arXiv  Detail & Related papers  (2024-03-21T17:58:56Z)
• Robust and Interpretable Medical Image Classifiers via Concept Bottleneck Models [49.95603725998561]
  We propose a new paradigm to build robust and interpretable medical image classifiers with natural language concepts. Specifically, we first query clinical concepts from GPT-4, then transform latent image features into explicit concepts with a vision-language model.
  arXiv  Detail & Related papers  (2023-10-04T21:57:09Z)
• Towards Foundation Models Learned from Anatomy in Medical Imaging via Self-Supervision [8.84494874768244]
  We envision a foundation model for medical imaging that is consciously and purposefully developed upon human anatomy. We devise a novel self-supervised learning (SSL) strategy that exploits the hierarchical nature of human anatomy.
  arXiv  Detail & Related papers  (2023-09-27T01:53:45Z)
• MAP: Domain Generalization via Meta-Learning on Anatomy-Consistent Pseudo-Modalities [12.194439938007672]
  We propose Meta learning on Anatomy-consistent Pseudo-modalities (MAP) MAP improves model generalizability by learning structural features. We evaluate our model on seven public datasets of various retinal imaging modalities.
  arXiv  Detail & Related papers  (2023-09-03T22:56:22Z)
• Anatomical Invariance Modeling and Semantic Alignment for Self-supervised Learning in 3D Medical Image Analysis [6.87667643104543]
  Self-supervised learning (SSL) has recently achieved promising performance for 3D medical image analysis tasks. Most current methods follow existing SSL paradigm originally designed for photographic or natural images. We propose a new self-supervised learning framework, namely Alice, that explicitly fulfills Anatomical invariance modeling and semantic alignment.
  arXiv  Detail & Related papers  (2023-02-11T06:36:20Z)
• Mine yOur owN Anatomy: Revisiting Medical Image Segmentation with Extremely Limited Labels [20.390832929798577]
  We introduce a novel semi-supervised medical image segmentation framework termed Mine yOur owN Anatomy (MONA) First, prior work argues that every pixel equally matters to the model training; we observe empirically that this alone is unlikely to define meaningful anatomical features. Second, we construct a set of objectives that encourage the model to be capable of decomposing medical images into a collection of anatomical features.
  arXiv  Detail & Related papers  (2022-09-27T15:50:31Z)
• LEAP: Learning Articulated Occupancy of People [56.35797895609303]
  We introduce LEAP (LEarning Articulated occupancy of People), a novel neural occupancy representation of the human body. Given a set of bone transformations and a query point in space, LEAP first maps the query point to a canonical space via learned linear blend skinning (LBS) functions. LEAP efficiently queries the occupancy value via an occupancy network that models accurate identity- and pose-dependent deformations in the canonical space.
  arXiv  Detail & Related papers  (2021-04-14T13:41:56Z)
• Generalized Organ Segmentation by Imitating One-shot Reasoning using Anatomical Correlation [55.1248480381153]
  We propose OrganNet which learns a generalized organ concept from a set of annotated organ classes and then transfer this concept to unseen classes. We show that OrganNet can effectively resist the wide variations in organ morphology and produce state-of-the-art results in one-shot segmentation task.
  arXiv  Detail & Related papers  (2021-03-30T13:41:12Z)
• Symbiotic Adversarial Learning for Attribute-based Person Search [86.7506832053208]
  We present a symbiotic adversarial learning framework, called SAL.Two GANs sit at the base of the framework in a symbiotic learning scheme. Specifically, two different types of generative adversarial networks learn collaboratively throughout the training process.
  arXiv  Detail & Related papers  (2020-07-19T07:24:45Z)
• Learning Semantics-enriched Representation via Self-discovery, Self-classification, and Self-restoration [12.609383051645887]
  We train deep models to learn semantically enriched visual representation by self-discovery, self-classification, and self-restoration of the anatomy underneath medical images. We examine our Semantic Genesis with all the publicly-available pre-trained models, by either self-supervision or fully supervision, on the six distinct target tasks. Our experiments demonstrate that Semantic Genesis significantly exceeds all of its 3D counterparts as well as the de facto ImageNet-based transfer learning in 2D.
  arXiv  Detail & Related papers  (2020-07-14T10:36:10Z)
• Global-Local Bidirectional Reasoning for Unsupervised Representation Learning of 3D Point Clouds [109.0016923028653]
  We learn point cloud representation by bidirectional reasoning between the local structures and the global shape without human supervision. We show that our unsupervised model surpasses the state-of-the-art supervised methods on both synthetic and real-world 3D object classification datasets.
  arXiv  Detail & Related papers  (2020-03-29T08:26:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.