Pin the Memory: Learning to Generalize Semantic Segmentation

• URL: http://arxiv.org/abs/2204.03609v1
• Date: Thu, 7 Apr 2022 17:34:01 GMT
• Title: Pin the Memory: Learning to Generalize Semantic Segmentation
• Authors: Jin Kim, Jiyoung Lee, Jungin Park, Dongbo Min, Kwanghoon Sohn
• Abstract summary: We present a novel memory-guided domain generalization method for semantic segmentation based on meta-learning framework. Our method abstracts the conceptual knowledge of semantic classes into categorical memory which is constant beyond the domains.
• Score: 68.367763672095
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rise of deep neural networks has led to several breakthroughs for semantic segmentation. In spite of this, a model trained on source domain often fails to work properly in new challenging domains, that is directly concerned with the generalization capability of the model. In this paper, we present a novel memory-guided domain generalization method for semantic segmentation based on meta-learning framework. Especially, our method abstracts the conceptual knowledge of semantic classes into categorical memory which is constant beyond the domains. Upon the meta-learning concept, we repeatedly train memory-guided networks and simulate virtual test to 1) learn how to memorize a domain-agnostic and distinct information of classes and 2) offer an externally settled memory as a class-guidance to reduce the ambiguity of representation in the test data of arbitrary unseen domain. To this end, we also propose memory divergence and feature cohesion losses, which encourage to learn memory reading and update processes for category-aware domain generalization. Extensive experiments for semantic segmentation demonstrate the superior generalization capability of our method over state-of-the-art works on various benchmarks.

Related papers

• Memory-Efficient Prompt Tuning for Incremental Histopathology Classification [69.46798702300042]
  We present a memory-efficient prompt tuning framework to cultivate model generalization potential in economical memory cost. We have extensively evaluated our framework with two histopathology tasks, i.e., breast cancer metastasis classification and epithelium-stroma tissue classification.
  arXiv  Detail & Related papers  (2024-01-22T03:24:45Z)
• Memory-Guided Semantic Learning Network for Temporal Sentence Grounding [55.31041933103645]
  We propose a memory-augmented network that learns and memorizes the rarely appeared content in TSG tasks. MGSL-Net consists of three main parts: a cross-modal inter-action module, a memory augmentation module, and a heterogeneous attention module.
  arXiv  Detail & Related papers  (2022-01-03T02:32:06Z)
• Meta-Learned Feature Critics for Domain Generalized Semantic Segmentation [38.81908956978064]
  We propose a novel meta-learning scheme with feature disentanglement ability, which derives domain-invariant features for semantic segmentation with domain generalization guarantees. Our results on benchmark datasets confirm the effectiveness and robustness of our proposed model.
  arXiv  Detail & Related papers  (2021-12-27T06:43:39Z)
• Hierarchical Variational Memory for Few-shot Learning Across Domains [120.87679627651153]
  We introduce a hierarchical prototype model, where each level of the prototype fetches corresponding information from the hierarchical memory. The model is endowed with the ability to flexibly rely on features at different semantic levels if the domain shift circumstances so demand. We conduct thorough ablation studies to demonstrate the effectiveness of each component in our model.
  arXiv  Detail & Related papers  (2021-12-15T15:01:29Z)
• A Style and Semantic Memory Mechanism for Domain Generalization [108.98041306507372]
  Intra-domain style invariance is of pivotal importance in improving the efficiency of domain generalization. We propose a novel "jury" mechanism, which is particularly effective in learning useful semantic feature commonalities among domains. Our proposed framework surpasses the state-of-the-art methods by clear margins.
  arXiv  Detail & Related papers  (2021-12-14T16:23:24Z)
• Semi-supervised Meta-learning with Disentanglement for Domain-generalised Medical Image Segmentation [15.351113774542839]
  Generalising models to new data from new centres (termed here domains) remains a challenge. We propose a novel semi-supervised meta-learning framework with disentanglement. We show that the proposed method is robust on different segmentation tasks and achieves state-of-the-art generalisation performance on two public benchmarks.
  arXiv  Detail & Related papers  (2021-06-24T19:50:07Z)
• Towards Recognizing New Semantic Concepts in New Visual Domains [9.701036831490768]
  We argue that it is crucial to design deep architectures that can operate in previously unseen visual domains and recognize novel semantic concepts. In the first part of the thesis, we describe different solutions to enable deep models to generalize to new visual domains. In the second part, we show how to extend the knowledge of a pretrained deep model to new semantic concepts, without access to the original training set.
  arXiv  Detail & Related papers  (2020-12-16T16:23:40Z)
• Learning to Learn Variational Semantic Memory [132.39737669936125]
  We introduce variational semantic memory into meta-learning to acquire long-term knowledge for few-shot learning. The semantic memory is grown from scratch and gradually consolidated by absorbing information from tasks it experiences. We formulate memory recall as the variational inference of a latent memory variable from addressed contents.
  arXiv  Detail & Related papers  (2020-10-20T15:05:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.