Distilled Reverse Attention Network for Open-world Compositional Zero-Shot Learning

• URL: http://arxiv.org/abs/2303.00404v1
• Date: Wed, 1 Mar 2023 10:52:20 GMT
• Title: Distilled Reverse Attention Network for Open-world Compositional Zero-Shot Learning
• Authors: Yun Li, Zhe Liu, Saurav Jha, Sally Cripps, Lina Yao
• Abstract summary: Open-World Compositional Zero-Shot Learning (OW-CZSL) aims to recognize new compositions of seen attributes and objects. OW-CZSL methods built on the conventional closed-world setting degrade severely due to the unconstrained OW test space. We propose a novel Distilled Reverse Attention Network to address the challenges.
• Score: 42.138756191997295
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Open-World Compositional Zero-Shot Learning (OW-CZSL) aims to recognize new compositions of seen attributes and objects. In OW-CZSL, methods built on the conventional closed-world setting degrade severely due to the unconstrained OW test space. While previous works alleviate the issue by pruning compositions according to external knowledge or correlations in seen pairs, they introduce biases that harm the generalization. Some methods thus predict state and object with independently constructed and trained classifiers, ignoring that attributes are highly context-dependent and visually entangled with objects. In this paper, we propose a novel Distilled Reverse Attention Network to address the challenges. We also model attributes and objects separately but with different motivations, capturing contextuality and locality, respectively. We further design a reverse-and-distill strategy that learns disentangled representations of elementary components in training data supervised by reverse attention and knowledge distillation. We conduct experiments on three datasets and consistently achieve state-of-the-art (SOTA) performance.

Related papers

• Attention Based Simple Primitives for Open World Compositional Zero-Shot Learning [12.558701595138928]
  Compositional Zero-Shot Learning (CZSL) aims to predict unknown compositions made up of attribute and object pairs. We are exploring Open World Compositional Zero-Shot Learning (OW-CZSL) in this study, where our test space encompasses all potential combinations of attributes and objects. Our approach involves utilizing the self-attention mechanism between attributes and objects to achieve better generalization from seen to unseen compositions.
  arXiv  Detail & Related papers  (2024-07-18T17:11:29Z)
• Contextual Interaction via Primitive-based Adversarial Training For Compositional Zero-shot Learning [23.757252768668497]
  Compositional Zero-shot Learning (CZSL) aims to identify novel compositions via known attribute-object pairs. The primary challenge in CZSL tasks lies in the significant discrepancies introduced by the complex interaction between the visual primitives of attribute and object. We propose a model-agnostic and Primitive-Based Adversarial training (PBadv) method to deal with this problem.
  arXiv  Detail & Related papers  (2024-06-21T08:18:30Z)
• Mitigating Object Dependencies: Improving Point Cloud Self-Supervised Learning through Object Exchange [50.45953583802282]
  We introduce a novel self-supervised learning (SSL) strategy for point cloud scene understanding. Our approach leverages both object patterns and contextual cues to produce robust features. Our experiments demonstrate the superiority of our method over existing SSL techniques.
  arXiv  Detail & Related papers  (2024-04-11T06:39:53Z)
• Hierarchical Visual Primitive Experts for Compositional Zero-Shot Learning [52.506434446439776]
  Compositional zero-shot learning (CZSL) aims to recognize compositions with prior knowledge of known primitives (attribute and object) We propose a simple and scalable framework called Composition Transformer (CoT) to address these issues. Our method achieves SoTA performance on several benchmarks, including MIT-States, C-GQA, and VAW-CZSL.
  arXiv  Detail & Related papers  (2023-08-08T03:24:21Z)
• Weakly-supervised Contrastive Learning for Unsupervised Object Discovery [52.696041556640516]
  Unsupervised object discovery is promising due to its ability to discover objects in a generic manner. We design a semantic-guided self-supervised learning model to extract high-level semantic features from images. We introduce Principal Component Analysis (PCA) to localize object regions.
  arXiv  Detail & Related papers  (2023-07-07T04:03:48Z)
• ProCC: Progressive Cross-primitive Compatibility for Open-World Compositional Zero-Shot Learning [29.591615811894265]
  Open-World Compositional Zero-shot Learning (OW-CZSL) aims to recognize novel compositions of state and object primitives in images with no priors on the compositional space. We propose a novel method, termed Progressive Cross-primitive Compatibility (ProCC), to mimic the human learning process for OW-CZSL tasks.
  arXiv  Detail & Related papers  (2022-11-19T10:09:46Z)
• Siamese Contrastive Embedding Network for Compositional Zero-Shot Learning [76.13542095170911]
  Compositional Zero-Shot Learning (CZSL) aims to recognize unseen compositions formed from seen state and object during training. We propose a novel Siamese Contrastive Embedding Network (SCEN) for unseen composition recognition. Our method significantly outperforms the state-of-the-art approaches on three challenging benchmark datasets.
  arXiv  Detail & Related papers  (2022-06-29T09:02:35Z)
• Disentangling Visual Embeddings for Attributes and Objects [38.27308243429424]
  We study the problem of compositional zero-shot learning for object-attribute recognition. Prior works use visual features extracted with a backbone network, pre-trained for object classification. We propose a novel architecture that can disentangle attribute and object features in the visual space.
  arXiv  Detail & Related papers  (2022-05-17T17:59:36Z)
• KG-SP: Knowledge Guided Simple Primitives for Open World Compositional Zero-Shot Learning [52.422873819371276]
  The goal of open-world compositional zero-shot learning (OW-CZSL) is to recognize compositions of state and objects in images. Here, we revisit a simple CZSL baseline and predict the primitives, i.e. states and objects, independently. We estimate the feasibility of each composition through external knowledge, using this prior to remove unfeasible compositions from the output space. Our model, Knowledge-Guided Simple Primitives (KG-SP), achieves state of the art in both OW-CZSL and pCZSL.
  arXiv  Detail & Related papers  (2022-05-13T17:18:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.