Ontology-aware Network for Zero-shot Sketch-based Image Retrieval
- URL: http://arxiv.org/abs/2302.10040v1
- Date: Mon, 20 Feb 2023 15:44:41 GMT
- Title: Ontology-aware Network for Zero-shot Sketch-based Image Retrieval
- Authors: Haoxiang Zhang, He Jiang, Ziqiang Wang, Deqiang Cheng
- Abstract summary: Zero-Shot Sketch-Based Image Retrieval (ZSSBIR) is an emerging task.
Recent work has begun to consider the triplet-based or contrast-based loss to mine inter-class information.
An Ontology-Aware Network (OAN) is proposed to respond to these issues.
- Score: 6.40453257995913
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Zero-Shot Sketch-Based Image Retrieval (ZSSBIR) is an emerging task. The
pioneering work focused on the modal gap but ignored inter-class information.
Although recent work has begun to consider the triplet-based or contrast-based
loss to mine inter-class information, positive and negative samples need to be
carefully selected, or the model is prone to lose modality-specific
information. To respond to these issues, an Ontology-Aware Network (OAN) is
proposed. Specifically, the smooth inter-class independence learning mechanism
is put forward to maintain inter-class peculiarity. Meanwhile,
distillation-based consistency preservation is utilized to keep
modality-specific information. Extensive experiments have demonstrated the
superior performance of our algorithm on two challenging Sketchy and Tu-Berlin
datasets.
Related papers
- CSTA: Spatial-Temporal Causal Adaptive Learning for Exemplar-Free Video Class-Incremental Learning [62.69917996026769]
A class-incremental learning task requires learning and preserving both spatial appearance and temporal action involvement.
We propose a framework that equips separate adapters to learn new class patterns, accommodating the incremental information requirements unique to each class.
A causal compensation mechanism is proposed to reduce the conflicts during increment and memorization for between different types of information.
arXiv Detail & Related papers (2025-01-13T11:34:55Z) - SUMI-IFL: An Information-Theoretic Framework for Image Forgery Localization with Sufficiency and Minimality Constraints [66.85363924364628]
Image forgery localization (IFL) is a crucial technique for preventing tampered image misuse and protecting social safety.
We introduce a novel information-theoretic IFL framework named SUMI-IFL that imposes sufficiency-view and minimality-view constraints on forgery feature representation.
arXiv Detail & Related papers (2024-12-13T09:08:02Z) - Relation-Aware Meta-Learning for Zero-shot Sketch-Based Image Retrieval [89.15541654536544]
Sketch-based image retrieval (SBIR) relies on free-hand sketches to retrieve natural photos within the same class.
To address this limitation, the task has evolved into Zero-Shot Sketch-Based Image Retrieval (ZS-SBIR)
We propose a novel framework for ZS-SBIR that employs a pair-based relation-aware quadruplet loss to bridge feature gaps.
arXiv Detail & Related papers (2024-11-28T09:35:27Z) - Decoupling the Class Label and the Target Concept in Machine Unlearning [81.69857244976123]
Machine unlearning aims to adjust a trained model to approximate a retrained one that excludes a portion of training data.
Previous studies showed that class-wise unlearning is successful in forgetting the knowledge of a target class.
We propose a general framework, namely, TARget-aware Forgetting (TARF)
arXiv Detail & Related papers (2024-06-12T14:53:30Z) - Symmetrical Bidirectional Knowledge Alignment for Zero-Shot Sketch-Based
Image Retrieval [69.46139774646308]
This paper studies the problem of zero-shot sketch-based image retrieval (ZS-SBIR)
It aims to use sketches from unseen categories as queries to match the images of the same category.
We propose a novel Symmetrical Bidirectional Knowledge Alignment for zero-shot sketch-based image retrieval (SBKA)
arXiv Detail & Related papers (2023-12-16T04:50:34Z) - DUET: Cross-modal Semantic Grounding for Contrastive Zero-shot Learning [37.48292304239107]
We present a transformer-based end-to-end ZSL method named DUET.
We develop a cross-modal semantic grounding network to investigate the model's capability of disentangling semantic attributes from the images.
We find that DUET can often achieve state-of-the-art performance, its components are effective and its predictions are interpretable.
arXiv Detail & Related papers (2022-07-04T11:12:12Z) - A Contrastive Distillation Approach for Incremental Semantic
Segmentation in Aerial Images [15.75291664088815]
A major issue concerning current deep neural architectures is known as catastrophic forgetting.
We propose a contrastive regularization, where any given input is compared with its augmented version.
We show the effectiveness of our solution on the Potsdam dataset, outperforming the incremental baseline in every test.
arXiv Detail & Related papers (2021-12-07T16:44:45Z) - Domain-Smoothing Network for Zero-Shot Sketch-Based Image Retrieval [66.37346493506737]
Zero-Shot Sketch-Based Image Retrieval (ZS-SBIR) is a novel cross-modal retrieval task.
We propose a novel Domain-Smoothing Network (DSN) for ZS-SBIR.
Our approach notably outperforms the state-of-the-art methods in both Sketchy and TU-Berlin datasets.
arXiv Detail & Related papers (2021-06-22T14:58:08Z) - CrossATNet - A Novel Cross-Attention Based Framework for Sketch-Based
Image Retrieval [30.249581102239645]
We propose a novel framework for cross-modal zero-shot learning (ZSL) in the context of sketch-based image retrieval (SBIR)
While we define a cross-modal triplet loss to ensure the discriminative nature of the shared space, an innovative cross-modal attention learning strategy is also proposed to guide feature extraction from the image domain.
arXiv Detail & Related papers (2021-04-20T12:11:12Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.