Memory Regulation and Alignment toward Generalizer RGB-Infrared Person
- URL: http://arxiv.org/abs/2109.08843v1
- Date: Sat, 18 Sep 2021 05:55:06 GMT
- Title: Memory Regulation and Alignment toward Generalizer RGB-Infrared Person
- Authors: Feng Chen, Fei Wu, Qi Wu, Zhiguo Wan
- Abstract summary: RGB-IR ReID always demands discriminative features, leading to over-rely feature sensitivity of seen classes.
We propose a novel multi-granularity memory regulation and alignment module (MG-MRA) to solve this issue.
Our method could alleviate the over-confidence of the model about discriminative features of seen classes.
- Score: 26.46821443401331
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The domain shift, coming from unneglectable modality gap and non-overlapped
identity classes between training and test sets, is a major issue of
RGB-Infrared person re-identification. A key to tackle the inherent issue --
domain shift -- is to enforce the data distributions of the two domains to be
similar. However, RGB-IR ReID always demands discriminative features, leading
to over-rely feature sensitivity of seen classes, \textit{e.g.}, via
attention-based feature alignment or metric learning. Therefore, predicting the
unseen query category from predefined training classes may not be accurate and
leads to a sub-optimal adversarial gradient. In this paper, we uncover it in a
more explainable way and propose a novel multi-granularity memory regulation
and alignment module (MG-MRA) to solve this issue. By explicitly incorporating
a latent variable attribute, from fine-grained to coarse semantic granularity,
into intermediate features, our method could alleviate the over-confidence of
the model about discriminative features of seen classes. Moreover, instead of
matching discriminative features by traversing nearest neighbor, sparse
attributes, \textit{i.e.}, global structural pattern, are recollected with
respect to features and assigned to measure pair-wise image similarity in
hashing. Extensive experiments on RegDB \cite{RegDB} and SYSU-MM01 \cite{SYSU}
show the superiority of the proposed method that outperforms existing
state-of-the-art methods. Our code is available in
https://github.com/Chenfeng1271/MGMRA.
Related papers
- Semantic-aware Representation Learning for Homography Estimation [28.70450397793246]
We propose SRMatcher, a detector-free feature matching method, which encourages the network to learn integrated semantic feature representation.
By reducing errors stemming from semantic inconsistencies in matching pairs, our proposed SRMatcher is able to deliver more accurate and realistic outcomes.
arXiv Detail & Related papers (2024-07-18T08:36:28Z) - Relevant Intrinsic Feature Enhancement Network for Few-Shot Semantic
Segmentation [34.257289290796315]
We propose the Relevant Intrinsic Feature Enhancement Network (RiFeNet) to improve semantic consistency of foreground instances.
RiFeNet surpasses the state-of-the-art methods on PASCAL-5i and COCO benchmarks.
arXiv Detail & Related papers (2023-12-11T16:02:57Z) - Dual Feature Augmentation Network for Generalized Zero-shot Learning [14.410978100610489]
Zero-shot learning (ZSL) aims to infer novel classes without training samples by transferring knowledge from seen classes.
Existing embedding-based approaches for ZSL typically employ attention mechanisms to locate attributes on an image.
We propose a novel Dual Feature Augmentation Network (DFAN), which comprises two feature augmentation modules.
arXiv Detail & Related papers (2023-09-25T02:37:52Z) - Exploring Fine-Grained Representation and Recomposition for Cloth-Changing Person Re-Identification [78.52704557647438]
We propose a novel FIne-grained Representation and Recomposition (FIRe$2$) framework to tackle both limitations without any auxiliary annotation or data.
Experiments demonstrate that FIRe$2$ can achieve state-of-the-art performance on five widely-used cloth-changing person Re-ID benchmarks.
arXiv Detail & Related papers (2023-08-21T12:59:48Z) - Regularizing Self-training for Unsupervised Domain Adaptation via
Structural Constraints [14.593782939242121]
We propose to incorporate structural cues from auxiliary modalities, such as depth, to regularise conventional self-training objectives.
Specifically, we introduce a contrastive pixel-level objectness constraint that pulls the pixel representations within a region of an object instance closer.
We show that our regularizer significantly improves top performing self-training methods in various UDA benchmarks for semantic segmentation.
arXiv Detail & Related papers (2023-04-29T00:12:26Z) - Complementary Random Masking for RGB-Thermal Semantic Segmentation [63.93784265195356]
RGB-thermal semantic segmentation is a potential solution to achieve reliable semantic scene understanding in adverse weather and lighting conditions.
This paper proposes 1) a complementary random masking strategy of RGB-T images and 2) self-distillation loss between clean and masked input modalities.
We achieve state-of-the-art performance over three RGB-T semantic segmentation benchmarks.
arXiv Detail & Related papers (2023-03-30T13:57:21Z) - Triplet Contrastive Learning for Unsupervised Vehicle Re-identification [55.445358749042384]
Part feature learning is a critical technology for fine semantic understanding in vehicle re-identification.
We propose a novel Triplet Contrastive Learning framework (TCL) which leverages cluster features to bridge the part features and global features.
arXiv Detail & Related papers (2023-01-23T15:52:12Z) - PatchMix Augmentation to Identify Causal Features in Few-shot Learning [55.64873998196191]
Few-shot learning aims to transfer knowledge learned from base with sufficient categories labelled data to novel categories with scarce known information.
We propose a novel data augmentation strategy dubbed as PatchMix that can break this spurious dependency.
We show that such an augmentation mechanism, different from existing ones, is able to identify the causal features.
arXiv Detail & Related papers (2022-11-29T08:41:29Z) - Measuring the Interpretability of Unsupervised Representations via
Quantized Reverse Probing [97.70862116338554]
We investigate the problem of measuring interpretability of self-supervised representations.
We formulate the latter as estimating the mutual information between the representation and a space of manually labelled concepts.
We use our method to evaluate a large number of self-supervised representations, ranking them by interpretability.
arXiv Detail & Related papers (2022-09-07T16:18:50Z) - SPCL: A New Framework for Domain Adaptive Semantic Segmentation via
Semantic Prototype-based Contrastive Learning [6.705297811617307]
Domain adaptation can help in transferring knowledge from a labeled source domain to an unlabeled target domain.
We propose a novel semantic prototype-based contrastive learning framework for fine-grained class alignment.
Our method is easy to implement and attains superior results compared to state-of-the-art approaches.
arXiv Detail & Related papers (2021-11-24T09:26:07Z) - Bi-directional Cross-Modality Feature Propagation with
Separation-and-Aggregation Gate for RGB-D Semantic Segmentation [59.94819184452694]
Depth information has proven to be a useful cue in the semantic segmentation of RGBD images for providing a geometric counterpart to the RGB representation.
Most existing works simply assume that depth measurements are accurate and well-aligned with the RGB pixels and models the problem as a cross-modal feature fusion.
In this paper, we propose a unified and efficient Crossmodality Guided to not only effectively recalibrate RGB feature responses, but also to distill accurate depth information via multiple stages and aggregate the two recalibrated representations alternatively.
arXiv Detail & Related papers (2020-07-17T18:35:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.