Symmetry-Aware Transformer-based Mirror Detection
- URL: http://arxiv.org/abs/2207.06332v1
- Date: Wed, 13 Jul 2022 16:40:01 GMT
- Title: Symmetry-Aware Transformer-based Mirror Detection
- Authors: Tianyu Huang, Bowen Dong, Jiaying Lin, Xiaohui Liu, Rynson W.H. Lau,
Wangmeng Zuo
- Abstract summary: We propose a dual-path Symmetry-Aware Transformer-based mirror detection Network (SATNet)
SATNet includes two novel modules: Symmetry-Aware Attention Module (SAAM) and Contrast and Fusion Decoder Module (CFDM)
Experimental results show that SATNet outperforms both RGB and RGB-D mirror detection methods on all available mirror detection datasets.
- Score: 85.47570468668955
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Mirror detection aims to identify the mirror regions in the given input
image. Existing works mainly focus on integrating the semantic features and
structural features to mine the similarity and discontinuity between mirror and
non-mirror regions, or introducing depth information to help analyze the
existence of mirrors. In this work, we observe that a real object typically
forms a loose symmetry relationship with its corresponding reflection in the
mirror, which is beneficial in distinguishing mirrors from real objects. Based
on this observation, we propose a dual-path Symmetry-Aware Transformer-based
mirror detection Network (SATNet), which includes two novel modules:
Symmetry-Aware Attention Module (SAAM) and Contrast and Fusion Decoder Module
(CFDM). Specifically, we first introduce the transformer backbone to model
global information aggregation in images, extracting multi-scale features in
two paths. We then feed the high-level dual-path features to SAAMs to capture
the symmetry relations. Finally, we fuse the dual-path features and refine our
prediction maps progressively with CFDMs to obtain the final mirror mask.
Experimental results show that SATNet outperforms both RGB and RGB-D mirror
detection methods on all available mirror detection datasets.
Related papers
- Reflecting Reality: Enabling Diffusion Models to Produce Faithful Mirror Reflections [26.02117310176884]
We tackle the problem of generating highly realistic and plausible mirror reflections using diffusion-based generative models.
To enable this, we create SynMirror, a large-scale dataset of diverse synthetic scenes with objects placed in front of mirrors.
We propose a novel depth-conditioned inpainting method called MirrorFusion, which generates high-quality geometrically consistent and photo-realistic mirror reflections.
arXiv Detail & Related papers (2024-09-23T02:59:07Z) - SBDet: A Symmetry-Breaking Object Detector via Relaxed Rotation-Equivariance [26.05910177212846]
Group Equivariant Convolution (GConv) empowers models to explore symmetries hidden in visual data, improving their performance.
Traditional GConv methods are limited by the strict operation rules in the group space, making it difficult to adapt to Symmetry-Breaking or non-rigid transformations.
We propose a novel Relaxed Rotation-Equivariant Network (R2Net) as the backbone and further develop the Symmetry-Breaking Object Detector (SBDet) for 2D object detection built upon it.
arXiv Detail & Related papers (2024-08-21T16:32:03Z) - MirrorGaussian: Reflecting 3D Gaussians for Reconstructing Mirror Reflections [58.003014868772254]
MirrorGaussian is the first method for mirror scene reconstruction with real-time rendering based on 3D Gaussian Splatting.
We introduce an intuitive dual-rendering strategy that enables differentiableization of both the real-world 3D Gaussians and the mirrored counterpart.
Our approach significantly outperforms existing methods, achieving state-of-the-art results.
arXiv Detail & Related papers (2024-05-20T09:58:03Z) - Adaptive Rotated Convolution for Rotated Object Detection [96.94590550217718]
We present Adaptive Rotated Convolution (ARC) module to handle rotated object detection problem.
In our ARC module, the convolution kernels rotate adaptively to extract object features with varying orientations in different images.
The proposed approach achieves state-of-the-art performance on the DOTA dataset with 81.77% mAP.
arXiv Detail & Related papers (2023-03-14T11:53:12Z) - Efficient Mirror Detection via Multi-level Heterogeneous Learning [39.091162729266294]
HetNet is a highly efficient mirror detection network.
HetNet follows an effective architecture that obtains specific information at different stages to detect mirrors.
Compared to the state-of-the-art method, HetNet runs 664$%$ faster and draws an average performance gain of 8.9$%$ on MAE, 3.1$%$ on IoU, and 2.0$%$ on F-measure.
arXiv Detail & Related papers (2022-11-28T18:51:11Z) - PSNet: Parallel Symmetric Network for Video Salient Object Detection [85.94443548452729]
We propose a VSOD network with up and down parallel symmetry, named PSNet.
Two parallel branches with different dominant modalities are set to achieve complete video saliency decoding.
arXiv Detail & Related papers (2022-10-12T04:11:48Z) - Reflection and Rotation Symmetry Detection via Equivariant Learning [40.61825212385055]
We introduce a group-equivariant convolutional network for symmetry detection, dubbed EquiSym.
We present a new dataset, DENse and DIverse symmetry (DENDI), which mitigates limitations of existing benchmarks for reflection and rotation symmetry detection.
Experiments show that our method achieves the state of the arts in symmetry detection on LDRS and DENDI datasets.
arXiv Detail & Related papers (2022-03-31T04:18:33Z) - Mirror-Yolo: A Novel Attention Focus, Instance Segmentation and Mirror Detection Model [6.048747739825864]
YOLOv4 achieves phenomenal results in terms of object detection accuracy and speed, but it still fails in detecting mirrors.
We propose Mirror-YOLO, which targets mirror detection, containing a novel attention focus mechanism for features acquisition.
arXiv Detail & Related papers (2022-02-17T08:03:48Z) - RRNet: Relational Reasoning Network with Parallel Multi-scale Attention
for Salient Object Detection in Optical Remote Sensing Images [82.1679766706423]
Salient object detection (SOD) for optical remote sensing images (RSIs) aims at locating and extracting visually distinctive objects/regions from the optical RSIs.
We propose a relational reasoning network with parallel multi-scale attention for SOD in optical RSIs.
Our proposed RRNet outperforms the existing state-of-the-art SOD competitors both qualitatively and quantitatively.
arXiv Detail & Related papers (2021-10-27T07:18:32Z) - Two-Stage Single Image Reflection Removal with Reflection-Aware Guidance [78.34235841168031]
We present a novel two-stage network with reflection-aware guidance (RAGNet) for single image reflection removal (SIRR)
RAG can be used (i) to mitigate the effect of reflection from the observation, and (ii) to generate mask in partial convolution for mitigating the effect of deviating from linear combination hypothesis.
Experiments on five commonly used datasets demonstrate the quantitative and qualitative superiority of our RAGNet in comparison to the state-of-the-art SIRR methods.
arXiv Detail & Related papers (2020-12-02T03:14:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.