Weakly Supervised Face and Whole Body Recognition in Turbulent
Environments
- URL: http://arxiv.org/abs/2308.11757v1
- Date: Tue, 22 Aug 2023 19:58:02 GMT
- Title: Weakly Supervised Face and Whole Body Recognition in Turbulent
Environments
- Authors: Kshitij Nikhal, Benjamin S. Riggan
- Abstract summary: We propose a new weakly supervised framework that generates domain representations, aligning turbulent and pristine images into a common subspace.
We also introduce a new tilt map estimator that predicts geometric distortions observed in turbulent images.
Our method does not require synthesizing turbulent-free images or ground-truth paired images, and requires significantly fewer annotated samples.
- Score: 2.2263723609685773
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Face and person recognition have recently achieved remarkable success under
challenging scenarios, such as off-pose and cross-spectrum matching. However,
long-range recognition systems are often hindered by atmospheric turbulence,
leading to spatially and temporally varying distortions in the image. Current
solutions rely on generative models to reconstruct a turbulent-free image, but
often preserve photo-realism instead of discriminative features that are
essential for recognition. This can be attributed to the lack of large-scale
datasets of turbulent and pristine paired images, necessary for optimal
reconstruction. To address this issue, we propose a new weakly supervised
framework that employs a parameter-efficient self-attention module to generate
domain agnostic representations, aligning turbulent and pristine images into a
common subspace. Additionally, we introduce a new tilt map estimator that
predicts geometric distortions observed in turbulent images. This estimate is
used to re-rank gallery matches, resulting in up to 13.86\% improvement in
rank-1 accuracy. Our method does not require synthesizing turbulent-free images
or ground-truth paired images, and requires significantly fewer annotated
samples, enabling more practical and rapid utility of increasingly large
datasets. We analyze our framework using two datasets -- Long-Range Face
Identification Dataset (LRFID) and BRIAR Government Collection 1 (BGC1) --
achieving enhanced discriminability under varying turbulence and standoff
distance.
Related papers
- Spatial-temporal Memories Enhanced Graph Autoencoder for Anomaly Detection in Dynamic Graphs [52.956235109354175]
Anomaly detection in dynamic graphs presents a significant challenge due to the temporal evolution of graph structures and attributes.
We introduce a novel Spatial-Temporal memories-enhanced graph autoencoder (STRIPE)
STRIPE has demonstrated a superior capability to discern anomalies by effectively leveraging the distinct spatial and temporal dynamics of dynamic graphs.
arXiv Detail & Related papers (2024-03-14T02:26:10Z) - Exposure Bracketing is All You Need for Unifying Image Restoration and Enhancement Tasks [50.822601495422916]
We propose to utilize exposure bracketing photography to unify image restoration and enhancement tasks.
Due to the difficulty in collecting real-world pairs, we suggest a solution that first pre-trains the model with synthetic paired data.
In particular, a temporally modulated recurrent network (TMRNet) and self-supervised adaptation method are proposed.
arXiv Detail & Related papers (2024-01-01T14:14:35Z) - Robustness and invariance properties of image classifiers [8.970032486260695]
Deep neural networks have achieved impressive results in many image classification tasks.
Deep networks are not robust to a large variety of semantic-preserving image modifications.
The poor robustness of image classifiers to small data distribution shifts raises serious concerns regarding their trustworthiness.
arXiv Detail & Related papers (2022-08-30T11:00:59Z) - Supervision by Denoising for Medical Image Segmentation [17.131944478890293]
We propose "supervision by denoising" (SUD), a framework that enables us to supervise models using their own soft labels.
SUD unifies averaging and spatial denoising techniques under a denoising framework and alternates denoising and model weight update steps.
As example applications, we apply SUD to two problems arising from biomedical imaging.
arXiv Detail & Related papers (2022-02-07T05:29:16Z) - LTT-GAN: Looking Through Turbulence by Inverting GANs [86.25869403782957]
We propose the first turbulence mitigation method that makes use of visual priors encapsulated by a well-trained GAN.
Based on the visual priors, we propose to learn to preserve the identity of restored images on a periodic contextual distance.
Our method significantly outperforms prior art in both the visual quality and face verification accuracy of restored results.
arXiv Detail & Related papers (2021-12-04T16:42:13Z) - Learning Discriminative Shrinkage Deep Networks for Image Deconvolution [122.79108159874426]
We propose an effective non-blind deconvolution approach by learning discriminative shrinkage functions to implicitly model these terms.
Experimental results show that the proposed method performs favorably against the state-of-the-art ones in terms of efficiency and accuracy.
arXiv Detail & Related papers (2021-11-27T12:12:57Z) - Spatially-Adaptive Image Restoration using Distortion-Guided Networks [51.89245800461537]
We present a learning-based solution for restoring images suffering from spatially-varying degradations.
We propose SPAIR, a network design that harnesses distortion-localization information and dynamically adjusts to difficult regions in the image.
arXiv Detail & Related papers (2021-08-19T11:02:25Z) - Image Fine-grained Inpainting [89.17316318927621]
We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields.
To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss.
We also employ a discriminator with local and global branches to ensure local-global contents consistency.
arXiv Detail & Related papers (2020-02-07T03:45:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.