TransCL: Transformer Makes Strong and Flexible Compressive Learning
- URL: http://arxiv.org/abs/2207.11972v1
- Date: Mon, 25 Jul 2022 08:21:48 GMT
- Title: TransCL: Transformer Makes Strong and Flexible Compressive Learning
- Authors: Chong Mou, Jian Zhang
- Abstract summary: Compressive learning (CL) is an emerging framework that integrates signal acquisition via compressed sensing (CS) and machine learning for inference tasks directly on a small number of measurements.
Previous attempts on CL are not only limited to a fixed CS ratio, but also limited to MNIST/CIFAR-like datasets and do not scale to complex real-world high-resolution (HR) data or vision tasks.
In this paper, a novel transformer-based compressive learning framework on large-scale images with arbitrary CS ratios, dubbed TransCL, is proposed.
- Score: 11.613886854794133
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Compressive learning (CL) is an emerging framework that integrates signal
acquisition via compressed sensing (CS) and machine learning for inference
tasks directly on a small number of measurements. It can be a promising
alternative to classical image-domain methods and enjoys great advantages in
memory saving and computational efficiency. However, previous attempts on CL
are not only limited to a fixed CS ratio, which lacks flexibility, but also
limited to MNIST/CIFAR-like datasets and do not scale to complex real-world
high-resolution (HR) data or vision tasks. In this paper, a novel
transformer-based compressive learning framework on large-scale images with
arbitrary CS ratios, dubbed TransCL, is proposed. Specifically, TransCL first
utilizes the strategy of learnable block-based compressed sensing and proposes
a flexible linear projection strategy to enable CL to be performed on
large-scale images in an efficient block-by-block manner with arbitrary CS
ratios. Then, regarding CS measurements from all blocks as a sequence, a pure
transformer-based backbone is deployed to perform vision tasks with various
task-oriented heads. Our sufficient analysis presents that TransCL exhibits
strong resistance to interference and robust adaptability to arbitrary CS
ratios. Extensive experiments for complex HR data demonstrate that the proposed
TransCL can achieve state-of-the-art performance in image classification and
semantic segmentation tasks. In particular, TransCL with a CS ratio of $10\%$
can obtain almost the same performance as when operating directly on the
original data and can still obtain satisfying performance even with an
extremely low CS ratio of $1\%$. The source codes of our proposed TransCL is
available at \url{https://github.com/MC-E/TransCL/}.
Related papers
- Transforming Image Super-Resolution: A ConvFormer-based Efficient
Approach [63.98380888730723]
We introduce the Convolutional Transformer layer (ConvFormer) and the ConvFormer-based Super-Resolution network (CFSR)
CFSR efficiently models long-range dependencies and extensive receptive fields with a slight computational cost.
It achieves 0.39 dB gains on Urban100 dataset for x2 SR task while containing 26% and 31% fewer parameters and FLOPs, respectively.
arXiv Detail & Related papers (2024-01-11T03:08:00Z) - AICT: An Adaptive Image Compression Transformer [18.05997169440533]
We propose a more straightforward yet effective Tranformer-based channel-wise auto-regressive prior model, resulting in an absolute image compression transformer (ICT)
The proposed ICT can capture both global and local contexts from the latent representations.
We leverage a learnable scaling module with a sandwich ConvNeXt-based pre/post-processor to accurately extract more compact latent representation.
arXiv Detail & Related papers (2023-07-12T11:32:02Z) - Binarized Spectral Compressive Imaging [59.18636040850608]
Existing deep learning models for hyperspectral image (HSI) reconstruction achieve good performance but require powerful hardwares with enormous memory and computational resources.
We propose a novel method, Binarized Spectral-Redistribution Network (BiSRNet)
BiSRNet is derived by using the proposed techniques to binarize the base model.
arXiv Detail & Related papers (2023-05-17T15:36:08Z) - Efficient Adversarial Contrastive Learning via Robustness-Aware Coreset
Selection [59.77647907277523]
Adversarial contrast learning (ACL) does not require expensive data annotations but outputs a robust representation that withstands adversarial attacks.
ACL needs tremendous running time to generate the adversarial variants of all training data.
This paper proposes a robustness-aware coreset selection (RCS) method to speed up ACL.
arXiv Detail & Related papers (2023-02-08T03:20:14Z) - Effective Invertible Arbitrary Image Rescaling [77.46732646918936]
Invertible Neural Networks (INN) are able to increase upscaling accuracy significantly by optimizing the downscaling and upscaling cycle jointly.
A simple and effective invertible arbitrary rescaling network (IARN) is proposed to achieve arbitrary image rescaling by training only one model in this work.
It is shown to achieve a state-of-the-art (SOTA) performance in bidirectional arbitrary rescaling without compromising perceptual quality in LR outputs.
arXiv Detail & Related papers (2022-09-26T22:22:30Z) - Cross-receptive Focused Inference Network for Lightweight Image
Super-Resolution [64.25751738088015]
Transformer-based methods have shown impressive performance in single image super-resolution (SISR) tasks.
Transformers that need to incorporate contextual information to extract features dynamically are neglected.
We propose a lightweight Cross-receptive Focused Inference Network (CFIN) that consists of a cascade of CT Blocks mixed with CNN and Transformer.
arXiv Detail & Related papers (2022-07-06T16:32:29Z) - CSformer: Bridging Convolution and Transformer for Compressive Sensing [65.22377493627687]
This paper proposes a hybrid framework that integrates the advantages of leveraging detailed spatial information from CNN and the global context provided by transformer for enhanced representation learning.
The proposed approach is an end-to-end compressive image sensing method, composed of adaptive sampling and recovery.
The experimental results demonstrate the effectiveness of the dedicated transformer-based architecture for compressive sensing.
arXiv Detail & Related papers (2021-12-31T04:37:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.