Beyond Quadratic: Linear-Time Change Detection with RWKV

• URL: http://arxiv.org/abs/2603.19606v1
• Date: Fri, 20 Mar 2026 03:23:02 GMT
• Title: Beyond Quadratic: Linear-Time Change Detection with RWKV
• Abstract summary: ChangeRWKV is a new architecture for remote sensing change detection.<n>It combines the parallelizable training of Transformers with the linear-time inference of RNNs.<n>It achieves state-of-the-art performance on the LEVIR-CD benchmark, with an 85.46% IoU and 92.16% F1 score.
• Score: 48.3313296326424
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing paradigms for remote sensing change detection are caught in a trade-off: CNNs excel at efficiency but lack global context, while Transformers capture long-range dependencies at a prohibitive computational cost. This paper introduces ChangeRWKV, a new architecture that reconciles this conflict. By building upon the Receptance Weighted Key Value (RWKV) framework, our ChangeRWKV uniquely combines the parallelizable training of Transformers with the linear-time inference of RNNs. Our approach core features two key innovations: a hierarchical RWKV encoder that builds multi-resolution feature representation, and a novel Spatial-Temporal Fusion Module (STFM) engineered to resolve spatial misalignments across scales while distilling fine-grained temporal discrepancies. ChangeRWKV not only achieves state-of-the-art performance on the LEVIR-CD benchmark, with an 85.46% IoU and 92.16% F1 score, but does so while drastically reducing parameters and FLOPs compared to previous leading methods. This work demonstrates a new, efficient, and powerful paradigm for operational-scale change detection. Our code and model are publicly available.

Related papers

• TRaM-VSR: Importance-Aware Token Routing and Merging for One-Step Diffusion Video Super-Resolution [52.96050253456634]
  TRaM-VSR is a framework for adaptive token allocation using context-aware video priors and network-levels.<n>It is shown that TRaM-VSR accelerates inference significantly while preserving state-of-the-art reconstruction quality and robust temporal consistency.
  arXiv  Detail & Related papers  (2026-07-24T11:57:26Z)
• Efficient RWKV-based Representation Learning for 3D Point Clouds [53.911842457391636]
  The textbfP-RWKV block bridges the gap between sequence modeling and irregular 3D geometry.<n>The P-RWKV block and its key sub-modules achieve competitive performance across various tasks with lower computational cost and inference latency.
  arXiv  Detail & Related papers  (2026-06-09T04:16:39Z)
• LiteVSR: Lightweight Adaptation of Frozen Diffusion Transformers for Video Super-Resolution [73.85089704122315]
  We propose a minimalist framework that performs Video Super-Resolution (VSR) using a frozen Transformer with a lightweight State-Aware Adapter.<n> LiteVSR achieves competitive restoration quality with only 11.25% trainable parameters and 12 GPU-hours of training on a single A100, while maintaining fast sampling (down to a single step) compatibility.
  arXiv  Detail & Related papers  (2026-06-08T09:23:42Z)
• VARestorer: One-Step VAR Distillation for Real-World Image Super-Resolution [76.19751531910039]
  We propose VARestorer, a framework that transforms a pre-trained text-to-image VAR model into a one-step ISR model.<n>VARestorer achieves state-of-the-art performance with 72.32 MUSIQ and 0.7669 CLIPIQA on DIV2K dataset.
  arXiv  Detail & Related papers  (2026-04-23T09:09:36Z)
• RWKV-UNet: Improving UNet with Long-Range Cooperation for Effective Medical Image Segmentation [70.79072961974141]
  We propose RWKV-UNet, a novel model that integrates the RWKV structure into the U-Net architecture.<n>This integration enhances the model's ability to capture long-range dependencies and to improve contextual understanding.<n> Experiments on 11 benchmark datasets show that the RWKV-UNet achieves state-of-the-art performance on various types of medical image segmentation tasks.
  arXiv  Detail & Related papers  (2025-01-14T22:03:00Z)
• Exploring Real&Synthetic Dataset and Linear Attention in Image Restoration [47.26304397935705]
  Image restoration aims to recover high-quality images from degraded inputs.<n>Existing methods lack a unified training benchmark for iterations and configurations.<n>We introduce a large-scale IR dataset called ReSyn, which employs a novel image filtering method based on image complexity.
  arXiv  Detail & Related papers  (2024-12-05T02:11:51Z)
• RRWKV: Capturing Long-range Dependencies in RWKV [0.0]
  The paper devises the Retrospected Receptance Weighted Key Value architecture via incorporating the retrospecting ability into the RWKV to effectively absorb information. RWKV has exploited a linearly tensor-product attention mechanism and achieved parallelized computations by deploying the time-sequential mode.
  arXiv  Detail & Related papers  (2023-06-08T13:17:06Z)
• RWKV: Reinventing RNNs for the Transformer Era [54.716108899349614]
  We propose a novel model architecture that combines the efficient parallelizable training of transformers with the efficient inference of RNNs. We scale our models as large as 14 billion parameters, by far the largest dense RNN ever trained, and find RWKV performs on par with similarly sized Transformers.
  arXiv  Detail & Related papers  (2023-05-22T13:57:41Z)
• Efficient Image Super-Resolution with Feature Interaction Weighted Hybrid Network [101.53907377000445]
  Lightweight image super-resolution aims to reconstruct high-resolution images from low-resolution images using low computational costs. Existing methods result in the loss of middle-layer features due to activation functions. We propose a Feature Interaction Weighted Hybrid Network (FIWHN) to minimize the impact of intermediate feature loss on reconstruction quality.
  arXiv  Detail & Related papers  (2022-12-29T05:57:29Z)
• Magic ELF: Image Deraining Meets Association Learning and Transformer [63.761812092934576]
  This paper aims to unify CNN and Transformer to take advantage of their learning merits for image deraining. A novel multi-input attention module (MAM) is proposed to associate rain removal and background recovery. Our proposed method (dubbed as ELF) outperforms the state-of-the-art approach (MPRNet) by 0.25 dB on average.
  arXiv  Detail & Related papers  (2022-07-21T12:50:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.