Uncertainty-Guided Spatial Pruning Architecture for Efficient Frame Interpolation

• URL: http://arxiv.org/abs/2307.16555v4
• Date: Sat, 28 Oct 2023 02:31:18 GMT
• Title: Uncertainty-Guided Spatial Pruning Architecture for Efficient Frame Interpolation
• Abstract summary: Video frame (VFI) model applies convolution operation to all locations, leading to redundant computations in regions with easy motion. We develop an Uncertainty-Guided Spatial Pruning architecture to skip redundant computation. Our method achieves state-of-the-art performance with lower FLOPs on multiple benchmarks.
• Score: 19.077966606237897
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The video frame interpolation (VFI) model applies the convolution operation to all locations, leading to redundant computations in regions with easy motion. We can use dynamic spatial pruning method to skip redundant computation, but this method cannot properly identify easy regions in VFI tasks without supervision. In this paper, we develop an Uncertainty-Guided Spatial Pruning (UGSP) architecture to skip redundant computation for efficient frame interpolation dynamically. Specifically, pixels with low uncertainty indicate easy regions, where the calculation can be reduced without bringing undesirable visual results. Therefore, we utilize uncertainty-generated mask labels to guide our UGSP in properly locating the easy region. Furthermore, we propose a self-contrast training strategy that leverages an auxiliary non-pruning branch to improve the performance of our UGSP. Extensive experiments show that UGSP maintains performance but reduces FLOPs by 34%/52%/30% compared to baseline without pruning on Vimeo90K/UCF101/MiddleBury datasets. In addition, our method achieves state-of-the-art performance with lower FLOPs on multiple benchmarks.

Related papers

• StaticSegFormer: An Efficient High-Performance Semantic Segmentation Based on Static Structured Pruning [17.894320802733755]
  Structured pruning enhances the efficiency of deep neural networks (DNNs) by eliminating groups of parameters during inference.<n>Recent dynamic structured pruning methods aim at reducing the performance drop, while lowering the FLOPs even more.<n>On a GPU platform such dynamic methods exhibit a surprisingly low frame rate far below a simple static approach.<n>We propose a static structured pruning method for attention layers, that both, a lower FLOPs and a high frame rate [fps] of the SegFormer network, the latter increased by up to 34% relative on the Cityscapes dataset.
  arXiv  Detail & Related papers  (2026-08-05T13:14:50Z)
• A Unified Variational Framework for Deep Weakly Supervised Image Segmentation [6.232598868087682]
  We propose a unified variational framework for image segmentation under sparse pixel-level supervision.<n>Our method is based on a simplex-constrained Potts model with a smooth perimeter regularizer.<n>Sparse labels are incorporated into the data fidelity term by constructing a fuzzy membership function.
  arXiv  Detail & Related papers  (2026-07-22T02:18:15Z)
• Selecting Samples on Graphs: A Unified Dataset Pruning Framework for Lossless Training Acceleration [71.77541419885925]
  We present a unified graph-based DP framework.<n>By modeling the dataset as a weighted graph, DP can be cast as a Maximum Weight Clique Problem (MWCP)<n>Our method outperforms existing DP methods while substantially reducing training cost, reducing training time by over 40% without sacrificing accuracy on ImageNet-1k with ResNet-50.
  arXiv  Detail & Related papers  (2026-06-11T05:13:32Z)
• Unbiased Gradient Estimation for Event Binning via Functional Backpropagation [64.88399635309918]
  We propose a novel framework for unbiased gradient estimation of arbitrary binning functions by synthesizing weak derivatives during backpropagation.<n>We achieve 9.4% lower EPE in self-supervised optical flow, and 5.1% lower RMS error in SLAM, demonstrating broad benefits for event-based visual perception.
  arXiv  Detail & Related papers  (2026-02-13T04:05:03Z)
• Rethinking Autoregressive Models for Lossless Image Compression via Hierarchical Parallelism and Progressive Adaptation [75.58269386927076]
  Autoregressive (AR) models are often dismissed as impractical due to prohibitive computational cost.<n>This work re-thinks this paradigm, introducing a framework built on hierarchical parallelism and progressive adaptation.<n> Experiments on diverse datasets (natural, satellite, medical) validate that our method achieves new state-of-the-art compression.
  arXiv  Detail & Related papers  (2025-11-14T06:27:58Z)
• The Unreasonable Effectiveness of Randomized Representations in Online Continual Graph Learning [23.73070470019403]
  Catastrophic forgetting is one of the main obstacles for Online Continual Graph Learning (OCGL)<n>We use a fixed, randomly encoder to generate robust and expressive node embeddings by aggregating neighborhood information.<n>By freezing the encoder, we eliminate drifts of the representation parameters, a key source of forgetting, obtaining embeddings that are both expressive and stable.
  arXiv  Detail & Related papers  (2025-10-08T09:44:14Z)
• EDFFDNet: Towards Accurate and Efficient Unsupervised Multi-Grid Image Registration [17.190325630307097]
  We propose an Exponential-Decay Free-Form Deformation Network (EDFFDNet), which employs free-form deformation with an exponential-decay basis function.<n>By transforming dense interactions into sparse ones, ASMA reduces parameters and improves accuracy.<n>Experiments demonstrate that EDFFDNet reduces parameters, memory, and total runtime by 70.5%, 32.6%, and 33.7%, respectively.<n>EDFFDNet-2 further improves PSNR by 1.06 dB while maintaining lower computational costs.
  arXiv  Detail & Related papers  (2025-09-09T12:30:51Z)
• SGAD: Semantic and Geometric-aware Descriptor for Local Feature Matching [16.683203139962153]
  We introduce the Semantic and Geometric-aware Descriptor Network (SGAD), which fundamentally rethinks area-based matching.<n>SGAD generates highly discriminative area descriptors that enable direct matching without complex graph optimization.<n>We further improve the performance of area matching through a novel supervision strategy that decomposes the area matching task into classification and ranking subtasks.
  arXiv  Detail & Related papers  (2025-08-04T10:46:53Z)
• Exploring Diffusion with Test-Time Training on Efficient Image Restoration [1.3830502387127932]
  DiffRWKVIR is a novel framework unifying Test-Time Training (TTT) with efficient diffusion.<n>Our method establishes a new paradigm for adaptive, high-efficiency image restoration with optimized hardware utilization.
  arXiv  Detail & Related papers  (2025-06-17T14:01:59Z)
• A projection-based framework for gradient-free and parallel learning [50.96641619247761]
  We introduce PJAX, a JAX-based software framework that enables this paradigm.<n>PJAX composes projection operators for elementary operations, automatically deriving the solution operators for the feasibility problems.<n>We train diverse architectures (MLPs, CNNs, RNNs) on standard benchmarks using PJAX, demonstrating its generality.
  arXiv  Detail & Related papers  (2025-06-06T08:44:56Z)
• Learning Coarse-to-Fine Pruning of Graph Convolutional Networks for Skeleton-based Recognition [5.656581242851759]
  Magnitude Pruning is a lightweight network design method which seeks to remove connections with the smallest magnitude. We devise a novel coarse-to-fine (CTF) method that gathers the advantages of structured and unstructured pruning. Our method relies on a novel CTF parametrization that models the mask of each connection as the Hadamard product.
  arXiv  Detail & Related papers  (2024-12-17T13:11:48Z)
• Structural Pruning via Spatial-aware Information Redundancy for Semantic Segmentation [34.554924043562295]
  We argue that most existing pruning methods, originally designed for image classification, overlook the fact that segmentation is a location-sensitive task. This paper proposes a novel approach, denoted as Spatial-aware Information Redundancy Filter Pruning, which aims to reduce feature redundancy between channels.
  arXiv  Detail & Related papers  (2024-12-17T08:41:50Z)
• FGP: Feature-Gradient-Prune for Efficient Convolutional Layer Pruning [16.91552023598741]
  This paper introduces a novel pruning method called Feature-Gradient Pruning (FGP) It integrates both feature-based and gradient-based information to more effectively evaluate the importance of channels across various target classes. Experiments conducted across multiple tasks and datasets show that FGP significantly reduces computational costs and minimizes accuracy loss.
  arXiv  Detail & Related papers  (2024-11-19T08:42:15Z)
• Fast Graph Sharpness-Aware Minimization for Enhancing and Accelerating Few-Shot Node Classification [53.727688136434345]
  Graph Neural Networks (GNNs) have shown superior performance in node classification. We present Fast Graph Sharpness-Aware Minimization (FGSAM) that integrates the rapid training of Multi-Layer Perceptrons with the superior performance of GNNs. Our proposed algorithm outperforms the standard SAM with lower computational costs in FSNC tasks.
  arXiv  Detail & Related papers  (2024-10-22T09:33:29Z)
• LeRF: Learning Resampling Function for Adaptive and Efficient Image Interpolation [64.34935748707673]
  Recent deep neural networks (DNNs) have made impressive progress in performance by introducing learned data priors. We propose a novel method of Learning Resampling (termed LeRF) which takes advantage of both the structural priors learned by DNNs and the locally continuous assumption. LeRF assigns spatially varying resampling functions to input image pixels and learns to predict the shapes of these resampling functions with a neural network.
  arXiv  Detail & Related papers  (2024-07-13T16:09:45Z)
• Magic ELF: Image Deraining Meets Association Learning and Transformer [63.761812092934576]
  This paper aims to unify CNN and Transformer to take advantage of their learning merits for image deraining. A novel multi-input attention module (MAM) is proposed to associate rain removal and background recovery. Our proposed method (dubbed as ELF) outperforms the state-of-the-art approach (MPRNet) by 0.25 dB on average.
  arXiv  Detail & Related papers  (2022-07-21T12:50:54Z)
• Exploring Motion Ambiguity and Alignment for High-Quality Video Frame Interpolation [46.02120172459727]
  We propose to relax the requirement of reconstructing an intermediate frame as close to the ground-truth (GT) as possible. We develop a texture consistency loss (TCL) upon the assumption that the interpolated content should maintain similar structures with their counterparts in the given frames.
  arXiv  Detail & Related papers  (2022-03-19T10:37:06Z)
• Federated Minimax Optimization: Improved Convergence Analyses and Algorithms [32.062312674333775]
  We consider non minimax optimization, is gaining prominence many modern machine learning applications such as GANs. We provide a novel and tighter analysis algorithm, improves convergence communication guarantees in the existing literature.
  arXiv  Detail & Related papers  (2022-03-09T16:21:31Z)
• Large-scale Optimization of Partial AUC in a Range of False Positive Rates [51.12047280149546]
  The area under the ROC curve (AUC) is one of the most widely used performance measures for classification models in machine learning. We develop an efficient approximated gradient descent method based on recent practical envelope smoothing technique. Our proposed algorithm can also be used to minimize the sum of some ranked range loss, which also lacks efficient solvers.
  arXiv  Detail & Related papers  (2022-03-03T03:46:18Z)
• Learning strides in convolutional neural networks [34.20666933112202]
  This work introduces DiffStride, the first downsampling layer with learnable strides. Experiments on audio and image classification show the generality and effectiveness of our solution.
  arXiv  Detail & Related papers  (2022-02-03T16:03:36Z)
• Fast and Accurate Neural CRF Constituency Parsing [16.90190521285297]
  This work presents a fast and accurate neural CRF constituency computation. We batchify the inside algorithm for loss by direct large tensor operations on GPU, and avoid the outside algorithm for computation via efficient back-propagation. Experiments on PTB, CTB5.1, and CTB7 show that our two-stage CRF achieves new state-of-the-art performance on both settings of w/o and w/ BERT.
  arXiv  Detail & Related papers  (2020-08-09T14:38:48Z)
• Real-Time High-Performance Semantic Image Segmentation of Urban Street Scenes [98.65457534223539]
  We propose a real-time high-performance DCNN-based method for robust semantic segmentation of urban street scenes. The proposed method achieves the accuracy of 73.6% and 68.0% mean Intersection over Union (mIoU) with the inference speed of 51.0 fps and 39.3 fps.
  arXiv  Detail & Related papers  (2020-03-11T08:45:53Z)
• Image Matching across Wide Baselines: From Paper to Practice [80.9424750998559]
  We introduce a comprehensive benchmark for local features and robust estimation algorithms. Our pipeline's modular structure allows easy integration, configuration, and combination of different methods. We show that with proper settings, classical solutions may still outperform the perceived state of the art.
  arXiv  Detail & Related papers  (2020-03-03T15:20:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.