LLA-FLOW: A Lightweight Local Aggregation on Cost Volume for Optical Flow Estimation

• URL: http://arxiv.org/abs/2304.08101v2
• Date: Tue, 18 Jul 2023 07:00:03 GMT
• Title: LLA-FLOW: A Lightweight Local Aggregation on Cost Volume for Optical Flow Estimation
• Authors: Jiawei Xu, Zongqing Lu, Qingmin Liao
• Abstract summary: Some methods insert stacked transformer modules that allow the network to use global information of cost volume for estimation. But the global information aggregation often incurs serious memory and time costs during training and inference, which hinders model deployment. We draw inspiration from the traditional local region constraint and design the local similarity aggregation (LSA) and the shifted local similarity aggregation (SLSA) Experiments on the final pass of Sintel show the lower cost required for our approach while maintaining competitive performance.
• Score: 35.922073542578055
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Lack of texture often causes ambiguity in matching, and handling this issue is an important challenge in optical flow estimation. Some methods insert stacked transformer modules that allow the network to use global information of cost volume for estimation. But the global information aggregation often incurs serious memory and time costs during training and inference, which hinders model deployment. We draw inspiration from the traditional local region constraint and design the local similarity aggregation (LSA) and the shifted local similarity aggregation (SLSA). The aggregation for cost volume is implemented with lightweight modules that act on the feature maps. Experiments on the final pass of Sintel show the lower cost required for our approach while maintaining competitive performance.

Related papers

• Can We Theoretically Quantify the Impacts of Local Updates on the Generalization Performance of Federated Learning? [50.03434441234569]
  Federated Learning (FL) has gained significant popularity due to its effectiveness in training machine learning models across diverse sites without requiring direct data sharing. While various algorithms have shown that FL with local updates is a communication-efficient distributed learning framework, the generalization performance of FL with local updates has received comparatively less attention.
  arXiv  Detail & Related papers  (2024-09-05T19:00:18Z)
• FedHPL: Efficient Heterogeneous Federated Learning with Prompt Tuning and Logit Distillation [32.305134875959226]
  Federated learning (FL) is a privacy-preserving paradigm that enables distributed clients to collaboratively train models with a central server. We propose FedHPL, a parameter-efficient unified $textbfFed$erated learning framework for $textbfH$eterogeneous settings. We show that our framework outperforms state-of-the-art FL approaches, with less overhead and training rounds.
  arXiv  Detail & Related papers  (2024-05-27T15:25:32Z)
• LW-FedSSL: Resource-efficient Layer-wise Federated Self-supervised Learning [14.413037571286564]
  We propose LW-FedSSL, a layer-wise self-supervised learning approach that allows edge devices to incrementally train a single layer of the model at a time. We introduce server-side calibration and representation alignment mechanisms to ensure LW-FedSSL delivers performance on par with conventional federated self-supervised learning (FedSSL) With the proposed mechanisms, LW-FedSSL achieves a $3.3 times$ reduction in memory usage, $2.1 times$ fewer computational operations (FLOPs), and a $3.2 times$ lower communication cost.
  arXiv  Detail & Related papers  (2024-01-22T01:57:31Z)
• Go beyond End-to-End Training: Boosting Greedy Local Learning with Context Supply [0.12187048691454236]
  greedy local learning partitions the network into gradient-isolated modules and trains supervisely based on local preliminary losses. As the number of segmentations of the gradient-isolated module increases, the performance of the local learning scheme degrades substantially. We propose a ContSup scheme, which incorporates context supply between isolated modules to compensate for information loss.
  arXiv  Detail & Related papers  (2023-12-12T10:25:31Z)
• TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models [52.454274602380124]
  Diffusion models heavily depend on the time-step $t$ to achieve satisfactory multi-round denoising. We propose a Temporal Feature Maintenance Quantization (TFMQ) framework building upon a Temporal Information Block. Powered by the pioneering block design, we devise temporal information aware reconstruction (TIAR) and finite set calibration (FSC) to align the full-precision temporal features.
  arXiv  Detail & Related papers  (2023-11-27T12:59:52Z)
• FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracy [84.45004766136663]
  Federated learning is an emerging distributed machine learning framework. It suffers from the non-vanishing biases introduced by the local inconsistent optimal and the rugged client-drifts by the local over-fitting. We propose a novel and practical method, FedSpeed, to alleviate the negative impacts posed by these problems.
  arXiv  Detail & Related papers  (2023-02-21T03:55:29Z)
• $\texttt{FedBC}$: Calibrating Global and Local Models via Federated Learning Beyond Consensus [66.62731854746856]
  In federated learning (FL), the objective of collaboratively learning a global model through aggregation of model updates across devices tends to oppose the goal of personalization via local information. In this work, we calibrate this tradeoff in a quantitative manner through a multi-criterion-based optimization. We demonstrate that $texttFedBC$ balances the global and local model test accuracy metrics across a suite datasets.
  arXiv  Detail & Related papers  (2022-06-22T02:42:04Z)
• Achieving Personalized Federated Learning with Sparse Local Models [75.76854544460981]
  Federated learning (FL) is vulnerable to heterogeneously distributed data. To counter this issue, personalized FL (PFL) was proposed to produce dedicated local models for each individual user. Existing PFL solutions either demonstrate unsatisfactory generalization towards different model architectures or cost enormous extra computation and memory. We proposeFedSpa, a novel PFL scheme that employs personalized sparse masks to customize sparse local models on the edge.
  arXiv  Detail & Related papers  (2022-01-27T08:43:11Z)
• DCVNet: Dilated Cost Volume Networks for Fast Optical Flow [5.526631378837701]
  The cost volume, capturing the similarity of possible correspondences across two input images, is a key ingredient in state-of-the-art optical flow approaches. We propose an alternative by constructing cost volumes with different dilation factors to capture small and large displacements simultaneously. A U-Net with skip connections is employed to convert the dilated cost volumes into weights between all possible captured displacements to get the optical flow.
  arXiv  Detail & Related papers  (2021-03-31T17:59:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.