FlowCaps: Optical Flow Estimation with Capsule Networks For Action Recognition

• URL: http://arxiv.org/abs/2011.03958v1
• Date: Sun, 8 Nov 2020 11:35:08 GMT
• Title: FlowCaps: Optical Flow Estimation with Capsule Networks For Action Recognition
• Authors: Vinoj Jayasundara, Debaditya Roy, Basura Fernando
• Abstract summary: Capsule networks (CapsNets) have recently shown promise to excel in most computer vision tasks. We propose a CapsNet-based architecture, termed FlowCaps, which attempts to achieve better correspondence matching via finer-grained, motion-specific, and more-interpretable encoding.
• Score: 22.712379018181398
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Capsule networks (CapsNets) have recently shown promise to excel in most computer vision tasks, especially pertaining to scene understanding. In this paper, we explore CapsNet's capabilities in optical flow estimation, a task at which convolutional neural networks (CNNs) have already outperformed other approaches. We propose a CapsNet-based architecture, termed FlowCaps, which attempts to a) achieve better correspondence matching via finer-grained, motion-specific, and more-interpretable encoding crucial for optical flow estimation, b) perform better-generalizable optical flow estimation, c) utilize lesser ground truth data, and d) significantly reduce the computational complexity in achieving good performance, in comparison to its CNN-counterparts.

Related papers

• Channel Fingerprint Construction for Massive MIMO: A Deep Conditional Generative Approach [65.47969413708344]
  We introduce the concept of CF twins and design a conditional generative diffusion model (CGDM)<n>We employ a variational inference technique to derive the evidence lower bound (ELBO) for the log-marginal distribution of the observed fine-grained CF conditioned on the coarse-grained CF.<n>We show that the proposed approach exhibits significant improvement in reconstruction performance compared to the baselines.
  arXiv  Detail & Related papers  (2025-05-12T01:36:06Z)
• Context-Aware Iteration Policy Network for Efficient Optical Flow Estimation [19.077966606237897]
  We develop a Context-Aware Iteration Policy Network for efficient optical flow estimation. Our method maintains performance while reducing FLOPs by about 40%/20% for the Sintel/ KITTI datasets.
  arXiv  Detail & Related papers  (2023-12-12T11:27:13Z)
• FlowX: Towards Explainable Graph Neural Networks via Message Flows [59.025023020402365]
  We investigate the explainability of graph neural networks (GNNs) as a step toward elucidating their working mechanisms. We propose a novel method here, known as FlowX, to explain GNNs by identifying important message flows. We then propose an information-controlled learning algorithm to train flow scores toward diverse explanation targets.
  arXiv  Detail & Related papers  (2022-06-26T22:48:15Z)
• Enabling Capsule Networks at the Edge through Approximate Softmax and Squash Operations [11.82717649039321]
  We propose to leverage approximate computing for designing approximate variants of the complex operations like softmax and squash. In our experiments, we evaluate tradeoffs between area, power consumption, and critical path delay of the designs implemented with the ASIC design flow.
  arXiv  Detail & Related papers  (2022-06-21T08:57:16Z)
• A Lightweight, Efficient and Explainable-by-Design Convolutional Neural Network for Internet Traffic Classification [9.365794791156972]
  This paper introduces a new Lightweight, Efficient and eXplainable-by-design convolutional neural network (LEXNet) for Internet traffic classification. LEXNet relies on a new residual block (for lightweight and efficiency purposes) and prototype layer (for explainability) Based on a commercial-grade dataset, our evaluation shows that LEXNet succeeds to maintain the same accuracy as the best performing state-of-the-art neural network.
  arXiv  Detail & Related papers  (2022-02-11T10:21:34Z)
• GMFlow: Learning Optical Flow via Global Matching [124.57850500778277]
  We propose a GMFlow framework for learning optical flow estimation. It consists of three main components: a customized Transformer for feature enhancement, a correlation and softmax layer for global feature matching, and a self-attention layer for flow propagation. Our new framework outperforms 32-iteration RAFT's performance on the challenging Sintel benchmark.
  arXiv  Detail & Related papers  (2021-11-26T18:59:56Z)
• Semi-supervised Network Embedding with Differentiable Deep Quantisation [81.49184987430333]
  We develop d-SNEQ, a differentiable quantisation method for network embedding. d-SNEQ incorporates a rank loss to equip the learned quantisation codes with rich high-order information. It is able to substantially compress the size of trained embeddings, thus reducing storage footprint and accelerating retrieval speed.
  arXiv  Detail & Related papers  (2021-08-20T11:53:05Z)
• OAS-Net: Occlusion Aware Sampling Network for Accurate Optical Flow [4.42249337449125]
  Existing deep networks have achieved satisfactory results by mostly employing a pyramidal coarse-to-fine paradigm. We propose a lightweight yet efficient optical flow network, named OAS-Net, for accurate optical flow. Experiments on Sintel and KITTI datasets demonstrate the effectiveness of proposed approaches.
  arXiv  Detail & Related papers  (2021-01-31T03:30:31Z)
• FDFlowNet: Fast Optical Flow Estimation using a Deep Lightweight Network [12.249680550252327]
  We present a lightweight yet effective model for real-time optical flow estimation, termed FDFlowNet (fast deep flownet) We achieve better or similar accuracy on the challenging KITTI and Sintel benchmarks while being about 2 times faster than PWC-Net.
  arXiv  Detail & Related papers  (2020-06-22T14:01:01Z)
• Network Adjustment: Channel Search Guided by FLOPs Utilization Ratio [101.84651388520584]
  This paper presents a new framework named network adjustment, which considers network accuracy as a function of FLOPs. Experiments on standard image classification datasets and a wide range of base networks demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-04-06T15:51:00Z)
• What Deep CNNs Benefit from Global Covariance Pooling: An Optimization Perspective [102.37204254403038]
  We make an attempt to understand what deep CNNs benefit from GCP in a viewpoint of optimization. We show that GCP can make the optimization landscape more smooth and the gradients more predictive. We conduct extensive experiments using various deep CNN models on diversified tasks, and the results provide strong support to our findings.
  arXiv  Detail & Related papers  (2020-03-25T07:00:45Z)
• ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions [76.05981545084738]
  We propose several ideas for enhancing a binary network to close its accuracy gap from real-valued networks without incurring any additional computational cost. We first construct a baseline network by modifying and binarizing a compact real-valued network with parameter-free shortcuts. We show that the proposed ReActNet outperforms all the state-of-the-arts by a large margin.
  arXiv  Detail & Related papers  (2020-03-07T02:12:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.