CountFormer: Multi-View Crowd Counting Transformer

• URL: http://arxiv.org/abs/2407.02047v1
• Date: Tue, 2 Jul 2024 08:19:48 GMT
• Title: CountFormer: Multi-View Crowd Counting Transformer
• Authors: Hong Mo, Xiong Zhang, Jianchao Tan, Cheng Yang, Qiong Gu, Bo Hang, Wenqi Ren,
• Abstract summary: We propose a 3D MVC framework called textbfCountFormer to elevate multi-view image-level features to a scene-level volume representation. By incorporating a camera encoding strategy, CountFormer successfully embeds camera parameters into the volume query and image-level features. The proposed method performs favorably against the state-of-the-art approaches across various widely used datasets.
• Score: 43.92763885594129
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-view counting (MVC) methods have shown their superiority over single-view counterparts, particularly in situations characterized by heavy occlusion and severe perspective distortions. However, hand-crafted heuristic features and identical camera layout requirements in conventional MVC methods limit their applicability and scalability in real-world scenarios.In this work, we propose a concise 3D MVC framework called \textbf{CountFormer}to elevate multi-view image-level features to a scene-level volume representation and estimate the 3D density map based on the volume features. By incorporating a camera encoding strategy, CountFormer successfully embeds camera parameters into the volume query and image-level features, enabling it to handle various camera layouts with significant differences.Furthermore, we introduce a feature lifting module capitalized on the attention mechanism to transform image-level features into a 3D volume representation for each camera view. Subsequently, the multi-view volume aggregation module attentively aggregates various multi-view volumes to create a comprehensive scene-level volume representation, allowing CountFormer to handle images captured by arbitrary dynamic camera layouts. The proposed method performs favorably against the state-of-the-art approaches across various widely used datasets, demonstrating its greater suitability for real-world deployment compared to conventional MVC frameworks.

Related papers

• Fancy123: One Image to High-Quality 3D Mesh Generation via Plug-and-Play Deformation [22.5996658181606]
  We propose Fancy123, featuring two enhancement modules and an unprojection operation to address the above three issues. The appearance enhancement module deforms the 2D multiview images to realign pixels for better multiview consistency. The fidelity enhancement module deforms the 3D mesh to match the input image. The unprojection of the input image and deformed multiview images onto LRM's generated mesh ensures high clarity.
  arXiv  Detail & Related papers  (2024-11-25T08:31:55Z)
• MVGenMaster: Scaling Multi-View Generation from Any Image via 3D Priors Enhanced Diffusion Model [87.71060849866093]
  We introduce MVGenMaster, a multi-view diffusion model enhanced with 3D priors to address versatile Novel View Synthesis (NVS) tasks. Our model features a simple yet effective pipeline that can generate up to 100 novel views conditioned on variable reference views and camera poses. We present several training and model modifications to strengthen the model with scaled-up datasets.
  arXiv  Detail & Related papers  (2024-11-25T07:34:23Z)
• A Global Depth-Range-Free Multi-View Stereo Transformer Network with Pose Embedding [76.44979557843367]
  We propose a novel multi-view stereo (MVS) framework that gets rid of the depth range prior. We introduce a Multi-view Disparity Attention (MDA) module to aggregate long-range context information. We explicitly estimate the quality of the current pixel corresponding to sampled points on the epipolar line of the source image.
  arXiv  Detail & Related papers  (2024-11-04T08:50:16Z)
• Scaling Multi-Camera 3D Object Detection through Weak-to-Strong Eliciting [32.66151412557986]
  We present a weak-to-strong eliciting framework aimed at enhancing surround refinement while maintaining robust monocular perception. Our framework employs weakly tuned experts trained on distinct subsets, and each is inherently biased toward specific camera configurations and scenarios. For MC3D-Det joint training, the elaborate dataset merge strategy is designed to solve the problem of inconsistent camera numbers and camera parameters.
  arXiv  Detail & Related papers  (2024-04-10T03:11:10Z)
• MuVieCAST: Multi-View Consistent Artistic Style Transfer [6.767885381740952]
  We introduce MuVieCAST, a modular multi-view consistent style transfer network architecture. MuVieCAST supports both sparse and dense views, making it versatile enough to handle a wide range of multi-view image datasets.
  arXiv  Detail & Related papers  (2023-12-08T14:01:03Z)
• ConsistNet: Enforcing 3D Consistency for Multi-view Images Diffusion [61.37481051263816]
  Given a single image of a 3D object, this paper proposes a method (named ConsistNet) that is able to generate multiple images of the same object. Our method effectively learns 3D consistency over a frozen Zero123 backbone and can generate 16 surrounding views of the object within 40 seconds on a single A100 GPU.
  arXiv  Detail & Related papers  (2023-10-16T12:29:29Z)
• MAIR: Multi-view Attention Inverse Rendering with 3D Spatially-Varying Lighting Estimation [13.325800282424598]
  We propose a scene-level inverse rendering framework that uses multi-view images to decompose the scene into geometry, a SVBRDF, and 3D spatially-varying lighting. Our experiments show that the proposed method achieves better performance than single-view-based methods, but also achieves robust performance on unseen real-world scene.
  arXiv  Detail & Related papers  (2023-03-22T08:07:28Z)
• Cross-View Cross-Scene Multi-View Crowd Counting [56.83882084112913]
  Multi-view crowd counting has been previously proposed to utilize multi-cameras to extend the field-of-view of a single camera. We propose a cross-view cross-scene (CVCS) multi-view crowd counting paradigm, where the training and testing occur on different scenes with arbitrary camera layouts.
  arXiv  Detail & Related papers  (2022-05-03T15:03:44Z)
• DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Cameras [63.186486240525554]
  DeepMultiCap is a novel method for multi-person performance capture using sparse multi-view cameras. Our method can capture time varying surface details without the need of using pre-scanned template models.
  arXiv  Detail & Related papers  (2021-05-01T14:32:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.