CoreNet: Conflict Resolution Network for Point-Pixel Misalignment and Sub-Task Suppression of 3D LiDAR-Camera Object Detection

• URL: http://arxiv.org/abs/2501.06550v1
• Date: Sat, 11 Jan 2025 14:08:40 GMT
• Title: CoreNet: Conflict Resolution Network for Point-Pixel Misalignment and Sub-Task Suppression of 3D LiDAR-Camera Object Detection
• Abstract summary: Current methods overlook two important conflicts: point-pixel misalignment and sub-task suppression.<n>We propose a novel method named Conflict Resolution Network (CoreNet) to address the aforementioned issues.<n> Experiments on the large-scale nuScenes dataset demonstrate the superiority of our proposed CoreNet.
• Score: 16.37397687985041
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Fusing multi-modality inputs from different sensors is an effective way to improve the performance of 3D object detection. However, current methods overlook two important conflicts: point-pixel misalignment and sub-task suppression. The former means a pixel feature from the opaque object is projected to multiple point features of the same ray in the world space, and the latter means the classification prediction and bounding box regression may cause mutual suppression. In this paper, we propose a novel method named Conflict Resolution Network (CoreNet) to address the aforementioned issues. Specifically, we first propose a dual-stream transformation module to tackle point-pixel misalignment. It consists of ray-based and point-based 2D-to-BEV transformations. Both of them achieve approximately unique mapping from the image space to the world space. Moreover, we introduce a task-specific predictor to tackle sub-task suppression. It uses the dual-branch structure which adopts class-specific query and Bbox-specific query to corresponding sub-tasks. Each task-specific query is constructed of task-specific feature and general feature, which allows the heads to adaptively select information of interest based on different sub-tasks. Experiments on the large-scale nuScenes dataset demonstrate the superiority of our proposed CoreNet, by achieving 75.6\% NDS and 73.3\% mAP on the nuScenes test set without test-time augmentation and model ensemble techniques. The ample ablation study also demonstrates the effectiveness of each component. The code is released on https://github.com/liyih/CoreNet.

Related papers

• DGNet: Dual-knowledge Guided Network for Infrared Small Target Detection [43.281292792735364]
  InfRared Small Target Detection (IRSTD) is a prominent and challenging task in computer vision.<n>We propose a novel Dual-knowledge Guided Network (DGNet) based on multiple generalizable texts.
  arXiv  Detail & Related papers  (2026-09-01T03:44:47Z)
• Dual-Thresholding Heatmaps to Cluster Proposals for Weakly Supervised Object Detection [19.807828545088082]
  Weakly supervised object detection (WSOD) has attracted significant attention in recent years.<n>We present a weakly supervised basic detection network (WSBDN), which augments each proposal with a background class representation.<n>We achieve mAP/mCorLoc scores of 58.5%/81.8% on VOC 2007 and 55.6%/80.5% on VOC 2012, performing favorably against the state-of-the-art WSOD methods.
  arXiv  Detail & Related papers  (2025-09-10T05:12:03Z)
• Project-and-Fuse: Improving RGB-D Semantic Segmentation via Graph Convolution Networks [21.713293775719414]
  We propose to fuse features from two modalities in a late fusion style, during which the geometric feature injection is guided by texture feature prior. At the 3D feature extraction stage, we argue that traditional CNNs are not efficient enough for depth maps. At projection matrix generation stage, we find the existence of Biased-Assignment and Ambiguous-Locality issues in the original pipeline.
  arXiv  Detail & Related papers  (2025-01-31T02:24:13Z)
• LESS: Label-Efficient and Single-Stage Referring 3D Segmentation [55.06002976797879]
  Referring 3D is a visual-language task that segments all points of the specified object from a 3D point cloud described by a sentence of query. We propose a novel Referring 3D pipeline, Label-Efficient and Single-Stage, dubbed LESS, which is only under the supervision of efficient binary mask. We achieve state-of-the-art performance on ScanRefer dataset by surpassing the previous methods about 3.7% mIoU using only binary labels.
  arXiv  Detail & Related papers  (2024-10-17T07:47:41Z)
• SegVG: Transferring Object Bounding Box to Segmentation for Visual Grounding [56.079013202051094]
  We present SegVG, a novel method transfers the box-level annotation as signals to provide an additional pixel-level supervision for Visual Grounding. This approach allows us to iteratively exploit the annotation as signals for both box-level regression and pixel-level segmentation.
  arXiv  Detail & Related papers  (2024-07-03T15:30:45Z)
• PV-SSD: A Multi-Modal Point Cloud Feature Fusion Method for Projection Features and Variable Receptive Field Voxel Features [1.5338480419018068]
  Real-time inference from extremely sparse 3D data is a formidable challenge. To address this problem, a typical class of approaches transforms the point cloud cast into a regular data representation. This paper proposes a multi-modal point cloud feature fusion method for projection features and variable receptive field voxel features.
  arXiv  Detail & Related papers  (2023-08-13T15:30:02Z)
• ComPtr: Towards Diverse Bi-source Dense Prediction Tasks via A Simple yet General Complementary Transformer [71.82644727907146]
  We propose a novel $underlineComP$lementary $underlinetr$ansformer, $textbfComPtr$, for diverse bi-source dense prediction tasks.<n>ComPtr treats different inputs equally and builds an efficient dense interaction model in the form of sequence-to-sequence on top of the transformer.
  arXiv  Detail & Related papers  (2023-07-23T15:17:45Z)
• Spatial-Temporal Graph Enhanced DETR Towards Multi-Frame 3D Object Detection [54.041049052843604]
  We present STEMD, a novel end-to-end framework that enhances the DETR-like paradigm for multi-frame 3D object detection. First, to model the inter-object spatial interaction and complex temporal dependencies, we introduce the spatial-temporal graph attention network. Finally, it poses a challenge for the network to distinguish between the positive query and other highly similar queries that are not the best match.
  arXiv  Detail & Related papers  (2023-07-01T13:53:14Z)
• Object Segmentation by Mining Cross-Modal Semantics [68.88086621181628]
  We propose a novel approach by mining the Cross-Modal Semantics to guide the fusion and decoding of multimodal features. Specifically, we propose a novel network, termed XMSNet, consisting of (1) all-round attentive fusion (AF), (2) coarse-to-fine decoder (CFD), and (3) cross-layer self-supervision.
  arXiv  Detail & Related papers  (2023-05-17T14:30:11Z)
• Density-Insensitive Unsupervised Domain Adaption on 3D Object Detection [19.703181080679176]
  3D object detection from point clouds is crucial in safety-critical autonomous driving. We propose a density-insensitive domain adaption framework to address the density-induced domain gap. Experimental results on three widely adopted 3D object detection datasets demonstrate that our proposed domain adaption method outperforms the state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-04-19T06:33:07Z)
• Task-Specific Context Decoupling for Object Detection [27.078743716924752]
  Exsiting methods usually leverage disentangled heads to learn different feature context for each task. We propose a novel Task-Specific COntext DEcoupling (TSCODE) head which further disentangles the feature encoding for two tasks. Our method stably improves different detectors by over 1.0 AP with less computational cost.
  arXiv  Detail & Related papers  (2023-03-02T08:02:14Z)
• Multi-patch Feature Pyramid Network for Weakly Supervised Object Detection in Optical Remote Sensing Images [39.25541709228373]
  We propose a new architecture for object detection with a multiple patch feature pyramid network (MPFP-Net) MPFP-Net is different from the current models that during training only pursue the most discriminative patches. We introduce an effective method to regularize the residual values and make the fusion transition layers strictly norm-preserving.
  arXiv  Detail & Related papers  (2021-08-18T09:25:39Z)
• Improving Point Cloud Semantic Segmentation by Learning 3D Object Detection [102.62963605429508]
  Point cloud semantic segmentation plays an essential role in autonomous driving. Current 3D semantic segmentation networks focus on convolutional architectures that perform great for well represented classes. We propose a novel Aware 3D Semantic Detection (DASS) framework that explicitly leverages localization features from an auxiliary 3D object detection task.
  arXiv  Detail & Related papers  (2020-09-22T14:17:40Z)
• Stereo RGB and Deeper LIDAR Based Network for 3D Object Detection [40.34710686994996]
  3D object detection has become an emerging task in autonomous driving scenarios. Previous works process 3D point clouds using either projection-based or voxel-based models. We propose the Stereo RGB and Deeper LIDAR framework which can utilize semantic and spatial information simultaneously.
  arXiv  Detail & Related papers  (2020-06-09T11:19:24Z)
• Learning 2D-3D Correspondences To Solve The Blind Perspective-n-Point Problem [98.92148855291363]
  This paper proposes a deep CNN model which simultaneously solves for both 6-DoF absolute camera pose 2D--3D correspondences. Tests on both real and simulated data have shown that our method substantially outperforms existing approaches.
  arXiv  Detail & Related papers  (2020-03-15T04:17:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.