HSI-Drive v2.0: More Data for New Challenges in Scene Understanding for Autonomous Driving

• URL: http://arxiv.org/abs/2411.17530v1
• Date: Tue, 26 Nov 2024 15:45:59 GMT
• Title: HSI-Drive v2.0: More Data for New Challenges in Scene Understanding for Autonomous Driving
• Abstract summary: We present the updated version of the HSI-Drive dataset aimed at developing automated driving systems (ADS) using hyperspectral imaging (HSI) The v2.0 version includes new annotated images from videos recorded during winter and fall in real driving scenarios. We show the improvements achieved over previously published results obtained on the v1.1 dataset, showcasing the enhanced performance of models trained on the new v2.0 dataset.
• Score: 1.2582887633807602
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present the updated version of the HSI-Drive dataset aimed at developing automated driving systems (ADS) using hyperspectral imaging (HSI). The v2.0 version includes new annotated images from videos recorded during winter and fall in real driving scenarios. Added to the spring and summer images included in the previous v1.1 version, the new dataset contains 752 images covering the four seasons. In this paper, we show the improvements achieved over previously published results obtained on the v1.1 dataset, showcasing the enhanced performance of models trained on the new v2.0 dataset. We also show the progress made in comprehensive scene understanding by experimenting with more capable image segmentation models. These models include new segmentation categories aimed at the identification of essential road safety objects such as the presence of vehicles and road signs, as well as highly vulnerable groups like pedestrians and cyclists. In addition, we provide evidence of the performance and robustness of the models when applied to segmenting HSI video sequences captured in various environments and conditions. Finally, for a correct assessment of the results described in this work, the constraints imposed by the processing platforms that can sensibly be deployed in vehicles for ADS must be taken into account. Thus, and although implementation details are out of the scope of this paper, we focus our research on the development of computationally efficient, lightweight ML models that can eventually operate at high throughput rates. The dataset and some examples of segmented videos are available in https://ipaccess.ehu.eus/HSI-Drive/.

Related papers

• SAVeD: A First-Person Social Media Video Dataset for ADAS-equipped vehicle Near-Miss and Crash Event Analyses [0.7874708385247353]
  This paper introduces SAVeD, a large-scale video dataset curated from publicly available social media content.<n>SAVED features 2,119 first-person videos, capturing ADAS vehicle operations in diverse locations, lighting conditions, and weather scenarios.<n>The dataset includes video frame-level annotations for collisions, evasive maneuvers, and disengagements, enabling analysis of both perception and decision-making failures.
  arXiv  Detail & Related papers  (2025-12-19T15:58:52Z)
• An object detection approach for lane change and overtake detection from motion profiles [3.545178658731506]
  In this paper, we address the identification of overtake and lane change maneuvers with a novel object detection approach applied to motion profiles. To train and test our model we created an internal dataset of motion profile images obtained from a heterogeneous set of dashcam videos. In addition to a standard object-detection approach, we show how the inclusion of CoordConvolution layers further improves the model performance.
  arXiv  Detail & Related papers  (2025-02-06T17:36:35Z)
• Label Correction for Road Segmentation Using Road-side Cameras [0.44241702149260353]
  Existing roadside camera infrastructure is utilized for collecting road data in varying weather conditions automatically. A novel semi-automatic annotation method for roadside cameras is proposed. The proposed method is validated with roadside camera data collected from 927 cameras across Finland over 4 month time period during winter.
  arXiv  Detail & Related papers  (2025-02-03T11:52:23Z)
• StreetCrafter: Street View Synthesis with Controllable Video Diffusion Models [59.55232046525733]
  We introduce StreetCrafter, a controllable video diffusion model that utilizes LiDAR point cloud renderings as pixel-level conditions. In addition, the utilization of pixel-level LiDAR conditions allows us to make accurate pixel-level edits to target scenes. Our model enables flexible control over viewpoint changes, enlarging the view for satisfying rendering regions.
  arXiv  Detail & Related papers  (2024-12-17T18:58:55Z)
• On-chip Hyperspectral Image Segmentation with Fully Convolutional Networks for Scene Understanding in Autonomous Driving [1.696186398088554]
  spectral reflectance of different objects in a driving scene beyond the visible spectrum can offer additional information. In this work we explore the use of snapshot, video-rate hyperspectral imaging (HSI) cameras in advanced driver assistance systems (ADAS) We analyze to what extent the spatial features codified by standard, tiny fully convolutional network (FCN) models can improve the performance of HSI segmentation systems.
  arXiv  Detail & Related papers  (2024-11-28T17:10:50Z)
• Self-Updating Vehicle Monitoring Framework Employing Distributed Acoustic Sensing towards Real-World Settings [5.306938463648908]
  We introduce a real-time semi-supervised vehicle monitoring framework tailored to urban settings. It requires only a small fraction of manual labels for initial training and exploits unlabeled data for model improvement. We propose a novel prior loss that incorporates the shapes of vehicular traces to track a single vehicle with varying speeds.
  arXiv  Detail & Related papers  (2024-09-16T13:10:58Z)
• WayveScenes101: A Dataset and Benchmark for Novel View Synthesis in Autonomous Driving [4.911903454560829]
  WayveScenes101 is a dataset designed to help the community advance the state of the art in novel view synthesis. The dataset comprises 101 driving scenes across a wide range of environmental conditions and driving scenarios.
  arXiv  Detail & Related papers  (2024-07-11T08:29:45Z)
• Beyond the Mud: Datasets and Benchmarks for Computer Vision in Off-Road Racing [38.719032057630024]
  Off-road motorcycle Racer Number dataset and Muddy Racer re-iDentification dataset feature over 6,300 images taken during off-road competitions. Off-the-shelf models transfer poorly, reaching only 15% end-to-end (E2E) F1 score on text spotting, and 33% rank-1 accuracy on ReID. Fine-tuning yields major improvements, bringing model performance to 53% F1 score for E2E text spotting and 79% rank-1 accuracy on ReID.
  arXiv  Detail & Related papers  (2024-02-12T19:42:05Z)
• MUSES: The Multi-Sensor Semantic Perception Dataset for Driving under Uncertainty [46.369657697892634]
  We introduce MUSES, the MUlti-SEnsor Semantic perception dataset for driving in adverse conditions under increased uncertainty. The dataset integrates a frame camera, a lidar, a radar, an event camera, and an IMU/GNSS sensor. MUSES proves both effective for training and challenging for evaluating models under diverse visual conditions.
  arXiv  Detail & Related papers  (2024-01-23T13:43:17Z)
• Towards Viewpoint Robustness in Bird's Eye View Segmentation [85.99907496019972]
  We study how AV perception models are affected by changes in camera viewpoint. Small changes to pitch, yaw, depth, or height of the camera at inference time lead to large drops in performance. We introduce a technique for novel view synthesis and use it to transform collected data to the viewpoint of target rigs.
  arXiv  Detail & Related papers  (2023-09-11T02:10:07Z)
• Street-View Image Generation from a Bird's-Eye View Layout [95.36869800896335]
  Bird's-Eye View (BEV) Perception has received increasing attention in recent years. Data-driven simulation for autonomous driving has been a focal point of recent research. We propose BEVGen, a conditional generative model that synthesizes realistic and spatially consistent surrounding images.
  arXiv  Detail & Related papers  (2023-01-11T18:39:34Z)
• 4Seasons: Benchmarking Visual SLAM and Long-Term Localization for Autonomous Driving in Challenging Conditions [54.59279160621111]
  We present a novel visual SLAM and long-term localization benchmark for autonomous driving in challenging conditions based on the large-scale 4Seasons dataset. The proposed benchmark provides drastic appearance variations caused by seasonal changes and diverse weather and illumination conditions. We introduce a new unified benchmark for jointly evaluating visual odometry, global place recognition, and map-based visual localization performance.
  arXiv  Detail & Related papers  (2022-12-31T13:52:36Z)
• A Fine-Grained Vehicle Detection (FGVD) Dataset for Unconstrained Roads [29.09167268252761]
  We introduce the first Fine-Grained Vehicle Detection dataset in the wild, captured from a moving camera mounted on a car. It contains 5502 scene images with 210 unique fine-grained labels of multiple vehicle types organized in a three-level hierarchy. While previous classification datasets also include makes for different kinds of cars, the FGVD dataset introduces new class labels for categorizing two-wheelers, autorickshaws, and trucks.
  arXiv  Detail & Related papers  (2022-12-30T06:50:15Z)
• SODA10M: Towards Large-Scale Object Detection Benchmark for Autonomous Driving [94.11868795445798]
  We release a Large-Scale Object Detection benchmark for Autonomous driving, named as SODA10M, containing 10 million unlabeled images and 20K images labeled with 6 representative object categories. To improve diversity, the images are collected every ten seconds per frame within 32 different cities under different weather conditions, periods and location scenes. We provide extensive experiments and deep analyses of existing supervised state-of-the-art detection models, popular self-supervised and semi-supervised approaches, and some insights about how to develop future models.
  arXiv  Detail & Related papers  (2021-06-21T13:55:57Z)
• Salient Objects in Clutter [130.63976772770368]
  This paper identifies and addresses a serious design bias of existing salient object detection (SOD) datasets. This design bias has led to a saturation in performance for state-of-the-art SOD models when evaluated on existing datasets. We propose a new high-quality dataset and update the previous saliency benchmark.
  arXiv  Detail & Related papers  (2021-05-07T03:49:26Z)
• Understanding Road Layout from Videos as a Whole [82.30800791500869]
  We formulate it as a top-view road attributes prediction problem and our goal is to predict these attributes for each frame both accurately and consistently. We exploit the following three novel aspects: leveraging camera motions in videos, including context cuesand incorporating long-term video information.
  arXiv  Detail & Related papers  (2020-07-02T00:59:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.