A Comparative Assessment of Multi-view fusion learning for Crop Classification

• URL: http://arxiv.org/abs/2308.05407v1
• Date: Thu, 10 Aug 2023 08:03:58 GMT
• Title: A Comparative Assessment of Multi-view fusion learning for Crop Classification
• Authors: Francisco Mena, Diego Arenas, Marlon Nuske, Andreas Dengel
• Abstract summary: This work assesses different fusion strategies for crop classification in the CropHarvest dataset. We present a comparison of multi-view fusion methods for three different datasets and show that, depending on the test region, different methods obtain the best performance.
• Score: 3.883984493622102
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With a rapidly increasing amount and diversity of remote sensing (RS) data sources, there is a strong need for multi-view learning modeling. This is a complex task when considering the differences in resolution, magnitude, and noise of RS data. The typical approach for merging multiple RS sources has been input-level fusion, but other - more advanced - fusion strategies may outperform this traditional approach. This work assesses different fusion strategies for crop classification in the CropHarvest dataset. The fusion methods proposed in this work outperform models based on individual views and previous fusion methods. We do not find one single fusion method that consistently outperforms all other approaches. Instead, we present a comparison of multi-view fusion methods for three different datasets and show that, depending on the test region, different methods obtain the best performance. Despite this, we suggest a preliminary criterion for the selection of fusion methods.

Related papers

• How Intermodal Interaction Affects the Performance of Deep Multimodal Fusion for Mixed-Type Time Series [3.6958071416494414]
  Mixed-type time series (MTTS) is a bimodal data type common in many domains, such as healthcare, finance, environmental monitoring, and social media. The integration of both modalities through multimodal fusion is a promising approach for processing MTTS. We present a comprehensive evaluation of several deep multimodal fusion approaches for MTTS forecasting.
  arXiv  Detail & Related papers  (2024-06-21T12:26:48Z)
• FusionBench: A Comprehensive Benchmark of Deep Model Fusion [78.80920533793595]
  Deep model fusion is a technique that unifies the predictions or parameters of several deep neural networks into a single model. FusionBench is the first comprehensive benchmark dedicated to deep model fusion.
  arXiv  Detail & Related papers  (2024-06-05T13:54:28Z)
• Convolutional autoencoder-based multimodal one-class classification [80.52334952912808]
  One-class classification refers to approaches of learning using data from a single class only. We propose a deep learning one-class classification method suitable for multimodal data.
  arXiv  Detail & Related papers  (2023-09-25T12:31:18Z)
• Central Similarity Multi-View Hashing for Multimedia Retrieval [14.766486538338498]
  We present a novel Central Similarity Multi-View Hashing (CSMVH) method to address the mentioned problems. On the MS COCO and NUS-WIDE, the proposed CSMVH performs better than the state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2023-08-26T05:43:29Z)
• Image Fusion Transformer [75.71025138448287]
  In image fusion, images obtained from different sensors are fused to generate a single image with enhanced information. In recent years, state-of-the-art methods have adopted Convolution Neural Networks (CNNs) to encode meaningful features for image fusion. We propose a novel Image Fusion Transformer (IFT) where we develop a transformer-based multi-scale fusion strategy.
  arXiv  Detail & Related papers  (2021-07-19T16:42:49Z)
• Attention Bottlenecks for Multimodal Fusion [90.75885715478054]
  Machine perception models are typically modality-specific and optimised for unimodal benchmarks. We introduce a novel transformer based architecture that uses fusion' for modality fusion at multiple layers. We conduct thorough ablation studies, and achieve state-of-the-art results on multiple audio-visual classification benchmarks.
  arXiv  Detail & Related papers  (2021-06-30T22:44:12Z)
• Multimodal Object Detection via Bayesian Fusion [59.31437166291557]
  We study multimodal object detection with RGB and thermal cameras, since the latter can provide much stronger object signatures under poor illumination. Our key contribution is a non-learned late-fusion method that fuses together bounding box detections from different modalities. We apply our approach to benchmarks containing both aligned (KAIST) and unaligned (FLIR) multimodal sensor data.
  arXiv  Detail & Related papers  (2021-04-07T04:03:20Z)
• Subspace-Based Feature Fusion From Hyperspectral And Multispectral Image For Land Cover Classification [17.705966155216945]
  A feature fusion method from hyperspectral (HS) and multispectral (MS) images for pixel-based classification is proposed. The proposed method first extracts spatial features from the MS image using morphological profiles. An algorithm based on combining alternating optimization (AO) and the alternating direction method of multipliers (ADMM) is developed to solve efficiently the feature fusion problem.
  arXiv  Detail & Related papers  (2021-02-22T17:59:18Z)
• Memory based fusion for multi-modal deep learning [39.29589204750581]
  Memory based Attentive Fusion layer fuses modes by incorporating both the current features and longterm dependencies in the data. We present a novel Memory based Attentive Fusion layer, which fuses modes by incorporating both the current features and longterm dependencies in the data.
  arXiv  Detail & Related papers  (2020-07-16T02:05:54Z)
• VMLoc: Variational Fusion For Learning-Based Multimodal Camera Localization [46.607930208613574]
  We propose an end-to-end framework, termed VMLoc, to fuse different sensor inputs into a common latent space. Unlike previous multimodal variational works directly adapting the objective function of vanilla variational auto-encoder, we show how camera localization can be accurately estimated.
  arXiv  Detail & Related papers  (2020-03-12T14:52:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.