Crop mapping from image time series: deep learning with multi-scale label hierarchies

• URL: http://arxiv.org/abs/2102.08820v1
• Date: Wed, 17 Feb 2021 15:27:49 GMT
• Title: Crop mapping from image time series: deep learning with multi-scale label hierarchies
• Authors: Mehmet Ozgur Turkoglu, Stefano D'Aronco, Gregor Perich, Frank Liebisch, Constantin Streit, Konrad Schindler, Jan Dirk Wegner
• Abstract summary: We develop a crop classification method that exploits expert knowledge and significantly improves the mapping of rare crop types. The three-level label hierarchy is encoded in a convolutional, recurrent neural network (convRNN) We validate the proposed method on a new, large dataset that we make public.
• Score: 22.58506027920305
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The aim of this paper is to map agricultural crops by classifying satellite image time series. Domain experts in agriculture work with crop type labels that are organised in a hierarchical tree structure, where coarse classes (like orchards) are subdivided into finer ones (like apples, pears, vines, etc.). We develop a crop classification method that exploits this expert knowledge and significantly improves the mapping of rare crop types. The three-level label hierarchy is encoded in a convolutional, recurrent neural network (convRNN), such that for each pixel the model predicts three labels at different level of granularity. This end-to-end trainable, hierarchical network architecture allows the model to learn joint feature representations of rare classes (e.g., apples, pears) at a coarser level (e.g., orchard), thereby boosting classification performance at the fine-grained level. Additionally, labelling at different granularity also makes it possible to adjust the output according to the classification scores; as coarser labels with high confidence are sometimes more useful for agricultural practice than fine-grained but very uncertain labels. We validate the proposed method on a new, large dataset that we make public. ZueriCrop covers an area of 50 km x 48 km in the Swiss cantons of Zurich and Thurgau with a total of 116'000 individual fields spanning 48 crop classes, and 28,000 (multi-temporal) image patches from Sentinel-2. We compare our proposed hierarchical convRNN model with several baselines, including methods designed for imbalanced class distributions. The hierarchical approach performs superior by at least 9.9 percentage points in F1-score.

Related papers

• A Capsule Network for Hierarchical Multi-Label Image Classification [2.507647327384289]
  Hierarchical multi-label classification applies when a multi-class image classification problem is arranged into smaller ones based upon a hierarchy or taxonomy. We propose a multi-label capsule network (ML-CapsNet) for hierarchical classification.
  arXiv  Detail & Related papers  (2022-09-13T04:17:08Z)
• Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework [75.79736930414715]
  We present a hierarchical multi-label representation learning framework that can leverage all available labels and preserve the hierarchical relationship between classes. We introduce novel hierarchy preserving losses, which jointly apply a hierarchical penalty to the contrastive loss, and enforce the hierarchy constraint.
  arXiv  Detail & Related papers  (2022-04-27T21:41:44Z)
• Label Relation Graphs Enhanced Hierarchical Residual Network for Hierarchical Multi-Granularity Classification [10.449261628173229]
  We study the HMC problem in which objects are labeled at any level of the hierarchy. We propose a hierarchical residual network (HRN) in which residual connections are added to features of children levels.
  arXiv  Detail & Related papers  (2022-01-10T07:17:24Z)
• Label Hierarchy Transition: Delving into Class Hierarchies to Enhance Deep Classifiers [40.993137740456014]
  We propose a unified probabilistic framework based on deep learning to address the challenges of hierarchical classification. The proposed framework can be readily adapted to any existing deep network with only minor modifications. We extend our proposed LHT framework to the skin lesion diagnosis task and validate its great potential in computer-aided diagnosis.
  arXiv  Detail & Related papers  (2021-12-04T14:58:36Z)
• Semi-Supervised Learning with Taxonomic Labels [42.02670649470055]
  We propose techniques to incorporate coarse taxonomic labels to train image classifiers in fine-grained domains. On the Semi-iNat dataset consisting of 810 species across three Kingdoms, incorporating Phylum labels improves the Species level classification accuracy by 6%. We propose a technique to select relevant data from a large collection of unlabeled images guided by the hierarchy which improves the robustness.
  arXiv  Detail & Related papers  (2021-11-23T00:50:25Z)
• MMF: Multi-Task Multi-Structure Fusion for Hierarchical Image Classification [10.713537820833665]
  We consider that different label structures provide a variety of prior knowledge for category recognition. We propose a multi-task multi-structure fusion model to integrate different label structures.
  arXiv  Detail & Related papers  (2021-07-02T02:53:35Z)
• Noisy Labels Can Induce Good Representations [53.47668632785373]
  We study how architecture affects learning with noisy labels. We show that training with noisy labels can induce useful hidden representations, even when the model generalizes poorly. This finding leads to a simple method to improve models trained on noisy labels.
  arXiv  Detail & Related papers  (2020-12-23T18:58:05Z)
• Grafit: Learning fine-grained image representations with coarse labels [114.17782143848315]
  This paper tackles the problem of learning a finer representation than the one provided by training labels. By jointly leveraging the coarse labels and the underlying fine-grained latent space, it significantly improves the accuracy of category-level retrieval methods.
  arXiv  Detail & Related papers  (2020-11-25T19:06:26Z)
• Exploring the Hierarchy in Relation Labels for Scene Graph Generation [75.88758055269948]
  The proposed method can improve several state-of-the-art baselines by a large margin (up to $33%$ relative gain) in terms of Recall@50. Experiments show that the proposed simple yet effective method can improve several state-of-the-art baselines by a large margin.
  arXiv  Detail & Related papers  (2020-09-12T17:36:53Z)
• Two-View Fine-grained Classification of Plant Species [66.75915278733197]
  We propose a novel method based on a two-view leaf image representation and a hierarchical classification strategy for fine-grained recognition of plant species. A deep metric based on Siamese convolutional neural networks is used to reduce the dependence on a large number of training samples and make the method scalable to new plant species.
  arXiv  Detail & Related papers  (2020-05-18T21:57:47Z)
• Hierarchical Image Classification using Entailment Cone Embeddings [68.82490011036263]
  We first inject label-hierarchy knowledge into an arbitrary CNN-based classifier. We empirically show that availability of such external semantic information in conjunction with the visual semantics from images boosts overall performance.
  arXiv  Detail & Related papers  (2020-04-02T10:22:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.