DT/MARS-CycleGAN: Improved Object Detection for MARS Phenotyping Robot

• URL: http://arxiv.org/abs/2310.12787v2
• Date: Fri, 20 Oct 2023 15:55:14 GMT
• Title: DT/MARS-CycleGAN: Improved Object Detection for MARS Phenotyping Robot
• Authors: David Liu, Zhengkun Li, Zihao Wu, Changying Li
• Abstract summary: This work proposes a novel Digital-Twin(DT)MARS-CycleGAN model for image augmentation to improve our Modular Agricultural Robotic System's crop object detection. In addition to the cycle consistency losses in the CycleGAN model, we designed and enforced a new DT-MARS loss in the deep learning model to penalize the inconsistency between real crop images captured by MARS and synthesized images sensed by DT MARS.
• Score: 11.869108981066429
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Robotic crop phenotyping has emerged as a key technology to assess crops' morphological and physiological traits at scale. These phenotypical measurements are essential for developing new crop varieties with the aim of increasing productivity and dealing with environmental challenges such as climate change. However, developing and deploying crop phenotyping robots face many challenges such as complex and variable crop shapes that complicate robotic object detection, dynamic and unstructured environments that baffle robotic control, and real-time computing and managing big data that challenge robotic hardware/software. This work specifically tackles the first challenge by proposing a novel Digital-Twin(DT)MARS-CycleGAN model for image augmentation to improve our Modular Agricultural Robotic System (MARS)'s crop object detection from complex and variable backgrounds. Our core idea is that in addition to the cycle consistency losses in the CycleGAN model, we designed and enforced a new DT-MARS loss in the deep learning model to penalize the inconsistency between real crop images captured by MARS and synthesized images sensed by DT MARS. Therefore, the generated synthesized crop images closely mimic real images in terms of realism, and they are employed to fine-tune object detectors such as YOLOv8. Extensive experiments demonstrated that our new DT/MARS-CycleGAN framework significantly boosts our MARS' crop object/row detector's performance, contributing to the field of robotic crop phenotyping.

Related papers

• Generating Diverse Agricultural Data for Vision-Based Farming Applications [74.79409721178489]
  This model is capable of simulating distinct growth stages of plants, diverse soil conditions, and randomized field arrangements under varying lighting conditions. Our dataset includes 12,000 images with semantic labels, offering a comprehensive resource for computer vision tasks in precision agriculture.
  arXiv  Detail & Related papers  (2024-03-27T08:42:47Z)
• Enhanced Droplet Analysis Using Generative Adversarial Networks [0.0]
  This work develops an image generator named DropletGAN to generate images of droplets. It is also used to develop a light droplet detector using the synthetic dataset. To the best of our knowledge, this work stands as the first to employ a generative model for augmenting droplet detection.
  arXiv  Detail & Related papers  (2024-02-24T21:20:53Z)
• GenFace: A Large-Scale Fine-Grained Face Forgery Benchmark and Cross Appearance-Edge Learning [50.7702397913573]
  The rapid advancement of photorealistic generators has reached a critical juncture where the discrepancy between authentic and manipulated images is increasingly indistinguishable. Although there have been a number of publicly available face forgery datasets, the forgery faces are mostly generated using GAN-based synthesis technology. We propose a large-scale, diverse, and fine-grained high-fidelity dataset, namely GenFace, to facilitate the advancement of deepfake detection.
  arXiv  Detail & Related papers  (2024-02-03T03:13:50Z)
• Real-time object detection and robotic manipulation for agriculture using a YOLO-based learning approach [8.482182765640022]
  This study presents a new framework that combines two separate architectures of convolutional neural networks (CNNs) Crop images in a simulated environment are subjected to random rotations, cropping, brightness, and contrast adjustments to create augmented images for dataset generation. The proposed method subsequently utilise the acquired image data via a visual geometry group model in order to reveal the grasping positions for the robotic manipulation.
  arXiv  Detail & Related papers  (2024-01-28T22:30:50Z)
• Robotic Handling of Compliant Food Objects by Robust Learning from Demonstration [79.76009817889397]
  We propose a robust learning policy based on Learning from Demonstration (LfD) for robotic grasping of food compliant objects. We present an LfD learning policy that automatically removes inconsistent demonstrations, and estimates the teacher's intended policy. The proposed approach has a vast range of potential applications in the aforementioned industry sectors.
  arXiv  Detail & Related papers  (2023-09-22T13:30:26Z)
• Towards Generating Large Synthetic Phytoplankton Datasets for Efficient Monitoring of Harmful Algal Blooms [77.25251419910205]
  Harmful algal blooms (HABs) cause significant fish deaths in aquaculture farms. Currently, the standard method to enumerate harmful algae and other phytoplankton is to manually observe and count them under a microscope. We employ Generative Adversarial Networks (GANs) to generate synthetic images.
  arXiv  Detail & Related papers  (2022-08-03T20:15:55Z)
• Generative Adversarial Networks for Image Augmentation in Agriculture: A Systematic Review [5.639656362091594]
  generative adversarial network (GAN) invented in 2014 in the computer vision community, provides suite of novel approaches that can learn good data representations. This paper presents an overview of the evolution of GAN architectures followed by a systematic review of their application to agriculture.
  arXiv  Detail & Related papers  (2022-04-10T15:33:05Z)
• Vision Transformers For Weeds and Crops Classification Of High Resolution UAV Images [3.1083892213758104]
  Vision Transformer (ViT) models can achieve competitive or better results without applying any convolution operations. Our experiments show that with small set of labelled training data, ViT models perform better compared to state-of-the-art CNN-based models.
  arXiv  Detail & Related papers  (2021-09-06T19:58:54Z)
• Potato Crop Stress Identification in Aerial Images using Deep Learning-based Object Detection [60.83360138070649]
  The paper presents an approach for analyzing aerial images of a potato crop using deep neural networks. The main objective is to demonstrate automated spatial recognition of a healthy versus stressed crop at a plant level. Experimental validation demonstrated the ability for distinguishing healthy and stressed plants in field images, achieving an average Dice coefficient of 0.74.
  arXiv  Detail & Related papers  (2021-06-14T21:57:40Z)
• Estimating Crop Primary Productivity with Sentinel-2 and Landsat 8 using Machine Learning Methods Trained with Radiative Transfer Simulations [58.17039841385472]
  We take advantage of all parallel developments in mechanistic modeling and satellite data availability for advanced monitoring of crop productivity. Our model successfully estimates gross primary productivity across a variety of C3 crop types and environmental conditions even though it does not use any local information from the corresponding sites. This highlights its potential to map crop productivity from new satellite sensors at a global scale with the help of current Earth observation cloud computing platforms.
  arXiv  Detail & Related papers  (2020-12-07T16:23:13Z)
• Multi-Spectral Image Synthesis for Crop/Weed Segmentation in Precision Farming [3.4788711710826083]
  We propose an alternative solution with respect to the common data augmentation methods, applying it to the problem of crop/weed segmentation in precision farming. We create semi-artificial samples by replacing the most relevant object classes (i.e., crop and weeds) with their synthesized counterparts. In addition to RGB data, we take into account also near-infrared (NIR) information, generating four channel multi-spectral synthetic images.
  arXiv  Detail & Related papers  (2020-09-12T08:49:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.