Push Past Green: Learning to Look Behind Plant Foliage by Moving It

• URL: http://arxiv.org/abs/2307.03175v2
• Date: Wed, 5 Jun 2024 17:59:01 GMT
• Title: Push Past Green: Learning to Look Behind Plant Foliage by Moving It
• Authors: Xiaoyu Zhang, Saurabh Gupta,
• Abstract summary: Partial visibility, extreme clutter, thin structures, and unknown geometry and dynamics for plants make such manipulation challenging. We use self-supervision to train SRPNet, a neural network that predicts what space is revealed on execution of a candidate action on a given plant. As SRPNet does not just predict how much space is revealed but also where it is revealed, we can execute a sequence of actions that incrementally reveal more and more space beneath the plant foliage.
• Score: 19.36396157137122
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autonomous agriculture applications (e.g., inspection, phenotyping, plucking fruits) require manipulating the plant foliage to look behind the leaves and the branches. Partial visibility, extreme clutter, thin structures, and unknown geometry and dynamics for plants make such manipulation challenging. We tackle these challenges through data-driven methods. We use self-supervision to train SRPNet, a neural network that predicts what space is revealed on execution of a candidate action on a given plant. We use SRPNet with the cross-entropy method to predict actions that are effective at revealing space beneath plant foliage. Furthermore, as SRPNet does not just predict how much space is revealed but also where it is revealed, we can execute a sequence of actions that incrementally reveal more and more space beneath the plant foliage. We experiment with a synthetic (vines) and a real plant (Dracaena) on a physical test-bed across 5 settings including 2 settings that test generalization to novel plant configurations. Our experiments reveal the effectiveness of our overall method, PPG, over a competitive hand-crafted exploration method, and the effectiveness of SRPNet over a hand-crafted dynamics model and relevant ablations.

Related papers

• PlantTrack: Task-Driven Plant Keypoint Tracking with Zero-Shot Sim2Real Transfer [4.923031976899536]
  Tracking plant features is crucial for various agricultural tasks like phenotyping, pruning, or harvesting. We propose PlantTrack where we utilize DINOv2 which provides high-dimensional features, and train a keypoint heatmap predictor network. We show that with as few as 20 synthetic images for training the keypoint predictor, we achieve zero-shot Sim2Real transfer, enabling effective tracking of plant features in real environments.
  arXiv  Detail & Related papers  (2024-07-23T20:40:17Z)
• Planning with Adaptive World Models for Autonomous Driving [50.4439896514353]
  Motion planners (MPs) are crucial for safe navigation in complex urban environments. nuPlan, a recently released MP benchmark, addresses this limitation by augmenting real-world driving logs with closed-loop simulation logic. We present AdaptiveDriver, a model-predictive control (MPC) based planner that unrolls different world models conditioned on BehaviorNet's predictions.
  arXiv  Detail & Related papers  (2024-06-15T18:53:45Z)
• Zoom in on the Plant: Fine-grained Analysis of Leaf, Stem and Vein Instances [3.399289369740637]
  We develop a model to extract fine-grained phenotypic information, such as leaf-, stem-, and vein instances. The underlying dataset RumexLeaves is made publicly available and is the first of its kind. We introduce an adapted metric POKS complying with the concept of keypoint-guided polylines.
  arXiv  Detail & Related papers  (2023-12-14T10:45:54Z)
• Semantics-Aware Next-best-view Planning for Efficient Search and Detection of Task-relevant Plant Parts [3.9074818653555554]
  To automate harvesting and de-leafing of tomato plants, it is important to search and detect the task-relevant plant parts. Current active-vision algorithms cannot differentiate between relevant and irrelevant plant parts. We propose a semantics-aware active-vision strategy that uses semantic information to identify the relevant plant parts.
  arXiv  Detail & Related papers  (2023-06-16T12:22:19Z)
• Semantic Image Segmentation with Deep Learning for Vine Leaf Phenotyping [59.0626764544669]
  In this study, we use Deep Learning methods to semantically segment grapevine leaves images in order to develop an automated object detection system for leaf phenotyping. Our work contributes to plant lifecycle monitoring through which dynamic traits such as growth and development can be captured and quantified.
  arXiv  Detail & Related papers  (2022-10-24T14:37:09Z)
• Auto-Parsing Network for Image Captioning and Visual Question Answering [101.77688388554097]
  We propose an Auto-Parsing Network (APN) to discover and exploit the input data's hidden tree structures. Specifically, we impose a Probabilistic Graphical Model (PGM) parameterized by the attention operations on each self-attention layer to incorporate sparse assumption.
  arXiv  Detail & Related papers  (2021-08-24T08:14:35Z)
• Potato Crop Stress Identification in Aerial Images using Deep Learning-based Object Detection [60.83360138070649]
  The paper presents an approach for analyzing aerial images of a potato crop using deep neural networks. The main objective is to demonstrate automated spatial recognition of a healthy versus stressed crop at a plant level. Experimental validation demonstrated the ability for distinguishing healthy and stressed plants in field images, achieving an average Dice coefficient of 0.74.
  arXiv  Detail & Related papers  (2021-06-14T21:57:40Z)
• Temporal Prediction and Evaluation of Brassica Growth in the Field using Conditional Generative Adversarial Networks [1.2926587870771542]
  The prediction of plant growth is a major challenge, as it is affected by numerous and highly variable environmental factors. This paper proposes a novel monitoring approach that comprises high- throughput imaging sensor measurements and their automatic analysis. Our approach's core is a novel machine learning-based growth model based on conditional generative adversarial networks.
  arXiv  Detail & Related papers  (2021-05-17T13:00:01Z)
• TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wild [77.59069361196404]
  TRiPOD is a novel method for predicting body dynamics based on graph attentional networks. To incorporate a real-world challenge, we learn an indicator representing whether an estimated body joint is visible/invisible at each frame. Our evaluation shows that TRiPOD outperforms all prior work and state-of-the-art specifically designed for each of the trajectory and pose forecasting tasks.
  arXiv  Detail & Related papers  (2021-04-08T20:01:00Z)
• Estimating Crop Primary Productivity with Sentinel-2 and Landsat 8 using Machine Learning Methods Trained with Radiative Transfer Simulations [58.17039841385472]
  We take advantage of all parallel developments in mechanistic modeling and satellite data availability for advanced monitoring of crop productivity. Our model successfully estimates gross primary productivity across a variety of C3 crop types and environmental conditions even though it does not use any local information from the corresponding sites. This highlights its potential to map crop productivity from new satellite sensors at a global scale with the help of current Earth observation cloud computing platforms.
  arXiv  Detail & Related papers  (2020-12-07T16:23:13Z)
• Deep Transfer Learning For Plant Center Localization [19.322420819302263]
  This paper investigates methods that estimate plant locations for a field-based crop using RGB aerial images captured using Unmanned Aerial Vehicles (UAVs) Deep learning approaches provide promising capability for locating plants observed in RGB images, but they require large quantities of labeled data (ground truth) for training. We propose a method for estimating plant centers by transferring an existing model to a new scenario using limited ground truth data.
  arXiv  Detail & Related papers  (2020-04-29T06:29:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.