Push Past Green: Learning to Look Behind Plant Foliage by Moving It

• URL: http://arxiv.org/abs/2307.03175v2
• Date: Wed, 5 Jun 2024 17:59:01 GMT
• Title: Push Past Green: Learning to Look Behind Plant Foliage by Moving It
• Authors: Xiaoyu Zhang, Saurabh Gupta,
• Abstract summary: Partial visibility, extreme clutter, thin structures, and unknown geometry and dynamics for plants make such manipulation challenging. We use self-supervision to train SRPNet, a neural network that predicts what space is revealed on execution of a candidate action on a given plant. As SRPNet does not just predict how much space is revealed but also where it is revealed, we can execute a sequence of actions that incrementally reveal more and more space beneath the plant foliage.
• Score: 19.36396157137122
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autonomous agriculture applications (e.g., inspection, phenotyping, plucking fruits) require manipulating the plant foliage to look behind the leaves and the branches. Partial visibility, extreme clutter, thin structures, and unknown geometry and dynamics for plants make such manipulation challenging. We tackle these challenges through data-driven methods. We use self-supervision to train SRPNet, a neural network that predicts what space is revealed on execution of a candidate action on a given plant. We use SRPNet with the cross-entropy method to predict actions that are effective at revealing space beneath plant foliage. Furthermore, as SRPNet does not just predict how much space is revealed but also where it is revealed, we can execute a sequence of actions that incrementally reveal more and more space beneath the plant foliage. We experiment with a synthetic (vines) and a real plant (Dracaena) on a physical test-bed across 5 settings including 2 settings that test generalization to novel plant configurations. Our experiments reveal the effectiveness of our overall method, PPG, over a competitive hand-crafted exploration method, and the effectiveness of SRPNet over a hand-crafted dynamics model and relevant ablations.

Related papers

• PlantCamo: Plant Camouflage Detection [60.685139083469956]
  This paper introduces a new challenging problem of Plant Camouflage Detection (PCD) To address this problem, we introduce the PlantCamo dataset, which comprises 1,250 images with camouflaged plants. We conduct a large-scale benchmark study using 20+ cutting-edge COD models on the proposed dataset. Our PCNet surpasses performance thanks to its multi-scale global feature enhancement and refinement.
  arXiv  Detail & Related papers  (2024-10-23T06:51:59Z)
• Polaris: Open-ended Interactive Robotic Manipulation via Syn2Real Visual Grounding and Large Language Models [53.22792173053473]
  We introduce an interactive robotic manipulation framework called Polaris. Polaris integrates perception and interaction by utilizing GPT-4 alongside grounded vision models. We propose a novel Synthetic-to-Real (Syn2Real) pose estimation pipeline.
  arXiv  Detail & Related papers  (2024-08-15T06:40:38Z)
• PlantTrack: Task-Driven Plant Keypoint Tracking with Zero-Shot Sim2Real Transfer [4.923031976899536]
  Tracking plant features is crucial for various agricultural tasks like phenotyping, pruning, or harvesting. We propose PlantTrack where we utilize DINOv2 which provides high-dimensional features, and train a keypoint heatmap predictor network. We show that with as few as 20 synthetic images for training the keypoint predictor, we achieve zero-shot Sim2Real transfer, enabling effective tracking of plant features in real environments.
  arXiv  Detail & Related papers  (2024-07-23T20:40:17Z)
• Planning with Adaptive World Models for Autonomous Driving [50.4439896514353]
  Motion planners (MPs) are crucial for safe navigation in complex urban environments. nuPlan, a recently released MP benchmark, addresses this limitation by augmenting real-world driving logs with closed-loop simulation logic. We present AdaptiveDriver, a model-predictive control (MPC) based planner that unrolls different world models conditioned on BehaviorNet's predictions.
  arXiv  Detail & Related papers  (2024-06-15T18:53:45Z)
• Zoom in on the Plant: Fine-grained Analysis of Leaf, Stem and Vein Instances [3.399289369740637]
  We develop a model to extract fine-grained phenotypic information, such as leaf-, stem-, and vein instances. The underlying dataset RumexLeaves is made publicly available and is the first of its kind. We introduce an adapted metric POKS complying with the concept of keypoint-guided polylines.
  arXiv  Detail & Related papers  (2023-12-14T10:45:54Z)
• Semantics-Aware Next-best-view Planning for Efficient Search and Detection of Task-relevant Plant Parts [3.9074818653555554]
  To automate harvesting and de-leafing of tomato plants, it is important to search and detect the task-relevant plant parts. Current active-vision algorithms cannot differentiate between relevant and irrelevant plant parts. We propose a semantics-aware active-vision strategy that uses semantic information to identify the relevant plant parts.
  arXiv  Detail & Related papers  (2023-06-16T12:22:19Z)
• Semantic Image Segmentation with Deep Learning for Vine Leaf Phenotyping [59.0626764544669]
  In this study, we use Deep Learning methods to semantically segment grapevine leaves images in order to develop an automated object detection system for leaf phenotyping. Our work contributes to plant lifecycle monitoring through which dynamic traits such as growth and development can be captured and quantified.
  arXiv  Detail & Related papers  (2022-10-24T14:37:09Z)
• TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wild [77.59069361196404]
  TRiPOD is a novel method for predicting body dynamics based on graph attentional networks. To incorporate a real-world challenge, we learn an indicator representing whether an estimated body joint is visible/invisible at each frame. Our evaluation shows that TRiPOD outperforms all prior work and state-of-the-art specifically designed for each of the trajectory and pose forecasting tasks.
  arXiv  Detail & Related papers  (2021-04-08T20:01:00Z)
• Estimating Crop Primary Productivity with Sentinel-2 and Landsat 8 using Machine Learning Methods Trained with Radiative Transfer Simulations [58.17039841385472]
  We take advantage of all parallel developments in mechanistic modeling and satellite data availability for advanced monitoring of crop productivity. Our model successfully estimates gross primary productivity across a variety of C3 crop types and environmental conditions even though it does not use any local information from the corresponding sites. This highlights its potential to map crop productivity from new satellite sensors at a global scale with the help of current Earth observation cloud computing platforms.
  arXiv  Detail & Related papers  (2020-12-07T16:23:13Z)
• Deep Transfer Learning For Plant Center Localization [19.322420819302263]
  This paper investigates methods that estimate plant locations for a field-based crop using RGB aerial images captured using Unmanned Aerial Vehicles (UAVs) Deep learning approaches provide promising capability for locating plants observed in RGB images, but they require large quantities of labeled data (ground truth) for training. We propose a method for estimating plant centers by transferring an existing model to a new scenario using limited ground truth data.
  arXiv  Detail & Related papers  (2020-04-29T06:29:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.