Self-Supervised Learning for Robotic Leaf Manipulation: A Hybrid Geometric-Neural Approach

• URL: http://arxiv.org/abs/2505.03702v3
• Date: Fri, 16 May 2025 18:23:01 GMT
• Title: Self-Supervised Learning for Robotic Leaf Manipulation: A Hybrid Geometric-Neural Approach
• Authors: Srecharan Selvam,
• Abstract summary: We propose a novel hybrid geometric-neural approach for autonomous leaf grasping.<n>Our method integrates traditional computer vision with neural networks through self-supervised learning.<n>Our approach achieves an 88.0% success rate in controlled environments and 84.7% in real greenhouse conditions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automating leaf manipulation in agricultural settings faces significant challenges, including the variability of plant morphologies and deformable leaves. We propose a novel hybrid geometric-neural approach for autonomous leaf grasping that combines traditional computer vision with neural networks through self-supervised learning. Our method integrates YOLOv8 for instance segmentation and RAFT-Stereo for 3D depth estimation to build rich leaf representations, which feed into both a geometric feature scoring pipeline and a neural refinement module (GraspPointCNN). The key innovation is our confidence-weighted fusion mechanism that dynamically balances the contribution of each approach based on prediction certainty. Our self-supervised framework uses the geometric pipeline as an expert teacher to automatically generate training data. Experiments demonstrate that our approach achieves an 88.0% success rate in controlled environments and 84.7% in real greenhouse conditions, significantly outperforming both purely geometric (75.3%) and neural (60.2%) methods. This work establishes a new paradigm for agricultural robotics where domain expertise is seamlessly integrated with machine learning capabilities, providing a foundation for fully automated crop monitoring systems.

Related papers

• Automated Multi-Class Crop Pathology Classification via Convolutional Neural Networks: A Deep Learning Approach for Real-Time Precision Agriculture [0.0]
  This research introduces a Convolutional Neural Network (CNN)-based image classification system designed to automate the detection and classification of eight common crop diseases.<n>The solution is deployed on an open-source, mobile-compatible platform, enabling real-time image-based diagnostics for farmers in remote areas.
  arXiv  Detail & Related papers  (2025-07-12T18:45:50Z)
• Automatic Fused Multimodal Deep Learning for Plant Identification [1.2289361708127877]
  We introduce a pioneering multimodal DL-based approach for plant classification with automatic modality fusion.<n>Our method achieves 82.61% accuracy on 979 classes of Multimodal-PlantCLEF, surpassing state-of-the-art methods and outperforming late fusion by 10.33%.
  arXiv  Detail & Related papers  (2024-06-03T15:43:29Z)
• Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learning [58.3994826169858]
  We introduce RoboFuME, a reset-free fine-tuning system for robotic reinforcement learning. Our insights are to utilize offline reinforcement learning techniques to ensure efficient online fine-tuning of a pre-trained policy. Our method can incorporate data from an existing robot dataset and improve on a target task within as little as 3 hours of autonomous real-world experience.
  arXiv  Detail & Related papers  (2023-10-23T17:50:08Z)
• ConCerNet: A Contrastive Learning Based Framework for Automated Conservation Law Discovery and Trustworthy Dynamical System Prediction [82.81767856234956]
  This paper proposes a new learning framework named ConCerNet to improve the trustworthiness of the DNN based dynamics modeling. We show that our method consistently outperforms the baseline neural networks in both coordinate error and conservation metrics.
  arXiv  Detail & Related papers  (2023-02-11T21:07:30Z)
• Mechanism of feature learning in deep fully connected networks and kernel machines that recursively learn features [15.29093374895364]
  We identify and characterize the mechanism through which deep fully connected neural networks learn gradient features. Our ansatz sheds light on various deep learning phenomena including emergence of spurious features and simplicity biases. To demonstrate the effectiveness of this feature learning mechanism, we use it to enable feature learning in classical, non-feature learning models.
  arXiv  Detail & Related papers  (2022-12-28T15:50:58Z)
• Masked World Models for Visual Control [90.13638482124567]
  We introduce a visual model-based RL framework that decouples visual representation learning and dynamics learning. We demonstrate that our approach achieves state-of-the-art performance on a variety of visual robotic tasks.
  arXiv  Detail & Related papers  (2022-06-28T18:42:27Z)
• Pushing the Limits of Learning-based Traversability Analysis for Autonomous Driving on CPU [1.841057463340778]
  This paper proposes and evaluates a real-time machine learning-based Traversability Analysis method. We show that integrating a new set of geometric and visual features and focusing on important implementation details enables a noticeable boost in performance and reliability. The proposed approach has been compared with state-of-the-art Deep Learning approaches on a public dataset of outdoor driving scenarios.
  arXiv  Detail & Related papers  (2022-06-07T07:57:34Z)
• Gradient-Based Trajectory Optimization With Learned Dynamics [80.41791191022139]
  We use machine learning techniques to learn a differentiable dynamics model of the system from data. We show that a neural network can model highly nonlinear behaviors accurately for large time horizons. In our hardware experiments, we demonstrate that our learned model can represent complex dynamics for both the Spot and Radio-controlled (RC) car.
  arXiv  Detail & Related papers  (2022-04-09T22:07:34Z)
• Plant Species Recognition with Optimized 3D Polynomial Neural Networks and Variably Overlapping Time-Coherent Sliding Window [3.867363075280544]
  This paper proposes a novel method, called Variably Overlapping Time-Coherent Sliding Window (VOTCSW), that transforms a dataset composed of images with variable size to a 3D representation with fixed size. By combining the VOTCSW method with the 3D extension of a recently proposed machine learning model called 1-Dimensional Polynomial Neural Networks, we were able to create a model that achieved a state-of-the-art accuracy of 99.9% on the dataset created by the EAGL-I system.
  arXiv  Detail & Related papers  (2022-03-04T23:37:12Z)
• Scene Synthesis via Uncertainty-Driven Attribute Synchronization [52.31834816911887]
  This paper introduces a novel neural scene synthesis approach that can capture diverse feature patterns of 3D scenes. Our method combines the strength of both neural network-based and conventional scene synthesis approaches.
  arXiv  Detail & Related papers  (2021-08-30T19:45:07Z)
• Towards an Automatic Analysis of CHO-K1 Suspension Growth in Microfluidic Single-cell Cultivation [63.94623495501023]
  We propose a novel Machine Learning architecture, which allows us to infuse a neural deep network with human-powered abstraction on the level of data. Specifically, we train a generative model simultaneously on natural and synthetic data, so that it learns a shared representation, from which a target variable, such as the cell count, can be reliably estimated.
  arXiv  Detail & Related papers  (2020-10-20T08:36:51Z)
• Automated Testing for Deep Learning Systems with Differential Behavior Criteria [5.653421430985333]
  We conducted a study on building an automated testing system for deep learning systems based on differential behavior criteria. By observing differential behaviors from three pre-trained models during each testing iteration, the input image that triggered erroneous feedback was registered as a corner-case. We explored other approaches based on differential behavior criteria to generate photo-realistic images for deep learning systems.
  arXiv  Detail & Related papers  (2019-12-31T10:31:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.