Optical flow-based branch segmentation for complex orchard environments
- URL: http://arxiv.org/abs/2202.13050v1
- Date: Sat, 26 Feb 2022 03:38:20 GMT
- Title: Optical flow-based branch segmentation for complex orchard environments
- Authors: Alexander You, Cindy Grimm, Joseph R. Davidson
- Abstract summary: We train a neural network system in simulation only using simulated RGB data and optical flow.
This resulting neural network is able to perform foreground segmentation of branches in a busy orchard environment without additional real-world training or using any special setup or equipment beyond a standard camera.
Our results show that our system is highly accurate and, when compared to a network using manually labeled RGBD data, achieves significantly more consistent and robust performance across environments that differ from the training set.
- Score: 73.11023209243326
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Machine vision is a critical subsystem for enabling robots to be able to
perform a variety of tasks in orchard environments. However, orchards are
highly visually complex environments, and computer vision algorithms operating
in them must be able to contend with variable lighting conditions and
background noise. Past work on enabling deep learning algorithms to operate in
these environments has typically required large amounts of hand-labeled data to
train a deep neural network or physically controlling the conditions under
which the environment is perceived. In this paper, we train a neural network
system in simulation only using simulated RGB data and optical flow. This
resulting neural network is able to perform foreground segmentation of branches
in a busy orchard environment without additional real-world training or using
any special setup or equipment beyond a standard camera. Our results show that
our system is highly accurate and, when compared to a network using manually
labeled RGBD data, achieves significantly more consistent and robust
performance across environments that differ from the training set.
Related papers
- RaSim: A Range-aware High-fidelity RGB-D Data Simulation Pipeline for Real-world Applications [55.24463002889]
We focus on depth data synthesis and develop a range-aware RGB-D data simulation pipeline (RaSim)
In particular, high-fidelity depth data is generated by imitating the imaging principle of real-world sensors.
RaSim can be directly applied to real-world scenarios without any finetuning and excel at downstream RGB-D perception tasks.
arXiv Detail & Related papers (2024-04-05T08:52:32Z) - Enhancing Graph Representation of the Environment through Local and
Cloud Computation [2.9465623430708905]
We propose a graph-based representation that provides a semantic representation of robot environments from multiple sources.
To acquire information from the environment, the framework combines classical computer vision tools with modern computer vision cloud services.
The proposed approach allows us to handle also small objects and integrate them into the semantic representation of the environment.
arXiv Detail & Related papers (2023-09-22T08:05:32Z) - Deep Multi-Emitter Spectrum Occupancy Mapping that is Robust to the
Number of Sensors, Noise and Threshold [32.880113150521154]
One of the primary goals in spectrum occupancy mapping is to create a system that is robust to assumptions about the number of sensors, occupancy threshold (in dBm), sensor noise, number of emitters and the propagation environment.
We show that such a system may be designed with neural networks using a process of aggregation to allow a variable number of sensors during training and testing.
arXiv Detail & Related papers (2022-11-27T14:08:11Z) - An in-depth experimental study of sensor usage and visual reasoning of
robots navigating in real environments [20.105395754497202]
We study the performance and reasoning capacities of real physical agents, trained in simulation and deployed to two different physical environments.
We show, that for the PointGoal task, an agent pre-trained on wide variety of tasks and fine-tuned on a simulated version of the target environment can reach competitive performance without modelling any sim2real transfer.
arXiv Detail & Related papers (2021-11-29T16:27:29Z) - Attention-based Neural Network for Driving Environment Complexity
Perception [123.93460670568554]
This paper proposes a novel attention-based neural network model to predict the complexity level of the surrounding driving environment.
It consists of a Yolo-v3 object detection algorithm, a heat map generation algorithm, CNN-based feature extractors, and attention-based feature extractors.
The proposed attention-based network achieves 91.22% average classification accuracy to classify the surrounding environment complexity.
arXiv Detail & Related papers (2021-06-21T17:27:11Z) - Learning to Continuously Optimize Wireless Resource In Episodically
Dynamic Environment [55.91291559442884]
This work develops a methodology that enables data-driven methods to continuously learn and optimize in a dynamic environment.
We propose to build the notion of continual learning into the modeling process of learning wireless systems.
Our design is based on a novel min-max formulation which ensures certain fairness" across different data samples.
arXiv Detail & Related papers (2020-11-16T08:24:34Z) - Object-based Illumination Estimation with Rendering-aware Neural
Networks [56.01734918693844]
We present a scheme for fast environment light estimation from the RGBD appearance of individual objects and their local image areas.
With the estimated lighting, virtual objects can be rendered in AR scenarios with shading that is consistent to the real scene.
arXiv Detail & Related papers (2020-08-06T08:23:19Z) - Stillleben: Realistic Scene Synthesis for Deep Learning in Robotics [33.30312206728974]
We describe a synthesis pipeline capable of producing training data for cluttered scene perception tasks.
Our approach arranges object meshes in physically realistic, dense scenes using physics simulation.
Our pipeline can be run online during training of a deep neural network.
arXiv Detail & Related papers (2020-05-12T10:11:00Z) - Transferable Active Grasping and Real Embodied Dataset [48.887567134129306]
We show how to search for feasible viewpoints for grasping by the use of hand-mounted RGB-D cameras.
A practical 3-stage transferable active grasping pipeline is developed, that is adaptive to unseen clutter scenes.
In our pipeline, we propose a novel mask-guided reward to overcome the sparse reward issue in grasping and ensure category-irrelevant behavior.
arXiv Detail & Related papers (2020-04-28T08:15:35Z) - Zero-Shot Reinforcement Learning with Deep Attention Convolutional
Neural Networks [12.282277258055542]
We show that a deep attention convolutional neural network (DACNN) with specific visual sensor configuration performs as well as training on a dataset with high domain and parameter variation at lower computational complexity.
Our new architecture adapts perception with respect to the control objective, resulting in zero-shot learning without pre-training a perception network.
arXiv Detail & Related papers (2020-01-02T19:41:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.