Controlling Fish Schools via Reinforcement Learning of Virtual Fish Movement

• URL: http://arxiv.org/abs/2603.16384v1
• Date: Tue, 17 Mar 2026 11:12:36 GMT
• Title: Controlling Fish Schools via Reinforcement Learning of Virtual Fish Movement
• Abstract summary: This study investigates a method to guide and control fish schools using virtual fish trained with reinforcement learning.<n>We utilize 2D virtual fish displayed on a screen to overcome technical challenges such as durability and movement constraints inherent in physical robotic agents.
• Score: 2.5782420501870296
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This study investigates a method to guide and control fish schools using virtual fish trained with reinforcement learning. We utilize 2D virtual fish displayed on a screen to overcome technical challenges such as durability and movement constraints inherent in physical robotic agents. To address the lack of detailed behavioral models for real fish, we adopt a model-free reinforcement learning approach. First, simulation results show that reinforcement learning can acquire effective movement policies even when simulated real fish frequently ignore the virtual stimulus. Second, real-world experiments with live fish confirm that the learned policy successfully guides fish schools toward specified target directions. Statistical analysis reveals that the proposed method significantly outperforms baseline conditions, including the absence of stimulus and a heuristic "stay-at-edge" strategy. This study provides an early demonstration of how reinforcement learning can be used to influence collective animal behavior through artificial agents.

Related papers

• A Novel Computer Vision Approach for Assessing Fish Responses to Intrusive Objects in Aquaculture [2.099922236065961]
  The aquaculture industry needs to address several challenges to secure sustainable seafood production that can serve an increasing global demand.<n>One major challenge is to ensure good fish health and acceptable welfare during production since the improvement of fish welfare is of vital importance in current and future production systems.<n>A novel approach for detecting, tracking, and estimating the 3D position of individual fish has thus been developed.
  arXiv  Detail & Related papers  (2026-05-28T14:46:51Z)
• Robots that learn to evaluate models of collective behavior [7.979981463085327]
  We introduce a reinforcement-learning-based framework that uses a biomimetic robotic fish (RoboFish) to evaluate computational models of live fish behavior through closed-loop interaction.<n>We trained policies in simulation using four distinct fish models-a simple constant-follow baseline, two rule-based models, and a biologically grounded convolutional neural network model-and transferred these policies to the real RoboFish setup.<n>We evaluate the fish models by quantifying the sim-to-real gaps, defined as the Wasserstein distance between simulated and real distributions of behavioral metrics such as goal-reaching performance, inter-individual distances, wall interactions
  arXiv  Detail & Related papers  (2026-04-08T17:11:29Z)
• Rethinking Camera Choice: An Empirical Study on Fisheye Camera Properties in Robotic Manipulation [53.27191803311681]
  We rigorously analyze the properties of wrist-mounted fisheye cameras for imitation learning.<n>Fisheye-trained policies unlock superior scene generalization when trained with sufficient environmental diversity.<n>Our findings provide concrete, actionable guidance for the large-scale collection and effective use of fisheye datasets in robotic learning.
  arXiv  Detail & Related papers  (2026-03-02T18:00:37Z)
• From Seeing to Experiencing: Scaling Navigation Foundation Models with Reinforcement Learning [59.88543114325153]
  We introduce the Seeing-to-Experiencing framework to scale the capability of navigation foundation models with reinforcement learning.<n>S2E combines the strengths of pre-training on videos and post-training through RL.<n>We establish a comprehensive end-to-end evaluation benchmark, NavBench-GS, built on photorealistic 3DGS reconstructions of real-world scenes.
  arXiv  Detail & Related papers  (2025-07-29T17:26:10Z)
• CBIL: Collective Behavior Imitation Learning for Fish from Real Videos [58.81930297206828]
  We present a scalable approach, Collective Behavior Imitation Learning (CBIL), for learning fish schooling behavior directly from videos.<n>MVAE effectively maps 2D observations to implicit states that are compact and expressive for following the imitation learning stage.<n>CBIL can be used for various animation tasks with the learned collective motion priors.
  arXiv  Detail & Related papers  (2025-03-31T21:15:00Z)
• Modular Neural Network Policies for Learning In-Flight Object Catching with a Robot Hand-Arm System [55.94648383147838]
  We present a modular framework designed to enable a robot hand-arm system to learn how to catch flying objects. Our framework consists of five core modules: (i) an object state estimator that learns object trajectory prediction, (ii) a catching pose quality network that learns to score and rank object poses for catching, (iii) a reaching control policy trained to move the robot hand to pre-catch poses, and (iv) a grasping control policy trained to perform soft catching motions. We conduct extensive evaluations of our framework in simulation for each module and the integrated system, to demonstrate high success rates of in-flight
  arXiv  Detail & Related papers  (2023-12-21T16:20:12Z)
• How to Track and Segment Fish without Human Annotations: A Self-Supervised Deep Learning Approach [3.0516727053033392]
  Training deep neural networks (DNNs) for fish tracking and segmentation requires high-quality labels.<n>We propose an unsupervised approach that relies on spatial and temporal variations in video data to generate noisy pseudo-ground-truth labels.<n>Our framework consists of three stages: (1) an optical flow model generates the pseudo labels using spatial and temporal consistency between frames, (2) a self-supervised model refines the pseudo-labels incrementally, and (3) a segmentation network uses the refined labels for training.
  arXiv  Detail & Related papers  (2022-08-23T01:01:27Z)
• Applications of Deep Learning in Fish Habitat Monitoring: A Tutorial and Survey [1.9249287163937976]
  Deep learning (DL) is a cutting-edge AI technology that has demonstrated unprecedented performance in analysing visual data. In this paper, we provide a tutorial that covers the key concepts of DL, which help the reader grasp a high-level understanding of how DL works. The tutorial also explains a step-by-step procedure on how DL algorithms should be developed for challenging applications such as underwater fish monitoring.
  arXiv  Detail & Related papers  (2022-06-11T01:59:54Z)
• A Survey of Fish Tracking Techniques Based on Computer Vision [11.994865945394139]
  This paper presents a review of the advancements of fish tracking technologies over the past seven years-2023. It explores diverse fish tracking techniques with an emphasis on fundamental localization and tracking methods. It also summarizes open-source datasets, evaluation metrics, challenges, and applications in fish tracking research.
  arXiv  Detail & Related papers  (2021-10-06T07:46:35Z)
• Visual Adversarial Imitation Learning using Variational Models [60.69745540036375]
  Reward function specification remains a major impediment for learning behaviors through deep reinforcement learning. Visual demonstrations of desired behaviors often presents an easier and more natural way to teach agents. We develop a variational model-based adversarial imitation learning algorithm.
  arXiv  Detail & Related papers  (2021-07-16T00:15:18Z)
• Movement Tracks for the Automatic Detection of Fish Behavior in Videos [63.85815474157357]
  We offer a dataset of sablefish (Anoplopoma fimbria) startle behaviors in underwater videos, and investigate the use of deep learning (DL) methods for behavior detection on it. Our proposed detection system identifies fish instances using DL-based frameworks, determines trajectory tracks, derives novel behavior-specific features, and employs Long Short-Term Memory (LSTM) networks to identify startle behavior in sablefish.
  arXiv  Detail & Related papers  (2020-11-28T05:51:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.