Task-specific Subnetwork Discovery in Reinforcement Learning for Autonomous Underwater Navigation

• URL: http://arxiv.org/abs/2604.21640v1
• Date: Thu, 23 Apr 2026 12:55:45 GMT
• Title: Task-specific Subnetwork Discovery in Reinforcement Learning for Autonomous Underwater Navigation
• Abstract summary: We analyze the internal structure of a pretrained multi-task reinforcement learning network in the HoloOcean simulator for underwater navigation.<n>We find that in a contextual multi-task reinforcement learning setting with related tasks, the network uses only about 1.5% of its weights to differentiate between tasks.
• Score: 4.692174333076031
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autonomous underwater vehicles are required to perform multiple tasks adaptively and in an explainable manner under dynamic, uncertain conditions and limited sensing, challenges that classical controllers struggle to address. This demands robust, generalizable, and inherently interpretable control policies for reliable long-term monitoring. Reinforcement learning, particularly multi-task RL, overcomes these limitations by leveraging shared representations to enable efficient adaptation across tasks and environments. However, while such policies show promising results in simulation and controlled experiments, they yet remain opaque and offer limited insight into the agent's internal decision-making, creating gaps in transparency, trust, and safety that hinder real-world deployment. The internal policy structure and task-specific specialization remain poorly understood. To address these gaps, we analyze the internal structure of a pretrained multi-task reinforcement learning network in the HoloOcean simulator for underwater navigation by identifying and comparing task-specific subnetworks responsible for navigating toward different species. We find that in a contextual multi-task reinforcement learning setting with related tasks, the network uses only about 1.5% of its weights to differentiate between tasks. Of these, approximately 85% connect the context-variable nodes in the input layer to the next hidden layer, highlighting the importance of context variables in such settings. Our approach provides insights into shared and specialized network components, useful for efficient model editing, transfer learning, and continual learning for underwater monitoring through a contextual multi-task reinforcement learning method.

Related papers

• Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills [53.15216425100295]
  We introduce Skill Self-Play (Skill-SP), a co-evolutionary framework comprising a proposer, a solver, and a skill controller.<n>The proposer generates challenging tasks conditioned on dynamically sampled skills; the solver explores candidate solutions to push its capability boundaries; and the skill controller collects execution feedback to update and expand the skill library.<n>This interactive co-evolution effectively bridges the gap between structured verification and open-ended exploration.
  arXiv  Detail & Related papers  (2026-07-24T17:59:22Z)
• CL-Anomaly: Layer-Adaptive Mixture-of-Experts with Multimodal Large Language Model for Continual Learning in Anomaly Detection [33.94199994115719]
  CL-Anomaly is a parameter-efficient fine-tuning framework for continuous learning for anomaly detection.<n>PrivLoRA physically isolates task-specific subspaces in the parameter space to prevent semantic entanglement of anomaly knowledge.<n> Layer-Adaptive Shared Experts maintain cross-task representations within a unified feature space.
  arXiv  Detail & Related papers  (2026-07-03T03:53:58Z)
• Contextual Multi-Task Reinforcement Learning for Autonomous Reef Monitoring [4.216733346030022]
  We employ a data-driven reinforcement learning approach to compensate for unknown dynamics and task variations.<n>We evaluate whether contextual multi-task reinforcement learning can efficiently learn robust and generalisable control policies.
  arXiv  Detail & Related papers  (2026-04-14T12:16:56Z)
• Learning with Preserving for Continual Multitask Learning [4.847042727427382]
  We introduce Learning with Preserving (LwP), a novel framework that shifts the focus from preserving task outputs to maintaining the shared representation space.<n>LwP not only mitigates catastrophic forgetting but also consistently outperforms state-of-the-art baselines in CMTL tasks.
  arXiv  Detail & Related papers  (2025-11-11T22:23:20Z)
• ThanoRA: Task Heterogeneity-Aware Multi-Task Low-Rank Adaptation [96.86211867758652]
  Low-Rank Adaptation (LoRA) is widely adopted for downstream fine-tuning of foundation models.<n>We propose ThanoRA, a Task Heterogeneity-Aware Multi-Task Low-Rank Adaptation framework.
  arXiv  Detail & Related papers  (2025-05-24T11:01:45Z)
• SwitchMT: An Adaptive Context Switching Methodology for Scalable Multi-Task Learning in Intelligent Autonomous Agents [5.343921650701002]
  We propose a novel adaptive task-switching methodology for RL-based multi-task learning in autonomous agents.<n>SwitchMT employs a Deep Spiking Q-Network with active dendrites and dueling structure to create specialized sub-networks.<n>It achieves superior performance in multi-task learning compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-04-18T08:12:59Z)
• Hierarchical Subspaces of Policies for Continual Offline Reinforcement Learning [19.463863037999054]
  We consider a Continual Reinforcement Learning setup, where a learning agent must continuously adapt to new tasks while retaining previously acquired skill sets.<n>We introduce HiSPO, a novel hierarchical framework designed specifically for continual learning in navigation settings from offline data.<n>We demonstrate, through a careful experimental study, the effectiveness of our method in both classical MuJoCo maze environments and complex video game-like navigation simulations.
  arXiv  Detail & Related papers  (2024-12-19T14:00:03Z)
• Sparsely Activated Mixture-of-Experts are Robust Multi-Task Learners [67.5865966762559]
  We study whether sparsely activated Mixture-of-Experts (MoE) improve multi-task learning. We devise task-aware gating functions to route examples from different tasks to specialized experts. This results in a sparsely activated multi-task model with a large number of parameters, but with the same computational cost as that of a dense model.
  arXiv  Detail & Related papers  (2022-04-16T00:56:12Z)
• Combining Modular Skills in Multitask Learning [149.8001096811708]
  A modular design encourages neural models to disentangle and recombine different facets of knowledge to generalise more systematically to new tasks. In this work, we assume each task is associated with a subset of latent discrete skills from a (potentially small) inventory. We find that the modular design of a network significantly increases sample efficiency in reinforcement learning and few-shot generalisation in supervised learning.
  arXiv  Detail & Related papers  (2022-02-28T16:07:19Z)
• Multi-Task Learning with Sequence-Conditioned Transporter Networks [67.57293592529517]
  We aim to solve multi-task learning through the lens of sequence-conditioning and weighted sampling. We propose a new suite of benchmark aimed at compositional tasks, MultiRavens, which allows defining custom task combinations. Second, we propose a vision-based end-to-end system architecture, Sequence-Conditioned Transporter Networks, which augments Goal-Conditioned Transporter Networks with sequence-conditioning and weighted sampling.
  arXiv  Detail & Related papers  (2021-09-15T21:19:11Z)
• Weakly-Supervised Reinforcement Learning for Controllable Behavior [126.04932929741538]
  Reinforcement learning (RL) is a powerful framework for learning to take actions to solve tasks. In many settings, an agent must winnow down the inconceivably large space of all possible tasks to the single task that it is currently being asked to solve. We introduce a framework for using weak supervision to automatically disentangle this semantically meaningful subspace of tasks from the enormous space of nonsensical "chaff" tasks.
  arXiv  Detail & Related papers  (2020-04-06T17:50:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.