Clock-state olfactory search in turbulent flows using Q-learning: The geometry of plume recovery

• URL: http://arxiv.org/abs/2605.15938v1
• Date: Fri, 15 May 2026 13:19:26 GMT
• Title: Clock-state olfactory search in turbulent flows using Q-learning: The geometry of plume recovery
• Abstract summary: We use Q-learning to train an olfactory search agent with a minimal memory of past observations.<n>This agent learns an interpretable strategy to recover the plume which combines well-known behaviors observed in insects.
• Score: 0.6393971049475565
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Finding an odor source in a turbulent flow requires effectively leveraging the history of olfactory observations into a robust navigation strategy. In this work, we use tabular Q-learning to train an olfactory search agent with a minimal memory of past observations: only a running clock since the last whiff. This agent learns an interpretable strategy to recover the plume which combines well-known behaviors observed in insects: surging, casting, and a return downwind. While achieving good performance on data from direct numerical simulations of turbulence, the agent is limited by an inability to adapt its strategy to the local intermittency level; we show that providing more flexibility improves robustness.

Related papers

• Robust Dynamic Expansion for Continual Learning under Backdoor Attacks via Purification and Selective Recovery [21.762207770794916]
  In practical scenarios, task streams collected from untrusted sources may contain backdoor-poisoned samples.<n>We propose a robust dynamic-expansion framework that integrates sample purification, selective recovery, and robust expert routing.
  arXiv  Detail & Related papers  (2026-09-06T02:52:44Z)
• CurateEvo: Data-Curation Evolving for Agentic Post-Training [53.68149869349268]
  We propose CurateEvo, a failure-driven dynamic evolution framework for agentic post-training data curation.<n>CurateEvo represents the curation strategy as executable code and iteratively rewrites it using failed trajectories from a held-out development set.<n>At each epoch, the evolved strategy transforms a fixed raw corpus into supervised fine-tuning data, reinforcement learning data, and an inference-time memory bank.
  arXiv  Detail & Related papers  (2026-07-07T11:07:00Z)
• Emergence of a Flow-Assisted Casting Strategy for Olfactory Navigation via Memory-Augmented Reinforcement Learning [0.1749935196721634]
  We investigate the navigation performance of Reinforcement Learning (RL) agents in unsteady flows under varying memory lengths and flow conditions.<n>The average speed toward the odor source exhibits a non-monotonic dependence on memory length, which can be explained by the "sector-search" model.
  arXiv  Detail & Related papers  (2026-05-16T03:32:48Z)
• TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training [53.93696896939915]
  Training tool-use agents typically rely on Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning (RL) on pass-rate-selected tasks.<n>We propose TopoCurate, an interaction-aware framework that projects multi-trial rollouts from the same task into a unified semantic quotient topology.<n>TopoCurate achieves consistent gains of 4.2% (SFT) and 6.9% (RL) over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-03-02T10:38:54Z)
• MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memory [46.632646462295234]
  We propose MemRL, a framework that enables agents to self-evolve via non-parametric reinforcement learning on episodic memory.<n>MemRL employs a Two-Phase Retrieval mechanism that filters candidates by semantic relevance and then selects them based on learned Q-values.<n>Our analysis experiments confirm that MemRL effectively reconciles the stability-plasticity dilemma, enabling continuous runtime improvement without weight updates.
  arXiv  Detail & Related papers  (2026-01-06T17:14:50Z)
• FlowSteer: Guiding Few-Step Image Synthesis with Authentic Trajectories [82.90132015584359]
  ReFlow has theoretical consistency with flow matching but suboptimal performance in practical scenarios.<n>We propose FlowSteer, a method unlocks the potential of ReFlow-based distillation by guiding the student along teacher's authentic generation trajectories.
  arXiv  Detail & Related papers  (2025-11-24T07:13:23Z)
• Q-learning with temporal memory to navigate turbulence [47.28605705112213]
  We consider the problem of olfactory searches in a turbulent environment.<n>We develop a reinforcement learning algorithm using a small set of interpretable olfactory states.<n>We show that two salient features of odor traces, discretized in few olfactory states, are sufficient to learn navigation in a realistic odor plume.
  arXiv  Detail & Related papers  (2024-04-26T15:51:20Z)
• Solving Offline Reinforcement Learning with Decision Tree Regression [0.0]
  This study presents a novel approach to addressing offline reinforcement learning problems by reframing them as regression tasks. We introduce two distinct frameworks: return-conditioned and return-weighted decision tree policies. Despite the simplification inherent in this reformulated approach to offline RL, our agents demonstrate performance that is at least on par with the established methods.
  arXiv  Detail & Related papers  (2024-01-21T23:50:46Z)
• Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimization [87.21285093582446]
  Diffusion Generative Flow Samplers (DGFS) is a sampling-based framework where the learning process can be tractably broken down into short partial trajectory segments. Our method takes inspiration from the theory developed for generative flow networks (GFlowNets)
  arXiv  Detail & Related papers  (2023-10-04T09:39:05Z)
• Emergent behavior and neural dynamics in artificial agents tracking turbulent plumes [1.8065361710947974]
  We use deep reinforcement learning to train recurrent neural network (RNN) agents to locate the source of simulated turbulent plumes. Our analyses suggest an intriguing experimentally testable hypothesis for tracking plumes in changing wind direction.
  arXiv  Detail & Related papers  (2021-09-25T20:57:02Z)
• Automatic Recall Machines: Internal Replay, Continual Learning and the Brain [104.38824285741248]
  Replay in neural networks involves training on sequential data with memorized samples, which counteracts forgetting of previous behavior caused by non-stationarity. We present a method where these auxiliary samples are generated on the fly, given only the model that is being trained for the assessed objective. Instead the implicit memory of learned samples within the assessed model itself is exploited.
  arXiv  Detail & Related papers  (2020-06-22T15:07:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.