CoFL: Continuous Flow Fields for Language-Conditioned Navigation

• URL: http://arxiv.org/abs/2603.02854v1
• Date: Tue, 03 Mar 2026 11:02:55 GMT
• Title: CoFL: Continuous Flow Fields for Language-Conditioned Navigation
• Abstract summary: CoFL is an end-to-end policy that maps a bird's-eye view (BEV) observation and a language instruction to a continuous flow field for navigation.<n>By training on a mixed distribution, CoFL significantly outperforms modular Vision-Language Model (VLM)-based planners.
• Score: 14.35468089349405
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Language-conditioned navigation pipelines often rely on brittle modular components or costly action-sequence generation. To address these limitations, we present CoFL, an end-to-end policy that directly maps a bird's-eye view (BEV) observation and a language instruction to a continuous flow field for navigation. Instead of predicting discrete action tokens or sampling action chunks via iterative denoising, CoFL outputs instantaneous velocities that can be queried at arbitrary 2D projected locations. Trajectories are obtained by numerical integration of the predicted field, producing smooth motion that remains reactive under closed-loop execution. To enable large-scale training, we build a dataset of over 500k BEV image-instruction pairs, each procedurally annotated with a flow field and a trajectory derived from BEV semantic maps built on Matterport3D and ScanNet. By training on a mixed distribution, CoFL significantly outperforms modular Vision-Language Model (VLM)-based planners and generative policy baselines on strictly unseen scenes. Finally, we deploy CoFL zero-shot in real-world experiments with overhead BEV observations across multiple layouts, maintaining reliable closed-loop control and a high success rate.

Related papers

• FlowPilot: Real-Time World-Action Modeling for Agile UAV Navigation [18.092834753587454]
  FlowPilot is a compact world-action model for real-time onboard UAV navigation from depth.<n>A dual-stream mixture-of-transformers couples video and action experts through shared attention, allowing future-scene prediction and trajectory generation to inform each other.
  arXiv  Detail & Related papers  (2026-08-01T12:43:31Z)
• PGN: Design and Implementation of a Vision-Language Navigation System Based on Pangu Multimodal Foundation Model [10.02560644098367]
  Vision-Language Navigation (VLN) requires an embodied agent to interpret a natural computation instruction and predict actions from temporally ordered visual observations.<n>This report presents PGN (Pangu Navigator), an offline VLN action-prediction system built on OpenPangu-7B.
  arXiv  Detail & Related papers  (2026-07-20T10:48:48Z)
• CLEAR: Closed-Loop Reinforcement Learning at Scale for End-to-End Autonomous Driving [66.77752555295355]
  We present CLEAR, a system that enables closed-loop training using Reinforcement Learning (RL) at scale for E2E-AD.<n>We show that with a simple reward, CLEAR significantly outperforms previous methods and sets new state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-07-03T00:34:50Z)
• CoFL-S: Spatially Queryable Sector Flow Fields for Local Language-Conditioned Navigation [13.76195083398865]
  CoFL-S is a low-level vision-language-action framework that predicts a language-conditioned flow field over the robot's local visible sector.<n>To train this low-level representation, we convert each VLN-CE episode, originally a whole-episode instruction paired with an action sequence, into frame-level local supervision.<n>For evaluation, we introduce a continuous-time Habitat benchmark that isolates low-level action interfaces from instruction decomposition.
  arXiv  Detail & Related papers  (2026-07-02T14:26:55Z)
• Learning Direct Control Policies with Flow Matching for Autonomous Driving [32.505127447635864]
  Flow-matching planner for autonomous driving generates actionable trajectories defined by acceleration and curvature.<n>Model is conditioned on a bird's-eye-view (BEV) control profiles of the surrounding scene.<n>We train exclusively on urban scenarios collected from a 2D traffic simulator with reactive agents.
  arXiv  Detail & Related papers  (2026-05-14T13:37:48Z)
• FineCog-Nav: Integrating Fine-grained Cognitive Modules for Zero-shot Multimodal UAV Navigation [20.13280678867453]
  FineCog-Nav organizes navigation into fine-grained modules for language processing, perception, attention, memory, imagination, reasoning, and decision-making.<n>FineCog-Nav consistently outperforms zero-shot baselines in instruction adherence, long-horizon planning, and generalization to unseen environments.
  arXiv  Detail & Related papers  (2026-04-17T17:59:48Z)
• Bench2Drive-VL: Benchmarks for Closed-Loop Autonomous Driving with Vision-Language Models [50.22099309218635]
  In autonomous driving, closed-loop evaluation has become widely recognized as a more reliable validation method than open-loop evaluation.<n>We present Bench2Drive-VL, an extension of Bench2Drive that brings closed-loop evaluation to VLM-based driving.
  arXiv  Detail & Related papers  (2026-04-01T11:38:46Z)
• ThinkJEPA: Empowering Latent World Models with Large Vision-Language Reasoning Model [53.15040805435013]
  Vision-language models (VLMs) provide strong semantic grounding and general knowledge by reasoning over uniformly sampled frames.<n>We propose a VLM-guided JEPA-style latent world modeling framework that combines dense-frame dynamics modeling with long-horizon semantic guidance.
  arXiv  Detail & Related papers  (2026-03-23T17:59:42Z)
• AgentVLN: Towards Agentic Vision-and-Language Navigation [78.739525400071]
  Vision-and-Language Navigation (VLN) requires an embodied agent to ground complex natural-language instructions into long-horizon navigation in unseen environments.<n>In this paper, we propose AgentVLN, a novel and efficient embodied navigation framework that can be deployed on edge computing platforms.
  arXiv  Detail & Related papers  (2026-03-18T12:43:47Z)
• LatentAM: Real-Time, Large-Scale Latent Gaussian Attention Mapping via Online Dictionary Learning [1.9229388624311596]
  LatentAM builds latent feature maps from streaming RGB-D observations for open-vocabulary robotic perception.<n>We propose an online dictionary learning approach that is both model-agnostic and pretraining-free.<n>Experiments on public benchmarks and a large-scale custom dataset demonstrate that LatentAM attains significantly better feature reconstruction fidelity.
  arXiv  Detail & Related papers  (2026-02-12T17:25:00Z)
• Steering Vision-Language-Action Models as Anti-Exploration: A Test-Time Scaling Approach [78.4812458793128]
  We propose textbfTACO, a test-time-scaling framework that applies a lightweight pseudo-count estimator as a high-fidelity verifier of action chunks.<n>Our method resembles the classical anti-exploration principle in offline reinforcement learning (RL), and being gradient-free, it incurs significant computational benefits.
  arXiv  Detail & Related papers  (2025-12-02T14:42:54Z)
• Vehicle-to-Infrastructure Collaborative Spatial Perception via Multimodal Large Language Models [41.00138090010061]
  A lightweight, plug-and-play bird's-eye view (BEV) injection connector is proposed to overcome the limitations of large language models (MLLMs)<n>Ray tracing is developed to generate RGB, LiDAR, GPS, and wireless signal data across varied sensing scenarios.<n> Simulation results show that the proposed BEV injection framework consistently improved performance across all tasks.
  arXiv  Detail & Related papers  (2025-09-04T02:57:47Z)
• VLN-R1: Vision-Language Navigation via Reinforcement Fine-Tuning [77.34267241692706]
  Vision-Language Navigation (VLN) is a core challenge in embodied AI, requiring agents to navigate real-world environments using natural language instructions.<n>We propose VLN-R1, an end-to-end framework that leverages Large Vision-Language Models (LVLM) to directly translate egocentric video streams into continuous navigation actions.
  arXiv  Detail & Related papers  (2025-06-20T17:59:59Z)
• FLEX: A Backbone for Diffusion-Based Modeling of Spatio-temporal Physical Systems [51.15230303652732]
  FLEX (F Low EXpert) is a backbone architecture for generative modeling of-temporal physical systems.<n>It reduces the variance of the velocity field in the diffusion model, which helps stabilize training.<n>It achieves accurate predictions for super-resolution and forecasting tasks using as few features as two reverse diffusion steps.
  arXiv  Detail & Related papers  (2025-05-23T00:07:59Z)
• BEVDiffLoc: End-to-End LiDAR Global Localization in BEV View based on Diffusion Model [8.720833232645155]
  Bird's-Eye-View (BEV) image is one of the most widely adopted data representations in autonomous driving.<n>We propose BEVDiffLoc, a novel framework that formulates LiDAR localization as a conditional generation of poses.
  arXiv  Detail & Related papers  (2025-03-14T13:17:43Z)
• Unified Human Localization and Trajectory Prediction with Monocular Vision [64.19384064365431]
  MonoTransmotion is a Transformer-based framework that uses only a monocular camera to jointly solve localization and prediction tasks.<n>We show that by jointly training both tasks with our unified framework, our method is more robust in real-world scenarios made of noisy inputs.
  arXiv  Detail & Related papers  (2025-03-05T14:18:39Z)
• BEVWorld: A Multimodal World Simulator for Autonomous Driving via Scene-Level BEV Latents [56.33989853438012]
  We propose BEVWorld, a framework that transforms multimodal sensor inputs into a unified and compact Bird's Eye View latent space for holistic environment modeling.<n>The proposed world model consists of two main components: a multi-modal tokenizer and a latent BEV sequence diffusion model.
  arXiv  Detail & Related papers  (2024-07-08T07:26:08Z)
• BEVBert: Multimodal Map Pre-training for Language-guided Navigation [75.23388288113817]
  We propose a new map-based pre-training paradigm that is spatial-aware for use in vision-and-language navigation (VLN) We build a local metric map to explicitly aggregate incomplete observations and remove duplicates, while modeling navigation dependency in a global topological map. Based on the hybrid map, we devise a pre-training framework to learn a multimodal map representation, which enhances spatial-aware cross-modal reasoning thereby facilitating the language-guided navigation goal.
  arXiv  Detail & Related papers  (2022-12-08T16:27:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.