LIME: Learning Intent-aware Camera Motion from Egocentric Video

• URL: http://arxiv.org/abs/2607.02417v1
• Date: Thu, 02 Jul 2026 16:48:43 GMT
• Title: LIME: Learning Intent-aware Camera Motion from Egocentric Video
• Abstract summary: We propose a vision-language camera-motion generator that combines an auto-regressive observation-gain output with a continuous flow-matching pose head.<n>We show that LIME can learn to actively choose camera poses from passive human video, turning ordinary egocentric recordings into supervision for intent-aware active perception.
• Score: 56.023644847494836
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autonomous robots often need to move their camera before they can act: to inspect an object, reveal an occluded region, or obtain a view that responds to a user's intent. While vision-language navigation translates instructions to base motion and vision-language-action policies map instructions to manipulation actions, language-conditioned camera motion remains comparatively underexplored as a first-class action. We formulate language-conditioned camera motion generation: given a current RGB observation and a free-form natural-language intent, predict a relative target camera pose for the next observation. This task is inherently non-trivial: viewpoint changes are driven by latent perceptual intentions, and a valid motion may operate at different semantic granularity, from entering a room to looking around a corner, inspecting a visible object, or revealing an occluded detail. To model this structure, we mine multi-intention camera-motion supervision from egocentric video, pairing plausible intents and observation-gain descriptions with relative SE(3) target poses. We propose LIME, a vision-language camera-motion generator that combines an auto-regressive observation-gain output with a continuous flow-matching pose head. This design lets the model jointly predict what the next view should reveal while representing multi-hypothesis target views. Across experiments and downstream robotic tasks, we show that LIME can learn to actively choose camera poses from passive human video, turning ordinary egocentric recordings into supervision for intent-aware active perception.

Related papers

• UniMoCa: Unifying Motion and Camera Controls as Visual Proxies for Faithful Human Video Generation [50.391925716274926]
  textbfUniMoCa is a representation-driven framework that unifies motion and camera controls in visual space.<n>We show that UniMoCa achieves substantial gains in human motion control, camera control, temporal consistency, and camera-aware robustness with minimal additional complexity.
  arXiv  Detail & Related papers  (2026-08-03T09:18:13Z)
• From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Model [72.8641426724502]
  Camera-Centric VLA (CamVLA) is a new VLA model that decouples manipulation controls from camera geometry.<n>CamVLA consistently improves success rates across diverse unseen viewpoints.
  arXiv  Detail & Related papers  (2026-07-06T17:59:59Z)
• Look-Before-Move: Narrative-Grounded World Visual Attention in Dynamic 3D Story Worlds [22.4160057556312]
  Look-Before-Move is a camera planning framework that separates observation specification from motion execution.<n>We construct a dynamic 3D Story World Benchmark based on StoryBlender, covering 50 stories, 457 scenes, and 1585 shots with animated characters.<n>Experiments show that our framework improves subject perception, intent consistency, and trajectory quality over representative baselines.
  arXiv  Detail & Related papers  (2026-06-25T12:38:39Z)
• A DVDrive Approach for doScenes Instructed Driving Challenge [42.41052099695495]
  This paper presents our submission to the doScenes Instructed Driving Challenge, built upon OmniDrive.<n>We adapt OmniDrive to the doScenes setting by training it on instruction-annotated nuScenes scenes.<n>To improve multi-view visual grounding, we introduce a DVPE-style divided-view perception module.
  arXiv  Detail & Related papers  (2026-06-19T17:26:58Z)
• MoRight: Motion Control Done Right [71.36903230523589]
  We introduce MoRight, a unified framework that addresses both limitations through disentangled motion modeling.<n>Object motion is specified in a canonical static-view and transferred to an arbitrary target camera viewpoint.<n>Experiments on three benchmarks demonstrate state-of-the-art performance in generation quality, motion controllability, and interaction awareness.
  arXiv  Detail & Related papers  (2026-04-08T17:59:22Z)
• Towards Understanding Camera Motions in Any Video [89.97247162415158]
  We introduce CameraBench, a large-scale dataset and benchmark designed to assess and improve camera motion understanding.<n>CameraBench consists of 3,000 diverse internet videos annotated by experts through a rigorous quality control process.<n>One of our contributions is a taxonomy of camera motion primitives, designed in collaboration with cinematographers.
  arXiv  Detail & Related papers  (2025-04-21T18:34:57Z)
• ChatCam: Empowering Camera Control through Conversational AI [67.31920821192323]
  ChatCam is a system that navigates camera movements through conversations with users. To achieve this, we propose CineGPT, a GPT-based autoregressive model for text-conditioned camera trajectory generation. We also develop an Anchor Determinator to ensure precise camera trajectory placement.
  arXiv  Detail & Related papers  (2024-09-25T20:13:41Z)
• Attentive and Contrastive Learning for Joint Depth and Motion Field Estimation [76.58256020932312]
  Estimating the motion of the camera together with the 3D structure of the scene from a monocular vision system is a complex task. We present a self-supervised learning framework for 3D object motion field estimation from monocular videos.
  arXiv  Detail & Related papers  (2021-10-13T16:45:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.