Contrastive Energy Fields for Inference-Time Procedure Planning in Instructional Videos

• URL: http://arxiv.org/abs/2608.16457v1
• Date: Mon, 17 Aug 2026 11:56:11 GMT
• Title: Contrastive Energy Fields for Inference-Time Procedure Planning in Instructional Videos
• Abstract summary: We propose CEFITO, a procedure planning approach that learns a predictor to express an action-conditioned representation space.<n>Unlike prior methods, CEFITO explicitly reasons over the action space by omitting irrelevant actions during inference-time planning.
• Score: 48.22554718706414
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Procedure planning seeks to estimate a sequence of actions to transition from an observed initial state to a given goal state. Current procedure planning approaches directly predict action sequences from latent representations using feed-forward neural networks or diffusion-based inference. These paradigms treat every action as plausible, lacking the ability to enforce task-specific logical constraints that render certain actions irrelevant or not plausible. We propose CEFITO, a procedure planning approach that learns a predictor to express an action-conditioned representation space. Based on this representation space, we formulate procedure planning as a task-constrained optimization problem. Unlike prior methods, CEFITO explicitly reasons over the action space by omitting irrelevant actions during inference-time planning. This reformulation enables effective procedure planning and achieves state-of-the-art accuracy on two established procedure planning benchmarks.

Related papers

• Occluding the Solution Space: Planner-Agnostic Adversarial Attacks on Tolerance-Aware Manipulation [57.9825059129472]
  We propose a planner-agnostic attack framework for tolerance-aware manipulation.<n>Our approach shifts the evaluation paradigm to task-level feasibility over goal regions.<n>Online, we formulate the attack as a budgeted maximum-coverage optimization.
  arXiv  Detail & Related papers  (2026-07-04T08:14:44Z)
• Coarse-to-Control: Action-Token Planning for Vision-Language-Action Models [101.44987067974999]
  Coarse-to-Control is a plan-execute VLA that introduces planning in the action-token space.<n>We show that action-token planning consistently improves over direct action generation.
  arXiv  Detail & Related papers  (2026-06-05T10:01:37Z)
• Future Predictive Success-or-Failure Classification for Long-Horizon Robotic Tasks [6.844121549749507]
  This paper proposes a future-predictive success-or-failure-classification method to obtain conditions automatically. The proposed method uses a long-horizon future-prediction method to enable success-or-failure classification without the execution of an action plan. The effectiveness of our method is demonstrated through classification and robotic-manipulation experiments.
  arXiv  Detail & Related papers  (2024-04-04T12:49:42Z)
• Unified Task and Motion Planning using Object-centric Abstractions of Motion Constraints [56.283944756315066]
  We propose an alternative TAMP approach that unifies task and motion planning into a single search. Our approach is based on an object-centric abstraction of motion constraints that permits leveraging the computational efficiency of off-the-shelf AI search to yield physically feasible plans.
  arXiv  Detail & Related papers  (2023-12-29T14:00:20Z)
• Planning as In-Painting: A Diffusion-Based Embodied Task Planning Framework for Environments under Uncertainty [56.30846158280031]
  Task planning for embodied AI has been one of the most challenging problems. We propose a task-agnostic method named 'planning as in-painting' The proposed framework achieves promising performances in various embodied AI tasks.
  arXiv  Detail & Related papers  (2023-12-02T10:07:17Z)
• Skip-Plan: Procedure Planning in Instructional Videos via Condensed Action Space Learning [85.84504287685884]
  Skip-Plan is a condensed action space learning method for procedure planning in instructional videos. By skipping uncertain nodes and edges in action chains, we transfer long and complex sequence functions into short but reliable ones. Our model explores all sorts of reliable sub-relations within an action sequence in the condensed action space.
  arXiv  Detail & Related papers  (2023-10-01T08:02:33Z)
• Event-Guided Procedure Planning from Instructional Videos with Text Supervision [31.82121743586165]
  We focus on the task of procedure planning from instructional videos with text supervision. A critical challenge of this task is the large semantic gap between observed visual states and unobserved intermediate actions. We propose a novel event-guided paradigm, which first infers events from the observed states and then plans out actions based on both the states and predicted events.
  arXiv  Detail & Related papers  (2023-08-17T09:43:28Z)
• On efficient computation in active inference [1.1470070927586016]
  We present a novel planning algorithm for finite temporal horizons with drastically lower computational complexity. We also simplify the process of setting an appropriate target distribution for new and existing active inference planning schemes.
  arXiv  Detail & Related papers  (2023-07-02T07:38:56Z)
• Efficient Belief Space Planning in High-Dimensional State Spaces using PIVOT: Predictive Incremental Variable Ordering Tactic [11.878820609988693]
  We examine the problem of online decision making under uncertainty, which we formulate as planning in the belief space. We call this approach PIVOT: Predictive Incremental Variable Ordering Tactic. Applying this tactic can also improve the state inference efficiency.
  arXiv  Detail & Related papers  (2021-12-29T07:30:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.