FoldingAgent: Inferring Parametric Origami Procedures from Demonstration Videos

• URL: http://arxiv.org/abs/2609.00377v1
• Date: Mon, 31 Aug 2026 21:09:22 GMT
• Title: FoldingAgent: Inferring Parametric Origami Procedures from Demonstration Videos
• Abstract summary: We present FoldingAgent, an agentic framework for inferring explicit parametric folding programs directly from origami demonstration videos.<n>Our framework leverages the reasoning power of a pre-trained Vision-Language Model equipped with a suite of specialized tools.<n>We evaluate our approach on PurelandFold, a newly curated benchmark of diverse Pureland origami videos with ground-truth geometry and action labels.
• Score: 26.43054143107089
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present FoldingAgent, an agentic framework for inferring explicit parametric folding programs directly from origami demonstration videos. Our framework leverages the reasoning power of a pre-trained Vision-Language Model (VLM) equipped with a suite of specialized tools that enable the agent to simulate geometric transitions, verify physical plausibility, retrieve and compare visual content, and evaluate its own predictions. To translate visual content into folding programs, we define a parametric space that consists of the paper's geometry and a set of parametric folding actions. Unlike models that predict static crease patterns, our agent operates sequentially and possesses the ability to re-plan its actions, effectively mitigating the compounding errors inherent in multi-step folding. Our approach takes a step toward closing the gap between human origami knowledge, which is primarily shared through unstructured visual demonstrations, and computational methods, which typically rely on structured, parametric representations such as a crease pattern or an executable parametric plan. We evaluate our approach on PurelandFold, a newly curated benchmark of diverse Pureland origami videos with ground-truth geometry and action labels. Our results demonstrate that by combining VLM reasoning with a set of specialized tools and physical simulation, we can successfully transform unstructured visual demonstrations into executable, physically plausible folding procedures.

Related papers

• TapSampling: Inference-Time Sampling with a Task-Progress-Understanding Verifier for Robotic Manipulation [61.35569005726248]
  Existing embodied control research demonstrates remarkable performance improvements by scaling training data and model size.<n>Non-deterministic generative models, such as diffusion and autoregressive models, have been widely adopted in the field of embodied control.<n>We propose textbfTapSampling, a plug-and-play framework for inference-time sampling.
  arXiv  Detail & Related papers  (2026-05-25T08:03:31Z)
• S-VAM: Shortcut Video-Action Model by Self-Distilling Geometric and Semantic Foresight [42.690473567612116]
  Video action models (VAMs) have emerged as a promising paradigm for robot learning.<n>We propose S-VAM, a shortcut video-action model that foresees coherent geometric and semantic representations via a single forward pass.
  arXiv  Detail & Related papers  (2026-03-17T07:21:28Z)
• Geometry-Aligned LLM Fine-Tuning for Sequential Narrow-Opening Planning [6.579320299248572]
  We study rigid-body motion planning through multiple sequential narrow openings.<n>To achieve this, we propose a geometry-aligned large language model (LLM) fine-tuning framework.<n>Our method achieves the highest success rate in both in-distribution and out-of-distribution environments.
  arXiv  Detail & Related papers  (2026-03-17T00:24:32Z)
• OrigamiBench: An Interactive Environment to Synthesize Flat-Foldable Origamis [28.493060188882314]
  Building AI systems that can plan, act, and create in the physical world requires more than pattern recognition.<n>Such systems must understand the causal mechanisms and constraints governing physical processes in order to guide sequential decisions.<n>This capability relies on internal representations, analogous to an internal language model, that relate observations, actions, and resulting environmental changes.<n>We introduce OrigamiBench, an interactive benchmark in which models iteratively propose folds and receive feedback on physical validity and similarity to a target configuration.
  arXiv  Detail & Related papers  (2026-03-14T09:33:29Z)
• Learn2Fold: Structured Origami Generation with World Model Planning [32.69934044200432]
  origami is governed by strict geometric axioms and hard kinematic constraints.<n>A single invalid crease or collision can invalidate the entire folding sequence.<n>We introduce Learn2Fold, a neuro-symbolic framework that formulates origami folding as conditional program induction over a crease-pattern graph.
  arXiv  Detail & Related papers  (2026-02-02T11:24:19Z)
• DiffusionSfM: Predicting Structure and Motion via Ray Origin and Endpoint Diffusion [53.70278210626701]
  We propose a data-driven multi-view reasoning approach that directly infers 3D scene geometry and camera poses from multi-view images.<n>Our framework, DiffusionSfM, parameterizes scene geometry and cameras as pixel-wise ray origins and endpoints in a global frame.<n>We empirically validate DiffusionSfM on both synthetic and real datasets, demonstrating that it outperforms classical and learning-based approaches.
  arXiv  Detail & Related papers  (2025-05-08T17:59:47Z)
• Fact :Teaching MLLMs with Faithful, Concise and Transferable Rationales [102.54274021830207]
  We introduce Fact, a novel paradigm designed to generate multimodal rationales that are faithful, concise, and transferable for teaching MLLMs. We filter rationales that can be transferred to end-to-end paradigms from programming paradigms to guarantee transferability. Our approach also reduces hallucinations owing to its high correlation between images and text.
  arXiv  Detail & Related papers  (2024-04-17T07:20:56Z)
• EgoPlan-Bench: Benchmarking Multimodal Large Language Models for Human-Level Planning [84.6451394629312]
  We introduce EgoPlan-Bench, a benchmark to evaluate the planning abilities of MLLMs in real-world scenarios. We show that EgoPlan-Bench poses significant challenges, highlighting a substantial scope for improvement in MLLMs to achieve human-level task planning. We also present EgoPlan-IT, a specialized instruction-tuning dataset that effectively enhances model performance on EgoPlan-Bench.
  arXiv  Detail & Related papers  (2023-12-11T03:35:58Z)
• Inverse Dynamics Pretraining Learns Good Representations for Multitask Imitation [66.86987509942607]
  We evaluate how such a paradigm should be done in imitation learning. We consider a setting where the pretraining corpus consists of multitask demonstrations. We argue that inverse dynamics modeling is well-suited to this setting.
  arXiv  Detail & Related papers  (2023-05-26T14:40:46Z)
• A Control-Centric Benchmark for Video Prediction [69.22614362800692]
  We propose a benchmark for action-conditioned video prediction in the form of a control benchmark. Our benchmark includes simulated environments with 11 task categories and 310 task instance definitions. We then leverage our benchmark to study the effects of scaling model size, quantity of training data, and model ensembling.
  arXiv  Detail & Related papers  (2023-04-26T17:59:45Z)
• Latent Space Roadmap for Visual Action Planning of Deformable and Rigid Object Manipulation [74.88956115580388]
  Planning is performed in a low-dimensional latent state space that embeds images. Our framework consists of two main components: a Visual Foresight Module (VFM) that generates a visual plan as a sequence of images, and an Action Proposal Network (APN) that predicts the actions between them.
  arXiv  Detail & Related papers  (2020-03-19T18:43:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.