Fugu-MT: arxivの論文翻訳

Summary

This week's theme links physically grounded demonstration generation with reinforcement-learning fine-tuning to address scarce robot data and limited demonstration coverage. Current-week evidence from DexAgent adds task-adaptive, verification-guided trajectory generation from a single human video. It builds on earlier work in contact-aware retargeting, trajectory optimization, and structured policy exploration.

Situation

The representative Introductions identify two linked bottlenecks. The first is obtaining scalable, dynamically feasible demonstrations for contact-rich manipulation. The second is improving policies beyond the coverage and quality of those demonstrations. SPIDER uses simulation-based sampling with gradually relaxed virtual contact guidance to bridge human–robot embodiment differences. Physics-Driven Data Generation for Contact-Rich Manipulation via Trajectory Optimization uses a small set of human demonstrations as global guidance and refines them locally into physically consistent trajectories across embodiments and physical parameters. Diffusion Policy Policy Optimization addresses the learning side by treating diffusion denoising as an MDP. This enables policy-gradient fine-tuning with structured exploration, evaluated in both simulation and hardware.

Current-week evidence from DexAgent (2609.35318v1) extends the data-generation direction to task-adaptive workflows for rigid, articulated, and deformable objects. Its Introduction describes property-specific simulation reconstruction, subgoal-dependent motion generation, and iterative physical-validity verification. Newly developed skills and verifiers are retained for reuse across videos. Together with the representative work, this points to two goals: making demonstration transfer physically reliable and adaptable, and using RL fine-tuning to improve performance beyond imitation.

Infographic (English)

Physics-Guided Robot Learning and RL Fine-Tuning situation infographic

Progress

DexPolicy: Scheduled Exploration for Trajectory-Guided Dexterous Manipulation <See Details on Fugu-MT>

DexPolicy studies explicit control of exploration scale for reinforcement learning in dexterous manipulation. Motivated by baseline PPO ending near its initial action noise after five million steps, it compares PPO, critic-free GRPO continuation, and flow-free PPO settings.

DexWeave: Learning Dexterous Humanoid Loco-Manipulation from Human Demonstrations <See Details on Fugu-MT>

DexWeave extends learning from human demonstrations to dexterous humanoid loco-manipulation. It couples interaction-consistent motion with anatomy-aware whole-body policy learning and deploys the policy on a physical Unitree G1 humanoid.

PrefPI: Preference-Guided Steering into Out-of-Distribution Behaviors <See Details on Fugu-MT>

PrefPI improves pretrained generative robot policies through iterative preference-guided steering. It adds preference supervision beyond demonstration imitation, reaching new behaviors on hardware with only 150 preference-labeled trajectories.

DexAgent: An Agentic Human2Sim2Robot Framework for Dexterous Manipulation with Self-Evolving Tool Library <See Details on Fugu-MT>

DexAgent converts a single human video into physically grounded robot training trajectories via simulation reconstruction and trajectory optimization. It adds verifier-guided refinement and a reusable tool library, and policies trained on its data achieve a 3.5× higher success rate than baselines across 11 real-world tasks.

DexTaG: Tactile-as-Guidance in Reinforcement Learning for Dexterous Manipulation <See Details on Fugu-MT>

DexTaG introduces a tactile-guided reinforcement learning framework for dexterous manipulation. It uses tactile guidance to address the kinematic gap that prevents glove-captured human motion from being executed directly on robot hands.

FoLD: Force-Informed Learning for Dexterous Articulated Object Manipulation <See Details on Fugu-MT>

FoLD brings force information into learning dexterous articulated-object manipulation. It computes compensatory force fields from human demonstrations and the robot's current interaction state, and it outperforms baselines across tasks and embodiments.

Outlook

Outlook Summary

The next direction is to turn physics-grounded demonstrations into policies that generalize across tasks and robot bodies. SPIDER depends on reconstruction quality, while trajectory-optimization work proposes reusing older datasets, training with synthetic images, and improving planning beyond demonstrated states. DexAgent and DexWeave broaden coverage, but reliable recovery remains unresolved. After pre-training, DPPO proposes planning or video prediction to reduce interaction needs, with further vision-based hardware evaluation still needed. This week's exploration-control and feedback-guided methods offer complementary routes to policy improvement. Whether they improve recovery with fewer interactions—and combine effectively with physics-generated data—remains an open research question.

Infographic (English)

Physics-Guided Robot Learning and RL Fine-Tuning outlook infographic

Three-Year Movement

Over three years, this scenario moves from generating physically feasible demonstrations toward a reusable workflow that connects inspected training data with locally tested robot policies. In the first year, researchers would check simulated motion batches before hardware trials, borrowing the logic of seed certification: inspect quality before committing resources to local testing. They would train policies on accepted batches and test one feedback method at a time, keeping enough history to trace hardware failures back to their training data. The first threshold is whether a reused skill or check reduces new human demonstrations without hurting hardware performance, with recovery from disturbed starting states measured separately.

In the second year, older motion datasets could enter this workflow after adaptation to a robot's body and controls, while another robot would still require its own trials. Planning or video prediction could help just outside demonstrated conditions, while the fast learned policy operates within the conditions where it has been tested. Contact sensing would support tasks that need it, but planning delays and hardware interaction counts would determine whether these additions actually reduce the work required.

By the third year, laboratories could maintain a modest registry of inspected batches, reusable checks, and policies qualified for related task families. A hardware failure would reveal a weak check, allowing researchers to reject similar faulty trajectories before the next training and testing cycle repeats the mistake. Visual training with synthetic images would enter only where hardware tests show that precise contact behavior survives the transfer. Selected humanoid skills might also accumulate, without implying that they can reliably form long action sequences.

The main monitoring cue is reduced adaptation effort alongside preserved hardware performance, rather than faster generation or a larger registry alone. Repeated hardware failures among accepted trajectories would undermine the inspection mechanism, especially if manual reconstruction and simulator adjustment erase the savings. If existing demonstration workflows work equally well with less total effort, the proposed gate would offer little advantage.

Over the three-year horizon, this scenario shifts attention from completing familiar tasks to recovering when execution goes wrong. Physics-grounded demonstration generation supplies feasible motions, while later learning uses controlled exploration and richer feedback to improve the resulting policies. Earlier work established larger supplies of usable trajectories and selected hardware transfers, but not reliable recovery far from demonstrated states.

The proposed mechanism is a change in evaluation: researchers would test whether a robot regains control after a slip, an unexpected contact, or an unfamiliar starting state. The monitoring cue is improved recovery under these disturbances, rather than familiar-task completion alone. Reusable physical-validity checks could help connect data generation with policy improvement, but the supplied account does not establish that this combination works. The weekly evidence consists of editorial summaries and provides no year-by-year progression, so a reliable combined pipeline remains a research direction rather than a demonstrated three-year outcome.

Over three years, this scenario asks whether the acceptance process for generated demonstrations becomes the organizing layer of robot learning, rather than remaining a supporting tool. In the first year, researchers would measure false acceptances—invalid trajectories that pass a check—and compare them with hardware outcomes to separate verification benefits from data or reconstruction gains. Laboratories would record why trajectories passed, test recovery outside demonstrated conditions, and examine whether checks transfer to another object or robot without weakening error detection.

In the second year, successful local checks could become shared infrastructure, with research groups contributing failure examples and hardware providers supplying calibration procedures for particular robot bodies. The mechanism resembles inspection in chip production: as output grows, consistent acceptance criteria become more useful because users cannot repeatedly examine every generated example themselves. More users could contribute failures that improve the checks, reducing repeated validation work and encouraging further reuse across participating laboratories. This loop would depend on bounded error rates under explicit hardware calibration, especially as testing expands beyond rigid objects, and the supplied evidence establishes no common acceptable ceiling.

By the third year, interchangeable generators could produce candidate trajectories, while a shared verification layer accepts training data for downstream policy-learning systems. Services could provide audited trajectories for defined task families, and participating benchmark organizers might require records showing how those trajectories were checked. Research would test whether this structure improves hardware recovery under matched training budgets, rather than attributing gains to verification when another component supplied the improvement. Faster adaptation would be most plausible for robot bodies and skills already covered by calibration tests, not for arbitrary new tasks.

The key monitoring cue is better hardware recovery with less repeated validation work, not simply a higher acceptance rate or a larger library of checks. Physical feasibility alone cannot establish recovery competence, and simulator-approved contacts may still fail when a robot attempts them on hardware. If matched-budget studies find no advantage over ordinary task-specific checks, verification may remain useful locally without becoming the field's shared acceptance layer.

1-Year / 3-Year Research-Application Infographic

Integrated scenario outlook infographic

References

This page was created using generative AI such as GPT-5, GPT-6.1 Sol, GPT-6 Astra, Claude Opus 5.5, Gemini 3, Grok 4.7, Fugu Ultra, GLM 5.3, Gemini 3.1 Flash Image, GPT Image 2.5 Sunburst, and their higher-end successor versions. No guarantee can be made regarding its contents.