RynnUnify: Towards Unified Embodied Brain
Abstract Overview
RynnUnify is a 16B unified embodied model designed to connect embodied understanding, long-horizon visual planning, and robot control through interleaved video-language reasoning. The architecture combines an 8B autoregressive reasoner with an 8B video generator, augmented with cross-chunk scene memory, temporal conditioning, and multiple generation modes to maintain coherence across successive subtasks. The paper also introduces an agentic extension that selectively invokes video prediction to reason about unobserved future states, alongside a world-action extension that translates understanding and prediction abilities into control policies. To evaluate these capabilities, the authors propose two new benchmarks, Rynn-VP for long-horizon visual planning and Rynn-GEU for anticipatory embodied reasoning, complemented by tests in simulation and on real robots.
Novelty
The paper's primary novelty is a unified embodied framework that explicitly interleaves reasoning with video prediction so that anticipated scene evolution directly guides subsequent subtask planning over long horizons. It further extends this foundation in two directions: an RL-trained agent that dynamically invokes video generation to reason about unobserved states, and a world-action model that combines visual world modeling with robot control.
Results
RynnUnify achieves the highest average embodied understanding score among evaluated unified multimodal models and improves the Rynn-VP overall score by 9.75 points over Cosmos 3 Nano. Its agentic extension improves the Rynn-GEU average score by 6.0 points over direct reasoning, while the world-action extension attains a 51.5% success rate on 20 selected composite RoboCasa365 tasks and a 46.7% average success rate across three real-robot tasks.
Key Points
- RynnUnify integrates an 8B autoregressive reasoner and an 8B video generator to enable interleaved video-language reasoning for long-horizon embodied planning.
- The authors introduce Rynn-VP and Rynn-GEU as dedicated benchmarks for evaluating sustained visual planning and reasoning about unobserved future states.
- The architecture supports an RL-trained agentic extension for selective video generation and a world-action model extension that achieves multi-step manipulation in simulation and on real hardware.