DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos

February 23, 2026

Here's something that caught my attention — researchers have developed DreamDojo, a groundbreaking robot world model trained on over 44,000 hours of human videos. Imagine a system that can simulate how actions play out in all sorts of environments, from everyday chores to complex dexterous tasks. According to /u/Secure-Technology-78, what’s impressive is how DreamDojo learns diverse interactions and fine motor controls just from watching people, even without lots of labeled action data. They use what's called continuous latent actions — kind of a universal language for movement — to transfer knowledge from videos to robots. After some post-training, the model shows a solid understanding of physics and control precision, and a distillation pipeline helps it run in real time at about 11 frames per second. So, whether it’s for teleoperation, policy testing, or planning, this work by /u/Secure-Technology-78 could really push forward the idea of versatile, open-world robots capable of tackling unpredictable, contact-rich tasks.

"Being able to simulate the outcomes of actions in varied environments will revolutionize the development of generalist agents at scale. However, modeling these world dynamics, especially for dexterous robotics tasks, poses significant challenges due to limited data coverage and scarce action labels.

As an endeavor towards this end, we introduce DreamDojo, a foundation world model that learns diverse interactions and dexterous controls from 44k hours of egocentric human videos. Our data mixture represents the largest video dataset to date for world model pretraining, spanning a wide range of daily scenarios with diverse objects and skills. To address the scarcity of action labels, we introduce continuous latent actions as unified proxy actions, enhancing interaction knowledge transfer from unlabeled videos.

After post-training on small-scale target robot data, DreamDojo demonstrates a strong understanding of physics and precise action controllability. We also devise a distillation pipeline that accelerates DreamDojo to a real-time speed of 10.81 FPS and further improves context consistency. Our work enables several important applications based on generative world models, including live teleoperation, policy evaluation, and model-based planning. Systematic evaluation on multiple challenging out-of-distribution (OOD) benchmarks verifies the significance of our method for simulating open-world, contact-rich tasks, paving the way for general-purpose robot world models."