Robot Learning 2025: Foundational Models for Robotics and Scaling DeepRL - RobotLearning Learning2Plan With Large Models

I discuss latent inverse models and their application in video generation and robotics. I explain how these models learn to predict future events by embedding an understanding of world dynamics, including complex elements like human motion. The models condition predictions on latent action models, allowing for controlled manipulation through learned actions. Berseth highlights the importance of a codebook for defining actions, often tailored to the specific task, and emphasizes the need for sufficient data and image fidelity for effective learning. I contrast this approach with previous inverse dynamics models, noting this version integrates action control. He also touches on the "Genie" project, emphasizing the significance of scale (data and image size) in capturing detailed dynamics, particularly for robotics, where fine-grained control is essential. I conclude by briefly explaining the training process of a latent dynamics model, which infers underlying states from image data.