Robot Learning 2025: Foundational Models for Robotics and Scaling DeepRL - Robot Learning: Methods and Considerations for Scaling Data Collection

I discuss data collection for robotics, focusing on the challenges and different methods. I start by asking how much data a robot needs to perform everyday tasks, comparing it to a child learning to cook. He then introduces two early methods of robot programming: keyframe programming, which is cumbersome and prone to errors, and teleoperation, where a user controls the robot's movements. Teleoperation, while better, still presents challenges like dynamics mismatch between the user and robot, latency issues, and the need for extensive operator training. I also mention the difficulty of collecting high-quality data, as a lot of data is of poor quality. I discuss many available datasets and the hopes to integrate large first- and third-person video datasets as a way to meet the data needs of large models. I conclude by discussing the trade-off between collecting data from experts that can have a high signal-to-noise ratio compared to using "in-the-wild" 3rd person video, which is plentiful but has a low signal-to-noise ratio.