Robot Learning 2025: Foundational Models for Robotics and Scaling DeepRL - Robot Learning: Sequential Decision Making

This lecture discusses the fundamental concepts of planning in the context of sequential decision-making problems and how they can be addressed with foundational models. It emphasizes that while in ideal scenarios, solutions to such problems might be readily available, in reality, agents often need to discover these solutions themselves. The lecture introduces the concept of planning as the process of finding actions that lead to the best possible states, which can then be maintained or improved upon in the future. The discussion then delves into the complexities of planning, highlighting the significant computational challenges, particularly when dealing with stochastic environments. The lecture explores the impact of factors like the branching factor and the depth of the search tree on the complexity of finding optimal solutions. Finally, the importance of identifying, exploiting, and learning structural properties within the planning problem, such as compositionality, productivity, and localism, is discussed to mitigate these computational challenges and improve the efficiency of planning algorithms.