Stanford AA289/ENGR319 - Robotics and Autonomous Systems Seminar - Stanford Seminar - Toward Scalable Autonomy - Aleksandra Faust

Aleksandra Faust is a Senior Staff Research Scientist and Reinforcement Learning research team co-founder at Google Brain Research. January 28, 2022 Reinforcement learning is a promising technique for training autonomous systems that perform complex tasks in the real world. However, training reinforcement learning agents is a tedious, human-in-the-loop process, requiring heavy engineering and often resulting in suboptimal results. In this talk we explore two main directions toward scalable reinforcement learning. First, we discuss several methods for zero-shot sim2real transfer for mobile and aerial navigation, including visual navigation and fully autonomous navigation on a severely resource constrained nano UAV. Second, we observe that the interaction between the human engineer and the agent under training as a decision-making process that the human agent performs, and consequently automate the training by learning a decision making policy. With that insight, we focus on zero-shot generalization and discuss learning RL loss functions and a compositional task curriculum that generalize to unseen tasks of evolving complexity. We show that across different applications, learning-to-learn methods improve reinforcement learning agents generalization and performance, and raise questions about nurture vs nature in training autonomous systems. View the full playlist: https://www.youtube.com/playlist?list=PLoROMvodv4rMeercb-kvGLUrOq4HR6BZD 0:00 Introduction 3:54 How to train goal reaching policies? 17:46 What about resource-constrained robots? 28:13 Training RL Agents is a (Sequential) Decision Making Problem 29:45 Learning Loss Functions 29:51 Evolving RL Algorithms 36:06 Generative Curriculum for Compositional Tasks 36:36 Autonomous web navigation 39:37 Learning to Navigate Web 43:19 Compositional Design of Environments (CODE) 46:58 Does it work in reality? RealED -- CODE w/ real primitives. 48:54 Better Generalization 51:01 Autonomous Scalable Systems Future #reinforcementlearning #autonomousrobot