°Õ¾±³Ù±ô±ð:ÌýSafe Robot Autonomy in Interactive Open-WorldÌýEnvironments, Accelerated by Generative AI
Date: Wednesday,ÌýMarch 20, 2024
°Õ¾±³¾±ð:Ìý11:00 AM - 12:00 PM
±Ê±ô²¹³¦±ð:ÌýEIT 3142
Join Zoom Meeting:Ìý
Meeting ID:Ìý965 8623 2115
±Ê²¹²õ²õ³¦´Ç»å±ð:ÌýFIROOZI24
Abstract:ÌýGenerative AI offers the potential to empower autonomous robots with the ability
(i) to connectÌýabstract concepts to 3D scene geometry
(ii) to enable complex intention prediction andÌýcollaboration capabilities for robots to interact intelligently with humans and other autonomousÌýagents.
However, there remain significant research challenges in how to integrate these generative AI technologies into the robot autonomy stack without compromising safety, reliability, and explainability. In this talk, I will present my recent research toward addressing these issues. In the first part, I will discuss how large pre-trained Vision-Language Models allow robots to connect abstract concepts to 3D scene geometry, enabling autonomous interaction with objects and 3D scenes. I'll demonstrate how language and feature distillation into 3D world representations, such as Neural Radiance Fields, facilitate robots’ open-vocabulary navigation and manipulation. Beyond interaction with objects and 3D scenes, robots must safely interact with other coexisting dynamic agents in the scene including other robots or humans. As they interact, they need to reason about and predict other agents' objectives and reactions. In the second part of the talk, I’ll delve into safe and intelligent multi-agent interactions modeled through distributed real-time optimization techniques such as model predictive control (MPC) and game-theoretic planning. I will detail how game-theoretic planning, closely related to distributed MPC, introduces an additional layer of abstraction to interactive autonomy by modeling agents' preferences and mutual influences. I will provide an overview of how to enhance these interaction modeling tools' capabilities with larg generative AI models. I’ll briefly touch on paths forward, including employing generative AI models like diffusion in distributed multi-robot learning to foster the emergence of cooperation; using generative AI as adversarial agents to induce robustness in autonomous systems, building new foundation models for interaction modeling, and evaluating safety as we integrate these foundation models into physical robotic systems.
Speaker Bio:ÌýRoya Firoozi is currently a postdoctoral researcher at Stanford University, Multi-Robot SystemsLaboratory. She received her bachelor's degree in mechanical engineering and her Ph.D. in control theory, with minors Ìýin optimization and machine learning from the University of California, Berkeley. Her research focuses on enabling safe robot autonomy powered by generative AI in interactive, open-world environments. Roya has received the NSF Postdoctoral Fellowship Award, the UC Berkeley Outstanding Graduate Student Instructor Award, and the UC Berkeley Chancellor’s Award. She was recognized as a rising star in EECS and Aerospace Engineering. Outside of her research, she has served as the robotics mentor at Stanford AI4ALL outreach program that aims to increase diversity in artificial intelligence.
Ìý