Stanford Online
Daniel S. Brown
May 6, 2022
In this talk I will discuss recent progress towards using human input to enable safe and robust AI systems. Much work on robust machine learning and control seeks to be resilient to, or completely remove the need for, human input. By contrast, my research seeks to directly and efficiently incorporate human input into the study of robust AI systems. One problem that arises when robots and other AI systems learn from human input is that there is often a large amount of uncertainty over the human’s true intent and the corresponding desired robot behavior. To address this problem, I will discuss prior and ongoing research along three main topics:
(1) how to enable AI systems to efficiently and accurately maintain uncertainty over human intent
(2) how to generate risk-averse behaviors that are robust to this uncertainty
(3) how robots and other AI systems can efficiently query for additional human input to actively reduce uncertainty and improve their performance.
My talk will conclude with a discussion of my long-term vision for safe and robust AI systems, including learning from multi-modal human input, interpretable and verifiable robustness, and developing techniques for human-in-the-loop robust machine learning that generalize beyond reward function uncertainty.
Source
nice presentation!