BI 106 Jacqueline Gottlieb and Robert Wilson: Deep Curiosity
Jackie and Bob discuss their research and thinking about curiosity. Jackie’s background is studying decision making and attention, recording neurons in nonhuman primates during eye movement tasks, and she’s broadly interested in how we adapt our ongoing behavior. Curiosity is crucial for this, so she recently has focused on behavioral strategies to exercise curiosity, developing tasks that test exploration, information sampling, uncertainty reduction, and intrinsic motivation. Bob’s background is developing computational models of reinforcement learning (including the exploration-exploitation tradeoff) and decision making, and he behavior and neuroimaging data in humans to test the models. He’s broadly interested in how and whether we can understand brains and cognition using mathematical models. Recently he’s been working on a model for curiosity known as deep exploration, which suggests we make decisions by deeply simulating a handful of scenarios and choosing based on the simulation outcomes. We also discuss how one should go about their career (qua curiosity), how eye movements compare with other windows into cognition, and whether we can and should create curious AI agents (Bob is an emphatic yes, and Jackie is slightly worried that will be the time to worry about AI). Jackie’s lab: Jacqueline Gottlieb Laboratory at Columbia University.Bob’s lab: Neuroscience of Reinforcement Learning and Decision Making.Twitter: Bob: @NRDLab (Jackie’s not on twitter).Related papersCuriosity, information demand and attentional priority.Balancing exploration and exploitation with information and randomization.Deep exploration as a unifying account of explore-exploit behavior.Bob mentions an influential talk by Benjamin Van Roy:Generalization and Exploration via Value Function Randomization. Timestamps: 0:00 – Intro4:15 – Central scientific interests8:32 – Advent of mathematical models12:15 – Career exploration vs. exploitation28:03 – Eye movements and active sensing35:53 – Status of eye movements in neuroscience44:16 – Why are we curious?50:26 – Curiosity vs. Exploration vs. Intrinsic motivation1:02:35 – Directed vs. random exploration1:06:16 – Deep exploration1:12:52 – How to know what to pay attention to1:19:49 – Does AI need curiosity?1:26:29 – What trait do you wish you had more of?