2012 Recipient: Peter Dayan

2012 Recipient: Peter Dayan

Dr. Peter Dayan is a pre-eminent researcher in Computational Neuroscience with a primary focus on the application of theoretical computational and mathematical methods to the understanding of neural systems. He has pioneered the use of Bayesian and other statistical and control theory methods from machine learning and artificial intelligence for building theories of neural function.  A major focus of his work is on understanding the ways in which animals and humans come to choose appropriate actions in the face of rewards and punishments, and the processes by which they come to form neural representations of the world. The models are informed and constrained by careful attention to neurobiological, psychological and ethological data and the models are both mathematically specified and computationally implemented. Dr. Dayan is a co-author of ‘Theoretical Neuroscience,’ a leading textbook in the field. It also provides one of the most influential sources of hypotheses about the function of the human mind and brain in current cognitive science.

Dr. Dayan’s early work focused on reinforcement learning.  This is an appealing learning paradigm because it does not require explicit instruction on what actions would have been ideal for an organism.  Instead, it only requires that an environment provide rewards depending upon sequences of actions.  Furthermore, through a process of comparing actions to internally generated predictions of rewards, these methods are able to learn even when external rewards are not immediately available.  Reinforcement learning integrates psychological insights from human and non-human animal learning with notions from control theory about optimal behavior. With colleagues, Dr. Dayan broadened these existing links using Bayesian ideas about uncertainty, and extended them into the domain of neuroscience by identifying the neuromodulator dopamine as a neural mechanism that plays a critical role in generating internal predictions of reward.

Dr. Dayan’s path-breaking work has been influential in several fields impinging on cognitive science, including machine learning, mathematics, neuroscience, and psychology.  The center of mass of his research program has been concerned with learning (self-supervised and reinforcement) and conditioning, and the influence of neuromodulation on this learning. However, he has also contributed significantly to the study of activity-dependent development and to key issues relating to population coding and dynamics. For example, in contrast with classical accounts, which suggest that the activity of large populations of neurons encode the value of the stimulus, he has articulated a view in which neural computation is akin to a Bayesian inference process, with population activity patterns representing uncertainty about stimuli in the form of probability distributions. Finally, Dr. Dayan has contributed to furthering our understanding of hippocampal function.  As an example, he has suggested that a key function of the replay of patterns of activity in the hippocampus that occurs during sleep is to maintain the representational relationship between this structure and the cortex so that episodic memory can continue to work over a lifetime, even as the coding of information changes with the acquisition of new knowledge. Critically, in all of his work, Dr. Dayan has been systematically attentive to the empirical findings, using them to constrain the simulations and theory closely, and, in so doing, has been able to articulate tractable and plausible biological accounts of the neural computations subserving learning and memory function, more generally.

Dr. Dayan’s academic career started at the University of Cambridge where he obtained a Bachelor of Arts (Hons) degree in Mathematics. This was followed by a PhD degree in artificial intelligence at the University of Edinburgh, which focused on statistical and neural network models of learning. He then went on to do a series of postdoctoral fellowships, a brief one with the MRC Research Centre in Brain and Behaviour at Oxford, followed by one at the Computational Neurobiology Laboratory at The Salk Institute, the Department of Computer Science at the University of Toronto, following which he became an assistant professor at MIT. Dr.  Dayan subsequently relocated to the Gatsby Computational Neuroscience Unit at University College London in 1998, assuming the position of Director of this Unit in 2002. He continues to direct the Gatsby Unit at present, and is a Professor of Computational Neuroscience at University College London.

Dr. Dayan has written approximately 200 publications, which have garnered in excess of 15,000 citations. Dr. Dayan has played a formidable role in fostering the growth of Computational Neuroscience as a discipline. He has mentored many junior investigators, served as an adviser to numerous advisory boards and selection panels, been on the editorial boards of multiple journals and participated as a member of program committees for a variety of academic conferences.

Selected Publications

Watkins, CJCH & Dayan, P (1992). Q-learning. Machine Learning, 8, 279-292.

Hinton, GE, Dayan, P, Frey, BJ &  Neal, RM (1995). The wake-sleep algorithm for unsupervised neural networks. Science, 268, 1158-1160.

Montague, PR, Dayan, P &  Sejnowski, TK (1996).  A framework for mesencephalic dopamine systems based on predictive Hebbian learning. Journal of Neuroscience, 16, 1936-1947.

Dayan, P, Kakade, S  & Montague, PR (2000). Learning and selective attention. Nature Neuroscience, 3, 1218-1223.

Daw, ND, Kakade, S  & Dayan, P (2002). Opponent interactions between serotonin and dopamine. Neural Networks, 15, 603-616.

Dayan, P  & Balleine, BW (2002). Reward, motivation and  reinforcement learning. Neuron, 36, 285-298.

Kali, S  & Dayan, P (2004) Off-line replay maintains declarative memories in a model of hippocampal-neocortical interactions. Nature Neuroscience, 7, 286-294.

Daw, ND, Niv, Y &  Dayan, P (2005) Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nature Neuroscience 8:1704-1711.

Yu, AJ  & Dayan, P (2005) Uncertainty, neuromodulation, and attention. Neuron 46:481-492.

Dayan, P, Niv, Y, Seymour, BJ &  Daw, ND (2006) The misbehavior of value and the discipline of the will. Neural Networks   19:1153-1160.

Niv, Y, Daw, ND, Joel, D &  Dayan, P (2007)  Tonic dopamine: Opportunity costs and the control of response vigor.   Psychopharmacology, 191:507-520.

Dayan, P  & Huys, QHM (2008) Serotonin, inhibition and negative mood. Public Library of Science: Computational Biology, 4(2):e4.

Huys, QJM  & Dayan, P (2009). A Bayesian formulation of behavioral control. Cognition, 113:314-328.

Schwartz, O, Sejnowski, TJ  & Dayan, P (2009) Perceptual organization in the tilt illusion. Journal of Vision, 9:1-20.

Dayan, P  & Solomon, JA (2010) Selective Bayes: Attentional load and crowding. Vision Research, 50:2248-2260.