I would also like to thank our co-sponsors the Economics Department the introduced sublunary program in neuroscience the interdisciplinary program in cognitive science the McDonough School of Business the psychology department and the Georgetown University lecture fund I am extremely honored to introduce professor glimmer today someone who has built bridges across the canyon that lies between the disciplines of neuroscience and economics these bridges have paved the way for the emerging field of neuro economics dr. Glenn sure received his bachelor’s and neuroscience from Princeton University in 1983 and his PhD in neuroscience from the University of Pennsylvania in 1989 his first four publications all his first author were based on research he conducted as an undergraduate on the rewarding effects of neuro peptides on the brain he then went to the University of Pennsylvania where he studied eye movement physiology for his PhD thesis he joined the faculty in neuroscience at NYU in 1994 continuing his studies on eye movements his research then moved into studying reward value and choice behavior he currently is professor of neuroscience economics and psychology and the director of the center of neuro economics at NYU amongst his many honors he has been a McKnight scholar a Klingon Steen foundation fellow and the Macdonald foundation 21st century scholar in 2004 he founded the Society for neuro economics to catalyze research at the interface of neuroscience psychology and economics he will share his own research in neuro economics with us today please join me in a warm welcome for professor Paul glib sure Thank You Justine can you all hear me in the back so before I begin today I want to thank you for inviting me to this very eclectic and broad audience this is kind of the worst thing that can happen to a neuro economist people expecting a lot from many different disciplines all at the same time to help me do the best I can I want to ask you to tell me what your disciplines are so I’m gonna ask for a show of hands I know Justine’s worried that I have not actually triggered my talk I can tell it I can see it in her face hang on okay I’m gonna ask you to commit to a discipline you can only pick one even I would only pick one I don’t know which one I would pick if you think of yourself and the choices will be neuroscientist psychologist economist or finance person neuroscientist psychologist economics oh this is really bad news finance person okay so I’ll do my best please stop me as I go if I say something that makes absolutely no sense to your discipline I will be doing my best to speak all four languages simultaneously it’s giving new meaning to the notion of speaking in tongues my plan today is to break the talk into three parts in the first part it’ll be easy because I what I’m gonna do is I’m gonna sort of lay out the logic of what no economics is why I think it’s important and how it’s grown in the last decade and a half in the second portion of the talk what I’ll do is lay out what’s becoming the standard model of the human choice architecture many labs have participated in the development of this model I’ll describe my labs contribution to it or some of my labs contribution to it the critical idea I want to get across is just what we know about the architecture for choice how that Maps went to standard neurobiological economic and psychological theories and in the last step I’ll show an example of what neuro economics can really do when you bring these together I’ll show an example of basic neuro physiological research and choice cortex what it tells us about the representational structure of value choice and utility and then make a novel prediction for a choice anomaly in humans and monkeys and show you that it actually exists as revealed by an understanding of the computational algorithm in choice cortex okay but first I want to go and present this this rationale you know as as a number of very famous economists have said to me famously George Bob Lucas once turned to me said why neuro economics and I want to explain why neuro economics and I want to explain it in two steps first I want to explain that over the course of the last 150 years there’s been a tremendous reconciliation across many different levels of scientific analysis if I can point at this I can’t really at the bottom of this chart is a

little map that I’ve labeled physics and honor and I’m showing you some of the basic logical objects the physicists would be comfortable with an electron an orbit electric field things like that these are the logical constructs from which physics is built lying above it is a different level of analysis chemistry which has things like molecules and transparency periodic tables these were the logical objects from which chemistry is built and lying above it the logical objects of biology things like genes things like heredity things like organs of course in the eighteen hundreds these were fully separate areas there was nothing about these areas that went together people were one or the other or the other now that began to change over the course of the early 1900’s when first physics and chemistry were largely united through the development of the wave equations by Schrodinger and his colleagues the development of the wave equations made it possible to look at objects within the domain of chemistry through the length of the wave equations at the level of physics and physical chemistry a particular sub branch of it that unites those two disciplines grew up now it’s important to remember that those two disciplines still exist the disciplines didn’t go away but the two disciplines were deeply and irrevocably changed they were changed we could check ideas in physics at the level of chemistry and import constraints from physics to chemistry and vice-versa the physics in chemistry we have today are very different from the physics and chemistry that existed before the wave equations were developed there’s no doubt about this that they’re stronger they’re richer and that they’re more predictive that happened again at the interface of chemistry and biology with the development of the discovery of DNA when Watson and Crick first discovered DNA and argued that this chemical object was the physical instantiation of the biological notion of the gene many people responded by saying that’s interesting but it’s irrelevant to biology when Watson was considered for tenure in the Harvard biology department many argued that what he did simply wasn’t biology that never would the theory of heredity be influenced by an understanding of genes of course we know that true now and so the challenge is to ask whether or not we can achieve that kind of intra level communication and dialogue when we try to bridge psychology neuroscience and economics in the study of human decision-making the argument I’ll make to you today is not only is that in principle possible but it is well underway and that we have fundamental insights that will serve the same role that the wave equations did that will bridge together these three disciplines and make each of them stronger not eliminate any of them but make each of them stronger now to make that point I want us to think about a storybook decision I want us to look at it from the point of view of an economist a psychologist and a neuroscientist to make it clear to you how different these three disciplines are how non-overlapping their set of logical objects are and what a weakness that is ok so let me tell this story I apologize to the graduate students in the lab in there in the room ok so here’s the story new graduate students go to a conference they meet at the meeting now it’s important that you know that the two graduate students are both have partners they love their partners they’re happy with their partners they’re thinking about staying with their partners forever and they meet each other at a conference they find themselves talking about utility theory or dopamine or attribution theory they they discover an instant chemistry and then they have to make a decision that night about whether or not they’re gonna wind up in each other’s rooms and for the purposes of our story let’s assume that these two graduate students spend the night together in the morning they’re miserable racked with guilt the satisfaction unhappiness they return home to their relationships complete basket cases they recover they meet again in the next year at the next conference is an important point for the economists and they do it again how do we think about that the decisions they made well for an economist the answer is simple for the last hundred years even more a critical feature of economic thought is what we care about is what we can measure what we can see what our observed variables people reveal their preferences by their choices these two people revealed their preferences was it a mistake well there’s some notions of mistake in economics but these guys did it twice so in no meaningful sense can an economist think that what these people did was make a mistake they prefer to sleep with this person than to not to and that’s the basic story from which we’ll extract their preference structure a psychologist looks at a very different group of statements by these

people they ask what were you thinking what were you feeling what does it make you feel like now what were your perceptions at the time how are those perceptions change these are internal mental state variables and a psychologist struggles to understand how it is that they made this mistake how it is that their mental life is at such dissonance with their behavior and how we can help them bring those two into alignment the fact that they slept together is kind of the least interesting part a neurobiologist might say might say a lot of things but let’s focus for the moment on the idea that a neurobiologist sees this behavior is entirely coherent looking through the lens of evolutionary theory what have these two people done they’ve maximized their behave their long-term evolutionary Fitness by having relationships with people who have different genetic patterns for the woman this ensures that her progeny have the broadest possible distribution of genetic background and for the man this ensures that he has the maximum number of progeny I want you to notice that nothing in this story talks about revealed preferences not a minute in this story talks about mental states what’s interesting about these three disciplines is that they give us such different answers to the notion of why people make the decisions they do what I want to urge you to do is think about the problems of modeling behavior a lot of different levels simultaneously down here at the bottom of my little cartoon here of different levels of analysis is physics chemistry and now biology biology is of course the level at which we begin to model decision making in a meaningful way and I think of these as kind of being the level at which Sherrington Zinn fights on which really modern neuroscience are founded lay above that of course is the level of psychology where people like Pavlov and Weber and Fechner and Stevens worked and above that of course the level of economics where well people like Samuel s’en have worked and what we’re really going to be asking is whether or not there isn’t some way to stitch these top three levels together in a deep way that strengthens all three levels that aligns those theories in a way that makes them more predictive and more powerful now we have this one problem believe me for me it’s a really big problem and that is that the social natural science boundary lies between here the level of nura of psychology and the level of economics at a university setting this is a particularly big deal because there there’s a different dean for here and for here so we’re trying not just to reconcile knowledge but reconcile Dean’s and those of you who are my age know that that’s hard but I want to stress that reductions of this type have succeeded before in fact they’ve always succeed partial reductions of this type just have always worked the relationship between physics and chemistry it’s a fact now we use it every day in chemical and physical labs across campuses across the world and there’s no arguing that Watson and Crick’s demonstration that this molecule DNA was the physical evidence the physical object of heredity changed the way biology works and what I’m arguing to you today is that these interdisciplinary linkages are going to show up in the next twenty or thirty years as we propagate up this chain oops sorry I went the wrong direction as we propagate up this chain there’s no doubt about it and I’m sure your evidence of it today but it’s just flying in the face of history to imagine this is an unavoidable this is an avoidable outcome okay so what I want to do now is give you a quick overview of how ideas of decision-making evolved in the last 50 years or so very quickly maybe even last 300 years in neuroscience and in economics because it’ll be obvious then how you put them together of course the critical insight for thinking about decision-making behavior organization of behavior of any kind for biologists is Rene Descartes Descartes argued fundamentally 350 years ago the physiologist could study behavior and the way they would do it was by breaking behavior into two categories the simple category which we now call reflexes because he used the French for breathless year and the complex category that were voluntary and were the product of the human soul it was the first of these two of the he argued scientists could meaningfully study and he argued that the way we would meaningfully study them was by tracing a pathway by which sensory information enters the nervous system is filtered or reflected and then passes out to produce movement in this famous cartoon from his treatise on man decart explains to us with what he imagines is a Euclidian style proof that the fire a contacts foot be that these rapidly moving particles of fire joggle

a little sensor on the bottom of B which pulls open a little hose on a little wire we ran in through this tube see which opens a pneumatic valve the pneuma travels back through this pipe activates this muscle and draws the foot away arcane and Greek sounding in some ways but the critical idea this notion that we understand action by following input and connection to output really is the core of the way neuroscientists have thought about the generation of behavior the person who codified that those of you who are neurobiologist know is this guy Chow Scott Sherrington who won the Nobel Prize in the 1940s Sharon can laid out in anatomical terms the Cartesian reflex he showed how sensors in the foot gather information pass it to the spinal cord make positive connections to motor neurons which activate muscles and draw feet away from fire and of course he studied much more complicated behaviors than that but you clearly established that the neurobiological tradition is one of tracing pathways tracing connections that’s how behavioral organization is studied by neurobiologists and what I want to stress to you is how different during the same time line the study of choice is from the point of view of an economist I could start with the work of Pascal a rough contemporary of de cartes who really laid the foundation for modern economics but instead I’m just going to cut to the most interesting and more recent version of economics a brand of economics called neoclassical economics which was really instantiated in the early 1900’s around the time of the wave equation and reached its first full bloom at the hands of this guy Paul Samuelson how many of you have heard of now Samuelson won the Nobel Prize for urging economists to focus on what they could observe and on what’s called in economics consistency Samuelson’s critical notion was that in understanding choice the first thing we have to look for is to determine whether a chooser is behaving in a logically consistent manner now we use the word rational to describe this logically consistent behavior that’s actually a technical term for an economist which is a terrible historical error it should have been called you know Shimoga Smoove or something because rational actually has common meaning and often these common and technical meanings are at war with each other and this leads to endless confusion but what I want to do is I want to explain to you Samuelson’s insight because it’s really critical if you’re not an economist that you understand the primitive most basic object that an economist thinks about okay so imagine this experiment let’s see this is an experiment I’m gonna do on Karen and I imagine that Karen has the following preference structure she prefers apples to oranges great she prefers oranges to pears great this implies of course that she prefers apples to pears that should be obvious now if Karen seemed consistent in her preferences this might be her preference structure she might prefer apples to oranges and oranges to pears but prefer pears to apples for an economist this is a truly terrible irrational awful thing to do anybody who feels this way should just either be retrained or shipped out and they’re right and here’s what so imagine this experiment so here we’re gonna make this these are actually modeled after two members of my lab but this is kind of fun so so this is um this is Karen and this is Justine so Karen’s got a pair and she’s got three scents Justine walks up to her and says I have an orange would you like to fill that would you like to buy that I’ll sell you this orange if you give me a pair well Karen looks I prefer orange I prefer oranges two pairs so that’s a good deal but Justine says you have to pay me one cent it’s a better deal but I want thumb for this so Karen says great that’s a great deal now I have an orange I have only two cents but the orange is better I’ve traded up and Justine now has that pair Oh Justine comes back to her she says well since that worked out so well for us what would you say to this Apple I’ll sell you this Apple for your pair plus one set Karen consults her internal preferences I prefer apples to oranges apples to oranges it’s a good deal she may the trade now she has this nice Apple and only one cent and Justine the orange now remember Karen’s inconsistent in her preferences which means Justine can whip out the original piece of fruit this pear that she got from Karen and offer it to her

for one cents and the Apple of course Karen then takes the pear she’s now got no money but she feels very well treated even though she’s now holding on to her original pear and of course if she hasn’t figured out what’s happened Justine takes all of her money now I stress this because this is a really core idea in economic anybody holds these preferences isn’t making any sense and all in economic theory is built on asking what are the representations that would be required in order to produce consistent choice what would your internal representations of value I’m gonna put internally in italics for an economist that doesn’t mean it’s inside you just internal to the theory representations of value necessary to avoid situations like this Samuelson won the Nobel Prize for showing what those internal representations would have to look like he showed but for a chooser who behaves consistently the internal representation of value has to be monotonic with regard to number of oranges number of apples and number of pears weekly monotonic and technical econ speak and that’s a really really big accomplishment because he tells us something about what’s going on inside the theory of representation when its user behaves consistently now that’s a series of ideas that have been expanded upon many times most importantly probably by von Neumann and Morgenstern they actually showed what the internal representations would have to look like when choosers faced not just objects of different values but probabilistic events they were actually able to show using a set of very beautiful axioms that choosers who are consistent with regard to probability who treat a 50% chance of something as half as good as a hundred percent chance of that same thing that these choosers would have to have a richer and more interesting kind of internal representation of value again internal to the theory and this internal representation has really come to be called utility so for an economist thinking about choice we think about goods as having values internal subjective idiosyncratic values but that are consistent we think essentially about those values as being multiplied by the probabilities that you’ll receive those Goods so if I asked you to check between an apple which was a value 3 to you but you’d only receive it with a 50/50 chance and an orange which was of value to to you which you’d received with an 80% chance you might well pick the orange this would reflect the higher value that aggregates value and probability this is what this is a decision variable in a general sense but the one that we usually call expected utility its utility because its value and the mixture of probability lets us use the word expected because we don’t really know what’s gonna happen okay so here’s the critical question you’ve got this theory of pathways you’ve got this theory of values the obvious question is how do you push these two things together the obvious way to begin would be to imagine well here’s our economics level representation here’s utility here’s choice utility that’s the economists know is derived from observations of choice and we imagine this operator this mathematical operator called the Arg max pick the best which lies between the utilities of the different goods we compare and the good that we actually select what we want to explore is the possibility that there’s a similar process going on at the neurobiological level that the representational theory that says this is the best way to produce consistent choice also tells us what lies down at the neurobiological level and we actually use the word here subjective value I’ll mention it a few times because there are important mathematical differences between the measurements we make neurobiologically and the choice derived theoretical objects like utility that we construct as economists and we respect that by not calling this object down here utility but rather subjective value and in fact we argue that these two I’ll show you in a bit are linearly correlated that’s a very strong claim but it’s what the data seems to suggest choice of course we think of as isomorphic to action in the neurobiological construct and so what we really want to do is ask whether or not we can build a representation like this at the neurobiological level let me pause here for the Economist this is one of my phys on the top you can’t read it side bar for economists any good economist is now angry I hope in the room because I have done something really terrible consistent choosers behave as if they represented a monotone utility function that’s you get that with your mother’s milk as an economist literally and you would never go past

this word as if and I am I am proposing now a new kind of theory ok this is not a traditional economic theory this is what we call a because theory we hypothesize that consistent choosers behave the way they do because they actually represent a monotone variable what we call SV of X in some discrete brain area ok for the economist I want to be really clear that means this theory can be tested at the behavioral level or at the physiologic level it can be falsified at either level it’s critically a theory that maps between levels it is different from traditional utility theory ok how do you do that well I think the first effort to do that is probably this experiment by Michael Platt an old postdoc of mine now the director of the Institute for neurosciences at Duke University in Michael’s first experiment what he did is he took a monkey he was recording from a single neuron in the parietal cortex that we had reason to believe was involved in the decision to look in this cartoon 10 degrees to the right so monkeys gonna choose where to look that’s easier than having him push buttons for purely technical reasons and we’re gonna look at neurons that seem to be involved in something about looking to the right now if we were tracing pathway as I tell you that these neurons receive input from the visual world and produce outputs that control movements of the eyes they lie about halfway down that process if I was trying to be a neuro economist I might hypothesize that these neurons lying halfway down the process might in fact represent in their action potential firing rates the utilities or let me say that properly the subjective values of the options the animals were considering so in this experiment what Michael did was basically train the monkeys to play a game of roulette two targets were illuminated one red one green after a variable delay the fixation light with probability 50/50 turned either red or green if it turned red the monkey looked at the red if it turned green the monkey looked at the green I’m gonna show you only the red trials why because they have all the same sensory input and all the same motor output nothing’s different about them now across blocks of trials what Michael very was the amount of reward for the red target and the amount of reward for the green target in the first block of 100 trials the monkey might get three tenths for looking at the red target in the second 4/10 I should have said of a milliliter of beriberi fruit juice which monkeys truly love this is a really good consumer good for monkeys 3.3 milliliters of beriberi fruit juice play 4 or 0.2 and what Michael simply gonna ask is the same pathways active it’s the same sensory to motor path but the values changing to the neurons know that and of course the answer is yes here what I’m plotting for you is the average activity of a single nerve cell in blue when the monkeys going to look at the red target and expects a small reward and in red when the monkeys going to look at that red target but expects a large reward in fact Michael ran many many different reward magnitudes on each neuron what I’m showing you here is a cartoon of that imagine there are seven of these lines we can cut out a lot of time and ask how firing rate varies as a function of expected reward magnitude here I’m plotting that for you firing rate this is actually the ratio of values and what I want you to see is that there’s a pretty pretty linear looking relationship and I want to stress that this is the entire dynamic coding range of this class of neuron these are cortical neurons they’re dynamic range is about zero to 100 Hertz so this isn’t something that’s having a little effect on this neuron of course Michael recognized that if we were thinking a bit like economists and thinking about expected utility type problems we ought to be able to vary the probability as well and he did that here what Michael varied was the probability that the read target would be the reinforced target here 50% here 80% here 20% and of course Michael got the same result and here you can see all 5 time periods but this is the only one that’s really important okay so great what Michaels experiments suggested was that individual neurons that lie halfway between sensory and motor represent something like subjective value something like utility and they imply that there’s some interaction amongst this representation that is involved in the process of making a choice to think about that let’s take a look at what this cortical area looks like and how we can think about it more in a more detailed fashion and to do that I need to remind you of the human brain so this is a slide I stole from David Van Essen and this is a human brain what david’s lab is going to do is using a computer

inflate the brain and i want to reveal to you what all the neurobiologists know that brains are basically flat cortex is a flat sheet like structure david makes just a few cuts and we flatten it out and so when we look at a sub-domain of cortex like the lateral intraparietal area where Michael was recording right up here we’re actually looking at a planar sheet of material now it turns out that these sheets of neurons are organized in a really really precise fashion we spent a few years beating on that organization in parietal area lipase equivalent in humans something we call IPS 1 but you know what this graph is unreadable and I slowly learned that my competitor Marty Sereno did a much better job of exactly the same experiment and so I’m going to show you Marty’s experiment because it’s like I don’t have to do for a analysis or anything so what what’s happening is human subjects are making a decision about which eye movement they want to make they’re doing it in random order but for our purposes let’s imagine they’re looking back and forth back and forth back and forth back and forth around the clock tracking this little spot as they look out choose to look at it at different times what you’re looking at here is the flattened Li P equivalent on the right side and the left side of one person’s brain and what I want you to see is as the dot sweeps up this band of activation sweeps out here and as the dot sweeps down on this side the band of activation sweeps down on this side what does this reveal to us this reveals that these sheets of cortex encode in a topographic map like fashion each of the eye movements a neuron right here likes attended something about a 10 degree rightward movement and Michael’s data says that what it what it actually encodes is not like but utility or something like utility so the idea then becomes that what we’re looking at is a topographic map of utility space so here I’m going to show you that now this is based on lots of other experiments this is the work of one of the economists in my labs he’s a random utility theorist by the name of Brian Webb and what Ryan’s built for you here is a simulation of what Li P might look like if you could stand above it and look down on it now this is about a hundred neuron by 500 neuron sheet really IP is a lot bigger each vertex on this surface is one neuron and the height in a moment is the firing rate of that neuron this is the display the monkeys looking at I’m gonna light up a target right there in a moment this is a time line this gray bar indicates where we are and this is the value of this target as it appears and dissapears these are all measurements we’ve made so there’s no subtlety here this is just a brute-force demo it’s just a brute-force demonstration of that and you can see a group of neurons over here become active one is active the most this is the guy who likes 10 degree movements the most his neighbors are a little bit active there’s a sort of spreading of activity and the height of this activity has to do with the expected utility or expected subjective value of this 10 degree rightward movement okay that’s true and we we’re pretty certain at this point that it is this is what we’d expect to see if we offer two options of equal value two points of activity on this decisional map each of which have equal Heights straightforward enough I think what happens if we offer a monkey a choice between two options having different values well we expect to see one peak be much taller than the other it’s the relative heights of these Peaks which are carrying information about value and it’s their location which is identifying if you like the choice set elements now how do we choose well here we have a little bit less data we have a lot of beautiful models and lots of data from different labs that seems to confirm the general features of these models it appears that what happens is that every point on these maps is connected with every other point in an inhibitory fashion so that these neurons essentially send out an inhibitory signal across the rest of the map which tries to keep all the other neurons from fire the neurons of this location are doing the same thing as we turn up the global inhibitory interaction across the network the peaks begin in essence to compete let me show you that again OOP no no don’t do that the peaks begin to compete you’ll see here is when we turn up the inhibitory connections this peak effectively suppresses that one

because it’s stronger and the result is a choice this is quite simply the neurobiological instantiation of The Economist’s Arg max operation and I think there’s widespread agreement that this basic mechanism is is in use in the brain in a lot of ways it’s usually called a winner-take-all mekin is although there’s some debate about whether it relies principally on excitatory or inhibitory connections if you’re a neuro geek like me in my neuro life that’s a really important thing but not not for today now let me stress that I’ve told you a story about Li P but we have every reason to believe this is a broadly distributed process for the control of eye movements there are at least three critical areas the lateral intraparietal area the frontal eye fields and the superior colliculus and we have a really strong reason to believe these areas are reciprocally interconnected that means that if I inject a value signal here in area Li P it propagates out to the other areas and if I increase the inhibitory Network strength in any one of these areas if forces convergence to a single movement in all of them you can see that here in this cartoon you’ll see the value signal first entrant at Li P and it propagates through the network to the frontal eye fields in the colliculus and then in this next step there the inhibitory burst which propagates through the network and then passes through a nonlinear threshold until Terr that’s very well understanding the colliculus which ends with the generation of the selected movement okay so the story is actually very simple and it’s the obvious and direct union of the stories I’ve told you so far there is a direct pathway that connects sensory to motor lying along it are representations of the things you’re thinking of getting but the firing rates along those pathways are something like the utilities and the process of choosing is using a local mechanism to identify the highest peak and not select the most desirable of the options it’s actually not that odd an idea and it really mates these two disciplines nicely let me stress that there aren’t a lot of new insights here here other than understanding the mechanism now that’s only going to be half the story and it’s going to be clear why moment this cluster of parietal and frontal areas this is actually for the reach choice system that I’ve been talking about right now these three areas they are critical in representing the values of the your options and choosing the one that has the highest value of course the process of really making a choice also requires that we store the values of everything we ever encountered that we have the ability to learn about new things that we have genetically and environmentally determine preferences that are stable over long periods of time and this calls for a large network of areas to participate in a very very complicated way in the storage learning and representation of value we think of this network as then passing those signals out to the triste network and if s as those signals out were pretty sure at this stage there are about 50 papers on this through this brain area of the medial ventromedial prefrontal cortex it seems to be the final point in the valuation network before passage to the choice network but I want to show you just a tiny bit about this because if I was sitting in the audience and an economist I would be more than a little distraught at this point because these are a lot of claims with very little data to support them other than me pointing in a lot of old papers the critical idea here is that in these valuation circuits there’s a representation of the value of each of the options you might encounter that guides you do this take the greatest activity process in your selection that’s real different from utility utility you can see my little arrows here utility really is produced by choice we derive utility from choices Economist’s subjective values different subjective value produces choice it’s causally responsible for choice in the theory if that’s true I ought to be able to go into the brain find an area that I think represents subjective value like we were just looking at but maybe in the pure valuation areas before choice ever happened and then use that value signal to predict the choices people would later make so that’s an experiment that three of my three members of my lab did I thought leaving is now a professor at Yale and Stephanie Lazaro and Rob Rutledge who were then graduate students and are now postdocs at University College London what they did was a two-step fMRI experiment in humans the first step was

to identify brain areas that ought to encode value and the way they did that was by running this very simple lottery task subject lies in the scanner for about ten minutes this lottery appears it means you have a 50% chance of winning $2 and a 50% chance of losing $2 after a delay we tell you which happened sometimes you win sometimes you lose we simply take the difference of the win versus lose and we ask what areas in the ventromedial prefrontal cortex of you we’re highly active for win versus lose now with that group of voxels the group of brain areas in hand this is what they look like across averaged across a group of people we’re gonna focus for today just on this one this em PFC area we can then go ahead and kind of weird backwards economic utility experiment so what we’re going to do is we’re going to start with a group of 20 real consumer goods and these decisions that are starters are going to be making are for real later on we’re gonna ask them to choose and one of their choices will be realized in the home with this consumer good here you can see we have some movies some books some stationery we slide someone to the scanner they stare straight ahead and a good consumer good appears they look at it for four seconds and it goes away this is a really dumb experiment another consumer good appears this is a poster about this big of this Monet that they can take home they’ve seen all these goods so they know they’re real and then all I’m gonna do is go into each one of the I’m gonna make twelve twenty of each of these measurements I’m going to average them so I’m gonna say what’s the brain activation in this value area for each of these goods so here’s a typical NYU undergraduate for better or worse this is his response to the moleskin notebook in the venture in the medial prefrontal cortex and alas this is his response to the Beethoven CD now what we can do is we can order this is gonna be an ordinal exercise for the economists in the room we can order by neural active I mean neural activation these 20 Goods so here’s this kid here’s the most kind notebook which he which is Mito prefrontal cortex love the Beethoven CD which is medial prefrontal cortex hates the Dali poster which he hates this is all very depressing really ok this is Dodge the movie dodge ball which I actually admit to liking but you know you wouldn’t know on your brain to reveal that ok so we generate this now we take them back into the lab they’re not in the scanner anymore and what we’re going to do next is simply have to make a series of choices which of these two would you rather have and we’ll show them lots of we’ll show them every possible pair twice and ask which would you rather have because this is going to allow us to construct a preference ordering what’s your behavioral preference now the critical idea next is pretty complicated so and I want to do it fairly quickly because I still want to show you one other thing the critical idea here is if two Goods have really different neural activation the highest and the lowest we ought to be able to predict that dead-on if two goods are really close to each other in their neural activations that ought to be pretty hard to predict so what we’ll do is we’ll sort all the neural pairs of neural data into pairs based on what we call their rank distance the easiest ones are the 19 apart and the hardest ones are the two apart and we’ll ask how accurately can we predict at each rank distance across the twenty people in this study this is the glass half empty glass half-full moment here’s 19 apart and you can see we can get about 85 to 87 percent correct here’s the one apart and here you can see one two three we’re basically operating completely a chance now that’s good because the implication of the theory is and these are for the economist let me be clear these are fully if I’m right these are fully cardinal measures of subjective value so that’s great down here I should be operating a chance because the random utility representation is highly overlapping here here it should be what should be a hundred percent at this point now we’ve actually done more complicated analyses of these because we asked people to make each choice twice we could ask whether they were consistent in their choice whether sometimes they preferred the Monet and sometimes the Clayton sometimes the Klimt and sometimes the Monet and vice versa and we asked whether from the variance of the neural signal we could predict this indifference and interestingly we could

not we could only do that if we added to the standard random utility model measurement error term and that measurement error term is sort of an aggregate measure of how badly the scanner samples this signal and now this is the bad news this is really glass less than this is glass 99% empty the signal-to-noise ratio on these experiments is on order thirty to one in the wrong direction so the scanners are terrible they’re terrible that’s not to say your money is badly spent on a scanner but but they’re the lowest signal-to-noise device on earth okay but what am I saying what’s the main point why did I tell you all of this this is the thing I really want to drive home we have this object in the because theories the economic because theories that we call subjective value and of course it has a partner called expected subjective value subjective value is linearly proportional to utility when choosers are consistent in their behavior when utility theory applies and tells us what the underlying representation should look like the theory says subjective value signals should have the same properties and this has now been done about ten times in different labs using different techniques and the answer is always yes the medial prefrontal cortex does clearly have the neural correlate of utility it can predict choice and it and causal if you disrupt it if you increase it if you decrease it if you lesion it you alter choice in fundamental ways predicted by the theory so the critical idea I want to get across is that we can project the economic theory down into the nervous system turning it into a because theory and what we get out is something that makes a lot of sense we have a network of areas involved in valuation I haven’t told you much about this but they include the we think hippocampus for sure they include the amygdala the ventral striatum the dorsal lateral prefrontal cortex the orbital frontal cortex we know something about how those areas go together not a lot they seem to produce a final common utility like representation at the level of the medial prefrontal cortex and I haven’t told you much about this at the level of the ventral striatum and that signal seems to be used by a pariah ttle frontal network which selects from that representation those objects in the current choice set and then forces a winner-take-all Arg max operation in order to extract the most highly valued of the options what don’t you see this is a really hybrid theory it has features of both of its parent theories now at this point it is perfectly reasonable for an economist in the audience to stop me and say okay I’m the first time it’s happened to me I we just got to this stage it was probably about May of making this claim without so many good experiments it was maybe eight years ago I was a Dartmouth it was in one of the many times Michael Gazzaniga was actually a professor at Dartmouth and he introduced me and I gave this talk one of the economists in the audience stood up and said so you’re telling us that we’re right there ok well that’s kind of true he said ok well that’s not news I said well but it’s in the brain he said yeah but we’re right you actually haven’t told me anything useful as an economist like so what okay vile I just don’t get it I knew that and and he was right the story I’ve told you really is a story about uniting two theories but it’s not a story that has particular impact on the way as economists we build models of how people choose now I want to show you that that’s not necessarily true as we come to understand this architecture better we’ve come to understand computational features of it which change the way we think about decision making at the economic level I’m going to just show you one example of that today too quickly but it’s one that I’m particularly proud of I have to say and it’s a study of representation in this area parietal area Li P so what I’m going to do is now stop for a moment and tell you a little bit about representation in the nervous system because we actually Asner as neurobiologists have lots of beautiful theories of representation in the nervous system one of the most important theories of representations we have in the nervous system which was developed well I guess originally probably by heart line in the last century but who we associate nowadays with the computational neurobiologist david hager is something called the normalized representation he had this interesting insight if we’re recording from a single neuron in cortical area of b1 that oh it fires when it sees a vertically oriented bar that is to say the action potential rate is proportional how big a vertically oriented bar is sitting in front of it it’s firing rate is actually higher if that vertical bar is surrounded by horizontal bars then if it’s surrounded by other vertically oriented bars it’s as if the other vertically oriented bars somehow suppress this neurons activity

and the horizontal Redington bar is augmented this is a huge puzzle for a long time he go recognized quite early on that there were two implications of this first just at a completely ad hoc arbitrary way we could model these interactions using something that we nowadays called divisive normalization what heater argued was that the firing rate of the neuron reflected the goodness-of-fit how close the stimulus was to the sensitivity of this neuron but divided by the sum of the goodness of fit of all the adjacent neurons that have the same sensitivity let me say that a slightly different way this neuron it’s activity which is the numerator is being divided or you can almost think of it as being subtracted away the more that nearby guys are seeing the same thing okay now why does that make sense I’m gonna try and do this super fast he and Eris Simoncelli realized that it made sense because adjacent locations in visual space in fact tend to see the same thing that’s kind of obvious if you have two little guys and their look in thing if you know that the first guy sees a vertically oriented bar you can probably guess that the second guys looking at a vertically oriented bar let’s take a more extreme case a ring of six of these detectors with one in the center if you know that all six see vertically oriented bars it’s extremely likely that the central one sees a vertically oriented bar and what he too realized was that that meant that the firing rate of that mil guy was redundant he was wasting spikes to tell you something you already knew in fact what he ought to do is fire fewer spikes because you already know that there’s probably a vertical bar there he just needs to fire enough to tell you that it’s what you expect of course if it’s not what you expect he has to fire a lot of extra spikes to communicate that and what arrow Simoncelli and his graduate student Adalia schwartz proved was that the normative theory of visual encoding which assumes that neurons and action potentials cost something to produce suggests that the most efficient way to encode information about the visual world is to represent not simply what you see but what you see divided by the stuff around it and the division process flex the statistical likelihood that you’re seeing the same thing so you would actually want some Gaussian like fall-off of correlation which would reflect the fall-off and spatial correlation of real-world images that these animals really see in their real lives okay so here’s the critical idea I’m skipping all this stuff the critical idea is then an efficient cortical representation takes this form the firing rate this is just a constant we’re not going to worry about this is something about what that neurons looking at plus a baseline firing rate for the neuron the unique zero for the economists for this neuron plus the sum of all the values of all the other stuff that’s being compared to it I’m going to really flesh this out in a moment plus this normalization constant which actually controls curvature we talked about that a bit okay so translate it into the language of Li P I’ve been telling you that area Li P is a value representation area that it represents objective value that’s true and what I really should think is that the firing rate associated with object one in the choice set is actually how good object one is divided by how good everything else in the choice that is it’s the relative value of the thing you’re looking at compared to the sum of all the things in the current choice sets now we can go into this is a really I can show you that this is a really efficient way to do this one way to say this is it D correlates the choice set that’s kind of interesting but the critical idea here is that as we increase the number of neurons that we use to encode information about the choice set we increase the cost of maintaining that representation here I’ve plotted cost as low here and high here as we increase the number of neurons we increase the accuracy with which we can encode information about the choice set and when we use cortical normalization we can bow this line out as far as we can and that bowing reflects taking advantage of the statistical correlations and the choice sets that you’ve encountered in your life and so

it’s the intersection of this metabolic cost constraint with this optimal encoding constraint which identifies a unique point which is the best encoding point okay I know that was complicated I’m gonna do two things now the next thing I’m gonna do is prove to you that this is actually what goes on in area Li P and then we’ll finish up so to do that Kenway lui in my lab trained a monkey on this simple task he’s staring straight ahead three targets light up one two three all go off except one he looked at that one gets a reward this targets always worth one unit of reward this targets always worth half a unit and this targets always worth two units that allows us to construct seven different conditions for this neuron and this monkey you’re looking only at the unit reward the unit reward divided by the unit reward plus a little reward plus a big reward plus both rewards and then these weird situations where the value of the targets outside the response field are going up but there’s nothing in the responsibility of course these are the ratios of value you extract and this is what you get when you record from single neuron scenario Li pey here’s the firing rate of the neuron when there’s a target value one in the response field then no other target here’s what happens when I turn on a second lower valued target the firing rate goes down a little bit a third lower valued target both at once and here are the same conditions we can see increasing suppression for the no target present case okay so this is by way of saying we now have really overwhelming evidence there are now a few papers on this let’s say that the firing rates and aerial IP have this weird normalization feature which we have a normative theory to predict who cares an economist has to care and here’s what an economist has to care I want to remind all the economists in the room that these are neurons these are random utility objects they have a mean firing rate with a variance the mean and variance are related as the mean firing rate drops the variance drops but not as fast as it should that means that as mean firing rates drop things get more confusable it’s a feature of every neuron that’s ever been studied every cortical neuron that’s ever been studied the second thing on one stress is that there’s an additive noise term for the Economist that’s fixed that all these neurons have okay keep those in mind now let’s imagine there’s a riff on Sheena Iyengar is famous too much choice experiment in the economics literature the business literature and here when I’m representing are three jellies that a consumer might face these are distributions of the internal utilities or firing rates let’s make em firing rates that we would observe on different trials this is the most likely value these are extreme high and extreme low values from different encounters with this jelly I present these three jellies and I asked my subject which do you prefer now in a normalized system we get this problem as we begin to add other jellies or swivel around the value of the lower valued jellies they crush down these upper two or distributions and crush them into each other so the result is that mucking with these undesirable jellies in the choice set ought to produce more and more stochastic choice in our choosers some have never been observed I’m really smoking here I know I apologize yeah I’m gonna even skip this blah blah blah come on come on come on okay so here’s the experiment I’m just gonna show you the monkey one cuz I’m so out of time here here’s the monkeys job find the good juice find the good target and get the juice target ones always going to be worth point one five six milliliters of berry berry juice target twos gonna range sometimes it’ll be 0.13 you should pick this one sometimes it’ll be point one eight two you should pick this one sometimes it’ll be the same they should produce a loads of choice choice function I’m gonna do it under two conditions Kenway is going to do it under two conditions when there’s a very very low value distractor and a medium low value distractor the monkeys will never pick these distractors their crummy options and here’s what was fine so this is when the distractor is very very low valued

you can see when they’re the same rewards b1 and b2 the monkey doesn’t care fifty-fifty when it’s point one eight two versus 0.15 the monkey picks point one eight to about 90% of the time he can find the right reward 90% of the time now what we’re gonna do is we’re gonna take the low valued distractor we’re gonna increase its value slightly not much not enough to ever have it chosen it may not be immediately obvious how huge and effective is this monkey has gone from being able to find the high valued target 90% of the time to being able to find it only 60% of the time it turns out this is robust we can produce this in several monkeys and we’ve now gotten in humans as well making consumer choices over candies so here’s my answer for an economist what are you what are you guys gonna do for us in the 1950s herb Simon argued that we aren’t as economist to take into account the costs of critical computes of cognitive computations if people satisfies but this is kind of hand waving at the time because we didn’t write out cost functions we didn’t say what it cost to do a computation we did just say gosh it would be neat if you could incorporate that but now nurse science has got to a place we can actually often write out normative theories of costs that means that we can ask not what behavior did the person make that was efficient with regard to the choice set but what behavior did the person make with regard to the choice set and the costs of the computation necessary to achieve efficient normalization efficient transitivity thinking of it this way these monkeys are violating rational choice they’re giving up juice and the argument is they’re giving up juice but not more than it would cost them neurobiologically to recover that juice there at the equilibrium point between the costs and the benefits of these two now this is a very traditional economic idea but it’s an idea we can really do now you know 50 years ago this was just a dream when Simon said it but this is the kind of thing that I think serious neuro economics make possible and it’s this Union whether we’re talking about in domains like learning theory where there been huge advances or whether we’re talking about domains like this choice theory that it really menaced see the mating of these two disciplines and the importing of constraints back and forth across the disciplinary boundaries to achieve goals that really central I think to both disciplines I think that’s what we really want to look forward to as these disciplines mature thank you so we still have time for a couple of questions so if anybody has a question please raise your hand and I will come over to here so I particularly interested in some of the statements you made early on about the I think your work was an inevitability of this kind of reduction and one person you didn’t mention is Kahneman so let’s compare two kinds of reduction here so the at the neurological level that you’ve been talking about and and I think it’s fair to characterize Cattleman’s work is more at the cognitive level I’m it’s more psychology and to play devil’s advocate and to prompt you I would argue that Kahneman work has had significant effects for the way in which economists go about their business okay so first I mean that was that’s a great question gets at agreements and disagreements right so for those of you who aren’t don’t know much about decision theory Danny Kahneman won the Nobel Prize in 2002 for work that he’s a psychologist it’s work he did with his colleague Amos Tversky in Israel and Stanford and they really wrote the modern model for behavioral economics they wrote the most predictive model for sort of general human behavior in decision-making it’s called prospect theory it’s a modification of standard expected utility theory that allows for things like subjective representation of probabilities and it’s been extremely impactful their paper and econometrics in 1979 even though this psychologist is the most cited paper in economics that’s not a methods paper period now when this sort of started 15 years ago Danny was a huge advocate of neuro economics because Danny’s view was that your economics would do exactly what we imagine it would do it would have this reductive synthesis and the reductive synthesis would align traditional economics which

he viewed as mostly busted with their theories including a distinction he later became very interested in what’s called the – system theory that system one system two or Fast and Slow he has a really fun new book about this theory and that there’d be a neurobiological instantiation of this as well and so he really worked to promote it he wrote the afterword for the first textbook which I added in their economics okay so over now read the next 10 years the neuro biologists have been drifting away from this to SystemVue the view of the neural biologists is that that’s not going to be reductively synthetic between economics and neuroscience and that there is not strong neurobiological evidence for two systems and we need extremely clear hair because Danny and I get into fights about this all the time when I show you that map of all those brain areas there are a lot of systems there so if Danny says I think there are eight systems I say no there probably 50 that contribute to the construction of value but that’s not really what he means he means that there are two completely independent decision-making modules in the human brain they have independent access to the motor control circuitry one of them is a high speed system one of them is a low speed system one involves cortex one involves something that’s not the cortex and I have to say that I just don’t think that’s right I mean I went in thinking that was right but the neurobiology doesn’t support that now that’s led me to go back to the psychology and ask how convinced I am that the psychology really supports fast and slow as opposed to gradient now if I really push Danny on this you know you the place where you can push me you can say well stuff that slow becomes fast with repetition he says yeah that’s true but that’s because the slow system passes it to the fast system well how does it do that I’m a psychologist okay that’s a fair answer I know I mean I’m asking mechanistic algorithm the question and he’s like I’m not an Arab I just don’t ask me a question like that so there’s no doubt about it that there’s a touch of common ground here this notion that there are multiple inputs danny’s notion that there are to use in your head that make decisions separately that they compete in a sort of aggressive game theoretic sense to control your behavior I just don’t think the neurobiology is there now the place this war was really fought was between my lab and Sun Cohen’s lab John is at Princeton and a colleague of of Danny’s over intertemporal choice discounting they published a very famous paper in which they argued that there were two systems involved one that liked immediate rewards and one that liked that was patient and rational and we published at exactly the same time well we published a couple years later a study that said the exact opposite and you know I I’ll leave it to you to go read those papers and see which one you believe I obviously believe mine I think that for the most part the view in the community right now is of some skepticism towards the two systems view and I mean I say that with all respect to Danny he’s been right more times than I’ve been right he’s even right more times than I’ve been so so I I’m very cautious here but I just don’t see it in the neurobiology and we have time for one more question hello like to know how does this apply to what makes a great poker player oh that’s too hard I could answer that with regard to blackjack see the nice thing about blackjack is is for an economist it’s not a game it’s an optimization problem right pokers a game because you’ve another player and you have to worry about his beliefs let me answer this way about eight nine years ago oh the nerd economic community got real interested in this distinction between blackjack and poker in blackjack the dealer is not your enemy right everybody who some of you may not have actually spent as many hours as I playing these games in blackjack they just deal cards and you have to decide whether you know hit or stay and you’re trying to get to 21 and knowing the size of the deck and the number of cards in the current distribution blahblahblah it’s just a math problem and there’s an optimal strategy and you’ll lose about 0.5 percent of the time poker there’s a guy sitting across from you and he’s screwing with you he’s trying to make you believe that he has

some set of beliefs he doesn’t have so that you’ll act on those false beliefs for an economist this is the difference between a standard micro economic optimization problem and a game in the sense that von Neumann Morgenstern and later John Nash won the Nobel Prize this is the guy from A Beautiful Mind meant the word game and so a lot of us got worried that games were different that they actually might be played by different systems and that the story I told you doesn’t apply to playing poker because it it really is a system about micro economic optimization not about gameplay now a bunch of labs did experiments to show that that wasn’t true but the one so I’ll tell you about one we did this was a postdoctoral fellow of mine at the time by the name of Michael Dorris Michael’s a professor well he’s a professor right now in Canada at Queen’s University but he’s going to the Chinese Academy of Sciences to be the first Anglo professor there in Shanghai it’s pretty neat and Michael in any case trained monkeys to play this game called work or shirk you can just think of it as like rock-paper-scissors and so the monkeys would play this game of work or shirk now what Nash proved and what he won the Nobel Prize for was showing this weird thing whenever you’re at what’s called a mixed strategy equilibrium that is sometimes I work sometimes I shirk sometimes I play rock star as I played papers sometimes I play scissors you do that because the subjective values or the utilities expected utilities of all those options are exactly equal if they are not exactly equal you would never mix you would pick the better one and that’s true even under circumstances when the game structure requires that you play rock 80% of the time and paper 20% so it’s not 50/50 anything that’s mixing always requires that the underlying expected values expected utilities be exactly equal even though the behaviors are not and so we thought oh well we’ll go into area I he will record from these neurons and see whether they encode the behavior or whether they encoded the expected utilities in which case they would nothing we could do that changed the behavior but left the equilibrium point alone would change the frame rates and they follow the Nash serum absolutely precisely they look exactly like the nash theorem predicted they should during gameplay now when we went at the data on a trial by trial basis in really careful detail what we noticed was that the firing rate although they were rock solid on average actually fluctuated a little bit and these fluctuations actually accounted for slight deviations of the monkey from – from 5050 it was as if the monkey was throwing dice but that the physical throwing of the dice were these fluctuating spike rates once he was parked at the Nash equilibrium so this if anything strengthened our conviction that the architecture I’ve just told you about is the same one for playing poker and that makes a good sense from an economists point of view that’s a natural for a neurobiologist who read Descartes that’s troubling because the car told us that really complicated behaviors like poker ought to be a different thing there are divine souls and I think the evidence is well I’m not going to rule on whether there are divine souls but they’re the same thing that does the Microsoft there are micro problems I’ve been showing you here today so I think that the story really holds there and I would lean you towards a bunch of these beautiful game theory papers but my lab and Dale Lee’s lab are the two big labs that have done it Glenn and I would like to thank you all for coming thank you