The following content is provided under a Creative Commons license Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu PROFESSOR: OK, good afternoon So in the last couple of weeks, we’ve gone through some of the major human capacities that you have and I have and most people have most of them, right? Which are the ability to see the world around us, to attend to what’s important, to learn lessons from life, to remember things, why we sometimes forget things And today, we’re going to talk about another huge dimension of human experience, of the human mind, of the human brain, language The ability to talk to one another I can talk, you can listen, and vice versa To read, to communicate, in the ways that are so remarkable Our life is flooded with language, right? That we hear, that we read, and so on So when psychologists try to divide up language and the study of both components, they talk about comprehension, understanding what somebody says to you when you’re an infant, or now, as an adult, or reading text, or producing speech, which you can do with your mouth by speaking, or you learn to write So you can both comprehend and produce language, both visually and orally And the thing that has greatly enamored linguists to figure out is how language that we speak is endlessly generative I’m not sure you can actually calculate out how many sentences we could possibly produce, but just for fun, people have done stuff And they said it would take 100 trillion years to memorize all the sentences we can produce I’m not sure that’s exactly the right number, but it’s a lot Because we can say a lot of different sentences and understand what a person is saying It’s an incredibly generative system for communication, one person to another So when people in psychology think about different aspects of language, levels of language, they often divide it into something like this Phonology is the sounds of language that we hear, that we acquire as infants, that we use to speak to one another Syntax, grammar And by this, this isn’t school grammar, not get the sentence right It’s just the understanding of the organization of words, and how that changes the meaning of the sentence Syntax Semantics, what do the words mean? What do the groups of words mean? Pragmatics And we’ll talk about each of these a little bit Sometimes a person is sarcastic So they go, that was an awesome lecture yesterday in psychology Somebody could say that, in a hundred trillion years And you go, yeah, yeah Right? So that’s the pragmatics versus semantics Then, of course, it’s not just sentences We have whole discussions And another aspect of language we’ll just touch on a little bit today is emotional comprehension and production The tone somebody speaks in, the tone you produce, are also informative So there’s many aspects of language We’ll start with the most basic, or phonology So phonemes are the words that people use to describe the building blocks of speech sounds So boy versus toy are different by one phoneme Phonemes are the building blocks of the sounds of language Humans use about 100 phonemes across languages, but any given language only uses a subset And this has really interesting consequences Of course, a language could only exist in the world if you could understand those phonemes Phonemes that are ununderstandable aren’t going to be a part of anybody’s language But different languages, for historical reasons, have ended up using different subsets of phonemes And we’ll come back to that When we’re babies, for about the first six months of our lives, we’re universal learners We know all the phonemes and respond to all of them As we become native language speakers and listeners, we become only good at the ones that are used in our language, and our ability to even recognize the ones that are not in our language disappears You can get it back some, but the older that you are, you never get it back fully So we’re universal learners at birth, but we become enslaved to the language that we are native speakers of and listen to within months of birth English has about 45 So it’s more than the number of letters we have Some people get confused a little bit that letters are phonemes They’re not So hot and cold, the oh sound in hot and the oh sound in cold are different sounds, same letters So there’s many more phonemes Even in English, when they say there’s about 45, depending on dialects, you could have a couple more, a couple less So again, we’re born to use all of them Then we use them or lose them It’s something, we don’t entirely lose it, but we’re never as good if we try to learn a distinction once we become older than that Humans understand speech at an incredible rate

The typical fast talking is about 180 words a minute, 14 phonemes a second in a continuous speech stream But people can go up to 250 words We’re incredibly fast language processors, incredibly fast language processors So how do we roughly conceptualize, in the most simple way, what we have to do to get to a word that we hear, to get to its meaning Well, acoustic information comes through the ear Those are wave forms, physical things, arriving at our ears We in some way figure out how those correspond to phonemes, and then map that in some way to words that have meanings in the world And the reason why machines understanding what you say is shockingly hard to do a good job on is for the following reason There’s a huge difference between a word by the time your mind understands what it is and the nature of the signal that arrived at your year So we talked a lot about that in terms of the retina in vision Right? You have this very impoverished stuff that arrives in your retina, and then the whole rest of your brain, millions and millions of neurons and synapses, reconstruct what’s out there from a very impoverished retinal input And the same thing happens in audition What gets to your ear is incredibly impoverished, and the rest of your brain has to brilliantly reconstruct what it heard and assign its meaning And so part of the challenge of your mind and brain to do that is this This is a representation of something like the physical signal that arrives at your ear And you could say, well, here’s one word Here’s another word What’s the problem? But this is all the signal that arrives at your ear for the word captain So here’s the C, all this stuff for the A, a period of long silence that sets up the P. Turns out– I’ll convince you of this in a moment– that our speech is full within words of silent breaks that are incredibly informative and alter our perception of what’s physically going on Then the little T. This long stuff for the AI This long stuff for the N So the physical signal to your ear barely resembles what seems like a single unit, captain And where’s that pause? A person doesn’t say, “cap,” or doesn’t say “ca,” one, two, three, “ptain.” Right? I mean, but the pause is happening, and it’s incredibly informative, as I’ll show you So that’s the incredible thing Your auditory system, your language system, has to reconstruct sound into phonemes, into words It’s not obvious in the physical signal of the ear It’s not even that the physical signal at the ear gives you much information at the outset about whether you’re hearing one word or two words or three words So here’s “what,” and “do you mean” as one big blend Where is “do”? Where is “you”? Where is “you mean”? And why does “what” look so separate, and these three words, “what do you mean,” get all garbled together And you have to unpack them by your brain and mind Here’s another example John said that the dog snapped at them The dog snapped Here’s what’s coming, roughly, to your ear, taken from spoken speech And here, again, notice that the boundaries that you can see here, the gaps of silence, occur not very related to the boundaries between the words that your mind ultimately understands So here’s the word “the,” “dog” Here’s S for “snapped” Here’s a good pause But that’s a pause between the P sound and the T sound That looks like a good word boundary To a machine, that’s a very good word boundary It’s not a word boundary for you You know snapped is one word And here comes “at.” “At” is kind of blended into the T. So there’s this physical signal doesn’t even give you breaks where words separate from one another It’s kind of a continuous mix of breaks and silences So here’s to convince you that a silence carries incredible information in it Let’s see, make sure I have the volume OK, so I’m going to play you this This is a physical signal to your ear of the word “say.” RECORDING: Say PROFESSOR: OK Yeah? I always worry because Tyler’s an expert in this, and I’m always like, please, Tyler, don’t embarrass me when I’m wrong You don’t have this pressure out there, but the teaching assistants are all experts in something or another And then they know more often than I do on that topic, and we hit their topic in the course I’m looking over there nervously So here’s “say.” Now, they’re going to take this thing and just separate this, just in the way you do a physical signal on a computer, just separate in time They’re just going to add a little bit of time And what do you hear? RECORDING: Stay PROFESSOR: The T sound that makes is simply adding silence

here to produce that Now, that’s what you hear all the time It’s not a random choice But isn’t that amazing? That silence is as ultimately informative for your mind as actual language sounds themselves All this is happening on the order of thousandths of a second, in bits of information That’s bad enough Here’s another amazingly tough thing So people like to compare consonants that have a lot of similarities with some interesting differences Ba and da This is really comparing B versus D. It’s throwing on the A just to have a vowel sound So here, again, is a representation of what’s coming to your ear Notice that all the difference between the B and the D, these two consonants, all the physical differences occurring in 40,000ths of a second 40,000ths of a second is all the physical difference that you get as you hear these two things RECORDING: Ba Da PROFESSOR: 40,000ths of a second, and yet you could hear the word bad, and you’re not going bad, dab Come on, that was thousandths of a second, right? And not only that, there’d be another word and another word in a sentence That’s why people use the word speech stream, and we like to think of it as a speech torrent Phonemes, phonemes, phonemes, breaks, all garbled together in your mind as you back and constantly, instantly, and brilliantly figure out what you’ve heard So let me give you another example because part of the challenge of this are phonemes occur in the context of words and sentences So make sure I have good volume on this OK, so let’s do this first one RECORDING: [INAUDIBLE] PROFESSOR: What was that? AUDIENCE: [INAUDIBLE] PROFESSOR: What? AUDIENCE: [INAUDIBLE] PROFESSOR: OK Here’s the same thing, and a person is going to say this now all by itself RECORDING: Little PROFESSOR: You got that, right? OK RECORDING: Mary saw the three little pigs in the corridor PROFESSOR: OK, but here’s the amazing thing The first sound you heard, and I’ll replay it, they took the word little, and they cut it out from the last sentence you heard When little is said all by itself, what comes to your ear is literally the first thing you heard Not this, because when you say a word in isolation, little, people say it differently But we almost never talk like that We almost always have a sentence And it turns out the way we say things, and therefore, the way you hear things, is hugely influenced by what comes before it and what comes after it Never mind things like the speed at which different people talk, the accent they talk, or the loudness of the background conversation But just simply the fact that you’re going to say little by itself How does that work? It’s amazing, right? Listen again If you heard little all by itself, and it’s exactly the same physical signal Maybe it’s not so– RECORDING: [INAUDIBLE] PROFESSOR: OK? Let me try that again RECORDING: [INAUDIBLE] PROFESSOR: That’s exactly the same physical signal, exactly the same physical signal, as– RECORDING: Mary saw the three little pigs in the corridor PROFESSOR: What you hear easily as little is exactly that Words in isolation– and there’s a great demonstration If you hear a word cut out, the reason why it sounds like nothing is your mind is already preparing how to interpret that, given what you heard just before it, and also a little bit what you hear just after it Your mind is constantly fixing up things because it knows what’s going on, what’s going into it, what’s coming out It’s not because it just simply hears the word itself If we lived in a world where people just say one word every half an hour, we could do this In a world where we’re talking to each other in sentences all the time, we have to do this Here’s another example of our brilliant auditory system for hearing these kinds of things So again, these are representations of how this physical signal arrives at your ear Here’s the word bet, bee, boat, and bird They all start with the same B sound That’s easy, right? But look what came to your ear Look at this signal, this signal, this signal Look at this one How different does this look? It’s different because what follows bird influences how this is said, but you can see that you can translate both of these into the B sound pretty easily Now, look at these words These all start with D. Look at how bet and debt look almost identical We said it’s only 40 milliseconds difference, 40,000ths of a second difference that physically moved through your ear But you can tell the word bet versus debt Look at the word D, how different it looks from word to word So your mind is constantly figuring out what you’re hearing The problem of invariance, and it’s constantly coming to you in incredibly variable ways So one of the tools we have, and an amazing human capacity, is what’s called categorical perception So we have all this huge range of physical sounds that can come to our ear

And kind of brilliantly, and you don’t even think about it– I mean, none of you go around bragging about your categorical perception abilities, right? Like when you go to spring break, right? And maybe some of you will go home, or you’ll see friends, and you won’t go, I have categorical perception I try to be modest about it But it’s an amazing human talent, and essential because what it does is this We hear this huge range of sounds, depending on accents, what comes before, what comes after And we have to decide that’s a B. And what you’re going to do is take this huge range of things, and just go, boom It’s a B or a D, and that’s all there is to it So here’s the way they do it So categorical perception Sounds can vary continuously We showed you some of this continuity In this experiment, on purpose, they’re going to vary sounds in a very continuous, physical way But in our heads, for ba and da here, in our heads, all the stuff that’s over here, even though they’re very different one from the other at a physical, specific level, we’ll say, ba I’m done with it We’re going to put it in one category for interpretation And you’re going to have a pretty sharp boundary here And then everything over here, all these physical signals, you’re going to call pa And so you’re not going to spend a lot of time going, that was a one third ba, 2/3 pa, probability You couldn’t do that, right? Because you couldn’t get through a word, nevermind a sentence You’re going to instantly group huge numbers of things that are different from one another into two categories that need to be separated So here’s a kind of experiment they do I’ll just give you the feeling of it Here’s what they’re varying, constant units Constant units, every stimulus here is different But every time you hear ba, and they’re constantly going to vary something by its physical property, you always call it a B. And then something happens, in about 20 milliseconds of information, and boom, you always call it a P. So change, change, change, change, change, in the physical signal, you call it one category A little bit more change, boom It’s the other category And you keep changing, but for you, it’s always a perfectly good P, even though every one of those are different So it’s an amazing fundamental capacity to take a huge, and, for all practical purposes, an infinite variety of inputs and say it’s a B or a P if that’s the world I’m working in Syntax That’s another way in which we understand things, the way that words are organized and the order So we could say, “The model embraced the designer and the photographer.” Imagine somebody saying to this But then they complete the sentence, “The model embraced the designer and the photographer laughed.” So people who study syntax really are interested in what happens in your mind as you hear, “The model embraced the designer and the photographer,” and some big group hug is going on, right? And then all of a sudden, you get this word laughed, and you have to rearrange the elements in your mind That’s what syntax is letting you do, to say, oh, the meaning is really quite different There’s only one hug going on, and one person laughing at them, right? But the meaning over time as somebody talks to you set up, you’re thinking this, boom Because you understand syntax, you understand what happens Semantics are kind of more straightforward than how we usually think about them They’re the meaning of words or sentences, morphemes And we can understand semantics even when the sentence is nonsense. “Colorless green ideas sleep furiously” is a famous example We can understand semantics even in bad grammatical sentences, right? “Fastly eat dinner, ballgame start soon.” Your fifth grade teacher would be up in arms to hear that, but you understand Eat now Go see the ball game So we get through a lot of stuff to get semantics And we can see interactions I stole this example from Ted Gibson because it took me 20 minutes to figure out what the sentence means You’re probably faster than I am Are you OK? I’m like the old man Where’s the missing words? The old man the boats So this is how you have to parse up the sentence It’s not always obvious People who study language love these things, where people are all confused Most of our lives, we don’t go through that much confusion But there’s some fun ones, sort of like the visual illusions, to point out how meaning and syntax can interact as you figure out things Here’s another example Jay Leno talked about sex with Lindsey Lohan OK, now wait a minute Did I miss that excellent episode of Jay Leno, or was she on the show, and talking about– she was saying I’d rather not talk about my recent conviction I’d rather talk about something less controversial So semantic All these things are constantly interacting in your mind It’s kind of amazing when you think about that, right? We said phonology’s amazing Semantics, syntax, boom They’re all happening in you And none of you are sitting there, going, just do phonology Please don’t ask me to do semantics We just had midterms, right? Brilliant So one of the ways that people try to convince themselves that these things are separable and gives a little bit more insight is Evoked Response Potentials Those are these things that you can measure on infants or adults that measure electrical activity in milliseconds and

the speed of language processing, surface electrodes that give you an index of a few hundred thousand neurons And this is an example you saw before, actually, earlier in the course, the so-called N400 So N means whether the wave is going up or down, negative or positive 400 means that it occurs about 400 milliseconds, about half a second, after the relevant event So here is this, let’s see, it was his first day at work That’s the baseline He spread the warm bread with socks Oh my gosh, socks Semantically bad And here comes the N400 And the control condition is you throw up this unexpected word, shoes, that’s weird, but not semantically, and you get quite a different response So here’s an N400 that seems to simply be about semantic expectation or congruence How about if you mess up syntax? The spoiled child throw the toys on the box Then you get a response at about 600 milliseconds to syntactic incongruity So people, this is just more evidence that in the brain, meaning or semantics and syntax or something about word order are processed somewhat differently in time and place Pragmatics is kind of fun It’s practical understanding The old joke, do you know what time it is? And the person says, yes They’re messing with your sense of pragmatics Two negatives make a positive Saying, yeah, yeah I’ve heard that one before, earlier in the lecture Now, here’s something kind of interesting, again, which is remarkable And if you didn’t do the experiment, you couldn’t know that our mind is like this Because you wouldn’t do things like this because it sounds too ridiculous, but your mind does it all the time So we’re going to think about the word “bugs.” And I have to set you up a little bit with paradigms you’ve heard before, but just remind you of these things So one kind of task that psychologists like to use to think about how people process words is a lexical decision task All you’re deciding is whether a word is a real word or a nonsense word So doctor is real Poctor is not Spy is a real word This spelling is not a real word So real word, not a real word We also talked about one other idea early in the course, that if, prior to processing a word like doctor, you get an unrelated word or a related word, you process doctor faster if you had a related word before it Semantic priming The ideas about nurse overlap a lot with the ideas about doctor And seeing this word before makes you process this word faster So you need those two things in mind What was the person doing? Is it a real word or not? And you’re going to mess with what’s in their mind before that, related or unrelated And here’s the experiment So you would hear, over headphones, something like this, as an example “Rumor has it that for years the government building had been plagued with problems.” Now, here’s a critical sentence “The man was not surprised when he found several spiders, roaches, and other bugs in the corner of his room.” So there’s two senses of the word, at least two meanings of the word bug, right? The insect you don’t want in your soup, or the microphone that somebody slipped into your room to listen to what’s going on Two different meanings for the word bug This is obviously about the insect So the person is hearing this over headphones, and on a computer monitor in front of them, after they hear the word bugs here, they see, for a lexical decision, the word “ant,” which is related to this meaning, or “spy,” which is related to the other meaning that’s not relevant for here, or “sew,” which is the control condition We’ll ignore that because it’s just a control condition I’ll ignore that for now So you’re making a lexical decision now on the related meaning or the unrelated meaning And here’s what they find If it’s the related meaning, and you only have about a half second from when that came on to when you had to do this task, so you saw the word bug and here comes this, you’re faster But if you get the unrelated meaning within a half a second of processing the word, you’re also faster What this means is, your mind has done exactly what you would think it wouldn’t do Your mind has looked up in your brain, and in your knowledge of vocabulary, every meaning of the word Even though the sentence is clearly setting you up for the word, the insect meaning, every meaning of your word is turned on and available to you Your mind does the opposite of what you might think You might think, well, I know– let’s look at the sentence I know, once I get spiders, roaches, and bugs By here, you know you’re in the insect world, not in the world of spies, right? But your mind, even then, looks up every possible word Because it’s cheaper for the mind to look up every word than to start to decide what the appropriate word is And then, if you wait another second and a half, you’re only

fast for the selected meaning And now you’re slowed down for the one that’s not context relevant So your mind looks up everything all the time Every meaning of every word you know Every time that word pops up, it seems like your mind looks up every meaning, and then the rest of your mind selects out the one that’s relevant and shuts down the ones that are not But you wouldn’t think that You would think, maybe I’ll just get the one I need I’m not going to get everything It’s like, I’m not going to put all my clothes on, walk outside, and take off everything that’s not relevant for the weather, right? It seems like a really inefficient way to get dressed, right? But that’s not the way your mind is Your mind is cheaper and more efficient to look up every meaning every time, and then pick the right one It’s more efficient, and that’s how people do it So people use this word exhaustive lexical access I look up every meaning of a word, and then I’ll pick out the correct one Another interesting aspect that could be brought into pragmatics, and we’ll talk a little bit more about this later in the course, but emotional intonation And it turns out where it’s almost all language processes are left dominance as we talked about many times in the course One stream of processing language, both visual and auditory, is right hemisphere dependent And that’s emotional intonation And that brings us to our reading for today, from Oliver Sacks And I have to say, I think Oliver Sacks is a phenomenal author, but there’s always been this sense of some of these stories are just a bit too true to be– just a bit cute, or whatever it is And so for years, I taught from this story, and I believed it’s likely true But I thought, well, it’s just a little cute And then came an experiment that showed it’s not only cute, but it’s exactly correct So for this, you appreciate this a bit more if you know a couple things about Ronald Reagan, who was president in the 1980s So what do you guys know about Ronald Reagan? What party? AUDIENCE: Republican PROFESSOR: There you go Was he considered a good communicator? AUDIENCE: Yes PROFESSOR: Yes He was considered– you know, you can have all kinds of views on Ronald Reagan’s things The public said, he’s an awesome communicator In fact, Barack Obama said he would like to communicate like Ronald Reagan Whether that’s to get Republican votes or not, that’s your judgment But he was considered an excellent communicator Was he considered, by and large, to be an extremely analytic thinker? He might have been I don’t know him But not so much Now, part of this is the prejudice or not, but he was from California, and he was an actor before he became president, or the governor president He also had some degree of– well, he ultimately got Alzheimer’s disease It’s a question about later in his administration, whether he was already working his way towards that So anyway, complicated things So he was a master communicator, but not necessarily the Barack Obama, Bill Clinton, Jimmy Carter version of an analytic power, is the general perception You never know whether these are right or wrong for the person But in the story, listening to the president’s speech, are patients with aphasia that Oliver Sacks described, and he’s getting laughter as they’re watching his speech And what happens is, some of the people with a left hemisphere lesion– so you have a language problem We’ll see a video of that pretty shortly They’re laughing because they think that the way he’s talking and his facial expressions, they find hilarious Now, why do they find it hilarious? They have damage to the parts of the brain that help them understand the content of his message on the left hemisphere And here’s what Sacks thinks is going on “The feelings I have, which all of us who work closely with aphasics have, is that you cannot lie to an aphasic He cannot grasp your words, and so be deceived by them But what he grasps, he grasps with infallible precision, namely, the expression that goes with the words, that total spontaneous, involuntary expressiveness which can never be simulated or faked, or words alone can hide all sometimes too easily.” So when you give a very public speech, especially under tough circumstances, that presidents sometimes have to do, we sometimes think there’s a tension between exactly what they’re saying and exactly what they’re thinking And what they feel they need to say, and exactly what they know is behind the scenes And so the idea is that these aphasics, they were saying this guy obviously doesn’t mean what he’s saying Look at his face Look at his tone I don’t know what he’s saying, but it’s hilarious, how he’s masking his face and speaking in a funny tone to tell something he truly doesn’t believe That’s their interpretation And yet, he has an opposite patient, a patient with a right hemisphere lesion We said that’s the side of the hemisphere that seems important for facial expressions and emotional intonation And she complains about the logic of the case he’s making, as if having– normally, when we talk to somebody, it’s both

Left hemisphere’s getting the content, analyzing the content Right hemisphere, something about the intonation True or not true? Are their eyes like this, and they can’t even face us as they tell us something they know is false But these patients, maybe because they have one channel of information knocked out because of injury, they become hypersensitive to the other one So I thought, well, that sounds fantastically interesting, but is that really true? So here in Boston, they did a study with a large number of patients that says they had superior identification of people who are lying to you– they had made video clips and audio clips of people who are lying to you– if they had a left hemisphere lesion So this is a paper So here’s the two examples So they have an example Let’s pick this one Where both, you see just a facial expression of a person talking The voice is turned down Your job is to decide whether a person is lying or not And some videos they’re lying Some videos they’re not So how would you do? I don’t know how you would do, but I can tell you how a group of Harvard undergraduates did 50% is chance, and here’s how they did That was one of their control groups And, in fact, for studies like this, they brought in many, many kinds of people to ask whether anybody is better Everybody’s pretty much terrible at directly telling whether a lie is going on Now, these are video clips of people you don’t know in circumstances you don’t know That could matter But everybody’s terrible at that Policemen are terrible FBI agents are terrible The only group that seems to be above chance, and not much above it, are Secret Service agents So if you’re friend with a Secret Service agent, and you want to deceive them, keep in mind that they seem to be above chance They’re not awesome, but they’re above chance But look who’s doing better than everybody, patients with the left hemisphere lesions And the interpretation is, all the content of the lie is gone All they see is your expression, or your expression and intonation And because that’s all they have to go on, they can tell more when you have the discomfort of lying than a person who’s partly persuaded by hearing the message you’re sending as you talk So I thought this was an overly cute example, but it turns out to be exactly replicable in a full study Any questions about that? OK So let’s talk a little bit about the brain basis of language We’ve talked a little bit about– we know that if you have damage here, you have Broca’s aphasia on the left hemisphere, trouble producing language Damage here, Wernicke’s aphasia, trouble comprehending language To a first approximation, and there’s a lot of research that has made this much more complicated, but a first approximation, the lesions are in frontal cortex or temporal cortex The speech in these patients are what’s called non-fluent or telegraphic They get single words out but have a hard time with anything like a sentence It can be very frustrating for these patients Wernicke’s patients seem much happier You’ll see a film in a moment Their speech is fluent, but it’s amazingly empty of content These patients have trouble producing speech, but their comprehension is decent These patients produce speech pretty well, but their comprehension So they’re mirror opposites And let’s see, what do I want? Yes So in this field, a frequently, incredibly frequently used picture to study aphasia is this specific picture It’s called the cookie theft picture Why it’s become widely used, I’m not exactly sure But once it’s widely used, then often everybody uses it So they’ll put a picture like this in front of an individual, and they’ll ask them to describe what’s going on It’s to elicit production And you would say something like, there’s two kids The kid’s on a stool Uh oh, is he going to fall over? Because they’re falling off the cookie jar, and the mother’s sort of not paying attention Her mind must be somewhere else because she’s letting the water splish splash here, right? You would say something like that So let’s see what patients with aphasia say, and other examples of their difficulty with language Yeah? AUDIENCE: [INAUDIBLE] PROFESSOR: Their writing tends to go pretty much with their spoken language That’s a really good question about– so the question is, what would their writing be like? It very much parallels their spoken language Yeah? AUDIENCE: [INAUDIBLE] PROFESSOR: Sorry? AUDIENCE: [INAUDIBLE] sign language? PROFESSOR: OK, so very good question Could not fluent aphasics learn sign language to bypass, in a sense, their problem, right? So I’m about to answer that in a moment, right on the next slide Not directly that, but the implication of that Because the important thing is– so let me show you this thing, and let me come back The question is, could you train an aphasic to use sign language to get around his or her language problem, if it’s

a speech problem? So people have studied sign language as a language for many reasons, but a huge one is this Because I’m going to interpret your question the way I need to, and then you can tell me if I did it right from your perspective Which is, in what sense does language equal speech? Could you communicate linguistically like, but just switch to your hands, if for some reason– and the answer is, that at the level of the problem of these patients, they could not do it Because these kinds of patients, the problem isn’t really speech There are people who have problems with the motor apparatus to move their mouth That does exist Their problem is in language processes that produce your speech intent, or language processes that extract meaning from sounds And those language processes that produce what you want to say, that comprehend what you hear or read, those are the same ones that are utilized, as far as we understand, whether you’re producing language in the typical, vocal way, or in the less typical, but well-studied, way of sign language Does that make sense? OK So you can’t get around it because it’s a language problem So both by brain imaging evidence, for example, here are patients who use sign language turning on Broca’s area and Wernicke’s area on tasks that are about comprehension and production It’s the same areas, whether it’s using your hands for sign language or using your mouth to talk or your ear to hear Further, there have been really fun studies where they took deaf infants and have shown– and it’s not easy to show, but I think most people are convinced that they babble with their hands The same way babies make little sounds to get themselves going We’ll talk more about infants in a moment Infants who grow up in an environment where they only learn sign language babble with their hands because speech, language is not about your mouth talking It’s about ideas that you can communicate by your production or comprehension, by your hand or by your mouth So language is separable from speech, because even without speech, you communicate in very much the same way, in terms of sign language, by many studies Yeah? AUDIENCE: So does that mean that people that [INAUDIBLE] imagined words or language? PROFESSOR: Say that again, sorry? AUDIENCE: Does that mean that they also couldn’t imagine words or language? PROFESSOR: Who couldn’t? AUDIENCE: People with Broca’s area aphasia PROFESSOR: I don’t think so We talked about this before, that– I may not be understanding your question exactly, but– in many ways, imagining something is the same part of the brain that does it When we imagine a face, we turn on the parts of our brain that sees I think when you imagine, if you– I do this sometimes I say, here’s what I’m going to say to that person, in my head And I practice that little speech That’s this part of the brain pretty much running There’s just a last little step of moving your mouth, but from the discussion we’re having now, that’s kind of a superficial thing of moving your mouth That’s not speech planning, speech thinking, speech intentions Those are our core process for production, for comprehension, and I think imagined stuff is exactly the same way Is that OK? Yeah Yeah? AUDIENCE: So if they can’t really imagine language, then how do they– what is their thinking like? [INAUDIBLE]? PROFESSOR: Yeah, that’s a good– so the question is, if they have these troubles with core language ideas, abilities, what’s their thinking like? I could just tell you, it’s a classic debate for humans, how deeply language equals thought But language [INAUDIBLE] in a sense of speech It’s just one of those things that people debate a lot because it depends what you mean by these things in many different ways But if language means something like comprehending ideas in the world, it’s hard to get around that for thinking [INAUDIBLE] It means planning actions in the world that change the world around you, and communicating that, it’s hard to not have that capacity for thinking in that way, right? So it’s at this level, not whether you use your hands or your mouth, it’s very hard to separate out what you might call thought and what you might call language But keep in mind that these patients that we see, it’s not like they lost all of language at all So even the Broca’s aphasic has trouble producing language, he’s producing some, and he comprehends it pretty well So it’s not like these patients have lost all of their language circuitry, and therefore don’t have thought available, right? They have a lot of language left in their head, and a lot of thought left in their mind Is that OK? So one interesting thing that people have wondered, and where brain imaging gives us evidence that we couldn’t have imagined 15 years ago, is this So if you have damage, let’s say, here’s a patient with damage to the left frontal area And they get better because they’ve had a stroke here or an injury here

They get better A question that people have wondered is, do they get better by sort of somehow recovering ability in a part of their brain that got injured? Or do they get better because they use an entirely different pathway in their brain? Now, until brain imaging, you couldn’t begin to touch that question But now we have some evidence, and I’ll make it just– and so here’s an example of a patient who has damage here They have him do a task that in healthy people turns on exactly that place And when he does it, he can do it OK And where does he do it? In his right hemisphere So this patient seems to switch which hemisphere he uses for language It takes time and training to recover that But it seems like he’s found an alternate pathway to produce a language somehow Turns out that the more people have studied these things, the more complicated it gets Maybe just diversity of people, diversity of injuries, or whatever So it doesn’t always work this way But often, it seems like when people have better recovery from a large injury to the speaking left hemisphere, they find a way, somehow, through practice and training, to get the right hemisphere to take over some of those responsibilities Language acquisition So one of the most amazing things about infants is they go from zero to speech in just a couple of years, and go through an unbelievable training and program of development So what happens? In two or three months, they perceive all the phonemes They notice this change in phonemes By six months, they start to ignore distinctions among sounds that are not used in their language In other words, other language sounds They start to babble, in some way getting their speech ready to go By eight months, they start to identify single words in a speech stream In a year, their babbling becomes more adult-like sounding But a little bit after that, they get up to about 50 words in understanding So comprehension always goes ahead of production Children understand words before they can speak them But they’re lagging Here comes a speech, telegraphic speech, short sentences at two years And then, finally, pretty complete speech, in many ways, by nine So it’s a long, long process, learning 10,000 words estimated by the age of six That’s a spectacular learning curve When they do experiments on infants where they measure things like preference or learning by sucking rates on a nipple because they can’t talk, they try to figure out, what is the infant hearing Within two hours of birth, they suck with more enthusiasm to their mother’s voice over another voice These are infants, this is their first visit to their mother You have to sign up to do this experiment Now, they might have heard their mother’s voice in a certain way in the womb, right? But they already have picked up, by three-day-old, they prefer language to many other sounds, like music We come to love music, but that’s not the urgent business of the infant By four days old, they start to notice the differences between languages This is amazing Their rate of learning is fantastic And by two months, they can tell the ba-ga distinction, subtle distinctions, 40 milliseconds information And then they develop a continuingly strong preference for the sounds of their native language Now, some years ago, people debated endlessly this left hemisphere specialization for language This is fantastically essential and striking in humans Does that grow over time, or is it in your mind and brain at birth? And the evidence is pretty compelling for many sources, but here is maybe the most Here’s fMRI on two-day-old infants You might ask, how do they instruct the infants to lie still in the scanner and participate in the experiment? And the answer is, at two days old, infants mostly just lay there and don’t do much They don’t do much And there’s a lot going on in their mind, but they don’t have much physical apparatus to do much So you could swaddle them very warmly, and just lay– you put them on a cradle into the fMRI scanner Their parents have signed up for them And you could play them sounds while they just lay there and do nothing They can’t push buttons They can’t do stuff But you can see what’s happening inside their brain And at two days old, here’s the left hemisphere response to language compared to the right So it’s there instantly, we believe It’s been there, actually in the womb as the child’s been developing This left hemisphere, genetically set up dominance for language in almost everybody fMRI, just for those of you who– again, we can start doing it at about age five, but from about three months to age five, it’s incredibly hard to do children in a scanner But the first couple days, it’s not so hard Now, a really interesting line of research has talked about motherese Have you heard this term? So how do grown-ups, even college students, talk to babies? Do you talk to babies just like you talk to everybody else? No Cute baby Baby, baby And everybody looks like an idiot, right? And you could just think, OK, that’s just people being

idiots around babies But it turns out there’s been beautiful, beautiful studies to show that, while you spontaneously do this– and you and I don’t take motherese or fatherese courses– but what we do spontaneously is that way of elongating sounds, the intonations we give, are incredibly perfect to help the baby learn language We’re exactly doing what they can understand And that, again, must be virtually in our genes, I think We hear it around, so it’s not zero But the short pauses, careful enunciation, exaggerated intonation, that’s– you could do experiments where you take language and you get rid of those, and the babies are like– they don’t know what’s going on They’re not interested You put them back in, boom Those are perfect for the baby It’s amazing, not only how babies are ready to go, but how adults who talk with them are ready to go I mean, it’s just amazing None of us will ever be as good a teacher in a classroom as all of you will be with a baby anytime you talk with them It’s just amazing So there’s beautiful experiments that show that every time you tinker with the way the mother spontaneously, or parents spontaneously, talk to indants, you make it worse, less interesting, and less informative for the infant So now let’s talk about a more formal experiment These are the classic experiment that nailed home this message, that at birth we’re universal understanders of all sounds in the world And as the years go along, we became only good at the sounds that are relevant, that are part of our own language A famous example in the US is a difficulty for Japanese native speakers for the R-L distinction But every language has some distinctions it doesn’t make, and they’re really hard when you get older So here’s the idea In English, we make a distinction between ba and da, and by 10 to 12 months, an infant can make that distinction And certainly, adults can That’s categorical perception that we talked about, that kind of division between ba and da But there’s sounds that occur in other language So we’re shown here And you’re going to hear– some of you may know Hindi sounds, and it’ll be a different experience for you Every year, when I first hear this clip I’m about to show you, I think something’s wrong Because they say, the infant’s going to tell the difference between da and da And I go, well, how are they going to do that? I can’t do that That’s because they really are different sounds you’re hearing, but in English, they’re not different sounds And so it all sounds the same to me I’m no longer a universal learner I haven’t been since quite a while So here we go I’m stunned, always, by the brilliance of infant researchers like this, who somehow, pretty convincingly in most cases, really tell you what the infant knows or pays attention to or thinks in pretty compelling ways, even though they can’t tell you themselves So one thing that’s come up in the US– I mean, the whole world is becoming ever more bilingual and trilingual There used to be some concerns, less now, and I’ll tell you, you don’t have to worry about it Whether it’s bad to learn two languages at once when you’re a kid And some parents worry about this, right? You’re going to be all messed up because you’re going to blend them all up and not do well in school And so there’s been quite a bit of research, actually, tracking and asking whether there’s any downside I mean, there’s huge upsides to knowing two or three languages Are there any downsides in terms of language development? And the objective evidence is pretty compelling that for babbling, for first words, for pairs of words at a time, [INAUDIBLE] at two months, that there’s really no cost So they followed, in Canada, for example, children who grew up learning English and French, English and ASL, French or English only Everybody does the same, pretty much There’s no cost, compared to the huge gain Sometimes some kids will get a little mixed up They’ll actually mix up languages because they’ll just mix them up when they’re pretty young But there’s no developmental delay of milestones or negative consequences There’s only the benefit of having two or three languages that you could speak So the evidence is overwhelming, empirically, that there’s no cost for learning more than one language at a time when you’re an infant One thing, there is a cost, a little bit, if you’re a boy So unknown why this happened I don’t think there’s any deep science about it But everybody has observed that, for example, number of new words understood during the first two years of life, that girls, on average, outperform boys in language acquisition early on And then things get closer, pretty much But the rate of learning on girls on average is faster than boys Nobody, I think, understands the specific basis of that So when people talk about critical periods in language acquisition, and what are fragile or resilient aspects of language Which aspects of learning language that seem very dependent on the age you are, and which are independent So phonology, including the production of language sounds,

seems very fragile Easy when you’re young, harder when you’re older Grammar is also age-sensitive, making subtle grammatical distinctions Learning about meaning or vocabulary, you’re perfectly fine when you’re an adult That doesn’t seem to show something like a sensitive period And here’s one famous study that established some of these things They looked at what age people arrive to different countries, from different countries to the US and started learning English These are people that didn’t know English before They’re all adults now, and they had them do quickly grammatical judgments And what they found is, if you came to the US when you– if you were born in the US, or you came to the age three to seven, you were very good at doing this But the later you came here, 8 to 10 years of age, 11 to 15, 17 to 39, your ability to make these fast, subtle grammatical judgments just went down Even though you’ve been here, in many cases, for many years You’re very smart Accents are like that, right? People come to the US at a certain point in their development They’re very skilled in English, have a big vocabulary, but the accent never disappears That’s another thing in terms of language production that has its fragile developmental early period So the last thing I’ll do, and then show you a movie for about eight minutes We get this occasional horrible experiments It’s not even an experiment of nature It’s just an experiment of horrible human behavior And the most famous one in language development, which had a huge impact on the field at the time, is a child, Genie I’ll show you You’ll see her in a movie clip in a moment They discovered her locked in a room, in a house, from age 20 months, approximately, until they discovered her at 13 She almost spoke to no one So the people taking care of her locked her in a closet, basically, and did not talk with her, or she had no language interactions until she was discovered and rescued And then they performed research for years saying could she learn things, what could she not learn She got single words, for example, but she never got particularly good at grammar [INAUDIBLE] syntax So a brutal way to test the idea that what’s acquirable later in life And what, if you don’t learn it early in life, is impossible to acquire fully? So I’ll end with this We’ll do this for five minutes or seven minutes I’ll stop there It’s summarized in your book Everybody knows about it because it’s this study you couldn’t do ethically in a million years You would never do, which is if you deprive a child of the critical period of language exposure And she never got lots of aspects of language going She got OK on a minimal vocabulary, grew in other ways But many of the things that, from a language perspective, she never got because she missed that critical period So that’s a bit of a sad story, obviously, but have a good spring break and a refreshing spring break And I’ll see you in 10 days