For most of our discussion of cognitive science, whether it's computational or biological, the emphasis has been on a introspective version of cognition. That is to say, we have certain problems that we want to solve or decisions that we want to make, and we mow those over or computers run algorithms, and then make attempt at solving the problem or making the decision. It's an extremely intellectual version of what cognition is like. What I would like to discuss today is a relatively recent, not completely recent but relatively recent. I believe highly important movement in cognitive science especially this larger biological turn in cognitive science to see cognition as something beyond the inner workings of the mind or brain, to go beyond, even if we're talking about biological systems, to go beyond the single organ of the brain. To think of cognition, and thinking as an activity that involves both brain and body, and depending on who one talks to, not only brain and body but the external world as well. So the cognitive science that we're going to be talking about today, and again, it's a very huge subject. We can only talk about some high points really just give you an introduction to some of the major themes of this area of thinking. What I'm going to do today is just try and give you some basis of understanding the point of view of their different names. Depending on the points of view of the researchers working in this area, they take different names to describe what they're doing. A common term is embodied cognition. The idea being that, one cannot understand cognition without taking account of the actions and sensations of the body. That's not the only name however for the various subcultures of people interested in the link between body and cognition. We'll talk a little bit about that. Just to give you a sense of the literature on this subject, it's rapidly expanding. I've put here a few books that I have found very useful. All of these books are, I could describe them as, at least in my view, classics of the genre. There's an early book on the relationship between bodily understanding and language called Metaphors We Live By, that's at the bottom right here by Lakoff and Johnson. George Lakoff also co-wrote with Rafael Nunez, a very interesting book, I believe it's called Where Mathematics Comes From, where he describes the understanding of what might be thought of as the most abstract subject in the world, mathematics. He and Nunez described that as having origins in bodily understanding. It's a very interesting book. The philosopher, Andy Clark has written a number of really wonderful books on the subject of extending the notion of cognitive science into the body and world. I think the best book to start with in understanding his point of view is this book called Being There, which describes the move from what I might call intellectual cognitive science to embodied cognitive science. Shaun Gallagher is a very active writer in this area, and he wrote an excellent book called How the Body Shapes the Mind. A collection of papers, a collection of essays really on the notion of thinking of cognition as going beyond the boundaries of the body, and including the external world itself is described in this collection, The Extended Mind. These are all very good books but they only scratch the surface of the literature on this subject that even in the last five or 10 years has expanded tremendously. They're now a couple of very good large handbook style reference books in embodied cognition, very useful. There have been conferences on the subject. So it's highly active area in cognitive science. What are the ways that we could describe what embodied cognition means? It's often easier to describe in a sense what it's not. We could begin with the, if you recall the image purveyed by Alan Turing in his description of the Turing test. When we were talking about the Turing test where a human being is in one room and a computer is in another room and one types questions. First of all, the image of the computer in that case, which is drawn really from the image of computers that Turing would've known about is a stationary instrument. A large information processing instrument. Nowadays, when I form images of the Turing test, I think that the human being in one room and a desktop computer as being in the other room possibly connected to the web but still a desktop computer. In other words, something which is really just an instrument that sits in one place, and whose response to Turing test questions will be dependent on its programming. It's not a biological system and not a system that moves around or takes action. Turing did mention in his paper the idea that maybe the only way to get a machine to pass the Turing test would be to have it move about in the world. Still the default image is of a stationary computer which is programmed to be intelligent, and which responds to questions in a way that mimics that of a human being. This is an abstracted and again, traditional computational and highly intellectualized version of what cognition is. The themes of embodied cognition are often presented in contrast to that view of cognition where the idea is first, and again, depending on the researchers one reads and the the various points of view that people take, none of these views are- None of these views are definitive of the themes of embodied cognition. Some people researching embodied cognition may hue to these themes more or less, but they're representative. So one is the idea that cognition is interwoven with action, that cognition is in part in the service of action or is at least complimentary to action. One thinks about things in order to take action, to interact with the world, to interact with other people, to interact with instruments or objects in the world. So the way in which one should study cognition is through its relationship with actions that people take as opposed to an emphasis on kind of solitary cognition. A second theme is that cognition is I say linked to, enabled by, constrained by parameters of the physical body, that there are tendencies to the way we think or can't think that are linked to both the affordances and limitations of the physical body. Certain ideas for example in physics or mathematics, might be especially natural to us or especially difficult for us because they jibe well with our physical experience. Finally, this view of extended cognition is that cognition is best understood as a system that spreads beyond the body and it's best understood as a systemic process that makes use of tools and objects outside the body like riding, like scissors, like professional tools stethoscopes or things like that. In other words, thinking of cognition is thinking of the cognitive system as containing and dependent upon the human or computer thinker, but also including the external objects that the thinker makes use of. So those are major themes of embodied cognition. Now, let me give you just a few examples to think with that illustrates some of these themes. Last time we were talking about children's cognition, and I mentioned to you this classic test by Piaget, to understand the behavior of certain kinds of objects, physical objects in the world. In particular, one of the major themes that Piaget is stressing in this particular experiment is the idea of reversibility or inverse operations. Again, just to remind you here's the experiment. You show a four-year-old, may be five-year-old a couple of identical glasses filled with water as on the left of this picture here and you ask them which glass has more water and they say it's the same, the glasses have the same amount of water. Then you take one glass in front of them, there's nothing hidden here. You take one glass and you pour it into this tall cylinder and then in this new situation represented by the panel two here you say, "Which glass has more water?" Young kids will say the cylinder, the tall cylinder has more water in it. Now, older kids would not make this mistake. Passed a point as kids get further into what Piaget called the concrete operational stage, they do not make this error anymore. One of the crucial insights about this as kids grow up is that, clearly you could take the cylinder in panel two and pour it back into the original glass and you'd be back in the same situation where you started, so there's a sense of of conservation or inverse operations here. Now, in her book Hearing Gesture, Susan Goldin Meadow described an extraordinarily interesting set of experiments that were done with young children involving this very task, and I'll see if I can describe this adequately. The idea is, one takes a certain group of children who are still in the early stage to the pre concrete operational stage of understanding this test. In other words, one takes a certain group of children who will say that in the second picture, the tall cylinder has more water in it. So you start out with a group of children who will answer that way. Now, you videotape them answering the question about which glass has more water. What you try to do is get some of the kids who were older edge of this belief. So some of the children when they're answering this question even though there are answering the question wrong, they gesture in a way that suggests inverse operation. That is, they'll answer the question saying, the tall cylinder has more water but they'll make a gesture that indicates this idea of pouring back and forth, and or you can invert the operation. They're still getting the answer wrong verbally, but they're making a gesture that looks like they're starting to get the understanding that in fact you could undo this operation. It turns out that the kids who make this gesture when tested subsequent you take the whole group and test them, I don't know couple of months later a few months later. The kids who were making this gesture are disproportionately more likely to have switched over to the correct understanding than the kids who didn't make the gesture. In other words, the kids who were making the gesture were signaling with their body that they were beginning to get an understanding of the idea of inverses even though verbally they were still giving the wrong answers. It's almost as though, or Goldin Meadow argues that it is, that the body is is preceding, is anticipating the verbal understanding that is soon to come. So one understands the task at a bodily level before one shows that understanding at a verbal level. It's a very interesting discussion of the relationship between body and cognition. A related question which I'm not sure is entirely resolved is, whether by encouraging certain gestures or even training certain gestures one can accelerate the transition from the wrong answer to the right answer. In other words, what we've just seen is that the kids who were making this gesture spontaneously are more likely to shift over in their view later on. What if you just tell the kid to do this? Just to this with your hand? Or the kids more likely to shift over. I don't know if that's quite resolved, but it is in any event. An extraordinarily interesting instance of the way in which body and mind interact. There's still other themes of embodied cognition, I mean one can begin with the idea that the body itself is something of a cognitive concept or construct. That is, how we view our own bodies is something that we continue to negotiate as we gather input day to day. You might think that by the time we're an adult, we're pretty clear on what our bodies consists of, and where they extend. But it turns out that in fact, at least in some instances, the body conception that one has is surprisingly malleable, surprisingly plastic. What I'm showing on the left here is a picture it's actually from a New York Times article in 2011. It's. The article is entitled Need an Extra Hand. This is one of a number of related experiments in which one takes a fake rubber arm and one can get people to identify bodily with the rubber arm. In this case, this young woman who's got her arms out and there's a third fake arm that has been placed on the table. The experimenter is stimulating both her right true arm and the fake rubber arms simultaneously. Given this simultaneous stimulation, the subject fairly quickly, and it doesn't take an extraordinarily long time, begins to have the distinct illusion that they have three arms. There are other versions of experiments that they'll identify with the rubber arm. There are other experiments where the subject just places their arms out on the table and the apparatus is set up so that instead of seeing their right arm they see on this side of the box in front. They placed their arms in two halves of the box and what they're actually seeing on the right half of the box is a fake rubber arm, not their own right arm. But it just looks like them. Then the same thing is done. That is one stimulates the rubber arm at the same time that one stimulates the biological arm. If you do that for a while, the person just begins to feel that this arm represented in the box is their arm. In fact, you can really do mean things to people, like if you take a hammer and slam the rubber arm, people will jump away. They're extremely scared because they've now begun to identify this arm as their own. There are medical conditions that some of them distressing to read about in which what the outside world and outside people would regard as untrue or dysfunctional views of the extent of their own body. Oliver Sacks, biologist and cognitive scientist, writes about his own experience with having a paralyzed leg fortunately for a temporary period of time. But he reports his own feeling that this leg that was sitting paralyzed in the bed with him was not his own. That is he refused to acknowledge for time that this inert leg could possibly be part of him. That's not an uncommon symptom in people who have paralysis in certain parts of their body that if you ask them, for example, whose leg is this? They'll confabulate answers. I mean they'll say things like, "I don't know, it's a stranger's leg, it's not mine, can't be mine." Even though intellectually you'd have to know that that's your leg but is deeply not felt as one's own leg. So these are among the kinds of situations in which one's body image can be surprisingly plastic, there are many other experiments along these lines. I would love to go into more detail about this because it's actually a fascinating area. There's some disorders of body perception that are rather hard to read about but involve, for example, people even without paralysis not believing or not wanting to believe that a part of the body is their own. But I would leave that to your outside reading. Another theme of embodied cognition is I mentioned early on is that cognition is seen as interwoven with action. This is an interesting instance of that thing. For those of you who have played Scrabble and I'm assuming that you've all at least seen the Scrabble game but the basic idea without going into details, you're given letter tiles and then you're asked to spell words from them on a board to create words on a game board. I mean you could build your own intuition about this. What do you do when you're playing Scrabble and you're given a set of seven tiles and they're put on this little rack here? What do you do when you're playing the game? The intellectualized purely computational version of cognitive science, the purist brute force artificial intelligence approach to this task would be to start thinking of all the anagrams, all the different ways of arranging this number and there's seven factorial ways of them generally. So that's going through a lot of different orderings of letters. What do you do when you're playing Scrabble? Most likely what you do is you move your hands. You take your hands and you start moving the tiles about on the rack to see if they suggest words to you. In other words, you're offloading some of the internal cognitive task to a mechanical task and a visual task that is by moving the tiles around you can perhaps can get to the point where a string of letters is sparking a visual memory of a word. But in this case, you're pure cognition aspects of your work. The intellectual version of your work is interwoven with and partially dependent upon the bodily actions that you're taking to solve the problem. The problem is solved in other words not by purely the manipulation of abstract information but the manipulation of information combined with manipulation of the outside world to facilitate information processing. That thing is a very interesting study. Again, you could take many of the more abstract tasks even the more rarefied tasks that we've talked about and that artificial intelligence has explored for a long time. Things like understanding mathematical problems, understanding physics, understanding mechanical systems. Perhaps understanding social systems. You could take a variety of those tasks and expand them by looking not only at the internal calculations that go on, but looking at the way in which people try to set up the world, try to take actions that will make the information processing easier. There's a very interesting experiment that was mentioned in a paper in Behavioral and Brain Sciences by Arthur Glenberg. I don't believe it's his experiment, but he talks about an experiment that I find extraordinarily interesting. Again, let me see if I can adequately describe this. As soon as you hear this experiment, first, if you just quietly identify with the subject, I think you'll see that this is exactly what would happen. Secondly, when you think about the experiment, you realize some weird result. So let me describe it. Person goes into a room, the subject is brought into a room, but just a plain old room with a certain objects of furniture around and they're told to stand in a particular location and look around the room, and remember where the furniture is. Then, they're blind folded. So the blindfold is put on them and they're told to point to the large chair. They do that without any problems. They're blindfolded, but they remember where the large chair was and they point to it. Then, secondly, they're told the following. They're told, don't do this, but imagine that you were to make a right face turn. Don't actually make the right face turn, but imagine that you were to make the right face turn. Stay exactly where you are. If you were to have made the right face turn and were then asked to point to the chair, how would you point? In that case, people get confused, they make mistakes. You could still accomplish the task, but it's much more difficult than it was before. Then finally, the last stage is they tell people, okay, make the right face turn. Now, make the right face turn. Now, point to the chair, no problem. So think about that. You're standing in one place blind-folded, point to the chair, no problem. Imagine that you were to turn and then show how you would point to the chair. If you did that, big problem. Do the actual turn, point to the chair, no problem. That's a little weird, isn't it? Think about that. From a computational standpoint, what's the difference between imagining the turn and doing the turn? It's just a matrix multiplication. Why should this be difficult? How is it that we exactly register the fact in our body that we have turned? This is a effect of bodily motion on a cognitive task. It's quite provocative because it suggests again that the state of the body and the way that one moves the body is deeply interwoven with performance on what you might think are purely cognitive tasks. The notion of extended cognition, which is a little bit different from embodied cognition, and there are strong debates over how useful or how correct this view is, is that the best way to analyze cognition, to think about it is as a part of an extended system so that a person with a calculator is a different cognitive system than a person without a calculator. Maybe the clearest case of this and intriguing case is on the upper right, a person who is vision impaired, blind, may use a walking stick and the stick is then perceived as an extended part of their body. So one could make a plausible argument, in this case, that the person, him or herself, is best analyzed as those parts of the person internal to the body navigating the world, extended by or including the walking stick object itself. The question of whether one analyzes the entire system including external objects as the cognitive system or not, or whether one excludes these external objects is in some cases more of academic interest than of interest to designers, for example. In either case, clearly one can do things in the presence of tools that one could not do without the tools present. That's clear. To some extent, it's a matter of definition and perhaps modeling. How you would model a system like this computationally whether you decide to extend the boundaries of the system to include the tools or just treat the system as constant, the biological entity and have means for grafting onto that biological system, the affordances of tools. I put a couple of other examples here. In general, when people are composing sentences, they don't do it in their head, they do it in the presence of a word processor or a pen and a notebook. Again, whether one says that they're a constitutive element of the act of composing sentences or literature, or whether they're just an additional element is a matter of debate. But there are lots and lots of people who report that they simply can't or would have a much harder time. Composing sentences without the aid of something like a word processor or notebook. There are people who compose music at an instrument. They do it with guitar, they do it with piano, and they would be hard-pressed to compose music without the presence of a musical instrument that they can play right there. So the general notion of extended cognition is one that goes beyond embodied cognition and it includes within the scope of action that the external objects that one makes use of in solving cognitive problems. Finally, I think a very interesting area of research in embodied cognition is designing the design of new kinds of tools that extend the body itself. When one talks about embodied cognition, the body itself is not a constant entity throughout history. Even these eyeglasses that I'm wearing are things that weren't available to people a millennium ago. So as far as I'm concerned, the eyeglasses are external to my body but there's sort of thought of as a natural extension, extremely common extension of the body. There's a great deal of research going on now. Some of it I find a lot of it remarkably interesting, and some of it going on the fringes that look into extending either the actuation possibilities of the body or the sensing, for instance, of the body through various means. Some of these involve things like cochlear implants for example are actually embedding within the body things that extend or enhance one's senses. In this case, one sense of hearing. The picture on the upper part of this slide is the cognitive scientists David Eagleman showing a vest that he designed. This is in a TED talk that you can access on the web. But this is the vest that translates incoming sounds into tactile stimuli in the surface of the vest. He's used this apparently with some success in providing a workaround for hearing impaired people. They can wear this vest, get tactile stimulation from the sound that surrounds them, and actually begin to interpret that tactile stimulation as meaningful sound, as what we would call meaningful sound. The brain and the sense is being used as a highly powerful transducer. It can take a signal in tactile stimulation, and under the right circumstances, interpret that signal as we would an input of sound. Another instance of extending the sensors, and this is kind of a homegrown and interesting subculture are the people who sometimes they refer to themselves as body matters or biohackers. That is people who are looking to extend there biological capacity through DIY means, through things they can do themselves. You can purchase on the web, you can purchase a kit in effect of powerful magnets that can be implanted into your fingers so that you can have a sense of magnetic fields in your fingers. I haven't done this, and I wouldn't recommend it, but some people are doing it. The subculture of people who were doing this, actually there's an interesting book by a reporter Kara Platoni, the book is called, We Have The Technology. She talks about the subculture of people doing things like this. You can see in the bottom right photo there that the person has a magnet implanted in I guess his finger and he can pick up a paper clip with it. One thing I will say is that, apparently the people who have done this don't seem, at least in the reading that I've done, they don't seem to regret it. That is they say, well, now I can feel all kinds of things in the world that I couldn't feel before magnetic fields, and that's interesting. In any event, a person with this kind of extension may have different cognitive affordances than a person without this kind of extension. Again, there are lots and lots of people becoming increasingly interested in designing tools and techniques whereby they can extend their sensory ability. There are other people very interested in the natural flip side to this in extending their actuation through new kinds of prosthetics. The picture on the bottom left here is of a performance artist from Australia named Stelarc, S-T-E-L-A-R-C. If you look him up on the web, you'll see he's done all kinds of weird things. Again, not that I'm recommending this, in one project that he did, he attached a third arm to his own arm and grew very adept at manipulating that third arm by twitching muscles in his stomach to the point where he could write his name for example with this third robotic arm. Here he's got a hand extension. He's done many, many experiments projects, I guess you could say artistic projects as he would describe them, that involve the toying with the notion of body integrity and extending the body in various ways. A much less controversial and perhaps in some ways much less disturbing body of work that goes on in this way are people designing new kinds of powerful prosthetics for people for example, with missing limbs. There's a an engineer scientist at the MIT Media Lab named Hugh Herr, H-E-R-R who lost the lower part of both his legs in a climbing accident, and who designed new robotic legs for himself. Again, he's given a TED talk and you can see some of his work on the web. one of the things that's especially interesting about this is that in some cases, he's designed extensions to his original legs. That is he's designed prosthetic legs that enabled him to do certain kinds of things in climbing that he was unable to do originally with his original body. So this is an interesting case of using prosthetics not just to replace, but to extend the abilities of the body, and perhaps in the same step, to extend one's cognitive capacities as well.
