So, as we now start to look at natural language processing particularly with neural class models, one of the key concepts that we're going to repeatedly utilize is this concept of word vectors. So, what this means is that every word in our vocabulary is going to be mapped to a vector, and then we will do our analysis of natural language in the context of these word vectors. This is a fairly novel and perhaps unusual concept, and so before we dive too deep into the natural language processing, I think it would be worthwhile to spend a little time just to get some deeper preparation or understanding for what we mean by word vectors. So to try to understand this, let's just think about a map of the globe. So, what I'm showing on this map are several different symbols, squares, triangles etc, and the thing that I want to communicate through those symbols is the idea that if two points on the globe are on this map are physically near by each other. So if two points on the globe are nearby each other, we would expect that the characteristics of the associated regions are similar. So, for example, you see the triangles in South America are near each other, where the squares and near Asia are near each other, and so, the idea is that the regions characterized by the two triangles since they're physically near each other, we would expect that the characteristics of perhaps the people, the geography, the history etc, of the two triangles would be similar to each other and the squares since they are physically proximate would be similar to each other, but very different from the regions characterized by the triangles. So, the concept here is that we can think about the globe as a breakdown into longitude and latitude. Two numbers, longitude and latitude. So, we can think about this as a two dimensional space. If two points in that space have similar longitude and latitude, we would expect that those regions on the globe are similar. If the associated longitude and latitude of two points is very different, we would expect those regions to be different. So, we have a concept of similarity manifested through proximity. So the way that we could think of this is that we're going to do a similar type of concept with words. So, every word in our vocabulary is going to be mapped to a point in a 2D space, and that the closer two words are in that mapping or in that two-dimensional space the more related or synonymous we would think the words are. The further apart two words are in this 2D space, the more dissimilar we would expect the words to be. So purely conceptually, we can think of a situation where we have a vocabulary of v words. So we have word one, word two, word, three, all the way up to word v. Then we can think about mapping every word to a two-dimensional space in longitude and latitude on long and lat, analogous to the way that we were thinking about the globe. So, the way that we would like to do this is that we would like to learn these two-dimensional vectors of the words in such a way that if two words are similar or synonymous or related to each other, we would want their associated longitude and latitude positions to be near each other. If two words are very dissimilar or unrelated, we would like them to be far apart in longitude and latitude. This is the basic concept in the geography of this word vector or this is called word to vec. So this is very difficult to read, but if you'd look closely at the screen you will see that every point in this two dimensional space corresponds to a word and the words are actually written here. If you look closely, you will see that words that are nearby each other are related and words that are far apart are not related. So the key, and this is a closer look, so this is now zooming in on that prior slide. So, that you can see that there is a relationship or a connection between words based upon their physical proximity, and so this concept of mapping words to vectors is fundamental to how we're going to do the modeling of natural language. So, the key thing to think about is that words are not numbers, they're not in the form of numbers. So, whenever we do modeling of natural language, that modeling is usually done in terms of algorithms and those algorithms like to word in terms of numbers. So, therefore what we need to achieve is a mapping of each word mapped to numbers, and then once we have achieved that we can then do analysis. So, the way that we're going to do this is that we're going to map or relate every word in our vocabulary to a vector which is like a point in space. However into motivate the concept, we thought about two-dimensional space much as we would think about the longitude and latitude of the globe, but that is purely just conceptual to help one understand it. In practice, we do not limit ourselves to two dimensional vectors. So each word is represented by a vector, that vector may be more than two dimensions. So, in fact, typically is larger than two-dimensional and the idea is that when words are similar, they should be nearby each other in this vector space, whenever they are unrelated they should be far apart from each other in this vector space. Within the context of learning, we're going to learn that the mapping of every word to a vector. So, when we talk about natural language processing modeling, one of the key aspects of this is going to be learning the mapping of every word to a vector, and subsequently in our discussions, we will describe many different ways of doing that. There are many ways that that can be achieved. So, I underscore that we will learn how to achieve this word to vector mapping, and as I said, this is not restricted to two-dimensional vectors. So usually, the vectors will be larger than two-dimensional. So, in this discussion, all we wanted to do was to introduce this concept of mapping each word to a vector. So, in the subsequent discussions we're going to describe how that will be achieved actually using many of the tools that we already know. But before we get into those technical details in this lesson, we wanted to first just introduce the concept of mapping words to vectors. So now as we move forward, we'll see how to achieve that mapping and then how we utilize it in natural language processing.
