In this module, we'll be going over some of the main concepts in TensorFlow 2. We're looking at tensors, variables, and the current API hierarchy. We'll then have a deep dive into the tf.DataAPI and learn how to create input pipelines from models that train on data that's both in-memory and also data that lives in multiple files. Lastly, we'll learn how feature columns are used to manipulate and prepare data so that it can be used to train neural network models. TensorFlow is an open-source, high-performance library for any numerical computation, not just for machine learning. In fact, people have used TensorFlow for all kinds of GPU computing. For example, you could use TensorFlow to solve partial differential equations which is super useful in physics fields like fluid dynamics. TensorFlow as a numeric programming library is appealing because you can write your computation code in a high-level language like Python and have it be executed in a very fast way at runtime. The way TensorFlow works is that you create a directed graph or a DAG to represent the computation that you want to do. The schematic that you see here, the nodes like those light green circles represent mathematical operations things like adding, subtracting and multiplying. You'll see small complex math functions like softmax and matrix multiplication which are great for machine learning. Connecting all of those nodes together are the edges which are the input and the output of those mathematical operations. The edges represent arrays of data flowing towards the output starting from the bottom of the arrays of raw input data. Sometimes we'll need to reshape your data before feeding it into the layers of a neural network like the ReLU layer you see here. More on ReLU later. Once inside that ReLU layer, the weight is then multiplied across that array of data in a map mole or matrix multiplication operation then a bias term is added and the data flows through to the activation function. I know what you're thinking, ReLU activation functions? Don't worry, let's start with the basics. I kept mentioning that array of data that's flowing around. What exactly does that mean? Actually where TensorFlow gets its name from. Starting on the far left, the simplest piece of data that you can have is called a scalar. That's a number like 3 or 5. Is what we call zero-dimensional or rank 0. Now we're not going to get very far passing around the single numbers in our flow, so let's upgrade. Rank 1 or a one-dimensional array is called a vector. Now in physics, a vector is something with magnitude and direction but in computer science, you use the word vector domain 1D arrays, like a series of numbers in a list. Let's keep going. A two-dimensional array is a matrix, a three-dimensional array, we'll call it a 3D tensor, scalar vector matrix, 3D tensor, 4D tensor, etc. A tensor is an N-dimensional array of data. Shared data in TensorFlow are tensors. They flow through the graph hence the name TensorFlow. Why does TensorFlow use directed graphs to represent computation? The answer is portability. The directed graph is a language independent-representation of the code in your model. You can build a DAG and Python stored in a saved model, restored in C++ program for low latency predictions. You can use this same Python code and executed both on CPUs, GPUs, and TPUs. This provides language and hardware portability. A lot of ways this is similar to how Java Virtual Machine or JVM and it's by code representation helps with the portability of Java code. As a developer, you write the code in a high-level language like Java, have an executed in different platforms by the JVM. Now the JVM is itself very efficient and targeted to the exact OS and hardware written in C or C++. Similar deal with TensorFlow. As a developer, you write your code in a high-level language like Python and have it executed in different platforms by the TensorFlow execution engine. Now the TensorFlow execution engine is very efficient and target toward the exact hardware chip and it's capabilities and it's written in C++. Portability between devices enables a lot of power and flexibility. For example, here's a common pattern. You train a TensorFlow model on the Cloud, on lots and lots and lots of powerful hardware, then you take the trained model and put it on a device out on the edge, perhaps a mobile phone or even an embedded chip. Then you can do predictions with the model right on the device itself offline. Have you had a chance to use the Google Translate app on an Android phone? The app can work completely offline because the train translation model is stored on the phone and is available for offline translation. Now I know what you're thinking due to the limitations of processing power on your phones, the edge model tends to be a bit smaller which means they're generally less powerful than what's on the cloud. However, the fact that TensorFlow allows for models to run on the edge means a much faster response during predictions. So TensorFlow is portable, powerful, production-ready software to do numeric computing. It's particularly popular for machine learning. It's the number one repository for machine learning on GitHub. Why is that? Well, it's popular among deep-learning researchers because of the community around it and the ability to extend it to do some pretty cool new things. It's popular among machine learning engineers because the ability to production analyzes models to do things at scale. The popularity among each of these groups increases the popularity in the other. Researchers want to see their methods being used widely and implementing them in TensorFlow is a way of ensuring that ML engineers want a future proof their code so that they can use newer models as soon as that are invented and TensorFlow can help them do that. Google open-sourced TensorFlow because again, empower many other companies and because Google saw the potential of his massive community support.
