Welcome to the third and final video of this week. In this video, we will be discussing decision-making in the self-driving car system. Recall that in the last video, we discussed the perception which forms the first step in performing a driving task. The other steps in driving include decision-making and then, finally, executing the decisions. In this video, we will categorize planning informally on the basis of the window of time over which the decision has to be made and discuss some examples. Then we will go over a simple intersection scenario and try to list out some of the various decisions needed to complete the driving task successfully. We will then categorize planning formerly based on the type of logic we use to make the decisions. So, is our logic made up of well-defined rules that react only to currently available information about the driving environment? Or is it also dependent on trajectory predictions of other agents? Let's get started. Making driving decisions falls under the bigger umbrella of planning. When we make driving decisions, we usually have three kinds of decisions to make. The first type is a long-term planning decision. A question such as, how do I navigate from New York to Los Angeles or from my home to work? By answering this question, we have a mission plan, a high-level plan for the entire driving task. Mapping applications that you use today are able to give you these driving instructions: which roads to take, which lanes to be in, and so on. But driving needs much more than that. The second type is a short-term driving decision with questions like, is it safe to change lanes now? Or when should I execute a left turn at an intersection? Driving also needs some immediate decisions or reactions. These decisions involve control and trajectory planning and answer questions like, how do I follow my lane on this curved road? What steering input should I apply? Should I accelerate or brake? If so, by how much. Let's discuss a very simple example of a driving task and try to think about what kind of decisions are involved. Note that throughout this specialization, we will assume right-handed driving for all scenarios. Suppose you are approaching an intersection on your way home. The long-term planning stage requires you to turn left at this intersection. Now, let's look at the intermediate and short-term decisions that need to be made. First, let's assume that the intersection is controlled. That is, it has traffic lights. Since you are turning left, you have to identify if you need to make a lane change into a left turning lane. Then, as you're approaching this intersection, you choose to slow down, and to do so smoothly so that the passengers don't experience discomfort. Nobody likes a jerky driver after all. You then come to a stop just before the stop line, before a pedestrian crossing. These decisions on lane changes and stopping locations are all short-term planning decisions. But wait! You also need to think and respond to situations that arise along the way. We still need object and event detection and response. What if a vehicle pulls into the turn lane in front of you? You would want to stop earlier to make room for the other vehicle. What if the stop lines weren't marked? You would have to approximately judge where the implied stop line is and stop before the pedestrian crossing. What if there were other vehicles behind you or even stalled in the intersection? How does the decision to execute a left turn change based on the many possible scenarios that can rapidly arise in normal driving? All of these decisions fall into the immediate decision category and requires safe reactions from the planning system. The end result is an exploding list of possible decisions to evaluate on different timescales, even for a simple left turn scenario. This amounts to talking about different cases for the same intersection crossing or scenarios. In each scenario, we need a consistent set of choices to be evaluated in real time and updated as new information about the scene becomes available. Furthermore, because decisions to change lanes affect where to drive and which cars to regulate our position relative to, even a seemingly simple driving scenario requires three or four levels of decisions, and must then still be executed with careful vehicle control. This example is really only scratching the surface of the constant stream of decisions needed for motion planning. The bottom line is, driving is complicated. Let's go ahead and discuss a possible structure to represent these decisions in software. One method to address the challenge of multilevel decision-making is reactive planning. In reactive planning, we define sets of rules that take into account the current state of the ego vehicle and other objects in the environment and produce immediate actions. So, these are rules that only consider the current state and not future predictions. Some examples of such rules would be, if there is a pedestrian on the road, stop. Or if the speed limit changes, adjust your speed to match it. In both of these rules, we just observe what is happening right now and make our decision based on immediately available information. But there are other kinds of planning as well. In predictive planning, we make predictions on how other agents in the environment, like vehicles and pedestrians, will move over time. We use this current state and prediction information to define all of our decisions. Some examples of rules in predictive planning would be, that car has stopped for the last 10 seconds. It's probably going to stay stopped for the next few seconds. So, perhaps there is a way that I can move past it safely. Or a pedestrian is jaywalking. They will enter our lane by the time I get close to them. Let me slow down and give them a chance to cross the road ahead of me. As you can see, this is a more natural way to think, and relates closely to how humans operate vehicles. We predict where other objects on the road will be in the future before we make our decisions. This type of planning, however, relies on accurate predictions of the actions of the other actors in the environment, which adds a significant layer of complexity to the perception tasks. Nonetheless, predictive planning is the predominant method for self-driving cars, as it greatly expands the scenarios a vehicle can handle safely. We will discuss all aspects of planning in a further detail in course four on planning for self-driving cars, where we will show you the methods to solve long-term, short-term, and immediate-term planning problems. All right. Let's summarize this video. We discussed the planning problem and the different types of planning based on the window of time over which we have to act. These types are long-term, short-term, and immediate planning. Then we discussed a simple intersection scenario, where we had to make a left turn. We concluded that driving is a really hard problem since we have so many variables and possibilities that result. Then we discussed two different planning approaches: reactive planning and predictive planning. This is just the tip of the iceberg. There's clearly so much more we need to think about before making a decision during the driving task. Once again, we will go through all of these ideas in much greater detail in course four. That brings us to the end of the first week of course one in our self-driving car specialization. Let's quickly recap what we learned this week. In this module, we explored the basic autonomous driving terminology that's useful for later weeks of the specialization. We then discussed the levels of automation, and came up with a taxonomy to characterize self-driving capabilities. Then we define the driving task and the major components of driving: perception, planning, and execution. We then listed the elements and agents in the environment we need to identify and track for perception. We also discussed why perception is so hard. Then we discussed planning with its different horizons, and looked at some decision-making approaches. In the next module, we will define the main components of self-driving cars, including both the hardware and software elements that make up a complete system. See you then.
