Module two, Concurrency Basics. Topic 1.1, Processes. So, we're going to start talking about what a process is. A lot of concurrent execution, a lot of these ideas came from operating systems. The job of an operating system is really to give concurrency, we'll talk about this, to allow programs to run concurrently. So, for instance, take the machine that I'm staring at right here. This run this PowerPoint presentation. It is running PowerPoint on it, but at the same time is doing other things in the background and the operating system is allowing them all to be running concurrently at the same time. So, concurrency issue, a lot of concurrency starts with this idea of processes, and then we'll talk about threads, and how they're related to processes, and then we'll get to Go, and how it implements those things. So, a process is basically an instance of a running program. There are things that every process has this unique to it, a big chunk of memory. So, if you've got multiple processes running on a machine, every process has it's own memory, it's virtual address space, and we're not going to talk about virtual memory, but it has an address space. It has addresses zero to two gig. This process owns them. It has code. Every process is going to have it's own code. It's going to have it's own stack. Stack is a region of memory that handles function calls mostly. Heap, which is another region of memory where you do memory allocation, stuff like that. Shared libraries, actually, shared libraries are shared. So, sometimes in fact, usually, those are shared between processes. So, the shared libraries are actually not unique to a process, they're shared by definition. So, they are shared by processes. But each process is going to have it's own stack, it's own code, it's own virtual address space, and a bunch of other things. Also, a process is going to have registers unique to it. Registers in case you don't know, basically, they're just store values inside the machine. They're like tiny little super fast memories. They just store one value, one word, say. A program counter. Program counter, it's the register that tells you what instruction you're executing right now or really the next instruction was the next one you're going execute. Data registers, the stack pointer, that's another register that tells you where you are on the stack, stuff like that. So, every process has this unique, was typically called context. A bunch of memory, a bunch of register values that are unique to the process and they're all needed to execute the program correctly. So, every processes in executing or running program. Now, an operating system does a lot of things. But what essentially an operating system is? Is something that allows multiple processes to execute concurrently. There's a lot of complexity behind that because they have to execute concurrently without bumping into each other. So, one process might say, look, I have addresses zero to two gig. The next process thinks it has the same set of addresses. The next process thinks it has the same set of addresses. So, they have to be able to access addresses. Maybe they all accessing address 1,000, but they're not accessing the same address 1,000. The operating system has to make sure this guy's 1,000 is different than this guy's 1,000, so they don't interfere with each other. So, nothing the operating system has to do is to make sure that these processes get fair use of the processor. Meaning this is concurrency. These processes are not actually executing in parallel or that's our assumption right now anyway. So, since they're concurrently executing, this process, and a process might get the CPU for a certain amount of time, and then the next process should get it's turn. So, a processes that switch quickly. So, like 20 milliseconds, that's typical number. I think that's a standard number and default number in Linux where process gets to use the processor, CPU core for 20 milliseconds, then the operating system says, "Okay. Next guy's turn." You can change that number, of course, but what happens is it's just moving them in and out so quickly that to a user's perspective, it looks like they're all running at the same time. So, that's the main job of an operating system and this task just to give it a name, it's called scheduling. Deciding which process runs at which time, that's called the scheduling task and operating systems do that. The user has the impression of parallelism even if it's not parallel. Although, operating systems can apply parallelism, the operating system can map something to a different core but generally, certainly, right now, we're talking about single core operating system. The operating system needs to give fair access to resources. So, when I say resources, I mean things in the system other than the processor itself. So, there's a processor itself. Maybe you give this guy 20 millisecond slice, that guy 20 millisecond slice, but also, other things like memory. This one gets to use this region of memory, this one gets to use that region of memory. IO devices, you get to use the screen now, now, you get your turn to using the screen, so on. So, the operating system is basically managing a pile of processes, and making sure they don't interfere with each other, and they get fair use of the resources, so they can all complete in a timely manner. Now, this is a picture of the test manager. So, if you have a Windows machine, you hit control, alt, delete, and you'll get a little prompt, the middle of the screen, it'll give you a bunch of options, you can select task manager. This is the task manager at the time when I made this slide. Now, task manager, what the task manager does is it shows you all the running processes. You can see there are many running processes. So, I can't even read it from here, but there are a lot of processes. So, typically, like if I were to look at the task manager for the machine, I'm looking at right now that's running the PowerPoint slides, it's got PowerPoint. That would be one of it's processes that's running, but there would be sort of 30 other processes running, doing other background things. Maybe reading email, maybe serving a web server on it, all kinds of background things, some security thing, checking for a tax, all that stuff would be going on at the same time. Now, when I say at the same time, it's all concurrent execution and maybe there's only one foreground task, like this machine right here that I'm looking at, it's got PowerPoint running on it, that's the foreground task, that's the foreground process. But in the background, it's got a bunch of other things that get their turn and they do their thing at same time. So, you can only see the processes running just by looking at the task manager on a Windows machine anyway. Now, let's say, you can see this on any machine, macOS, Linux, macOS just for your information, it's basically a Linux. So, you can look at that. If you wanted to see that, go to the command line, type PS, it lists all the processes running or do a PS-L, you can see them all and you will see similar information of this. Although, in a less easily readable format, you'll see the same information. But this is the idea that an operating system is allowing all these different processes to execute at the same time or virtually at same time concurrently. Thank you. Module two, Concurrency Basics. Topic 1.2, Scheduling. So, we're talking about how an operating system allows these processes to all execute concurrently, virtually at the same time, giving the illusion of parallel execution to a user. So, what happens is the main task of an operating system is scheduling. So, you might have many processes that are running like in this case, I have three processes that I want to run, process one, two, and three. This little chart that I'm showing is execution over time. So, time is decreasing down. These little vertical bars, they're colored, and they're each one's aligned with a process. They're just showing how basically the operating system is making sure that one process gets a turn, so process one first, then it gives process two a turn, then it gives process three a turn, then maybe goes back to one again, so on. Now, the scheduling decisions, there are many different scheduling algorithms, ways of doing scheduling. Like for instance, what we're showing here is basically what's called round robin. So, you give each one a fair amount of time. Like in this case, I'm just going 1, 2, 3, 1, 2, 3, 1, 2, 3, over and over, everybody gets the same slice over time, sort of a fair way to distribute. This is not necessarily how scheduling is done. Like for instance, maybe you have a higher priority process. Some processes are more important than others. For instance, some processes maybe they're interacting with the human, so they need to be prioritized better than something in the background. So, in that case, the scheduling algorithm built in the operating system might consider that and say, "Well, you're higher priority. I will let you have more time than I let this other priority task run or something like that or I'll let you maybe process one's higher priority, so it occurs, it gets to be scheduled more frequently than process two or three, something like that." There are many different scheduling algorithms and you get a lot of this especially with embedded systems where you get these certain tasks in an embedded system or an IoT device that are critical like if you take like a car. Let's take a car. Cars got many processors actually and lots of things running. Say, it's got a processor that is processors doing anti-lock braking. So, you hit the brake, and the car's got the pedals, the brake pads have to come down on the wheel, and break it. Anti-lock braking, that would be considered a high priority process. Now, on the other hand, maybe you got a stereo running. That is a low priority process. So, what the processor should do if they were both running on the same processor, which they typically are not, but if they were, then you would say that anti-lock braking thing, that's got to have higher priority. So, even if you're in the middle of playing some stereo music, that task would have to be stopped. So, the operating system will say, "Okay. I'm putting you aside. Somebody just hit the brakes. We've got to go to the high priority task, the anti-lock braking." So, there are many different scheduling algorithms for operating systems to handle prioritization, and meeting deadlines, and all this, and we won't go into it, but scheduling is the main task in operating system. Now, when the operating system moves from one process to another, when it starts a process running and then it starts another process running, that act of switching is called a context switch. So, what we've shown here, we'll graph where times increasing down, you got process A, is one column on the left, process B is another column on the right. Now, when you get the vertical line like at the beginning process A is running, vertical line on a process A. Then there's this blue area that's labeled context switch, and that's where it's switching from process A to process B, that's the operating system, changing from process A to process B. Then you can see process B runs at vertical line on the process B and then there's another context switch from process B back to process A again, that's the operating system. So, basically what's happening here is that, when you're running a process, when you want to stop it like you are running process A, you want to start running process B, what you have to do is take the state, the current state of process A and save it somewhere for later use. Then you have to bring in the state of process B, let process B run and then later when process B is running you want go back to A, you take that state of process A that you saved and you bring that back in so it can start from where it left off. Now, when I said the state, that's called the Context. Okay? It includes all those things that are unique to a process. So, it's virtual memory system like which parts of memory it can access which I can't, or this register values, the program counter value, stack pointer, all that stuff, it's code. Right? All that stuff is unique to a process. So, we call that the state, memory and registers associated uniquely with the process. When you do a context switch, you take that context, all that state information, you save it to memory somewhere for the process that you're throwing out. Then for process B if you're bringing that one in, you would find its state and memory and bring it all back in, bring in its virtual address system and bring in its register values and all that so, the process B can continue from where it left off. Then when you go back to process A, you do a swap again, process B state, its context is saved to memory somewhere. Then process A state is brought in from memory back into the registers where it belongs and process A can continue where I left off. So, this context switch is performed by the operating system. So, if we look at the chart that show there, right? There's a time when process A is running, then there is context switch, then process B then the context switch in process A. Those two contexts which times, that's the operating system itself running, what's called the Kernel. The Kernel of the operating system is sort of main code of the operating system was thinking like that. The context switch is the Kernel of the operating system running, it's executing at that time and typically what happens is, you got a timer. So, these processes all have timers. When the operating system starts process A running, it sets a timer, maybe 10, maybe 20 millisecond timer, and when that timer goes off, process A is stopped and the operating system is executed. Then it does the context switch and then it starts process B and it sets a timer again. So, when the timer runs out again then it does, the operating system runs again does the context switch and the process continues wherever, they are in the process. This whole procedure continues forever or as long as the machine is powered on. Thank you. Module two, Concurrency Basics, topic 1.3: Threads and Goroutines. So, we talked about what a process is, now I want to define what a thread is. So, threads versus processes used to be that there were only processes. Now, this is way back when Units was first created, right? They just had processes first and they used to say, "Look, it it one downside of processes, is it that the context switching time?" Can be long. Now because the context switch requires taking data, writing it to memory and then reading stuff from memory back into registers. So, there's a lot of memory access and memory access can be slow. So, switching during a context switch between two processes, that can be slow, and you want the operating system to be fast, don't want to waste time doing that. So, the switching process rather, often way to speed it up. People said look, let's make threads. Now, they were called the actually originally called lightweight processes but what a thread is, it's like a process but it has less context. It shares some of the contexts with other threads in the process. So, one process can have multiple threads inside it, okay? This thread share a decent amount of context. So, if you look at the top picture, I show a process and it's got these two blocks of things that are unique to the process, is virtual memory, would page table that sort of thing, file descriptors, it's stack, it's program counter, always there is registers, those are all unique to a process. The nodes I separated them to two, I have those little orange block and the green block, right? All of those are unique to a process. Now, once you start having threads, you can have multiple threads in one process. So, you can see the green chunk, the green rectangle as a thread. So, you could think that first but that process picture is just a process with one thread in it, one green thread and with one code, and each thread is associated with a piece of code that is executing. So, since the stack is associated with the code executing, you got a stack and you got some register, data registers that are unique to thread. But in the bottom picture, what I'm showing is one process with three threads inside it. Now, each one of these threads does have unique context, the stack is unique for every thread, the code is unique for every thread, the data registers are unique for every thread. So, there is unique context between different threads but they also share contexts like that stuff in orange. That's the virtual memory system, that's the same, right? The file descriptors those are the same. So, this is an exhaustive list these pictures but the point though is that these threads, they have unique context but is much less than the unique context between two different processes. So, when you do a context switch between two processes, that might take you a long time because there's a lot of context, this unique. But with two threads, when you go from one thread to the next in the same process, it's much faster, because there's less contexts, less data that you have to write to memory and read back from memory when you do a context switch. So, and now what happens in operating systems is that they instead of scheduling processes, they scheduled thread by thread. So, they say, okay, this thread runs and that's red ones then that one, where in the old days, it used to be process by process now they've gone to this thread level granularity, it's the same problem but the level granularity is different, they do one thread at a time and they schedule accordingly. Now, Goroutines then we'll move into Go. All that other stuff that I talked about, it was generic independent of a particular language, it was talking about operating systems in general. Now we're going to talk a little bit about Go. Go has what are called Goroutines. Goroutines are basically a thread but in Go. Many Goroutines execute within a single operating system thread. So, remember there was this hierarchy where I said look there's a process and it's got many threads in it, that's from the operating system point of view. You can have many threads inside a process, maybe process only has one thread which is called main thread, but it might have multiple threads. Now, what Go does is it says look, it takes process with one thread in it, one main thread and you can have multiple Goroutine that are all executing in that same thread. So, from the operating system point of view, all it does is schedule the main thread but then within Go, you can have multiple Goroutines that are executing within that thread all alternating within that thread. So, maybe Goroutine one executes, then two, then three, or the while from the operating system point of view, nothing is changing, it's got its main thread running. But inside a Go, Go can start switching basically these Goroutines which are like threads inside the main thread. So, that process of doing the switching determining which Goroutine executing and what time, that scheduling process, that's done by what's called the Go Runtime Scheduler. The Go Runtime Scheduler, it does scheduling within an operating system thread. So, you see how the scheduling task is separated between the operating system and the Go Runtime Scheduling. The operating system schedules which thread, which operating system thread runs at a time. Then once the main thread is running, your Go program is running within a main thread, an operating system thread. Once that's running, the Runtime Scheduler can choose different Goroutines to execute at different times underneath that main thread. Now, the Go Runtime Scheduler use a logical processor. This logical processor is mapped to a thread. So, typically like default, you're going to have one logical process are mapped to a main thread and then the Goroutine is going to run on that logical processor which means it's running in that main thread. Then notice it since all these Goroutines are basically running in one thread, one main thread, there is not an opportunity for actual parallelism like this, it's all concurrent. You're executing one Goroutine, or the next, or the next at a time. But what you can do and we'll touch on this later, is you can as a programmer, you can say look, I don't want one logical process I want to logical processors or three or four, and you would do this according to how many cores you had presumably that would be sort of the obvious way. You might say look I got four cores I'm going to have four logical processes. Then what will happen is the Go Runtime Scheduler, it will take these Goroutine and map them to different logical processes, each logical processor can be mapped to a different operating system thread and the operating system combat those, made those threads to different cores. So, you can allow, there is a way to change the setting. So, that you don't have just one logical process and you have multiple, assuming and if you had cores, multiple cores that would allow you to exploit those cores during execution. Now still this task of hardware mapping, determining this Goroutine which core is at on, that task is still not done by the programmer, but the programmer can determine how many logical processors that are going to be used. So, and by default it'll be one in which case it's got to be concurrent behavior but you can increase that if you want to which would allow the Go Runtime and the operating system to do parallel scheduling if it wanted to. Thank you.
