When we talked about influence diagram we included in the influence diagram nodes that represent the agent's utility function and those utility functions, we said, indicate an agent's preferences regarding the state of the world or different aspects of the state of the world. What are these utility functions and where do they come from? So, utility functions are necessary for our ability to compare complex scenarios that involve uncertainty or risk. It's not difficult for a person to say that they prefer an outcome where they get four million to one where they prefer three million. But it's not quite as easy to encode a more complicated preference that allows us to compare the utility of these two lotteries, as they're called. Where the one on the left gives the agent, gave the agent $4 million with probability 0.2 and this the one on the right gives the agents $3 million with probability 0.25. Which of those lotteries do we prefer if we had to make that decision. It turns out that the way to formalize the decisions making process of an agent in this type of scenario is by ascribing a numerical utility to these different outcomes to the outcome of 4 million to the outcome 3 million and to the outcome of $0 and then we can use the principle of maximum expected utility to decide between these two different lotteries. Specifically, we can then compare 0.2 times the utility of the outcome 4 million plus. 0.8 to the utility of the outcome $0 versus the converse which is 0.25, ver, versus the utility, expected utility for the second lottery which is 0.25 times the utility of $3 million + 0.75 * the utility of $0. And we can compare these two expressions and decide whethere we prefer the one on the right, the one, the one on the left, the one on the right or, or they're equally good in our view. Now, it might be natural to assume that utilities should be linear in the amount of payoff that we get so that $5 is preferred about half as much as $10. It turns out that it's not actually the case for most people and one example of that is this decision making situation over here where on the left the agent has the option of getting 4 million dollars with a probability of 0.8 and on the right they have the option of getting $3 million with certainty. Most people tend to prefer the lottery on the right, but if one computes the expected payoff of these two different lotteries we can see that the expected payoff over here is is 4 million * 0.8 which is 3.2 million. Where as on the right we have an expected payoff of 3 million so the expected payoff on this side is higher and nevertheless people prefer the lottery on the right. Another example, very famous example of this type, of this type of preference is what's called the St. Petersburg Paradox. St Petersburg Paradox is an imaginary game that one can play where a fair coin is tossed repeatedly until it comes up heads for the first time. And if it comes up heads for the first time on the N-th cost you get 2^N dollars. So what's the expected pay off in this case? Well the probability that it comes up heads ont eh first toss is half and then you get $2. The probability that it comes up heads for the first time is a, a quarter and the pay off here is $4. A, a pro, third tosses A time eight, times $8 and, it's easy to see that, the expected pay-off over here is infinite. So in principal people might, be willing to pay any amount to pay this, to play this game, because their expected pay-off is bigger than any amount, that there, that they would paying to play, but the fact is, that for most people. The value of playing this game is approximately $2 which is a strong indication that their preferences are not linear in the amount of money that they earn. So, let's try and quantify that, using this notion which is called the utility curve. The utility curve in this case has, as the x axis, the dollar amount that you get. And on the y axis, the utility that an agent describes to that. And now let's compare a few different, scenarios here. So first let's, let's look at the utility of getting $500. So if we go up from 500 to the utility curve, we can see that the utility of this outcome is going to fall over here. So this is going to be the utility of $500. But now lets look at a decision a situation that involves some risk so lets look at a set of lotteries where I get zero dollars with probability one minus P and a thousand dollars with probability P. Because of the linearity of expected utility all these lotteries are going to sit on this line over here, where depending on the value of P, I have a different weighted combination between getting the utility of $0 and utility of $1000. So, for high values of p1, = 1. we'll be sitting on this side of the curve and otherwise for example, for low values of P, we will be sitting close to here. Specifically, what happens if we look at the probability P equals 0.5? Well, in that case, we would have. This point on the curve over here. Now the important thing to notice is that the utility of this point where I get $1000 probability 50% and $zero probability 50%. That utility in this example is considerably lower than the utility of $500. So, I prefer to get the $500 for certain which is what most people would say. Now if we look at what the lottery is worth, that is, the risky version, we can see that, that sits over here and might for example be corresponding to getting $400 with certainty. So that $400 is called the certainty equivalent of this lottery over here. That is, it's the amount that you'd be willing to trade for this lottery in terms of getting that money for certain. The difference. Between these two numbers. The expected reward and the utility of of that lottery is called, the insurance premium or the risk premium. And it's called that because that's where insurance companies make their money. Because, of a persons willingness to take less money with certainty over a more risky proposition. So we can see that this kind of a curve that has this shape, this concave shape is, is representing a risk profile which is risk averse. That is a person is willing to pay for taking less risk. Other profiles would of this, of this curve would represent different behaviors. So for example if the utility was linear in in the reward, that would be a behavior that was called risk neutral. Conversely if we had a curve that looked like this, which is a convex function That would be risk seeking. . And our risk seeking behavior occurs for example in Las Vegas, where one is willing, or in other gambling situations, where one is willing to actually take a loss in terms of the expected reward for the small chance of a getting a really high pay-off. . Now it turns out that people often have a utility curve that looks like the following. So if the X axis is the amount of money that we get and we arbitrarily raise the zero point over here which is at once current state. And we ask how much do you prefer to earn money, and how much do you prefer to lose money? What's your utility for these different changes to one's state? We can see that ones preferences for earning money typically exhibit a form of diminishing returns. Which give us this concave utility curve which suggests risk adverse behavior in the sense that we would prefer a certain amount of we would prefer to get money with certainty relative to the expected relative to the payoff equivalent uncertain lottery. Now, on the negative side of the spectrum many people exhibit some kind of behavior that is actually more risk seeking. Which means that many people would prefer a small probability of a large loss. Relative to a small loss, that you get with certainty and that's a that's a, that's a behaviour that one often sees. More importantly, in this region of the space, which is close to one's current state the behaviors often risk neutral. That is small small losses, or small gains on the order of a small number of, of dollars. And of course, it depends on one's one's base line. are often something that you don't really care about having the uncertainty. And the expected pay-off is often very close to the, to the utility of the expected pay-off. Now one final important observation regarding utility functions is that one's utility often depends on many, many things, not just on the monetary gain. So, in all of the attributes that effect the preferences must be integrated into a single utility function. This is something that many people find very painful because it forces us to do things like, umm, put human life or the loss of human life on the same scale as monetary gain. The point is even if we don't do this explicitly, even if we decline to put human life for example on the same scale as monetary gain, de facto our decisions are indicating that we're making those decisions. So for example when an airline chooses not to run maintenance on the airplane, every single. Time that the airplane lands, that's a financial decision, because that would be too costly. But at the same time, it also definitely increases the chance of loss of human life because of, because of an accident. Now, it's not just, big companies that make these decisions. We make these decisions ourselves so we don't change the tires on our cars every month, or every week, because that would be too costly. But, clearly, having better tires is something that is likely to increase our chances of surviving an accident or a skid. So, these trade offs are ones that we make all the time whether we recognize it or not. And so its important when we think about a decision making situation to list out for ourselves all of the different things that could affect our decision, money, time, pleasure and many, many other attributes and think about how we could bring them together into a single utility function. Specifically in the context of human life people have spent a lot of time thinking about how to bring human life, into ones utility function, and what turns out to be the wrong strategy, in terms of reflecting peoples preferences, is to have the utility for the monolithic event of someone's death, and that turns out to be a very difficult thing to contemplate. What what seems like a better strategy in general is this notion of a micromort, which is a one in a million chance of death. And so one puts the risk explicitly into the utility function. And, and so, what is a one in a million chance of death worth? Well, back in 1980. So, a while ago, People did, this, this study. And it turns out that a micromort was worth approximately $20 of, $20 in 1980 dollars. And, so of course you can account for inflation but it's not a huge amount of money. And that turns out to be a much better way of ranking people's utility for outcomes that involve risk to human life than asking about the utility of death. The second way that people use a medical decision making situations specifically for accounting for human life is this notion of equality, or equality adjusted life year. So each quality adjusted life year, which is a year adjusted for one's quality of life, has a certain utility associated with it which allows it to be compared with other aspects that effect our utility in the decision making situation. One example from a real world situation is in this context of prenatal diagnosis. Where researchers did extensive work in eliciting utility functions that involve prenatal testing. So, relevant variables in this scenario include the, whether the baby is going to end up with some kind of genetic disorder. And specifically, the one they focused on was down syndrome. But at the same time, there's other aspects that effect one's utility so, for example, the pain of testing for Down's syndrome is one aspect. The comfort of knowledge that you know what, what you're going, what's going to happen, is something that also the result of contributes towards utility function. Prenatal testing runs the risk of the loss of the fetus. And that is also clearly a component of one's utility function. And at at the same time, the potential for future pregnancy. That is whether there will be a future pregnancy or not is another component of one's utility function. So, if we think about the space here, the utility function depends in complicated way on a large number on these five variables, and this is fairly high dimensional space over which to elicit to elicit utilities. Fortunately it turns out that many people have a lot of structure in their utility function and specifically they can break down the utility function as a sum of sub utilities just as we had in the context of the influence diagram and for many people that decomposition looks like the utility of the testing. The a separate component for the utility of the peace of mind of knowledge, and then we have. These two pair-wise utility terms, the first of which is a term that depends simultaneously on Down's syndrome and the loss of the fetus. And the second is, the utility that depends on the loss of the fetus and the potential for future pregnancy. So people's utility function for many people, decomposes in this way, which, it turns out, we can actually think about as a graphical model that has singleton terms, as well as these, pair-wise terms over here. And that allows us to considerably reduce the number of terms that we need to list in order to get a usable utility function. So, to summarize our utility function is what we can use to determine preferences about decisions that involve risk or uncertainty. in order to define or elicit a utility function, we generally need to consider multiple factors all of which affect our utility. In most cases, the relationship between these different factors, the. Between say money and the utility or, or micromorts and the utility. This relationship is usually a non-linear one and the shape of the utility curve determines one's attitude towards risk. Finally, the actual utility function is usually a multi attribute utility that integrates all of these different factors. And it often helps to decompose this utility function into tractable pieces, often as a sum of these pieces which allows us to make this elicitation problem much more manageable.
