Hi I'm Professor Paula Lance and welcome to a new module in our series and we're now going to be focusing on program evaluation and why it matters so much for public policy and public administration and the important role of data analytics in the process. Here's our roadmap for the next few weeks. First, I'm going to introduce you to different types of program and policy evaluation. To do this, we're going to use a program evaluation framework and I'm sorry but there's just a lot of jargon and terms we're going to have to define and then we'll use them a lot. Also, we will talk in detail about the important role of data analytics in program and policy evaluation. We'll talk about some common designs used and associated data analysis and data visualization techniques. And also, we're going to have a lot of real world examples and a guest. All right, so what is program evaluation? We talked about this a little bit already. It is a descriptive or retrospective type of policy analysis. And this means that we're not as we talked about before with prospective policy analysis, we're trying to predict what would the future look like if a policy or a set of policies were to be implemented. With program evaluation, we're asking did a public program policy or other kind of intervention that was implemented work. It can involve analysis of the implementation aspects of a program user experiences but also more commonly it focuses on the impact and the effectiveness of policies and programs and interventions, their costs. And also, it's important to look at different kinds of ethical and value issues, inequities that might arise, etc. So why is program evaluation so important? Well, first of all, let's review our four pillars of public administration. Program evaluation has to do with accountability to these four important pillars. First of all effectiveness, government needs to know if the kinds of policies that are being implemented and reformed programs that are being implemented are working well. Are they meeting their goals and also to make sure they're actually not hurting people. Also, the pillars of economics and efficiency, of course, are so important. And government were always working in the scenario of scarce resources. And so, we want to make sure that these resources are being used and allocated efficiently, meaning that there's value for the amount of resources that are being expended to achieve a specific goal. And then also, equity, always an important pillar. And again, we need to know if programs or policies that are being implemented are raising any kind of equity issues or if they're actually meeting the goals of reducing inequities. There's some other reasons that program evaluation is really important to what government does. First of all, it can bring positive attention through the media, but through other venues as well to selected policies and programs, it's a great way for government to communicate with the public about what it's doing. And also, program evaluation is important from a research sort of perspective, it can help contribute to theory, development and a number of different disciplines, empirical research and really high level evidence based policy design and dissemination. All right, you've seen this before but I'm going to remind you of the important role of data analytics and the policy process and how policy analysis comes into play. So as you might recall the first step in the policy making process is the assessment of conditions when we use data to do that. And it ties back to those core functions of public administration, all seven of them, but especially for the functions of reporting and planning and directing. Okay, the second step in the policy cycle is an assessment of policy options. We've already talked about the important role of prospective policy analysis and policy simulation modeling for forecasting what might the world look like if certain policies were implemented. Then policymakers have to make a policy choice and again, prospective policy analysis. And we've done exercises in this policy options analysis. The role of data is so important in comparing and contrasting different policy options and making the choice then policy implementation and that's what we're going to be focusing on here and also policy evaluation. This again, is retrospective policy analysis. So for the policy implementation, we can engage in what's called process evaluation or implementation evaluation or implementation science. And then for the evaluation part we will be engaging in what's called outcome or submitted evaluation. Don't worry, I'm going to define all these terms and give you plenty of examples for all of these different kinds of evaluation. But let's go through an example quickly with the hope to really underscore you, how important this kind of policy analysis, policy evaluation is to what government does through policy. Let's take the example of tobacco excise taxes. Now, as we talked about before, this is an example of a formal policy using the tax policy lover and it's based on the theory in economics that there's a negative price elasticity of demand between price and tobacco consumption. And in this case it means a unit change in price as a corresponding negative impact or decrease in consumption of tobacco products. So important policy question is, what's the level of taxation needed to make the price of tobacco products have a price elasticity that actually produces a public health benefit. So the price of a pack of cigarettes varies greatly around the world. And that's because different countries or states or provinces have implemented different levels of taxation onto the price of cigarettes. Well, dozens of tobacco tax policy evaluations have been conducted. Most of them have been done using what we call a time series analysis to estimate the change in demand following different levels of price increase through taxation. The overall findings do show that price through taxation decreases demand. Just what economic theory predicts. However, the price does matter, the price elasticity does vary by tax level. So, it's been shown in some research that only 38 countries in the world, covering only 14% of the global population, actually have tobacco taxes that are sufficiently high to have a public health impact. Which is defined as taxing cigarettes to cover at least 75% of the costs of tobacco-related health care, loss of labor and other costs to the public. This is research done by the World Health Organization. And so, the World Health Organization also recommends that countries or sub country units, states, provinces, etc, raise their tobacco taxes to a level that will meet the public health impact goals and do that as a way of part of a broader strategy. And the point here is that it's not just simply about the tax, but they also have based on evidence policy recommendations for how to implement that tax. And the World Health Organization suggests, do make it simple, don't price different tobacco products like cigars or smokeless tobacco or vaping products which are nicotine products, not tobacco products or cigarettes differently, have a simple tax structure. Ensure that the taxes decrease affordability by also accounting for the impact of inflation and economic growth. Already made that point. And so to sum up the point is that this type of policy evaluation research is so important for recommendations and really understanding how policy can have the most value and the best impact on its goals
