Welcome back. In this section we will continue to talk about causal loop diagrams, focusing particularly on the type of information that is used to develop them, as well as the strengths and weaknesses associated with this methodology. First we will talk a bit about the types of information and sources of data that you need to determine a causal relationship, including the direction that the influence is going in, and also the polarity, or how one variable effects the other. So you can find this type of information in the literature, perhaps there are articles that have published on the topic that you're interested in. You can also check secondary data sources, and some authors actually collect primary data, both qualitative and quantitative to develop a causal loop diagram. Often the data sources are actually mixed and the process of putting it together often produces knowledge on its own, so causal loop diagrams can be developed in a participatory, iterative process. And it is not to underestimate how much information can actually be gained through that. Because you're mixing data sources and because some of the information will come from your own participants, it is important to document where you get this information from. And it is also important to understand that often, you can refine the diagram, even in the absence of physical data. You might not have data about a specific relationship, but it is helpful to discuss it nonetheless with your stakeholders and arrive to at least a hypothesis about the direction of causality. And documentation of the data sources will be also very important, since you will be using causal loop diagrams for displaying mental models and potentially also for scenario building, so it is important that the data that you use doesn't need you to specify the relationships that you're interested in. Although as mentioned before, data isn't always necessary and it's not about obtaining precise point estimates, but really about understanding feedback and the dynamics of the topic at hand, or the dynamics of the relationships that you're interested in. Here are two examples of recent articles that use various data sources for the development of the causal loop diagrams. These are two articles that are also found in your course resources. On the left hand side, you will see an example from Agnes's work in Uganda on understanding the dynamics of neonatal mortality. And you'll see that what she has proposed is a six stage process which includes both some preliminary work, both working with stakeholders and using existing data. It involves a formative phase, which includes field studies and it also involves actual case studies, empirical investigation using real life context and data collection to verify the model relationships. It's worth noting that the causal loop diagram is developed after the first and second phases and that the relations can be further defined through that, but that the case study and the empirical investigation will especially be feeding into the system dynamics model, which will be discussed in subsequent lectures. On the right hand side you will see an example from my own work which was looking to explore dual practice and its management in Kampala, Uganda. And in this research the main question was, how did dual practice management develop and adapt in Uganda? And to develop the causal loop diagram, I used mostly qualitative data, which was collected through in-depth interviews. And for historical events, I also used secondary data, which I tried to piece together from various accounts of dual practice in the private sector in Uganda. And as you see it's really part of a larger study, so it's embedded in a larger study. These are two examples, but if you look in the literature, it's worth noting and seeing the various ways in which authors develop causal loop diagrams and also how they combine various sources of information. As I mentioned earlier, documentation of the data sources and how you obtain the relationships in the causal loop diagram is very important. And what's also important is to actually verify the relationships in your causal loop diagram. And when you are thinking about trying to establish rigor and trustworthiness of the data that you're using, you should apply the same criteria that you would if you were doing any other type of research. For example, for evaluating trustworthiness in qualitative research, a key resource is Lincoln and Guba's four criteria of credibility, transferability, dependability, and confirmability. So there's one example of evaluative criteria that you can use to establish trustworthiness. In Agnes's work, we see also a process of stakeholder validation, which is similar perhaps also to member checking. So through this process she breaks down her causal loop diagrams into smaller loops so that they're easier to understand and to evaluate. Then she facilitates brainstorming sessions to test for the clarity of the model, whether the variables exist and whether the representation of the relationships is accurate. And you can see in the box below an example of one small loop, and an associated verbal statement. So stakeholders would be presented both with the image and the verbal statement, and they would be asked to discuss and see whether it is accurate, what information is missing. So this is one approach to also engaging your stakeholders and continuing to engage them in a participatory fashion through the entire phases of developing and refining your causal loop diagram. Although it is important to note that when doing stakeholder validation, it's important to also reflect and document your observations on power relationships and potentially political influences that might occur when you're engaging a variety of stakeholders. So as in any qualitative research, it's important to know who the information is coming from and to document it in a reflexive way. But really, the purpose of doing this is both to document where the data is coming from and to continue ensuring rigor, because the main thing that we try to avoid in doing so, is to mis-specify the relationships that we're interested in. So we discussed where the data comes from and what types of information one uses to develop causal loop diagrams. Now I'll move to discuss the strength and weaknesses of the causal loop diagram methodology. So first on the strengths, causal loop diagrams are useful because they help to provide an integrated view of the system through illustrating internal mental models and making them explicit. And in doing so, they can help to reveal feedback that is unexpected and also potential areas for further exploration. The way that it has been used in public health, causal loop diagrams usually complement traditional data collection and analysis, and it helps to make use of available data, even when it is perhaps incomplete. But it provides a platform for theorizing, developing hypotheses for untested relationships, and also scenario building. So for example, if you identify a feedback loop that is undesirable, you can perhaps hypothesize about changing a relationship or introducing a policy that would change the feedback, and you can build scenarios based on that. If causal loop diagrams are developed in a participatory fashion, they can also help to facilitate consensus-building and brainstorming, and they help a group of stakeholders arrive at a shared understanding. And in the process, also owning the causal loop diagram, so it gives them a sense of ownership. This is particularly important if you're interested in perhaps building on it with a system dynamics model, which involves a more extensive process, and ownership and stakeholder participation will be important for that as well. And causal loop diagrams, as it can help engage stakeholders, it also can bring together different perspectives and data from different disciplines. So it provides a common platform and a common language for people from engineering and public health, for example, and policymakers to sit together and to formulate a mental model that they agree upon. And as I mentioned before, they can be used as a foundation for quantitative modeling, especially stack and flow diagrams, which will be discussed in subsequent lectures. In this slide is a repetition from the first section of this lecture and I'm just putting it up here to remind you that causal loop diagrams can have different roles at different parts of the research and policy process. This methodology however does not come without some challenges, and I hinted at some previously, but to summarize them, one of the challenges with causal loop diagrams is its generalizability. And this is because the diagram on its own cannot really capture the context. So while it is important to have a diagram, it is also necessary to have an accompanying narrative and tables to describe the assumptions, the process and the context. And often times, you can't necessarily duplicate or replicate a diagram from one context to another. Another issue might be that participatory development of a causal loop diagram can be challenging. It is difficult sometimes to blend perspectives and assumptions, especially when there are power dynamics involved and there can be tensions about multiple ways to frame the system. As you'll remember, there can be multiple causal loop diagrams, so it might be difficult to reach agreement on one particular diagram to display an issue or to bind a problem. Then finally, there are issues with the diagram itself. It's not a perfect tool. As I mentioned, it is necessary but not sufficient on its own. The variables, relationships and polarity can be ambiguous, especially if they're hypothetical, theoretical relationships. And there can also be missing causal factors or missing links, both due to lack of data and also because the system needs boundaries, so a causal diagram inherently cannot display everything. There might be non-causal relationships, so we might make mistakes about the causality that we determine whether it's based on data or not. And the validation process, which is really a process essentially to ensure rigor, can be difficult and potentially time-consuming. And causal loop diagrams might be difficult to communicate if they're too complicated. So, while it's difficult to have missing factors, if you try to include everything in your diagram, it becomes what is colloquially known as a spaghetti diagram, which looks like this. Where although the relationships might be well-specified, it's quite difficult to understand, especially for stakeholders that weren't part of this process. So it's important to reach a balance between the participatory process, how one draws boundaries around the system, and how one specifies the problem, so that the diagram itself can also be used for communication purposes. This concludes the section where we discussed where the data comes from for causal loop diagrams. And also where I presented some of the strengths and weaknesses of causal loop diagrams. We look forward to discussing these further in the live talks that are upcoming. Thank you. [MUSIC]
