Earlier this year, Dr. George Kaplan, a pioneering social epidemiologist and expert in complex systems modeling, spoke with Dr. Ellicott Matthay about complex systems models in population health research. His insights are especially relevant now, as researchers grapple with how to best inform policy decisions to reduce the impact of COVID-19. Infectious disease modeling – describing the spread of disease and the consequences of hypothetical interventions – is a domain where complex systems modeling has seen many applications. Dr. Kaplan's work has emphasized potential applications of these same tools to a broad set of problems, including social determinants of health.
EM: How would you describe complex systems modeling briefly to non-researchers?
Dr. Kaplan: Complex systems modeling approaches attempt to deal with the multi-level, dynamic, and multi-faceted nature of real-world phenomena that determine a variety of health outcomes and inequities. Complex system modelers try to capture the network of causes and pathways that come together to determine health outcomes – the opposite of what you might think of as the magic bullet approach, which tries to look for a single determinant of a complex problem. Complex systems bring together these considerations into a simulation model that allows you to explore how different outcomes arise from various factors all working simultaneously and potentially influencing each other.
In fact, all the types of analyses that we do, whether they are regression or some other method, are really all attempts to understand the “what if” question. That is: what if we changed the distribution of some risk factor or intervened in some particular way? They are simulations also. So, in that sense, complex systems approaches are consistent with other approaches. Where they differ, I think, is in the understanding and the embracing of complexity. You get a more comprehensive answer that applies to the whole population.
EM: Why would someone consider using complex systems modeling over other more traditional regression approaches, specifically to study social programs and policies?
Dr. Kaplan: First, we know that there are many policy relevant questions that cannot be addressed with randomized controlled trials. There are also many situations where the results of randomized controlled trials don’t match up with what we observe once we trot out a policy or intervention. What’s interesting about complex systems models, I think, is that they are not attempts to isolate single causes, but rather to understand how things come together and how in some cases the whole is greater than the sum of its parts. I think it is the recognition and embrace of the multitude of things operating in the real world or a simulated world that distinguishes complex systems modeling from most other approaches.
EM: In terms of the questions complex systems models can answer, it seems that there are a pool of questions that can be answered around how social systems lead to poor health and health inequalities, as well as a separate pool of questions that can be answered around deciding among different policy options. Is that an accurate characterization, and are there other categories of questions that I missed?
Dr. Kaplan: I think complex systems models have the potential to address both kinds of questions. For example, consider Christakis’s work on social networks in the Framingham study on obesity. We could consider creating a model which puts in place what he’s talking about: What if we start manipulating the first order, second order, third order, and more distant connections people have? What if we clustered people like they are in communities in different ways? How would that affect the outcomes? So, etiologic questions can be addressed within the model. The problem is these models are very difficult to put together, and there are a lot of decisions that have to get made in order to put those models together. The devil is in the details, but that’s true for any kind of analytic investigation you might do.
A variety of questions can also be asked about the potential impacts of different policy options, within a properly constituted simulation model. The problem is getting a model that everyone will agree is a properly constituted one.
EM: Would you agree that with complex system models, because there is more flexibility, there is also more room for wrong decisions?
Dr. Kaplan: When building a complex systems model, everything has to be made explicit – there is no hand-waving in the assumptions. So in a sense, that exposes the model to greater scrutiny. It is also true that the greater the granularity of a model, of an approach in general, the more room there is for agreement or disagreement about the way things are characterized. With increasing granularity comes the burden of exposing all the assumptions that you are making, which may not be such a bad thing.
EM: Are there questions that complex systems modeling shouldn’t be used for?
Dr. Kaplan: There’s a tradeoff between the complexity of the model – the weight of the assumptions you have to make – and of the utility of the model. If there really is no available evidence on which to base the model, we can’t really be very confident about the conclusions that come up.
EM: Are there any other considerations or limitations of complex systems modeling that you think are important to note?
Dr. Kaplan: There are several considerations that go in to developing a complex systems model. The primary consideration is asking the right question. Others include, but are not limited to, shared language and scholarship and data suitability issues. A shared language is particularly important when developing simulation models that cross disciplinary boundaries. Useful models should also be as simple of possible and not simpler - sort of taking off on Einstein’s quote about “everything should as simple as possible but not simpler”.
I think another major issue is a training issue. We don’t have the collection of expertise that brings together, say, population scientists and computer scientists to really use these tools in the most fruitful way. I think it requires both kinds of sensitivities.
EM: Are there any particular examples of complex systems modeling work that you have found to be particularly compelling, high quality, or influential for policy or practice?
Dr. Kaplan: I think the most influential examples probably have to do with infectious disease interventions and immunization. They’re able to model what happens if you have people stay at home versus going to work, for example, in terms of the spread of the flu. In another example, we tried to model Black-white disparities in BMI. The model uses education, neighborhood characteristics, social networks, physical activity, diet, quality of food – a whole variety of things – and then creates a number of scenarios. In one scenario, we shift the distribution of educational attainment upwards, and in 20 years, eliminate Black-White differentials in BMI. There are a lot of assumptions in it, but all the parameters are based on existing studies. I was pretty impressed by that.
EM: At Evidence for Action, we are seeking to fund research that will deliver definitive results on whether a particular program, policy, or practice, or system can improve population health. How actionable do you think the results of any individual complex systems modeling study are, and are there ways to make complex systems studies more actionable?
Dr. Kaplan: I think that question is probably applicable to just about any method. I’d say, there are a couple of things. One is to involve the community when building population models. This tends to make them more realistic, because they include people’s experience. The other thing that makes them more potentially useful is to base them solidly, where possible, on existing data and plausible interventions. For plausible interventions, it’s actually kind of a dual-edged sword. On the one hand you don’t want to be stuck in what’s possible, but on the other hand you have to ask: what can we possibly do?
EM: Is there anything else you think is important to emphasize when considering complex systems modeling as an option for understanding the health effects of social programs and policies?
Dr. Kaplan: There is a universe to be explored here using these modeling techniques. It would be a mistake to get into a war about which is the right method to use, because we know that there are many ways to look at things. Complex systems provide an additional window into complex phenomena and if we believe, as I do, that most natural, social, and physical systems are very complex, then to ignore that complexity is to miss quite a bit of the picture.