Why we do not always simulate
By Vincent de Gooyert, Inge Bleijenbergh, Hubert Korzilius, Brigit Fokkinga, Monic Lansu, Stephan Raaijmakers, Etiënne Rouwette, Merel van der Wal
With great interest we have been following and participating in discussions on what ‘good’ and ‘appropriate’ system dynamics modeling is. The system dynamics society, including its conferences and the wiSDom blog provide a stimulating environment for such discussions. However, we noticed with growing discomfort over the last few years a more and more vocal expression of two axioms in these discussions: the idea that every system dynamics project should be accompanied by the collection of quantitative data, and the idea that every system dynamics project should be accompanied by a quantified computer simulation model. The main argument being that not including these two elements is detrimental for the project’s rigor. We fully support the continuous strive for rigor, but we contend that system dynamics projects without quantitative data and quantified computer simulation models can be just as rigorous. In this memo we explain why, in an attempt to advance the debate.
Our main argument is that the answer to the question what ‘appropriate’ system dynamics modeling is, depends on the research question being asked. For some research questions, quantitative data and models are indispensable. However, for other research questions they are not relevant at all, let alone that they would increase rigor. We think that the call for data and models comes from overlooking this latter category of research questions. We are hesitant to use terms as ontology and epistemology, as these terms often come with a wealth of obfuscating language, but we believe this is exactly where the views differ. We see a group of system dynamicists with a positivistic stance striving for rigor in terms of quantitative data and models, and a (smaller) group of system dynamicists with a constructivist approach working on a research agenda that is substantially different.
In the positivist research tradition, different methods are used to get ever closer to an objective reality. Triangulation is an important means to increase validity and computer models and quantitative data provide great ways to check the validity of mental models of decision makers. After all, there is a myriad of biases and heuristics that distort mental models, and the human brain is notoriously incapable of calculating the implications of feedback, delays and non-linearities. In the constructivist tradition there is less interest in the ‘essence’ of objective reality. Rather, it focuses on the meanings that people ascribe to their environment, and how this socially constructed meaning continuously is both the cause and consequence of their behavior. While system dynamicists with a positivist stance triangulate to decrease subjectivity, system dynamicists with a constructivist approach are interested in exactly that subjectivity that positivists try to avoid. Perceptions are valuable, not as a distorted image of reality, but as a complete answer to a research question as for example ‘what meaning do members of an organization ascribe to a particular problem?’ Collecting quantitative data and running computer models are not helpful to answer such questions, it is even completely irrelevant as this would take us further away from an answer to our question how people perceive their environment.
Group model building can be a very exciting part of SD research with a constructivist approach, as it allows researchers to witness the social construction of reality first-hand. Mapping the structure of a system is a great way of sharing worldviews, surfacing and resolving conflicting worldviews, and collaboratively making sense of an equivocal environment. Critical studies, a subgroup of constructivist research, explicitly give voice to worldviews from minority groups. An important aspect of understanding social reality as socially constructed is that not all worldviews get equal space and attention. Powerful actors tend to dominate the social construction of reality. Group model building is one way to challenge the hegemony of dominant worldviews by being inclusive in terms of the stakeholders that are heard and the space each stakeholder gets to contribute to the discussion. This may help to challenge taken for granted assumptions. Sometimes collecting quantitative data supports putting dominant worldviews under discussion, but it is good to be aware that these dominant worldviews affect how qualitative data is translated into quantitative data.
Some suggest that deciding whether to translate a qualitative model into a quantitative model is a trade-off between additional resources and additional insights: is the added value of the quantitative model worth the additional investment? We find it important to note that our objection against the call for quantitative data and models is of a different nature. As stated, for us the difference is in the research questions. Some research questions require data and model runs, but if the research question is about the perception of reality itself, these tools add rigor nor relevance. The terms rigor and validity get a different meaning when used in constructivist research. For us, validity is not about ‘is this model a good representation of objective reality’, but about ‘is this model a good representation of how these participants give meaning to their reality’. There are various ways of increasing confidence in the validity of our models in this sense, including giving participants ample room to adjust the model, and regularly asking the question whether the participants see their own worldview represented in the way it is visualized.
A positivist system dynamicist may respond to our argumentation above by saying: I agree that the confusion stems from a difference in the kind of questions that we try to answer, and to avoid that confusion I would like you to refrain from using the labels ‘system dynamics’ and ‘group model building’ when doing research in the constructivist tradition, let’s confine the use of those labels to ‘proper’ positivistic system dynamics. To those we say: no can do. After all, we are interested in the dynamics of systems, and we build models with groups. We are educated, motivated, inspired by and emotionally engaged with the rich past and future of system dynamics and use these terms to connect to it. We would find it worrisome if dogmas would prevent researchers from answering new types of questions, and from creatively using parts of our body of knowledge for their ends. It might be confusing to have both positivist and constructionist flavors of system dynamics, yet it is also enriching. After all, system dynamics is a field in which we all work with and for people who are primarily meaning giving creatures. The language of system dynamicists with a constructivist approach is still evolving and we are discovering what it is that we do while we do it, we welcome any reflections that readers might have. To all system dynamicists with a constructivist approach we say: let’s work together and collaboratively improve our understanding of the opportunities and challenges of qualitative and quantitative system dynamics, and become more proficient in describing and justifying our approach.