First, look at the data
I recently received an e-mail from a college student, someone I’d never heard of before, wondering whether system dynamics was the right approach for studying a particular issue: how the government’s publication of secondary school rankings in his country might influence parents’ attitudes and behaviors. The student wondered whether I could give him “some advice on creating equations in SD”.
I’m not sure how this student had heard of system dynamics, but it was clear he had never built a model and was thinking of SD as a piece of software, perhaps something like a simple spreadsheet program. As much as I believe in mentoring (see my April 2019 blog post), I was not prepared to give this fellow much of my attention and wanted only to steer him in a useful direction. I advised him to look into SD resources for beginners, including textbooks and the pre-conference Summer School. I also advised him to gather data on his issue of interest and do some statistical analysis.
I want to talk about this second piece of advice and why I gave it. For more than 20 years I’ve written about the importance of evidence—both structural and behavioral—as a foundation for reliable and scientific SD modeling. I think that every SD model should start with a scan of the evidence surrounding the issue at hand, including gathering as much historical data as possible. Any theorizing should emerge from this evidence, rather than from preconceptions and “common knowledge” that may have little or no basis in fact. Statistical analysis can be useful in exploring the data, if done and interpreted carefully. As statistician Edward Tufte said, “Correlation is not causation, but it sure is a hint.”
As a master’s student many years ago, I took a class in exploratory data analysis. The professor told us, “First, before you start theorizing, look at the data. Plot it all, look at the relationships, and think carefully about what you see. This is the best way to avoid going down the wrong road.” I took his advice to heart, and I think it would be good for our field if we all did so. Let’s look before we leap—it’s the scientific thing to do.