skip to content
Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors
Search in posts
Search in pages

Go Home

Modeling for Implementation: an Illusion of Control?

Modeling for Implementation: an Illusion of Control?

Every so often, I see someone in our field drawing attention to the fact that many client-based system dynamics modeling projects do not lead to effective implementation of their findings, and they wonder what can be done to fix this problem.  This is not a new idea, but rather one that has been expressed for more than 40 years.

The first important example was Ed Roberts’ 1977 paper, “Strategies for effective implementation of complex corporate models”, which advocated extensive client involvement, substantive real-world detail and metrics in the model, actionable findings, and supporting the client team into the implementation phase. 

Thirty years later, in 2007, came an even more assertive article from Andreas Grössler, “System dynamics projects that failed to make an impact” (System Dynamics Review 23:4). Grössler argues that System Dynamics projects should be viewed as nothing less than organizational interventions, involving both analysis and action planning. He asks for more research to be done on how modeling can be smoothly integrated into large-scale organizational change efforts.  

Jorgen Randers later amplified this message in his 2019 paper, “The great challenge for system dynamics on the path forward: implementation and real impact” (System Dynamics Review 35:1). Randers says that we need to focus much more on the question of implementation and to develop an effective implementation methodology to stand alongside our modeling methodology.

These are important thoughts on implementation, and I do not doubt the authors when they point to cases of successful implementation as evidence that we can do better.  I must confess, however, that I get uncomfortable when someone speaks of modeling as merely the lead-in to the broader task of organizational change.  I ask myself, “Have I not set my sights high enough?  Why can I not see beyond today’s modeling to the more profound organizational reverberations to follow?”  

But this is silly. Over more than 30 years of private and public sector consulting (often side by side with major-league consulting firms), I have never witnessed profound organizational change happening, not even with longtime clients. I have never expected that any of my models would lead to a cascading change of organizational structure or culture. I really only expect to complete a high-quality model that the client finds useful and clear.

Why would I expect more than that? What ends up happening after the model is completed is largely out of my hands.  I was not trained in organizational intervention, and even those who do claim expertise have a mixed track record.

I have decided that it is sufficient to do a good job of System Dynamics modeling for the client without any particular expectation of implementation afterward. Developing a useful model takes all of my attention. Neither the client nor I can know with certainty what the next step will be after the current modeling project is complete, nor in fact whether this work will continue to be supported by the organization.

So, to answer the question in this blog’s title—yes, I would say that a System Dynamics modeler who believes he or she knows how to maximize the chances of implementation is under an illusion of control. Changing an organization is an incredibly difficult undertaking, and it seems presumptuous to think that one’s modeling skills extend into social engineering or executive leadership. 

I plan to stick with my only real skill set, which is modeling for clients, and to keep trying to do a better job of it. The only way I “model for implementation” is by satisfying the client and putting them in a position to take a more confident next step, whatever it may be.

Recent Posts

Upcoming Events

2024 Student Chapter Monthly Meeting

2024 Student Chapter Monthly Meeting

Join us for the Monthly Meeting of the Student Chapter at the System Dynamics Society! This virtual gathering provides an excellent opportunity for students passionate about system dynamics to connect, share ideas, and collaborate on projects. Access the session via...

Recent Business cases

Join us

Webinar Q&A | Local Level COVID Models

Webinar Q&A | Local Level COVID Models

We had an insightful Webinar with the participation of The COVID 19 Localisation Modelling Group. Kim Warren and Maurice Glucksman and the formidable youth Farrah Farnejad, Quinn Kennedy, Harshita Magroria, and Brahmani Nutakki joined us for a great discussion about the positive impacts of using a localized System Dynamics model to understand COVID realities locally.

You can watch the webinar recording here and download the presentation here. And don’t forget to check the Understanding Your Local COVID-19 Outbreak FREE course that allows you to apply the model to your own locality!

Here are the answers to questions asked live during the Webinar. Please refer to the event page to get more information about the project.

Learn more about the Seminar Series.

Q&A Webinar Local Level COVID Models: Bringing Youth to the Table

Answers by Kim Warren and Maurice Glucksman

  1. Do any of the models consider bigger system impacts on the health of the community and the economy?

These are big important issues but beyond the scope of what we are doing. Our work on mass localizing is starting to address some of the macro issues but only as it relates to the pandemic. Our collaboration with the Emergent Alliance is aimed at wider societal impacts on this and Kim especially is currently engaged with the NHS on Mental Health issues as they relate to the wider impact of COVID-19.

  1. Can you repeat what equation you used to graph/visualize the map?

The data we use comes from publicly available sources that produce the maps.

  1. How can this model help decision-makers make better decisions?

Good question.  We deliberately focused on young people who are captains of their ship and uniquely in this pandemic their actions and decisions are very important for reducing the impact on others through asymptomatic transmission  — even though as a group they are less directly impacted because of severe illness and death in young people is a tiny fraction of older people. So if you define decision-makers as the largest group of people impacting others then it’s very big and it is starting to percolate through the schools and social networks. If you define decision-makers as the traditional healthcare policy organizations it could also be big but not yet.

  1. What data needs to be available to fit a localized model?

Simply 10-year age-group population numbers, and daily series for reported cases, deaths, and hospital cases. You also make estimates of some localization-specific differences. This is all documented in our free webinars and course.

  1. What do the models tell us about the longer-term outlook?

The models tell us radically different stories for every local area and the story is hugely uncertain because of reinfections and mutations. In some cases, the pandemic is over, as the slums of Mumbai, Here in London we are just getting started but vaccines might save us. There is no one story except that you can’t analyze or manage the pandemic well at a national or even state level.

  1. Can we see the model structures? What software did you use?

This is all freely available and documented in our webinars and course.

  1. Quinn — Did you update the model based on your learning to see if you could successfully predict what happened from Jul-Dec 2020 case? And what policies would you recommend going forward?

You need to ask Quinn – will pass this to him. We are constantly revising the model as new information comes in. The pace of revisions has slowed but continues.

  1. Could this be applied to other areas of concern say Climate Change or Food Security to demonstrate inequities?

We have used the principle of localization in Supply Chain, Operations, Marketing, Sales, Strategy so we know it is widely applicable. Issues like Climate change are global by their nature. However, our approach can certainly be used to tackle widely-found environmental challenges – a generic model of common issues, easily configurable to specific local conditions. Others in our field are doing great work on such issues!

  1. Does the definition of local ever change because of events?

Yes. If you look at historical examples of pandemics the maps show each pandemic is a dynamic self-organizing animal without a brain flowing like a swollen river breaking through its banks and spreading out across the landscape influenced by topography, demographics, infrastructure. Inevitably the definition of a local area changes as that happens. Our challenge as analysts is to select a local area that is stable enough to get insights about policies that will help.

  1. How are you defining youth? Including young adults?  What’s the age range?

See question 3.  Youth is our focus because they are important and have been disenfranchised in the pandemic. There is no barrier for anyone of any age to use the models.

  1. Do any of the models include vaccinations and their consequences?

Yes. You can see examples of this analysis in the Westminster presentation.

  1. Farrah — You did a fantastic job. When calibrating your model what data did you find most difficult to find and you missed the most?

You have to ask Farrah! I will pass the question to her but short of that in our Webinars, we go through the data quality issues in detail. The least reliable data is paradoxically the most widely reported: new cases. This is perhaps partly why unfortunately there are so many problems managing the pandemic — it’s like trying to drive your car by looking at google maps on your phone with bad reception and ignoring the view of the road outside. Opportunities for misinterpretation are constant. Death rates are mostly more reliable, but still under-reported to varying degrees, often substantial! The most reliable information is seroprevalence surveys – but these are very occasional [no daily time-series] These allow us to ‘back into’ what must be happening with the stock of susceptible and total stock of infected people. The WHO, CDC, etc will tell you these are the key high leverage drivers of strategy and management of the pandemic but rarely if ever reported.

  1. Have local public health authorities and policymakers who have an influence on policies that affect these communities been involved in the discussion of these results?

This has been limited so far partly because of our focus on young people who have to work hard to win the trust of authorities before their views are taken into account, but it has started to percolate through in surprising ways and we are optimistic that over time the Greta Thunberg effect may take hold in the pandemic.

  1. Many people think that a more effective vaccine is better for them. Is there any way to show that a more rapid roll-out of a less effective vaccine is of greater benefit to everyone?

Farrah is working on a one vs two-dose strategy for Westminster and other boroughs in London. Our working hypothesis is there is no one-size-fits-all strategy. To see why it’s useful to do a thought experiment imagining two districts right next to each other (like Mumbai) where one district has many susceptible people and the other hardly any. The best dosing strategy in these two districts is highly unlikely to be the same.

  1. Can this be broadened to be a ‘system of systems’ where local models are threaded together to look at the local effects, compounded together / interacting?

This is one objective of the ‘mass localization’ project we are working on now.   There will be ‘meta’ interactions between locations that we can’t see working from one location.  It’s the forest for the parable of the trees.

  1. Good to have such a worldwide community for modeling important material

The insights from every location are informing the others. We believe this is very powerful and we hope it is creating an enduring network between young people that would never have developed otherwise.

  1. This is impressive work by the young team led by Prof. Kim. The model has been validated or not?

The model has been validated in dozens of locations globally and provides good insights into radically different locations. Data quality is always a major issue.  Validation is ongoing updating parameters as new mutations emerge and regional differences in the proportion of asymptomatic infections as well as the varying impact of differing vaccines. Occasionally new structures emerge — a good example is a migration that occurred last summer in the Dharavi slum — we could not replicate the pandemic outbreak without allowing half the population to leave the slum. Our belief is no possibility of correctly validating any model of the pandemic that is not local and not developed in collaboration with people actually living in that location or at the very least with excellent access to people who are living there. 

  1. We see so much data on a daily basis, without it being connected in the way models such as this work. Do we feel there would be a benefit if models such as these were more widely shared, and what may those benefits be? 

Tough question… we don’t know. We are sharing our work as open-source with the hope that it will help. The evidence is it has helped our students. They have spoken in this presentation about how the modeling helps them not only understand the pandemic but it has spilled over into other areas in their educations and the way they think about the world. Whether that will have long-term benefits it’s hard to say. I think from our perspective, working on this project has helped us stay sane in what would otherwise be mind-numbingly boring lockdowns, and from a selfish perspective has been a great learning experience and a great way to mobilize our personal networks to have a positive impact on society.

  1. Maurice & Kim – great work! Any tips on how to engage youth effectively?

Thank you. Many tips but mainly motivation. Our key resource was paradoxically having Covid 19 as a research assistant — that motivated all of us to try to solve this and we learned a great deal from the virus about how to make a problem-solving effort viral. It may be possible to leverage other networks, e.g. teachers, but they have had other pressures from COVID.

  1. Wondering about the potential of applying this localization principle could be applied to Doughnut Economic metrics linking social foundation metrics with ecological ceiling metrics to demonstrate complex relationships.

Apparently, donughts are a big reason why diabetes is a major global issue — that’s a complex chain of interaction we can blame on runnin’ on Dunkin’ and Krispy Kreme 😉

  1. Could use it for predicting the new COVID variant found out in Amazonas, Brazil?

We are working with Guttenberg Ferreira Passos and Niraldo Nascimento right now. Suggest you reach out to them and get involved. Let us know if you need us to connect you.

  1. How does System Dynamics modeling compare to Agent-Based Modelling of infection?

We are not agent-based modeling experts but have an amateur understanding of how they work. Agent-based models are excellent for understanding the dynamics of disease transmission – especially for geospatial patterns – because the agents capture the infection pathways and can show emergent dynamics you might miss if you are working only with and SEIR Stock and flow. I believe they are complementary and both offer insight.

  1. How have you raised awareness and gained interest in System Dynamics (COVID) modeling across institutions and what’s next?

We have worked like hell to generate awareness and I would say its early days but the signs are positive. Our work with the NHS in the UK, MIT students in the MISTI program, engagement with Junior Achievement, and most recently the Emergent Alliance are good lead indicators but we can do much more.

  1. Lockdowns serve to restrict the movement between locales and thereby enhance the relevance of the model

Lockdowns, shielding of vulnerable, hygiene measures, testing and tracing policies, vaccines, hospital capacity are amongst the policy levers directly available.

  1. The final model appears to use Silico. What is this?

Silico is a relatively new System Dynamics Simulation software package. We use it because it is free for personal use as long as you are happy to share your model as an open-source resource, it is easy to teach and use and very intuitive… check it out in our webinars and course. Check their website.

Recent Posts

Upcoming Events

2024 Student Chapter Monthly Meeting

2024 Student Chapter Monthly Meeting

Join us for the Monthly Meeting of the Student Chapter at the System Dynamics Society! This virtual gathering provides an excellent opportunity for students passionate about system dynamics to connect, share ideas, and collaborate on projects. Access the session via...

Recent Business cases

Join us

Models, Maps, and Levels of Evidence

Models, Maps, and Levels of Evidence

Anyone who has attended a System Dynamics conference in recent years, or has read past posts in the System Dynamics Society blog, is surely aware of differences of opinion on the value of qualitative maps as opposed to quantified simulations.

What you may not know is that this debate has been going on for a long time, stretching back to the early 1980s, when Eric Wolstenholme and others asserted that one might be able to infer dynamics from qualitative maps.  In 1990, Peter Senge’s book The Fifth Discipline upped the ante by suggesting that certain problems might be categorized according to System Archetypes, which some took to mean that one could go directly to solutions without first simulating.  This became known as Systems Thinking.

In 2000, Geoff Coyle wrote a paper (SDR 2000, 16:3) that pointedly asked whether there might be situations so uncertain that quantified modeling cannot tell us more than qualitative mapping alone.  Rogelio Oliva and I wrote a rejoinder (SDR 2001, 17:4) that challenged this idea and sought to reclaim Jay Forrester’s original view of simulation as the necessary test bed for any dynamic hypothesis that might conceivably lead to policy decisions.

Even now, 20 years later, the debate continues.  We see in our conferences increasing use of Group Model Building that goes no further than a qualitative map, and from which the authors claim to have derived dynamic insights.

Some have been dismayed by this development—which appears to be the further expansion of Systems Thinking—fearing it is diluting and dumbing down the fundamentals of our field.  The modelers accuse the mappers of lacking rigor, while the mappers say that good group process has a rigor of its own.

What can we do about this long-brewing pot of trouble?

  Is there any way out of the impasse?

I’d like to suggest a possibility.  Several years ago, I published a paper (SDR 2014, 30:1-2) describing how a “levels of evidence” approach—a standard for classifying work in the biomedical sciences—might be applied in SD.  To achieve an “A” level of evidence, one would need both strong structural and behavioral evidence and the ability to reliably test one’s model.

  Structural evidence comes from conversations with subject matter experts and focused studies of cause and effect.  Behavioral evidence comes from a comparison of model output with time series data and historical records.   Work with strong support for structure but not behavior, or behavior but not structure, would achieve at best a “B” rating.  Work with strong support for neither would get a “C” rating.

In the biomedical sciences, works that have “B” or “C” ratings can still be presented at conferences and even appear in prestigious journals.  A rating less than “A” does not mean poor quality but rather lack of full, iron-clad reliability for drawing conclusions and making decisions; that is, something more exploratory or tentative.  Its level of evidence is designated in the conference proceedings or at the top of the paper so that the audience knows what they are dealing with—a decisive work (one from which decisions can be made with confidence) or something less than that.

It seems to me we can apply the Levels of Evidence filter objectively across both simulation models and qualitative maps.  Let’s start with the simple fact that a simulation model can be tested formally, while a qualitative map cannot.  It follows that the best possible rating for a simulation model is “A”, while the best possible for a qualitative map is “B”.

If we can agree on this much, then we may be able to find a way for simulation models and qualitative maps to coexist.  It would require acknowledgment from the advocates of qualitative maps that their work can never be considered decisive.  And, it would require acknowledgment from the advocates of simulation that a model lacking sufficient evidence may be no more reliable than a well-developed qualitative map.

Recent Posts

Upcoming Events

Restricted content

Monthly meetings of the Policy Council occur on the 4th Wednesday of the month with the exception of conference month and holiday weeks.  All Society members are welcome to attend.  Login to access the zoom link which will be displayed here to members 15 minutes prior...

Recent Business cases

Join us