tecnicas de modelagem

This forum supports discussion of the Brazil Chapter. Discussion is in Portuguese. It is currently unmoderated, and anybody who is signed in may post to the forum.
Post Reply
Ricardo Pietrobon
Posts: 5
Joined: Thu Sep 03, 2009 6:44 pm

tecnicas de modelagem

Post by Ricardo Pietrobon » Sat Sep 26, 2009 2:42 pm

From: "SDMAIL Jay Forrester" <jforestr@MIT.EDU>
Date: February 12, 2008 5:30:12 AM EST
To: System Dynamics Mailing List <sdmail@lists.systemdynamics.org>
Cc: System Dynamics Mailing List <sdmail@lists.systemdynamics.org>
Subject: REPLY Do marginal models marginalize modeling? (SD6740)

Posted by Jay Forrester <jforestr@MIT.EDU>

Posted by Jack Harich <register@thwink.org>

My point is that "good modeling practices," good enough to reliably solve
difficult problem, do not yet exist. Solving tough problems with SD is still
way too dependent on the brilliance of the modeler. But bright modelers are
in short supply.

I see here a wishing for something that has likewise not been achieved in any other
profession. A model is a theory of behavior that explains the problem at hand.
There are no rules for obtaining a theory leading to a Nobel Prize in physics that
is not "dependent on the brilliance of the modeler." In a more everyday realm,
there is no way to teach engineers to be the one to design the most successful
airplane. Success in an engineering design depends on understanding the underlying
concepts. Beyond the basics, a successful engineer builds on a basic education and
grows from repeated apprenticeships, from trial and error, from learning from past
failures, and being winnowed down from the large number of engineering graduates
who are not willing, or able, or competent, to go through the entire learning
process.

In this discussion thread, there have been comments about how most are not able to
devote time to getting a Ph.D. in system dynamics, as if that were enough to
guarantee success. A full scale academic program in engineering or in medicine
is only the beginning of the process of becoming a true expert.

The emphasis in these discussions needs to be less on how to find a quick path to
successful modeling and much more on how to establish educational and apprenticeship
programs to train experts.

The field is unable to consistently solve the great social problems of our
time, as Forrester alludes to in "System dynamics: The next fifty years" when
he asks: "Why is there so little impact of system dynamics in the most important
social questions?" My answer is because the foundation every field needs for
success is not yet mature. In particular, SD's foundation lacks a repeatable
process. Therefore "more widespread use" of existing modeling practices will
not lead to SD success.

The foundation is never "mature" in the sense that it will not be improving,
however, I see in this discussion a failure to recognize and discuss the
foundations that already exist.

I hope we will have here a discussion of the presently identified foundations,
because it appears that many people who consider themselves working in system
dynamics are no aware of the foundations, or, at least, have not internalized
them.

As discussed above, it is extremely over-optimistic to expect that there will
ever be a repeatable process that leads everyone to create fully effective
models. We should be looking to the other professions for guides to creating
successful practitioners--look at how a heart surgeon has been trained, look
at how an engineer who designs a space ship or a chemical refinery or even a
successful automobile has been moved from novice to expert.

Hypothesis: Without a repeatable process, most modelers cannot solve difficult
social problems.

Not true, if one means a step-by-step process that anyone can follow. The
difficult social problems require a person with a broad and deep understanding
of the social situation coupled with a successful past career of building up
through progressively more difficult modeling situations.


Corollary: Without a repeatable process, most difficult problem models will
be low quality.

Without a solid foundation in underlying fundamentals and an extensive
background in apprenticeships and practice, "most difficult problem models
will be low quality."


This hypothesis can be tested. One way would be to writeup a case, on a problem
with difficulty level similar to the urban decay problem (Forrester 1969). The
case has all the clues and data you need to build a rough model to solve the
problem. But the case also has a lot of chaff or noise: irrelevant and attractive
data that leads to symptomatic solutions that will of course fail. It should not
be at all obvious what the backbone of the model should be.

This might be a very good kind of education. One might well spend two or three
years dealing with a sequence of such exercises.


Then you run an experiment with the case. The control group spends 30 minutes (more?
a lecture Q&A? a course?) reading an irrelevant short article. The treatment group
spends that time reading an article on how to execute a repeatable process, such as
the System Improvement Process. Then both groups receive the case and try to solve
the problem with SD. The treatment group is allowed to refer to the written process
as they go. This of course may require separate rooms for the two groups.

Does this not suggest that the process is far simpler than it really is? My Urban
Dynamics model had two ingredients--my 40 years dealing with dynamics in engineering
and management, and six weeks of discussions, a long half day every week, with a group
who had been fighting the battle of decaying cities, while I was spending about 30
hours per week trying to identify a useful model in what they were saying.

Certainly, the 40 years of background will not be required. Much of that experience
was without the visible goal of modeling social systems. Most of it was dealing with
dynamics through the mathematics of differential equations rather than the more
realistic and easier to understand medium of integrations. Most of the time was
without the clarity of underlying basic principles that are now available and that
are now teachable. So, we should expect that the 40 years can be shrunk to something
more like 10 years. At least an intensive year should be devoted to truly
understanding and appreciating the underlying principles of dynamics that are now
available.


Such an attack would allow the process to be iteratively improved until the
experimental results were stunning, and the hypothesis was proven to satisfaction.
The output would be (1) a formal process, a best practice that we could spread, and
(2) a series of cases that can be used to teach/test this best practice.

Democracy succeeds because it relies on the rule of law, not men. SD too can succeed
if it comes to rely on the use of process, not men.

But law needs courts to interpret laws and processes to enforce them. In
addition to truly understanding the available underlying principles, there
are processes that are not receiving much discussion. The following were
offered in my conference talk on the next 50 years:

" How often do you see a paper that shows all of the following characteristics?

1. The paper starts with a clear description of the system shortcoming to be improved.
2. It displays a compact model that shows how the difficulty is being caused.
3. It is based on a model that is completely endogenous with no external time series to drive it.
4. It argues for the model being generic and descriptive of other members of a class of systems to which the system at hand belongs.
5. It shows how the model behavior fits other members of the class as policies followed by those other members are tested.
6. It arrives at recommended policies that the author is willing to defend.
7. It discusses how the recommended policies differ from past practice.
8. It examines why the proposed policies will be resisted.
9. It recognizes how to overcome antagonism and resistance to the proposed policies."
Granted, my own models and publications do not meet all of these tests.

Jay W. Forrester
Professor of Management
Sloan School, MIT
Posted by Jay Forrester <jforestr@MIT.EDU>
posting date Mon, 11 Feb 2008 14:42:50 -0500
_______________________________________________

Jack Harich
Posts: 54
Joined: Mon Jan 12, 2009 10:56 am
Location: Atlanta, Georgia US
Contact:

Re: tecnicas de modelagem

Post by Jack Harich » Wed Sep 30, 2009 5:53 pm

Ricardo has resurrected what to me was a highly educational thread. For those unfamiliar with it, here’s identification of who said what, which helps to follow the discussion. Text by Harich is normal, text by Forrester is italic.
From: "SDMAIL Jay Forrester" <jforestr@MIT.EDU>
Date: February 12, 2008 5:30:12 AM EST
To: System Dynamics Mailing List <sdmail@lists.systemdynamics.org>
Cc: System Dynamics Mailing List <sdmail@lists.systemdynamics.org>
Subject: REPLY Do marginal models marginalize modeling? (SD6740)

Posted by Jay Forrester <jforestr@MIT.EDU>

Posted by Jack Harich <register@thwink.org>

My point is that "good modeling practices," good enough to reliably solve difficult problem, do not yet exist. Solving tough problems with SD is still way too dependent on the brilliance of the modeler. But bright modelers are in short supply.

I see here a wishing for something that has likewise not been achieved in any other profession. A model is a theory of behavior that explains the problem at hand. There are no rules for obtaining a theory leading to a Nobel Prize in physics that is not "dependent on the brilliance of the modeler." In a more everyday realm, there is no way to teach engineers to be the one to design the most successful airplane. Success in an engineering design depends on understanding the underlying concepts. Beyond the basics, a successful engineer builds on a basic education and grows from repeated apprenticeships, from trial and error, from learning from past failures, and being winnowed down from the large number of engineering graduates who are not willing, or able, or competent, to go through the entire learning process.

In this discussion thread, there have been comments about how most are not able to devote time to getting a Ph.D. in system dynamics, as if that were enough to guarantee success. A full scale academic program in engineering or in medicine is only the beginning of the process of becoming a true expert.

The emphasis in these discussions needs to be less on how to find a quick path to successful modeling and much more on how to establish educational and apprenticeship programs to train experts.


The field is unable to consistently solve the great social problems of our time, as Forrester alludes to in "System dynamics: The next fifty years" when he asks: "Why is there so little impact of system dynamics in the most important social questions?" My answer is because the foundation every field needs for success is not yet mature. In particular, SD's foundation lacks a repeatable process. Therefore "more widespread use" of existing modeling practices will not lead to SD success.

The foundation is never "mature" in the sense that it will not be improving, however, I see in this discussion a failure to recognize and discuss the foundations that already exist.

I hope we will have here a discussion of the presently identified foundations, because it appears that many people who consider themselves working in system dynamics are no aware of the foundations, or, at least, have not internalized them.

As discussed above, it is extremely over-optimistic to expect that there will ever be a repeatable process that leads everyone to create fully effective models. We should be looking to the other professions for guides to creating successful practitioners--look at how a heart surgeon has been trained, look at how an engineer who designs a space ship or a chemical refinery or even a successful automobile has been moved from novice to expert.


Hypothesis: Without a repeatable process, most modelers cannot solve difficult social problems.

Not true, if one means a step-by-step process that anyone can follow. The difficult social problems require a person with a broad and deep understanding of the social situation coupled with a successful past career of building up through progressively more difficult modeling situations.

Corollary: Without a repeatable process, most difficult problem models will be low quality.

Without a solid foundation in underlying fundamentals and an extensive background in apprenticeships and practice, "most difficult problem models will be low quality."

This hypothesis can be tested. One way would be to writeup a case, on a problem with difficulty level similar to the urban decay problem (Forrester 1969). The case has all the clues and data you need to build a rough model to solve the problem. But the case also has a lot of chaff or noise: irrelevant and attractive data that leads to symptomatic solutions that will of course fail. It should not be at all obvious what the backbone of the model should be.

This might be a very good kind of education. One might well spend two or three years dealing with a sequence of such exercises.

Then you run an experiment with the case. The control group spends 30 minutes (more? a lecture Q&A? a course?) reading an irrelevant short article. The treatment group spends that time reading an article on how to execute a repeatable process, such as the System Improvement Process. Then both groups receive the case and try to solve the problem with SD. The treatment group is allowed to refer to the written process as they go. This of course may require separate rooms for the two groups.

Does this not suggest that the process is far simpler than it really is? My Urban Dynamics model had two ingredients--my 40 years dealing with dynamics in engineering and management, and six weeks of discussions, a long half day every week, with a group who had been fighting the battle of decaying cities, while I was spending about 30 hours per week trying to identify a useful model in what they were saying.

Certainly, the 40 years of background will not be required. Much of that experience was without the visible goal of modeling social systems. Most of it was dealing with dynamics through the mathematics of differential equations rather than the more realistic and easier to understand medium of integrations. Most of the time was without the clarity of underlying basic principles that are now available and that are now teachable. So, we should expect that the 40 years can be shrunk to something more like 10 years. At least an intensive year should be devoted to truly understanding and appreciating the underlying principles of dynamics that are now available.


Such an attack would allow the process to be iteratively improved until the experimental results were stunning, and the hypothesis was proven to satisfaction. The output would be (1) a formal process, a best practice that we could spread, and (2) a series of cases that can be used to teach/test this best practice.

Democracy succeeds because it relies on the rule of law, not men. SD too can succeed if it comes to rely on the use of process, not men.

But law needs courts to interpret laws and processes to enforce them. In addition to truly understanding the available underlying principles, there are processes that are not receiving much discussion. The following were offered in my conference talk on the next 50 years:

"How often do you see a paper that shows all of the following characteristics?

1. The paper starts with a clear description of the system shortcoming to be improved.
2. It displays a compact model that shows how the difficulty is being caused.
3. It is based on a model that is completely endogenous with no external time series to drive it.
4. It argues for the model being generic and descriptive of other members of a class of systems to which the system at hand belongs.
5. It shows how the model behavior fits other members of the class as policies followed by those other members are tested.
6. It arrives at recommended policies that the author is willing to defend.
7. It discusses how the recommended policies differ from past practice.
8. It examines why the proposed policies will be resisted.
9. It recognizes how to overcome antagonism and resistance to the proposed policies."
Granted, my own models and publications do not meet all of these tests.


Jay W. Forrester
Professor of Management
Sloan School, MIT
Posted by Jay Forrester <jforestr@MIT.EDU>
posting date Mon, 11 Feb 2008 14:42:50 -0500

Jack Harich
Posts: 54
Joined: Mon Jan 12, 2009 10:56 am
Location: Atlanta, Georgia US
Contact:

Re: tecnicas de modelagem

Post by Jack Harich » Wed Sep 30, 2009 10:11 pm

Forgive my inability to converse in anything other than English. Ricardo replied to me that:
I thought the characteristics of an ideal article were especially helpful for me, but any other information on this might be of help as well. At this point I am particularly interested in the evaluating quality standards that might contribute to an overall increase in the productivity and quality of simulation models (SD and ABM), and would appreciate any additional comments you might have in this regard.
In the long thread of February 2008 Forrester, I, and a dozen others were grappling with how we can do better. How can SD help to solve more problems, especially (these are my favorite) large pressing social problems, like the ones Forrester tackled long ago: urban decay and environmental sustainability?

The gist of my suggestions was SD practitioners need a repeatable process that drives overall problem solving effort. Modeling is merely one of many tools used to execute the process. This is a new way of thinking.

Currently SD modelers think in terms of “How can I develop a dynamic hypothesis and model it so we can get some new insights that will help us solve this problem?” As you can see, that gives little guidance. Even superb textbooks like Sterman’s Business Dynamics lack what I consider a mature repeatable process (by comparison to other fields), though this point is sure to be controversial.

Forrester’s reply that “There are no rules for obtaining a theory leading to a Nobel Prize in physics that is not ‘dependent on the brilliance of the modeler.’ " is a bit of a straw man and reveals a tinge of resistance to anything other than his universal solution of education, education, education to the puzzle of why SD has not achieved its latent potential. No process automatically guarantees 100% success and Nobel prize quality work by its users. But other fields, such as large project management, new plant engineering, software engineering, medical diagnosis, surgery, and aircraft design, use repeatable formal processes with great success, with no requirement that users be brilliant. They accomplish extraordinary results with ordinary people. This demonstrates the same is possible with the problems SD modelers attempt to solve, though the challenge is harder for problems in the soft sciences.

Ricardo says “I am particularly interested in the evaluating quality standards that might contribute to an overall increase in the productivity and quality of simulation models (SD and ABM)…”

“Quality standards” implies testing to confirm a model meets a list of quality requirements. I’m sure you already know this, but I see testing as falling into the trap Deming warned us about decades ago. You can’t test in quality by removing defects, such as by following the very fine checklist Forrester offers at the end of his post. If you want high quality reliably, you have to prevent defects from appearing in the first place.

That is what a formal process that fits the problem type can do. A process is a repeatable series of steps to achieve a goal. The right steps ask a series of questions that, if answered reasonably well, prevent defects from appearing at that step. Input to the next step can then be assumed to be defect free. The result is work products that do not accumulate defects (such as wrong assumptions leading to wrong conclusions, as in the case of modeling). When the work product gets to the end of the process, there is no need for quality testing, because that’s already been done by asking the right question at each fork in the long road to solving the problem.

I’m more than a tad passionate about this, and I hope it shows, because process maturity has been a boon to so many fields, companies, and projects.

Ricardo, if you’d like to see my latest work on a process for solving difficult social problems that uses SD as its main tool, see this manuscript in progress. The old title was Process Is Everything. Despite the new title, which elevates the book to a higher level of problem, the book is all about the System Improvement Process with the global environmental sustainability problem as an application example. Only the first four chapters are done, so you will have a short skim or read. This of course is not the silver bullet that will turn SD into a wundertool overnight, but it should serve as a modest example of what’s possible.

The first four chapters are:
1. The Paradox
2. Process Is Everything
3. The System Improvement Process
4. Two Hundred Years of the Wrong Process

Hope this helps,

Jack

James Thompson
Posts: 44
Joined: Sun Jan 11, 2009 11:41 am
Location: Farmington, CT USA

Re: tecnicas de modelagem

Post by James Thompson » Thu Oct 01, 2009 8:55 am

My dissertation research investigates how and under what conditions clients learn in system dynamics consulting engagements. In the spirit of this thread, I would say that as a profession SD is more similar to medicine than engineering. How many cures are there in medicine? We work with complex social problems in a changing environment and, while we may aspire to holisitic practice, our treatments are not likely to lead to immortality for our client firms.

In my view the maturity of the field is expressed in our tolerance for ambiguity, not in some recipe for success.
Jim Thompson
jimthompson@live.com
jpthompson@wpi.edu
+1 860.796.2428

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest