President Elect
Submitted for consideration at the 2003 summer policy council meeting
June 30, 2003
Our annual conference is under scrutiny now both in terms of how the site selection takes place and the content of the conference. The conference is a very big part of Society activities and the budget and is one of the cornerstones for bringing in sufficient revenue to fund the central office. As we decrease the level of funding for the central office in order to keep the Society financially stable it is critical that we adjust the conference activities to accommodate this change in paid effort.
This report has three parts: The first is a recommendation on the way in which the program should be developed. The second is a recommendation on what should be included in the conference under the responsibility of the central office. The third part is some specific guidelines for program development for 2004 along with a motion that I believe should be discussed and adopted during the Policy Council meeting in July. It also contains a motion to appoint a committee to revise the first two parts and then make a recommendation to be adopted at the Winter 2004 Policy Council meeting.
Originally conference program development was the responsibility of the program chairs with whatever administrative support they could muster themselves. When we went to a professionally run central office, that office began to provide administrative support. Now, if the volunteer efforts flags, the central office will step in and pick up the slack. For any one conference this is clearly good, but it also tends to lead to eroding goals. Over time more and more of the responsibility for the program has gone to the central office staff and program support is now a significant source of labor cost for the Society. In addition, many of the decisions around program design are being assumed by the central office, rather than the scholars and practitioners who should hold that responsibility.
Each year are typically 250-350 works (abstracts, presentations and full papers) submitted to the conference. Forming these into a program is not an easy thing to do. The momentum policy that we had in place for dealing with this was to assign reviewers to judge the submitted works and select an appropriate session for them. The Program Chair then looks through the recommendations and manages papers for which there are conflicting judgments.
In developing the program for 2003 I took a much more hands on approach and personally looked through every submission making notes before trying to reconcile the submitted reviews. This was a tremendous amount of work, and is not generally practical, but it did confirm what others have always said: There is a great deal of variation in the standards by which different individual reviewers judge the works they review.
This is not a criticism of the reviews. Most were thoughtful and provided a number of good comments to the authors. It is just that the people writing the reviews have diverse backgrounds and expectations and that means that the judgments they make about how a work should fit into the program can be very different, and are also likely to be different from those of the program committee members. I think the review process is a wonderful way to get people feedback on their work, though I do have concerns about the cost of maintaining it.
For me, the conclusion that comes out of this experience is that submitted work needs to be organized and judged by a set of people with a great deal of experience and a shared understanding of what constitutes good work that should be shown to others. This is, of course, exactly how the Review is run, with people selected to review submissions based on their experience in the subject area. That degree of rigor is not practical for the conference, but I think it is possible to move in that direction.
In order to gather a set of people to help structure the program I believe there need to be a small number of people who take on the responsibility of managing a subset of the papers submitted. This year, the conference program has been divided by thread and this seems the natural way to distribute the work. Each work should be submitted to a thread, and each thread should have a chair that would be responsible for deciding what to do with that work. The Program Chair or chairs, would be responsible for organizing the program by thread, but the Thread Chairs would be responsible for the individual sessions (what papers to include in what order with what session chair). Contributed plenary sessions would be formed by the Program chair in consultation with the Thread Chairs. The conference theme would most likely be treated as a separate, possibly special, thread.
In addition to the Thread Chairs having a shared vision and common understanding, I believe it would be very valuable for them to have some consistency from year to year. Thus it would be useful if a Thread Chair would serve for several years, much like the members of the Policy Council do. Whether the Thread Chairs are chosen in a manner similar to the Policy Council, or actually are made up of Policy Council members ex-officio is something that should be considered.
The responsibility of a Thread Chair would be to review every submission to that thread and determine how best to place it. This review would include consideration of the results of the individual reviews to the extent that those are available. The Thread Chair would then build a program for that thread, and the Program Chair would figure out how to fit that with the other threads. In this manner we can be confident that every paper has received good faith review by a competent person. In cases where the Thread Chair disagrees with an individual review, the Thread Chair gets to decide.
I was a program co-chair in 1987 and then the chair in 2003. In 1987 we accepted everything that was at least marginally related to system dynamics for presentation at a parallel session. In 2003 we accepted all but a handful of submissions that were clearly related to system dynamics for presentation at parallel and poster sessions. The number of submissions is definitely up. The average quality of submission has moved up a little. Were we to be more selective, the average quality of presentations could be moved up significantly.
Balancing the selection of high quality presentations with the desire to be inclusive and get people feedback on their work is a difficult problem. It is my strong belief that our current structuring of the non-plenary program into parallel and poster presentations is not the right way to go. It is true that the parallel sessions are pretty good, and posters can be viewed selectively. From that perspective the current solution is reasonable, but we are putting into the proceedings of our conferences pretty much all work that is related to system dynamics independent of quality. I think we need to be more selective about what is included in the proceedings.
As an alternative to Poster Sessions I would propose that we include Research Sessions. These sessions would be organized and presented by a well-respected academic or practitioner and would summarize work that has been done and is ongoing in any research area. The presenter would take one slide from each person wanting to show what research they are doing and present that slide, putting it in context, and introduce the author. The session audience, including all the authors, would get to find out who is doing what, and attach a face to a name to enable ongoing communications. Authors would be welcome to bring copies of their papers to share with others, though the papers themselves would not be included in the proceedings.
Any work that is related to system dynamics would be accepted into a Research Session. The author of any work not accepted for parallel or plenary presentation would also be given the chance to have their work presented in a research session. (That is, unless a work is rejected as not being on-topic it will be included in a Research Session if the author wants it to be.) The organizer of the research session would be welcome to provide constructive criticism and point out methodological problems with any submitted work. A research session could easily handle 15 individual works in one hour. Since the presenter is someone fairly well known it is likely that a number of people would want to attend just to get an overview of what is happening in an area.
Research sessions would run as parallel sessions either with other Research Sessions or in conjunction with presented papers.
People like the current printed abstract proceedings as they are a useful guide to what to see during the conference. These proceedings would continue to be done as they are now except that only parallel and plenary presentations would be included. The Research Sessions would have a single abstract for the entire session and list the names of contributing participants, but not include abstracts of the individual works (or the titles of contributed works).
The electronic proceedings would consist of all plenary and parallel papers and also a paper or presentation for each Research Session if the presenter of that session decided to do this. Each author would also be given the opportunity to provide a location where updates on a work might be found. This material would be put up on the web with updates allowed until 1 month after the conference after which time it would remain static for 8 years. One the site is fixed the society offices might also choose to create a small number CDs if they could sell them above the cost of creation (which should be modest). No CD would be distributed at the conference.
The details of formatting for the web proceedings would need to be worked out so as to minimize the labor involved. Fundamentally, not much more is needed than a list of papers with links to the papers in pdf or html format. Links the author’s want to include could be put directly into their papers. It does not seem like this needs to be fancy, just accessible and archival.
Submission of an abstract would qualify a person to participate in a Research Session. To be considered for a plenary or parallel presentation a full paper would need to be submitted. Based on this year’s experience, submissions of slide presentations and other things not in the form of academic papers are just not accessible to those trying to review them.
Every submission would be made to a Thread. If the author feels that more than one thread might be appropriate they would be allowed up to two alternatives. The submission would, however, be director to the Thread Chair of the first thread chosen who might, in turn pass it to another.
Abstracts would not be sent out for review but simply placed into a Research Session by the Thread Chair.
Full papers would be sent out for review to two reviewers. I don’t believe it is worth the extra effort to make these reviews blind, though the reviewers would be anonymous. Those reviews would ask for comments on the paper, and a recommendation for disposition. That recommendation would be one of: Reject not related to system dynamics, Reject recommend inclusion in a research session, Accept Parallel, Accept Plenary. The additional information of content quality, writing quality and of interest to would also be included. Recommendations for a session and other information would be dropped as would comments to the chair. This is a slight simplification of the current system.
I would propose we have only a single submission deadline of March 1. Anyone requiring an early decision on acceptance to the conference could request that and the responsible Track chair could make the decision. Early acceptances will only guarantee inclusion in at least a Research Session and these are easier to deal with by exception than with additional deadlines.
Following the submission there should be 1 month for reviewers to review and 1 month to construct the program. Acceptances would go out on May 1. The preliminary program would be published on May 15.
The annual conference is a major source of funds for the Society. At the same time it also represents the only opportunity for most people working in the field to get together and meet the well-known members of the community. Thus, we would like to keep the conference fee as low as possible. To make money, this means that we need to keep conference costs down.
There are two main sources of conference costs. One is central office activity, which is driven by the program and the logistics of negotiating contracts and managing registrations. The second is the direct cost of the venue.
The proposed revisions in the way programs are constructed, and the elimination of CD proceedings, will decrease the central office costs modestly. It is possible that more can be done there, though it will be a matter of balancing benefits with the costs. There is no fat to be trimmed and we are also constrained to some extent by being on a yearly budget cycle.
Mostly, then, this proposal is aimed at the direct costs of running the conference. Of the things that are currently included in the conference fee those that can be removed are:
· Free coffee during breaks (making sure coffee can be purchased conveniently)
· Banquet
· Handouts can be minimized (no bags)
· Reception
For New York these amount to about $150 per person. Thus, if they were simply to be removed, other things equal, we could have dropped the registration fee from $375 to $225 and come out even. Other things are not, of course, completely equal. Some of these activities are partially or wholly sponsored and it is not clear if this sponsorship would be available without them.
It is my recommendation that we budget for and execute a Spartan conference. If attendance is high, then profitability should also be high. The money from a successful attendance should not be used to provide more amenities to attendees.
I have no objection to conference frills, just their cost. We should give conference attendees the lowest possible registration fee. If the conference organizers, or others, desire and can figure out a way to pay for more frills than these could certainly be added. The Society would neither contract for or pay for anything beyond the basic conference.
In terms of affordability I think it is also important that a more explicit process for subsidizing people who can’t afford registration and other fees be put in place. It would be fairly easy to add $25 into the registration fee that could be used for this purpose. However, we need a committee to actually decide whom to award money and on what basis. Right now the central office is continually burdened with requests to which they try to find compassionate solutions. It would be much better if requestors could just be referred to the chair of the conference scholarships committee (Roberta endorses this idea).
I am not proposing that we redo conferences based on the above discussion immediately. However, there are some very practical considerations that have to be accounted for in 2004. The most important of these is a budget that has less staff available for program support than this year and last. With this in mind, I would propose the following:
Moved: That Michael Kennedy and Graham Winch be approved as the Program Chairs for the 2004 Conference.
Moved: That the Program Chairs for 2004 be strongly encouraged to appoint Thread Chairs to manage the program in a manner consistent with that described above.
(For the record I volunteer to act as a Thread Chair for 2004).
Moved: That the Program Chairs in consultation with the Conference Chair, Thread Chairs, Central Office and VP Meetings, be strongly encouraged to decide, before October, whether to use the parallel/poster format currently in use or a variation on that.
Moved: That the Program Chair in consultation with the Central office adjust the submission and review process to meet their needs and not be strictly bound by the standardized program guidelines adopted last year.
Longer term, I believe that it is important that we adopt a stronger set of changes to the structuring of our programs and therefore I would propose the following:
Moved: That a committee chaired by Bob Eberlein and including Jac Vennix, Paal Davidsend, George Richardson, Peter Milling, John Sterman, Jim Lyneis, Andy Ford and Roberta Spencer be appointed to make a strong recommendation on the future structure of conference programs. Their report should be submitted for a vote at the 2004 winter Policy Council meeting.
Others who wish to participate in this discussion are welcome to do so. The new guidelines would supersede the standardized program guidelines adopted last year which turn seem to have some problems.