ETS Reduces Costs With Questions Inventory Management
In the mid-1990s, ETS started a transition from paper-and-pencil testing to computer-based testing for all of its graduate-level tests. Computer-based tests, offered on an almost daily basis at many test locations around the world and throughout the year, draw questions from a weekly pool of test items many times larger than the test itself, and that weekly pool itself gets refreshed from an even larger inventory of items. This inventory represents a substantial investment to ETS, because the items must be written and honed for precision, reviewed for cultural and gender bias, and pre-tested on large samples of test takers. Both item security and millions of dollars in item development costs are affected by the way in which the item inventory is managed.
An SD model was developed to project the number of available test items under different assumptions about security risk, and also to look at policies for more cost-effective inventory management. The model revealed that in cases of higher security risk, under existing policies, the number of available items could stagnate at an unacceptably low level. Model tests showed that a modest and safe amount of “recycling” items used in the past could neutralize this potential problem. Moreover, the model showed that, even when security risk was not high, the policy of recycling could effectively reduce ETS item creation costs and help their bottom line. The model analysis was presented to the ETS executive board and affected their decisions regarding test item inventory policy.
Client | Educational Testing Service (ETS), the world’s leading developer and provider of standardized educational tests |
---|---|
Author/Consultant | Homer J |
Publication
Homer J. Structure, Data, and Compelling Conclusions: Notes from the Field. System Dynamics Review, 13(4): 293-309, 1997. |
OTHER SUCCESSFUL APPLICATIONS
System Dynamics Unravels ICU Tensions at the Portuguese Oncology Institute
System Dynamics Unravels ICU Tensions at the Portuguese Oncology Institute EXECUTIVE Summary The Portuguese Oncology Institute (IPO) faced a critical challenge in its intensive care unit (ICU) where doctors and nurses experienced high turnover due to tensions with...
System Dynamics Helps Farmers Escape Poverty Trap in Guatemala
System Dynamics Helps Farmers Escape Poverty Trap in Guatemala EXECUTIVE Summary Guatemala holds the 4th highest global ranking for chronic malnutrition, and climate change is intensifying the challenges subsistence farmers face in providing food for their families....
Twinings Uses System Dynamics Games to Enhance HR Capability
Twinings Uses System Dynamics Games to Enhance HR Capability “Realistic simulation is a powerful approach to building capability. The business simulation developed [by Dashboard Simulations and Lane4] gave [Twinings staff] an experience that called for them to develop...
Upcoming Events

NATO Computer Assisted Analysis, eXercise, and eXperimentation Forum (CA2X2 Forum)
On behalf of the NATO Modelling & Simulation Centre of Excellence, we are pleased to confirm that the 2023 Edition of the NATO Computer Assisted Analysis, eXercise, and eXperimentation Forum (CA2X2 Forum) will be held from 3-5 October 2023 at the ERGIFE PALACE...
Recent Posts
Call for Presenters: Seminar Series
Call for Presenters: Seminar Series We at the System Dynamics Society are continually seeking vibrant and knowledgeable presenters for our ongoing Seminar Series. As we unfold the calendar, there’s always a place for more insights, experiences, and expertise to enrich...
Honoring Excellence: A Glimpse into the Awards of the International System Dynamics Conference
Honoring Excellence: A Glimpse into the Awards of the International System Dynamics Conference The International System Dynamics Conference brings together experts, practitioners, and students to exchange ideas, showcase real-world applications, and celebrate...
From Bergen to Global: UiB’s System Dynamics Group
From Bergen to Global: UiB’s System Dynamics Group The System Dynamics Group, an autonomous research group at the University of Bergen (UiB) was established in 1971 by professor emeritus Svein Nordbotten. Inspired by the work of Jay W. Forrester, Nordbotten...
Join us
OTHER SUCCESSFUL APPLICATIONS
System Dynamics Unravels ICU Tensions at the Portuguese Oncology Institute
System Dynamics Unravels ICU Tensions at the Portuguese Oncology Institute EXECUTIVE Summary The Portuguese Oncology Institute (IPO) faced a critical challenge in its intensive care unit (ICU) where doctors and nurses experienced high turnover due to tensions with...
System Dynamics Helps Farmers Escape Poverty Trap in Guatemala
System Dynamics Helps Farmers Escape Poverty Trap in Guatemala EXECUTIVE Summary Guatemala holds the 4th highest global ranking for chronic malnutrition, and climate change is intensifying the challenges subsistence farmers face in providing food for their families....
Twinings Uses System Dynamics Games to Enhance HR Capability
Twinings Uses System Dynamics Games to Enhance HR Capability “Realistic simulation is a powerful approach to building capability. The business simulation developed [by Dashboard Simulations and Lane4] gave [Twinings staff] an experience that called for them to develop...
Recent Posts
Call for Presenters: Seminar Series
Call for Presenters: Seminar Series We at the System Dynamics Society are continually seeking vibrant and knowledgeable presenters for our ongoing Seminar Series. As we unfold the calendar, there’s always a place for more insights, experiences, and expertise to enrich...
Honoring Excellence: A Glimpse into the Awards of the International System Dynamics Conference
Honoring Excellence: A Glimpse into the Awards of the International System Dynamics Conference The International System Dynamics Conference brings together experts, practitioners, and students to exchange ideas, showcase real-world applications, and celebrate...
From Bergen to Global: UiB’s System Dynamics Group
From Bergen to Global: UiB’s System Dynamics Group The System Dynamics Group, an autonomous research group at the University of Bergen (UiB) was established in 1971 by professor emeritus Svein Nordbotten. Inspired by the work of Jay W. Forrester, Nordbotten...
Upcoming Events



NATO Computer Assisted Analysis, eXercise, and eXperimentation Forum (CA2X2 Forum)
On behalf of the NATO Modelling & Simulation Centre of Excellence, we are pleased to confirm that the 2023 Edition of the NATO Computer Assisted Analysis, eXercise, and eXperimentation Forum (CA2X2 Forum) will be held from 3-5 October 2023 at the ERGIFE PALACE...