ETS Reduces Costs With Questions Inventory Management
In the mid-1990s, ETS started a transition from paper-and-pencil testing to computer-based testing for all of its graduate-level tests. Computer-based tests, offered on an almost daily basis at many test locations around the world and throughout the year, draw questions from a weekly pool of test items many times larger than the test itself, and that weekly pool itself gets refreshed from an even larger inventory of items. This inventory represents a substantial investment to ETS, because the items must be written and honed for precision, reviewed for cultural and gender bias, and pre-tested on large samples of test takers. Both item security and millions of dollars in item development costs are affected by the way in which the item inventory is managed.
An SD model was developed to project the number of available test items under different assumptions about security risk, and also to look at policies for more cost-effective inventory management. The model revealed that in cases of higher security risk, under existing policies, the number of available items could stagnate at an unacceptably low level. Model tests showed that a modest and safe amount of “recycling” items used in the past could neutralize this potential problem. Moreover, the model showed that, even when security risk was not high, the policy of recycling could effectively reduce ETS item creation costs and help their bottom line. The model analysis was presented to the ETS executive board and affected their decisions regarding test item inventory policy.
Client | Educational Testing Service (ETS), the world’s leading developer and provider of standardized educational tests |
---|---|
Author/Consultant | Homer J |
Publication
Homer J. Structure, Data, and Compelling Conclusions: Notes from the Field. System Dynamics Review, 13(4): 293-309, 1997. |
OTHER SUCCESSFUL APPLICATIONS
Decarbonization of Student Transportation via Electric School Bus Transition
Decarbonization of Student Transportation via Electric School Bus Transition The webinar on “Decarbonization of Student Transportation via Electric School Bus Transition” offered a deep dive into the complexities and potential solutions surrounding the...
Management Design for Planted Forests in Japan Using System Dynamics
Management Design for Planted Forests in Japan Using System Dynamics EXECUTIVE Summary The simulation model offers a novel approach to sustainable forest management in Japan, enabling detailed analysis of labor requirements and changes in forest conditions. It enables...
How Did En-ROADS Get 755,000 users? Lessons on Modeling, Interface Design, and Facilitation
How Did En-ROADS Get 755,000 users? Lessons on Modeling, Interface Design, and Facilitation Achieving widespread engagement is a significant challenge with a System Dynamics model. Yet, En-ROADS, the climate solutions simulator co-developed by Climate Interactive and...
Upcoming Events
![2024 Conference Material Submission Deadline](https://b2104009.smushcdn.com/2104009/wp-content/uploads/2021/10/Event-Images-600-x-600-4.jpg?lossy=2&strip=1&webp=1)
2024 Conference Material Submission Deadline
Submit conference materials to share with attendees by this date (always updateable!) for the Online Conference Website. Requirements and options vary by session type. Materials include: Paper File in PDF format (a full paper, extended abstract, or slides for...
Recent Posts
How Did En-ROADS Get 755,000 users? Lessons on Modeling, Interface Design, and Facilitation
How Did En-ROADS Get 755,000 users? Lessons on Modeling, Interface Design, and Facilitation Achieving widespread engagement is a significant challenge with a System Dynamics model. Yet, En-ROADS, the climate solutions simulator co-developed by Climate Interactive and...
Society Governance Updates
Society Governance Updates Welcome, Allyson! New President Allyson Beall King joined the Policy Council as our 2024 President. Her primary role is as director of the Washington State University School of the Environment, which focuses on regional ecologies and our...
Call for Presenters: Seminar Series
Call for Presenters: Seminar Series We at the System Dynamics Society are continually seeking vibrant and knowledgeable presenters for our ongoing Seminar Series. As we unfold the calendar, there’s always a place for more insights, experiences, and expertise to enrich...
Join us
OTHER SUCCESSFUL APPLICATIONS
Decarbonization of Student Transportation via Electric School Bus Transition
Decarbonization of Student Transportation via Electric School Bus Transition The webinar on “Decarbonization of Student Transportation via Electric School Bus Transition” offered a deep dive into the complexities and potential solutions surrounding the...
Management Design for Planted Forests in Japan Using System Dynamics
Management Design for Planted Forests in Japan Using System Dynamics EXECUTIVE Summary The simulation model offers a novel approach to sustainable forest management in Japan, enabling detailed analysis of labor requirements and changes in forest conditions. It enables...
How Did En-ROADS Get 755,000 users? Lessons on Modeling, Interface Design, and Facilitation
How Did En-ROADS Get 755,000 users? Lessons on Modeling, Interface Design, and Facilitation Achieving widespread engagement is a significant challenge with a System Dynamics model. Yet, En-ROADS, the climate solutions simulator co-developed by Climate Interactive and...
Recent Posts
How Did En-ROADS Get 755,000 users? Lessons on Modeling, Interface Design, and Facilitation
How Did En-ROADS Get 755,000 users? Lessons on Modeling, Interface Design, and Facilitation Achieving widespread engagement is a significant challenge with a System Dynamics model. Yet, En-ROADS, the climate solutions simulator co-developed by Climate Interactive and...
Society Governance Updates
Society Governance Updates Welcome, Allyson! New President Allyson Beall King joined the Policy Council as our 2024 President. Her primary role is as director of the Washington State University School of the Environment, which focuses on regional ecologies and our...
Call for Presenters: Seminar Series
Call for Presenters: Seminar Series We at the System Dynamics Society are continually seeking vibrant and knowledgeable presenters for our ongoing Seminar Series. As we unfold the calendar, there’s always a place for more insights, experiences, and expertise to enrich...
Upcoming Events
![2024 Conference Material Submission Deadline](https://b2104009.smushcdn.com/2104009/wp-content/uploads/2021/10/Event-Images-600-x-600-4.jpg?lossy=2&strip=1&webp=1)
2024 Conference Material Submission Deadline
Submit conference materials to share with attendees by this date (always updateable!) for the Online Conference Website. Requirements and options vary by session type. Materials include: Paper File in PDF format (a full paper, extended abstract, or slides for...