Journal of Educational Evaluation for Health Professions

Software report

Conducting simulation studies for computerized adaptive testing using SimulCAT: an instructional piece: Kyung (Chris) Tyek Han; J Educ Eval Health Prof. 2018;15:20. Published online August 17, 2018; DOI: https://doi.org/10.3352/jeehp.2018.15.20

28,987 View
244 Download
7 Web of Science
7 Crossref

Computerized adaptive testing (CAT) technology is widely used in a variety of licensing and certification examinations administered to health professionals in the United States. Many more countries worldwide are expected to adopt CAT for their national licensing examinations for health professionals due to its reduced test time and more accurate estimation of a test-taker’s performance ability. Continuous improvements to CAT algorithms promote the stability and reliability of the results of such examinations. For this reason, conducting simulation studies is a critically important component of evaluating the design of CAT programs and their implementation. This report introduces the principles of SimulCAT, a software program developed for conducting CAT simulation studies. The key evaluation criteria for CAT simulation studies are explained and some guidelines are offered for practitioners and test developers. A step-by-step tutorial example of a SimulCAT run is also presented. The SimulCAT program supports most of the methods used for the 3 key components of item selection in CAT: the item selection criterion, item exposure control, and content balancing. Methods for determining the test length (fixed or variable) and score estimation algorithms are also covered. The simulation studies presented include output files for the response string, item use, standard error of estimation, Newton-Raphson iteration information, theta estimation, the full response matrix, and the true standard error of estimation. In CAT simulations, one condition cannot be generalized to another; therefore, it is recommended that practitioners perform CAT simulation studies in each stage of CAT development.

Citations

Citations to this article as recorded by

Presidential address: improving item validity and adopting computer-based testing, clinical skills assessments, artificial intelligence, and virtual reality in health professions licensing examinations in Korea
Hyunjoo Pai
Journal of Educational Evaluation for Health Professions.2023; 20: 8. CrossRef
Students’ perceptions of Computerised Adaptive Testing in higher education
Proya Ramgovind, Shamola Pramjeeth
The Independent Journal of Teaching and Learning.2023; 18(2): 109. CrossRef
Preliminary Development of an Item Bank and an Adaptive Test in Mathematical Knowledge for University Students
Fernanda Belén Ghio, Manuel Bruzzone, Luis Rojas-Torres, Marcos Cupani
European Journal of Science and Mathematics Education.2022; 10(3): 352. CrossRef
Evaluating a Computerized Adaptive Testing Version of a Cognitive Ability Test Using a Simulation Study
Ioannis Tsaousis, Georgios D. Sideridis, Hannan M. AlGhamdi
Journal of Psychoeducational Assessment.2021; 39(8): 954. CrossRef
Exploring Counselor‐Client Agreement on Clients’ Work Capacity in Established and Consultative Dyads
Uma Chandrika Millner, Diane Brandt, Leighton Chan, Alan Jette, Elizabeth Marfeo, Pengsheng Ni, Elizabeth Rasch, E. Sally Rogers
Journal of Employment Counseling.2020; 57(3): 98. CrossRef
Development of a Computerized Adaptive Testing for Internet Addiction
Yong Zhang, Daxun Wang, Xuliang Gao, Yan Cai, Dongbo Tu
Frontiers in Psychology.2019;[Epub] CrossRef
Updates from 2018: Being indexed in Embase, becoming an affiliated journal of the World Federation for Medical Education, implementing an optional open data policy, adopting principles of transparency and best practice in scholarly publishing, and appreci
Sun Huh
Journal of Educational Evaluation for Health Professions.2018; 15: 36. CrossRef

First
Prev
Page of 1
Next
Last

Search