Journal of Educational Evaluation for Health Professions

Research article

Post-hoc simulation study of computerized adaptive testing for the Korean Medical Licensing Examination: Dong Gi Seo, Jeongwook Choi; J Educ Eval Health Prof. 2018;15:14. Published online May 17, 2018; DOI: https://doi.org/10.3352/jeehp.2018.15.14; Correction in: J Educ Eval Health Prof 2018;15(0):27

36,356 View
321 Download
8 Web of Science
7 Crossref

Purpose
Computerized adaptive testing (CAT) has been adopted in licensing examinations because it improves the efficiency and accuracy of the tests, as shown in many studies. This simulation study investigated CAT scoring and item selection methods for the Korean Medical Licensing Examination (KMLE).
Methods
This study used a post-hoc (real data) simulation design. The item bank used in this study included all items from the January 2017 KMLE. All CAT algorithms for this study were implemented using the ‘catR’ package in the R program.
Results
In terms of accuracy, the Rasch and 2-parametric logistic (PL) models performed better than the 3PL model. The ‘modal a posteriori’ and ‘expected a posterior’ methods provided more accurate estimates than maximum likelihood estimation or weighted likelihood estimation. Furthermore, maximum posterior weighted information and minimum expected posterior variance performed better than other item selection methods. In terms of efficiency, the Rasch model is recommended to reduce test length.
Conclusion
Before implementing live CAT, a simulation study should be performed under varied test conditions. Based on a simulation study, and based on the results, specific scoring and item selection methods should be predetermined.

Citations

Citations to this article as recorded by

Presidential address: improving item validity and adopting computer-based testing, clinical skills assessments, artificial intelligence, and virtual reality in health professions licensing examinations in Korea
Hyunjoo Pai
Journal of Educational Evaluation for Health Professions.2023; 20: 8. CrossRef
Developing Computerized Adaptive Testing for a National Health Professionals Exam: An Attempt from Psychometric Simulations
Lingling Xu, Zhehan Jiang, Yuting Han, Haiying Liang, Jinying Ouyang
Perspectives on Medical Education.2023;[Epub] CrossRef
Optimizing Computer Adaptive Test Performance: A Hybrid Simulation Study to Customize the Administration Rules of the CAT-EyeQ in Macular Edema Patients
T. Petra Rausch-Koster, Michiel A. J. Luijten, Frank D. Verbraak, Ger H. M. B. van Rens, Ruth M. A. van Nispen
Translational Vision Science & Technology.2022; 11(11): 14. CrossRef
The accuracy and consistency of mastery for each content domain using the Rasch and deterministic inputs, noisy “and” gate diagnostic classification models: a simulation study and a real-world analysis using data from the Korean Medical Licensing Examinat
Dong Gi Seo, Jae Kum Kim
Journal of Educational Evaluation for Health Professions.2021; 18: 15. CrossRef
Linear programming method to construct equated item sets for the implementation of periodical computer-based testing for the Korean Medical Licensing Examination
Dong Gi Seo, Myeong Gi Kim, Na Hui Kim, Hye Sook Shin, Hyun Jung Kim
Journal of Educational Evaluation for Health Professions.2018; 15: 26. CrossRef
Funding information of the article entitled “Post-hoc simulation study of computerized adaptive testing for the Korean Medical Licensing Examination”
Dong Gi Seo, Jeongwook Choi
Journal of Educational Evaluation for Health Professions.2018; 15: 27. CrossRef
Updates from 2018: Being indexed in Embase, becoming an affiliated journal of the World Federation for Medical Education, implementing an optional open data policy, adopting principles of transparency and best practice in scholarly publishing, and appreci
Sun Huh
Journal of Educational Evaluation for Health Professions.2018; 15: 36. CrossRef

Research Articles

Impact of a novel, resource appropriate resuscitation curriculum on Nicaraguan resident physician’s management of cardiac arrest: Breena R. Taira, Aristides Orue, Edward Stapleton, Luis Lovato, Sitaram Vangala, Lucia Solorzano Tinoco, Orlando Morales; J Educ Eval Health Prof. 2016;13:25. Published online June 9, 2016; DOI: https://doi.org/10.3352/jeehp.2016.13.25

27,582 View
206 Download
4 Web of Science
6 Crossref

Abstract PDF

Purpose
Project Strengthening Emergency Medicine, Investing in Learners in Latin America (SEMILLA) created a novel, language and resource appropriate course for the resuscitation of cardiac arrest for Nicaraguan resident physicians. We hypothesized that participation in the Project SEMILLA resuscitation program would significantly improve the physician’s management of simulated code scenarios. Methods: Thirteen Nicaraguan resident physicians were evaluated while managing simulated cardiac arrest scenarios before, immediately, and at 6 months after participating in the Project SEMILLA resuscitation program. This project was completed in 2014 in Leon, Nicaragua. The Cardiac Arrest Simulation Test (CASTest), a validated scoring system, was used to evaluate performance on a standardized simulated cardiac arrest scenario. Mixed effect logistic regression models were constructed to assess outcomes. Results: On the pre-course simulation exam, only 7.7% of subjects passed the test. Immediately post-course, the subjects achieved a 30.8% pass rate and at 6 months after the course, the pass rate was 46.2%. Compared with pre-test scores, the odds of passing the CASTest at 6 months after the course were 21.7 times higher (95% CI 4.2 to 112.8, P<0.001). Statistically significant improvement was also seen on the number of critical items completed (OR=3.75, 95% CI 2.71-5.19), total items completed (OR=4.55, 95% CI 3.4-6.11), and number of “excellent” scores on a Likert scale (OR=2.66, 95% CI 1.85-3.81). Conclusions: Nicaraguan resident physicians demonstrate improved ability to manage simulated cardiac arrest scenarios after participation in the Project SEMILLA resuscitation course and retain these skills.

Citations

Citations to this article as recorded by

Симуляційні технології у навчанні інтернів та безперервній професійній освіті дипломованих лікарів-анестезіологів
Kateryna Bielka, Мariya Smirnova, Hanna Fomina
The Ukrainian Scientific Medical Youth Journal.2023; 139(2): 64. CrossRef
Analysis of Unstructured Text-Based Data Using Machine Learning Techniques: The Case of Pediatric Emergency Department Records in Nicaragua
Giulia Lorenzoni, Silvia Bressan, Corrado Lanera, Danila Azzolina, Liviana Da Dalt, Dario Gregori
Medical Care Research and Review.2021; 78(2): 138. CrossRef
Assessing the impact of resuscitation residents on the treatment of cardiopulmonary resuscitation patients
David M. Lee, David A. Berger, Patrick A. Wloszczynski, Patrick Karabon, Lihua Qu, Michael J. Burla
The American Journal of Emergency Medicine.2021; 41: 46. CrossRef
Resuscitation Resident Impact in the Treatment of Sepsis
Michael J Burla, Nashid Shinthia, Judith A Boura, Lihua QU, David A Berger
Cureus.2020;[Epub] CrossRef
A Review of Anesthesia Simulation in Low-Income Countries
Oleg Turkot, Michael C. Banks, Seung Woo Lee, Adam Dodson, Shirley Duarte, Mwemezi Kaino, Howard Nelson-Williams, Serkan Toy, John Sampson
Current Anesthesiology Reports.2019; 9(1): 1. CrossRef
Community-based in situ simulation: bringing simulation to the masses
Barbara M. Walsh, Marc A. Auerbach, Marcie N. Gawel, Linda L. Brown, Bobbi J. Byrne, Aaron Calhoun
Advances in Simulation.2019;[Epub] CrossRef

Modifiable variables in physical therapy education programs associated with first-time and three-year National Physical Therapy Examination pass rates in the United States: Chad Cook, Chalee Engelhard, Michel D. Landry, Christine McCallum; J Educ Eval Health Prof. 2015;12:44. Published online September 23, 2015; DOI: https://doi.org/10.3352/jeehp.2015.12.44

28,257 View
182 Download
9 Web of Science
19 Crossref

Abstract PDF

Purpose
This study aimed to examine the modifiable programmatic characteristics reflected in the Commission on Accreditation in Physical Therapy Education (CAPTE) Annual Accreditation Report for all accredited programs that reported pass rates on the National Physical Therapist Examination, and to build a predictive model for first-time and three-year ultimate pass rates. Methods: This observational study analyzed programmatic information from the 185 CAPTE-accredited physical therapy programs in the United States and Puerto Rico out of a total of 193 programs that provided the first-time and three-year ultimate pass rates in 2011. Fourteen predictive variables representing student selection and composition, clinical education length and design, and general program length and design were analyzed against first-time pass rates and ultimate pass rates on the NPTE. Univariate and multivariate multinomial regression analysis for first-time pass rates and logistic regression analysis for three-year ultimate pass rates were performed. Results: The variables associated with the first-time pass rate in the multivariate analysis were the mean undergraduate grade point average (GPA) and the average age of the cohort. Multivariate analysis showed that mean undergraduate GPA was associated with the three-year ultimate pass rate. Conclusions: Mean undergraduate GPA was found to be the only modifiable predictor for both first-time and three-year pass rates among CAPTE-accredited physical therapy programs.

Citations

Citations to this article as recorded by

Challenges and Opportunities for Physical Therapist Education Research: Results From an American Council of Academic Physical Therapy Needs Assessment Survey
Tara Dickson, Jody Eckert, Patrick Pabian, Jamie Greco
Journal of Physical Therapy Education.2024; 38(1): 80. CrossRef
Predictive Value of In-Program Academic Variables for Future Clinical Education Performance in Physical Therapist Education: An Exploratory Study
Marisa Birkmeier, Joyce Maring, Rebecca Pinkus, Ellen Costello
Journal of Physical Therapy Education.2024; 38(1): 44. CrossRef
A Systematic Review of Variables Used in Physical Therapist Education Program Admissions Part 2: Noncognitive Variables
Andrea N. Bowens
Journal of Physical Therapy Education.2024;[Epub] CrossRef
Predictors of National Physical Therapy Examination Failure in Graduates of a Blended Doctor of Physical Therapy Program
Melissa J. Lazinski, Kathleen Rockefeller, M. Samuel Cheng
Journal of Physical Therapy Education.2023; 37(1): 52. CrossRef
Board of Certification Examination Achievement Gaps as a Barrier to Diversifying the Athletic Training Profession
Nicolette A. Harris, Lindsey E. Eberman
Journal of Athletic Training.2023; 58(1): 1. CrossRef
How Doctor of Physical Therapy Students Overcome Academic Challenges to Achieve First-Attempt Success on the National Physical Therapy Examination: A Mixed Methods Study
Jane Baldwin, Catherine Schmidt, Laura Plummer, Perman Gochyyev, Jillian E. Battista, Simran Kaur, Keshrie Naidoo
Education Sciences.2023; 13(5): 430. CrossRef
Predictors of Success in the National Physical Therapy Examination for Hispanic Doctor of Physical Therapy Students
Kristian J. Pamias-Velázquez, Brent Harper
Journal of Physical Therapy Education.2023; 37(3): 243. CrossRef
A Systematic Review of Variables Used in Physical Therapist Education Program Admissions Part 1: Cognitive Variables
Andrea N. Bowens
Journal of Physical Therapy Education.2023;[Epub] CrossRef
A National Study on the Relationship Between Programmatic Factors and Athletic Training Education Board of Certification Pass Rates
Leslie A. Rippon, Rong Chen, Robert J. Kelchen, Richard J. Boergers
Athletic Training Education Journal.2022; 17(1): 37. CrossRef
Predictors of Success on the National Physical Therapy Examination in 2 US Accelerated-Hybrid Doctor of Physical Therapy Programs
Breanna Reynolds, Casey Unverzagt, Alex Koszalinski, Roberta Gatlin, Jill Seale, Kendra Gagnon, Kareaion Eaton, Shane L. Koppenhaver
Journal of Physical Therapy Education.2022; 36(3): 225. CrossRef
Virtual Reality Instructional Design in Orthopedic Physical Therapy Education: A Randomized Controlled Trial
Aaron J. Hartstein, Kory Zimney, Margaret Verkuyl, Jean Yockey, Patti Berg-Poppe
Journal of Physical Therapy Education.2022; 36(2): 176. CrossRef
Evaluation of Educational Interventions Based on Average Treatment Effect: A Case Study
Jingyu Liang, Jie Liu
Mathematics.2022; 10(22): 4333. CrossRef
Faculty and programmatic influences on the percentage of graduates of color from professional physical therapy programs in the United States
Tara Dickson, Jason Zafereo
Advances in Health Sciences Education.2021; 26(1): 215. CrossRef
Predictors of research productivity among physical therapy programs in the United States: an observational study
David M. Rowland, Amanda A. Murphy, Hannah R. Manik, Chris Y. Lane, Deborah L. Givens, Chad E. Cook, Alessandra Narciso Garcia
BMC Medical Education.2020;[Epub] CrossRef
Characteristics of Professional Physical Therapist Faculty and Doctor of Physical Therapy Programs, 2008-2017: Influences on Graduation Rates and First-Time National Physical Therapy Examination Pass Rates
Tara Dickson, Barrett Taylor, Jason Zafereo
Physical Therapy.2020; 100(11): 1930. CrossRef
Predicting Success for Student Physical Therapists on the National Physical Therapy Examination: Systematic Review and Meta-Analysis
Mitch Wolden, Brent Hill, Sara Voorhees
Physical Therapy.2020; 100(1): 73. CrossRef
Current Practices and Perceptions of Admission Criteria at Physical Therapist Education Programs in the United States
Katy Mitchell, Jennifer Ellison, Peggy Gleeson
Journal of Physical Therapy Education.2019; 33(1): 55. CrossRef
Data-driven Decision Making in Doctor of Physical Therapy Curricula Part I: Program-level Analysis
Dawn Roller, Michael Wininger, John Leard, Barbara Crane
Journal of Physical Therapy Education.2018; 32(4): 368. CrossRef
20th Pauline Cerasoli Lecture: The Sunk Cost Fallacy
Chad Cook
Journal of Physical Therapy Education.2017; 31(3): 10. CrossRef

First
Prev
Page of 1
Next
Last

Search