Journal of Educational Evaluation for Health Professions

Research articles

Improving item pool utilization for health professions examinations under variable-length computerized adaptive testing designs: a shadow-test approach: Hwanggyu Lim, Kyung (Chris) Tyek Han; J Educ Eval Health Prof. 2025;22:35. Published online November 3, 2025; DOI: https://doi.org/10.3352/jeehp.2025.22.35

2,863 View
188 Download

Abstract PDF Supplementary Material: Purpose
The shadow-test approach to computerized adaptive testing (CAT) ensures content validity in health professions examinations but may suffer from poor item pool utilization in variable-length designs, increasing operational costs and security risks. This study aimed to address this challenge by developing algorithms that enhance the sustainability of shadow CAT in variable-length design.
Methods
A simulation study was conducted to evaluate 3 proposed modifications of the α-stratification method designed to improve item pool utilization. These methods, which integrated randomesque selection and multiple-form strategies, were compared with 2 baseline algorithms within a variable-length shadow CAT framework. Performance was assessed in terms of measurement precision, pool utilization, and test efficiency.
Results
The proposed modifications significantly outperformed the baseline methods across all measures of item pool utilization and exposure control. The most effective method (Modification 2) reduced the proportion of unused items from 35.6% to 5.0% and produced more uniform item exposure rates. These substantial gains in operational sustainability were achieved while maintaining measurement precision comparable to the baseline methods.
Conclusion
The proposed algorithms effectively mitigate poor item pool utilization in shadow CAT under variable-length design. This enhanced framework provides a robust, secure, and sustainable solution for high-stakes adaptive assessments in the health professions that remain content-valid, precise, and operationally efficient.

The impact of differential item functioning on ability estimation using the Korean Medical Licensing Examination with computerized adaptive testing: a post-hoc simulation study: Dogyeong Kim, Jeongwook Choi, Dong Gi Seo; J Educ Eval Health Prof. 2025;22:31. Published online October 10, 2025; DOI: https://doi.org/10.3352/jeehp.2025.22.31

2,974 View
175 Download

Abstract PDF Supplementary Material: Purpose
This study examined the impact of differential item functioning (DIF) on ability estimation in a computerized adaptive testing (CAT) environment using real response data from the 2017 Korean Medical Licensing Examination (KMLE). We hypothesized that excluding gender-based DIF items would improve estimation accuracy, particularly for examinees at the extremes of the ability scale.
Methods
The study was conducted in 2 steps: (1) DIF detection and (2) post-hoc simulation. The analysis used data from 3,259 examinees who completed all 360 dichotomous items. Gender-based DIF was detected with the residual-based DIF method (reference group: males; focal group: females). Two CAT conditions (all items vs. DIF-excluded) were compared against a “true θ” estimated from a fixed-form test of 264 non-DIF items. Accuracy was evaluated using bias, root mean square error (RMSE), and correlation with true θ.
Results
In the CAT condition excluding DIF items, accuracy improved, with RMSE reduced and correlation with true θ increased. However, bias was slightly larger in magnitude. Gender-specific analyses showed that DIF removal reduced the underestimation of female ability but increased the underestimation of male ability, yielding estimates that were fairer across genders. When DIF items were included, estimation errors were more pronounced at both low and high ability levels.
Conclusion
Managing DIF in CAT-based high-stakes examinations can enhance fairness and precision. Using real examinee data, this study provides practical evidence of the implications of DIF for CAT-based measurement and supports fairness-oriented test design.

Technical report

Feasibility of applying computerized adaptive testing to the Clinical Medical Science Comprehensive Examination in Korea: a psychometric study: Jeongwook Choi, Sung-Soo Jung, Eun Kwang Choi, Kyung Sik Kim, Dong Gi Seo; J Educ Eval Health Prof. 2025;22:29. Published online October 1, 2025; DOI: https://doi.org/10.3352/jeehp.2025.22.29

1,277 View
187 Download

Abstract PDF Supplementary Material: Purpose
This study aimed to investigate the feasibility of transitioning the Clinical Medical Science Comprehensive Examination (CMSCE) to computerized adaptive testing (CAT) in Korea, thereby providing greater opportunities for medical students to accurately compare their clinical competencies with peers nationwide and to monitor their own progress.
Methods
A medical self-assessment using CAT was conducted from March to June 2023, involving 1,541 medical students who volunteered from 40 medical colleges in Korea. An item bank consisting of 1,145 items from previously administered CMSCE examinations (2019–2021) hosted by the Medical Education Assessment Corporation was established. Items were selected through 2-stage filtering, based on classical test theory (discrimination index above 0.15) and item response theory (discrimination parameter estimates above 0.6 and difficulty parameter estimates between –5 and +5). Maximum Fisher information was employed as the item selection method, and maximum likelihood estimation was used for ability estimation.
Results
The CAT was successfully administered without significant issues. The stopping rule was set at a standard error of measurement of 0.25, with a maximum of 50 items for ability estimation. The mean ability score was 0.55, with an average of 28 items administered per student. Students at extreme ability levels reached the maximum of 50 items due to the limited availability of items at appropriate difficulty levels.
Conclusion
The medical self-assessment CAT, the first of its kind in Korea, was successfully implemented nationwide without significant problems. These results indicate strong potential for expanding the use of CAT in medical education assessments.

Software report

Special article on the 20th anniversary of the journal

The irtQ R package: a user-friendly tool for item response theory-based test data analysis and calibration: Hwanggyu Lim, Kyungseok Kang; J Educ Eval Health Prof. 2024;21:23. Published online September 12, 2024; DOI: https://doi.org/10.3352/jeehp.2024.21.23

5,622 View
319 Download
3 Web of Science
2 Crossref

Abstract PDF Supplementary Material

Computerized adaptive testing (CAT) has become a widely adopted test design for high-stakes licensing and certification exams, particularly in the health professions in the United States, due to its ability to tailor test difficulty in real time, reducing testing time while providing precise ability estimates. A key component of CAT is item response theory (IRT), which facilitates the dynamic selection of items based on examinees' ability levels during a test. Accurate estimation of item and ability parameters is essential for successful CAT implementation, necessitating convenient and reliable software to ensure precise parameter estimation. This paper introduces the irtQ R package (http://CRAN.R-project.org/), which simplifies IRT-based analysis and item calibration under unidimensional IRT models. While it does not directly simulate CAT, it provides essential tools to support CAT development, including parameter estimation using marginal maximum likelihood estimation via the expectation-maximization algorithm, pretest item calibration through fixed item parameter calibration and fixed ability parameter calibration methods, and examinee ability estimation. The package also enables users to compute item and test characteristic curves and information functions necessary for evaluating the psychometric properties of a test. This paper illustrates the key features of the irtQ package through examples using simulated datasets, demonstrating its utility in IRT applications such as test data analysis and ability scoring. By providing a user-friendly environment for IRT analysis, irtQ significantly enhances the capacity for efficient adaptive testing research and operations. Finally, the paper highlights additional core functionalities of irtQ, emphasizing its broader applicability to the development and operation of IRT-based assessments.

Citations

Citations to this article as recorded by

Development of a CAT based Diagnostic System for Assessing Basic Academic Skills in Undergraduate Students
Woo-Jin Han, Jeongwook Choi, Dong-Gi Seo
The Korean Association of General Education.2025; 19(3): 177. CrossRef
Feasibility of applying computerized adaptive testing to the Clinical Medical Science Comprehensive Examination in Korea: a psychometric study
Jeongwook Choi, Sung-Soo Jung, Eun Kwang Choi, Kyung Sik Kim, Dong Gi Seo
Journal of Educational Evaluation for Health Professions.2025; 22: 29. CrossRef

Research article

Special article on the 20th anniversary of the journal

Comparison of real data and simulated data analysis of a stopping rule based on the standard error of measurement in computerized adaptive testing for medical examinations in Korea: a psychometric study: Dong Gi Seo, Jeongwook Choi, Jinha Kim; J Educ Eval Health Prof. 2024;21:18. Published online July 9, 2024; DOI: https://doi.org/10.3352/jeehp.2024.21.18

3,643 View
363 Download
2 Web of Science
2 Crossref

Abstract PDF Supplementary Material

Purpose
This study aimed to compare and evaluate the efficiency and accuracy of computerized adaptive testing (CAT) under 2 stopping rules (standard error of measurement [SEM]=0.3 and 0.25) using both real and simulated data in medical examinations in Korea.
Methods
This study employed post-hoc simulation and real data analysis to explore the optimal stopping rule for CAT in medical examinations. The real data were obtained from the responses of 3rd-year medical students during examinations in 2020 at Hallym University College of Medicine. Simulated data were generated using estimated parameters from a real item bank in R. Outcome variables included the number of examinees’ passing or failing with SEM values of 0.25 and 0.30, the number of items administered, and the correlation. The consistency of real CAT result was evaluated by examining consistency of pass or fail based on a cut score of 0.0. The efficiency of all CAT designs was assessed by comparing the average number of items administered under both stopping rules.
Results
Both SEM 0.25 and SEM 0.30 provided a good balance between accuracy and efficiency in CAT. The real data showed minimal differences in pass/fail outcomes between the 2 SEM conditions, with a high correlation (r=0.99) between ability estimates. The simulation results confirmed these findings, indicating similar average item numbers between real and simulated data.
Conclusion
The findings suggest that both SEM 0.25 and 0.30 are effective termination criteria in the context of the Rasch model, balancing accuracy and efficiency in CAT.

Citations

Citations to this article as recorded by

AI-enhanced adaptive testing with cognitive diagnostic feedback and its association with performance in undergraduate surgical education: a pilot study
Nuno Silva Gonçalves, Carlos Collares, José Miguel Pêgo
Frontiers in Behavioral Neuroscience.2026;[Epub] CrossRef
Feasibility of applying computerized adaptive testing to the Clinical Medical Science Comprehensive Examination in Korea: a psychometric study
Jeongwook Choi, Sung-Soo Jung, Eun Kwang Choi, Kyung Sik Kim, Dong Gi Seo
Journal of Educational Evaluation for Health Professions.2025; 22: 29. CrossRef

Software report

Introduction to the LIVECAT web-based computerized adaptive testing platform: Dong Gi Seo, Jeongwook Choi; J Educ Eval Health Prof. 2020;17:27. Published online September 29, 2020; DOI: https://doi.org/10.3352/jeehp.2020.17.27

8,566 View
161 Download
7 Web of Science
9 Crossref

Abstract PDF Supplementary Material

This study introduces LIVECAT, a web-based computerized adaptive testing platform. This platform provides many functions, including writing item content, managing an item bank, creating and administering a test, reporting test results, and providing information about a test and examinees. The LIVECAT provides examination administrators with an easy and flexible environment for composing and managing examinations. It is available at http://www.thecatkorea.com/. Several tools were used to program LIVECAT, as follows: operating system, Amazon Linux; web server, nginx 1.18; WAS, Apache Tomcat 8.5; database, Amazon RDMS—Maria DB; and languages, JAVA8, HTML5/CSS, Javascript, and jQuery. The LIVECAT platform can be used to implement several item response theory (IRT) models such as the Rasch and 1-, 2-, 3-parameter logistic models. The administrator can choose a specific model of test construction in LIVECAT. Multimedia data such as images, audio files, and movies can be uploaded to items in LIVECAT. Two scoring methods (maximum likelihood estimation and expected a posteriori) are available in LIVECAT and the maximum Fisher information item selection method is applied to every IRT model in LIVECAT. The LIVECAT platform showed equal or better performance compared with a conventional test platform. The LIVECAT platform enables users without psychometric expertise to easily implement and perform computerized adaptive testing at their institutions. The most recent LIVECAT version only provides a dichotomous item response model and the basic components of CAT. Shortly, LIVECAT will include advanced functions, such as polytomous item response models, weighted likelihood estimation method, and content balancing method.

Citations

Citations to this article as recorded by

A Systematic Review on Computerized Adaptive Testing
Hümeyra Demir, Selahattin Gelbal
Erzincan Üniversitesi Eğitim Fakültesi Dergisi.2025; 27(1): 137. CrossRef
Development of a CAT based Diagnostic System for Assessing Basic Academic Skills in Undergraduate Students
Woo-Jin Han, Jeongwook Choi, Dong-Gi Seo
The Korean Association of General Education.2025; 19(3): 177. CrossRef
Feasibility of applying computerized adaptive testing to the Clinical Medical Science Comprehensive Examination in Korea: a psychometric study
Jeongwook Choi, Sung-Soo Jung, Eun Kwang Choi, Kyung Sik Kim, Dong Gi Seo
Journal of Educational Evaluation for Health Professions.2025; 22: 29. CrossRef
Comparison of real data and simulated data analysis of a stopping rule based on the standard error of measurement in computerized adaptive testing for medical examinations in Korea: a psychometric study
Dong Gi Seo, Jeongwook Choi, Jinha Kim
Journal of Educational Evaluation for Health Professions.2024; 21: 18. CrossRef
Educational Technology in the University: A Comprehensive Look at the Role of a Professor and Artificial Intelligence
Cheolkyu Shin, Dong Gi Seo, Seoyeon Jin, Soo Hwa Lee, Hyun Je Park
IEEE Access.2024; 12: 116727. CrossRef
The irtQ R package: a user-friendly tool for item response theory-based test data analysis and calibration
Hwanggyu Lim, Kyungseok Kang
Journal of Educational Evaluation for Health Professions.2024; 21: 23. CrossRef
Presidential address: improving item validity and adopting computer-based testing, clinical skills assessments, artificial intelligence, and virtual reality in health professions licensing examinations in Korea
Hyunjoo Pai
Journal of Educational Evaluation for Health Professions.2023; 20: 8. CrossRef
Patient-reported outcome measures in cancer care: Integration with computerized adaptive testing
Minyu Liang, Zengjie Ye
Asia-Pacific Journal of Oncology Nursing.2023; 10(12): 100323. CrossRef
Development of a character qualities test for medical students in Korea using polytomous item response theory and factor analysis: a preliminary scale development study
Yera Hur, Dong Gi Seo
Journal of Educational Evaluation for Health Professions.2023; 20: 20. CrossRef

Review

Components of the item selection algorithm in computerized adaptive testing: Kyung (Chris) Tyek Han; J Educ Eval Health Prof. 2018;15:7. Published online March 24, 2018; DOI: https://doi.org/10.3352/jeehp.2018.15.7

47,879 View
559 Download
25 Web of Science
19 Crossref

Abstract PDF Supplementary Material

Computerized adaptive testing (CAT) greatly improves measurement efficiency in high-stakes testing operations through the selection and administration of test items with the difficulty level that is most relevant to each individual test taker. This paper explains the 3 components of a conventional CAT item selection algorithm: test content balancing, the item selection criterion, and item exposure control. Several noteworthy methodologies underlie each component. The test script method and constrained CAT method are used for test content balancing. Item selection criteria include the maximized Fisher information criterion, the b-matching method, the astratification method, the weighted likelihood information criterion, the efficiency balanced information criterion, and the KullbackLeibler information criterion. The randomesque method, the Sympson-Hetter method, the unconditional and conditional multinomial methods, and the fade-away method are used for item exposure control. Several holistic approaches to CAT use automated test assembly methods, such as the shadow test approach and the weighted deviation model. Item usage and exposure count vary depending on the item selection criterion and exposure control method. Finally, other important factors to consider when determining an appropriate CAT design are the computer resources requirement, the size of item pools, and the test length. The logic of CAT is now being adopted in the field of adaptive learning, which integrates the learning aspect and the (formative) assessment aspect of education into a continuous, individualized learning experience. Therefore, the algorithms and technologies described in this review may be able to help medical health educators and high-stakes test developers to adopt CAT more actively and efficiently.

Citations

Citations to this article as recorded by

Efficiency of PROMIS MCAT Assessments for Orthopaedic Care
Michael Bass, Scott Morris, Sheng Zhang
Measurement: Interdisciplinary Research and Perspectives.2025; 23(2): 124. CrossRef
Computerized Adaptive Testing Framework Based on Excitation Block and Gumbel-Softmax
Chengsong Liu, Yan Wei
IEEE Access.2025; 13: 3475. CrossRef
From Development to Validation: Exploring the Efficiency of Numetrive, a Computerized Adaptive Assessment of Numerical Reasoning
Marianna Karagianni, Ioannis Tsaousis
Behavioral Sciences.2025; 15(3): 268. CrossRef
Patient reported outcome measures: from the classics to AI
Conrad J Harrison, Ryan W Trickett
Journal of Hand Surgery (European Volume).2025; 50(6): 807. CrossRef
A hybrid model based on learning automata and cuckoo search for optimizing test item selection in computerized adaptive testing
Chanjuan Jin, Weiming Pan
Scientific Reports.2025;[Epub] CrossRef
Maximin criterion for item selection in computerized adaptive testing
Jyun-Hong Chen, Hsiu-Yi Chao
Behavior Research Methods.2025;[Epub] CrossRef
Development and initial validation of DESCAQ: A Rasch‐based computer‐adaptive questionnaire for assessing digital eye strain
Mariano González‐Pérez, Ana Barrio, Rosario Susi, Carlos Pérez‐Garmendia, Beatriz Antona
Ophthalmic and Physiological Optics.2025; 45(6): 1326. CrossRef
Development of a CAT based Diagnostic System for Assessing Basic Academic Skills in Undergraduate Students
Woo-Jin Han, Jeongwook Choi, Dong-Gi Seo
The Korean Association of General Education.2025; 19(3): 177. CrossRef
Utilizing Real-Time Test Data to Solve Attenuation Paradox in Computerized Adaptive Testing to Enhance Optimal Design
Jyun-Hong Chen, Hsiu-Yi Chao
Journal of Educational and Behavioral Statistics.2024; 49(4): 630. CrossRef
A Context-based Question Selection Model to Support the Adaptive Assessment of Learning: A study of online learning assessment in elementary schools in Indonesia
Umi Laili Yuhana, Eko Mulyanto Yuniarno, Wenny Rahayu, Eric Pardede
Education and Information Technologies.2024; 29(8): 9517. CrossRef
A shortened test is feasible: Evaluating a large-scale multistage adaptive English language assessment
Shangchao Min, Kyoungwon Bishop
Language Testing.2024; 41(3): 627. CrossRef
Implementing Computer Adaptive Testing for High-Stakes Assessment: A Shift for Examinations Council of Lesotho
Musa Adekunle Ayanwale, Julia Chere-Masopha, Mapulane Mochekele, Malebohang Catherine Morena
International Journal of New Education.2024;[Epub] CrossRef
The Effects of Different Item Selection Methods on Test Information and Test Efficiency in Computer Adaptive Testing
Merve ŞAHİN KÜRŞAD
Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi.2023; 14(1): 33. CrossRef
Presidential address: improving item validity and adopting computer-based testing, clinical skills assessments, artificial intelligence, and virtual reality in health professions licensing examinations in Korea
Hyunjoo Pai
Journal of Educational Evaluation for Health Professions.2023; 20: 8. CrossRef
Remote Symptom Monitoring With Ecological Momentary Computerized Adaptive Testing: Pilot Cohort Study of a Platform for Frequent, Low-Burden, and Personalized Patient-Reported Outcome Measures
Conrad Harrison, Ryan Trickett, Justin Wormald, Thomas Dobbs, Przemysław Lis, Vesselin Popov, David J Beard, Jeremy Rodrigues
Journal of Medical Internet Research.2023; 25: e47179. CrossRef
Evaluating a Computerized Adaptive Testing Version of a Cognitive Ability Test Using a Simulation Study
Ioannis Tsaousis, Georgios D. Sideridis, Hannan M. AlGhamdi
Journal of Psychoeducational Assessment.2021; 39(8): 954. CrossRef
Developing Multistage Tests Using D-Scoring Method
Kyung (Chris) T. Han, Dimiter M. Dimitrov, Faisal Al-Mashary
Educational and Psychological Measurement.2019; 79(5): 988. CrossRef
Conducting simulation studies for computerized adaptive testing using SimulCAT: an instructional piece
Kyung (Chris) Tyek Han
Journal of Educational Evaluation for Health Professions.2018; 15: 20. CrossRef
Updates from 2018: Being indexed in Embase, becoming an affiliated journal of the World Federation for Medical Education, implementing an optional open data policy, adopting principles of transparency and best practice in scholarly publishing, and appreci
Sun Huh
Journal of Educational Evaluation for Health Professions.2018; 15: 36. CrossRef

Research Article

Estimation of an Examinee's Ability in the Web-Based Computerized Adaptive Testing Program IRT-CAT: Yoon-Hwan Lee, Jung-Ho Park, In-Yong Park; J Educ Eval Health Prof. 2006;3:4. Published online November 22, 2006; DOI: https://doi.org/10.3352/jeehp.2006.3.4

56,667 View
178 Download
5 Crossref

Abstract PDF

We developed a program to estimate an examinee's ability in order to provide freely available access to a web-based computerized adaptive testing (CAT) program. We used PHP and Java Script as the program languages, PostgresSQL as the database management system on an Apache web server and Linux as the operating system. A system which allows for user input and searching within inputted items and creates tests was constructed. We performed an ability estimation on each test based on a Rasch model and 2- or 3-parametric logistic models. Our system provides an algorithm for a web-based CAT, replacing previous personal computer-based ones, and makes it possible to estimate an examinee?占퐏 ability immediately at the end of test.

Citations

Citations to this article as recorded by

Analysis on Validity and Academic Competency of Mock Test for Korean Medicine National Licensing Examination Using Item Response Theory
Han Chae, Eunbyul Cho, SeonKyoung Kim, DaHye Choi, Seul Lee
Keimyung Medical Journal.2023; 42(1): 7. CrossRef
Accuracy and Efficiency of Web-based Assessment Platform (LIVECAT) for Computerized Adaptive Testing
Do-Gyeong Kim, Dong-Gi Seo
The Journal of Korean Institute of Information Technology.2020; 18(4): 77. CrossRef
Computer‐Based Testing and Construction of an Item Bank Database for Medical Education in Korea
Sun Huh
Korean Medical Education Review.2014; 16(1): 11. CrossRef
Can computerized tests be introduced to the Korean Medical Licensing Examination?
Sun Huh
Journal of the Korean Medical Association.2012; 55(2): 124. CrossRef
Application of Computerized Adaptive Testing in Medical Education
Sun Huh
Korean Journal of Medical Education.2009; 21(2): 97. CrossRef

Original Articles

Correlations between the scores of computerized adaptive testing, paper and pencil tests, and the Korean Medical Licensing Examination: Mee Young Kim, Yoon Hwan Lee, Sun Huh; J Educ Eval Health Prof. 2005;2(1):113-118. Published online June 30, 2005; DOI: https://doi.org/10.3352/jeehp.2005.2.1.113

44,584 View
167 Download
3 Crossref

Abstract PDF

To evaluate the usefulness of computerized adaptive testing (CAT) in medical school, the General Examination for senior medical students was administered as a paper and pencil test (P&P) and using CAT. The General Examination is a graduate examination, which is also a preliminary examination for the Korean Medical Licensing Examination (KMLE). The correlations between the results of the CAT and P&P and KMLE were analyzed. The correlation between the CAT and P&P was 0.8013 (p=0.000); that between the CAT and P&P was 0.7861 (p=0.000); and that between the CAT and KMLE was 0.6436 (p=0.000). Six out of 12 students with an ability estimate below 0.52 failed the KMLE. The results showed that CAT could replace P&P in medical school. The ability of CAT to predict whether students would pass the KMLE was 0.5 when the criterion of the theta value was set at -0.52 that was chosen arbitrarily for the prediction of pass or failure.

Citations

Citations to this article as recorded by

Analysis on Validity and Academic Competency of Mock Test for Korean Medicine National Licensing Examination Using Item Response Theory
Han Chae, Eunbyul Cho, SeonKyoung Kim, DaHye Choi, Seul Lee
Keimyung Medical Journal.2023; 42(1): 7. CrossRef
Application of Computerized Adaptive Testing in Medical Education
Sun Huh
Korean Journal of Medical Education.2009; 21(2): 97. CrossRef
Estimation of an Examinee's Ability in the Web-Based Computerized Adaptive Testing Program IRT-CAT
Yoon-Hwan Lee, Jung-Ho Park, In-Yong Park
Journal of Educational Evaluation for Health Professions.2006; 3: 4. CrossRef

Students' Attitude toward and Acceptability of Computerized Adaptive Testing in Medical School and their Effect on the Examinees' Ability: Mee Young Kim, Sun Huh; J Educ Eval Health Prof. 2005;2(1):105-111. Published online June 30, 2005; DOI: https://doi.org/10.3352/jeehp.2005.2.1.105

33,778 View
178 Download
3 Crossref

Abstract PDF

An examinee's ability can be evaluated precisely using computerized adaptive testing (CAT), which is shorter than written tests and more efficient in terms of the duration of the examination. We used CAT for the second General Examination of 98 senior students in medical college on November 27, 2004. We prepared 1,050 pre-calibrated test items according to item response theory, which had been used for the General Examination administered to senior students in 2003. The computer was programmed to pose questions until the standard error of the ability estimate was smaller than 0.01. To determine the students' attitude toward and evaluation of CAT, we conducted surveys before and after the examination, via the Web. The mean of the students' ability estimates was 0.3513 and its standard deviation was 0.9097 (range -2.4680 to +2.5310). There was no significant difference in the ability estimates according to the responses of students to items concerning their experience with CAT, their ability to use a computer, or their anxiety before and after the examination (p>0.05). Many students were unhappy that they could not recheck their responses (49%), and some stated that there were too few examination items (24%). Of the students, 79 % had no complaints concerning using a computer and 63% wanted to expand the use of CAT. These results indicate that CAT can be implemented in medical schools without causing difficulties for users.

Citations

Citations to this article as recorded by

Computer‐Based Testing and Construction of an Item Bank Database for Medical Education in Korea
Sun Huh
Korean Medical Education Review.2014; 16(1): 11. CrossRef
Can computerized tests be introduced to the Korean Medical Licensing Examination?
Sun Huh
Journal of the Korean Medical Association.2012; 55(2): 124. CrossRef
Application of Computerized Adaptive Testing in Medical Education
Sun Huh
Korean Journal of Medical Education.2009; 21(2): 97. CrossRef

First
Prev
Page of 1
Next
Last

Search