jeehp Search


J Educ Eval Health Prof > Epub ahead of print
J Educ Eval Health Prof. 2022; 19: 26.
Published online September 8, 2022.
[Epub ahead of print]
Acceptability of the 8-case objective structured clinical examination of medical students in Korea using generalizability theory: a reliability study
Song Yi Park1,2  , Sang-Hwa Lee2  , Min-Jeong Kim3  , Ki-Hwan Ji4  , Ji Ho Ryu5 
1Department of Emergency Medicine, Dong-A University, College of Medicine, Busan, Korea
2Department of Medical Education, Dong-A University, College of Medicine, Busan, Korea
3Department of Medical Education and Neurology, Kosin University, College of Medicine, Busan, Korea
4Department of Neurology, Busan Paik Hospital, Inje University, College of Medicine, Busan, Korea
5Department of Emergency Medicine, Pusan National University, School of Medicine, Busan, Korea
Correspondence  Min-Jeong Kim ,Email:
Editor:  Sun Huh, Hallym University, Korea
Submitted: June 13, 2022  Accepted after revision: August 19, 2022
This study investigated whether the reliability was acceptable when the number of cases in the objective structured clinical examination (OSCE) decreased from 12 to 8 using generalizability theory (GT).
This psychometric study analyzed the OSCE data of 439 fourth-year medical students conducted in the Busan and Gyeongnam areas of South Korea from July 12 to 15, 2021. The generalizability study (G-study) considered 3 facets—students (p), cases (c), and items (i)—and designed the analysis as p(i:c) due to items being nested in a case. The acceptable generalizability (G) coefficient was set to 0.70. The G-study and decision study (D-study) were performed using G String IV version 6.3.8 (
All G coefficients except for July 14th (0.69) were above 0.70. The major sources of variance components (VCs) were items nested in cases (i:c), from 51.34% to 57.70%, and residual error (pi:c), from 39.55% to 43.26%. The proportion of VCs in cases was negligible, ranging from 0.00% to 2.03%.
The case numbers decreased in the 2021 Busan and Gyeongnam OSCE. However, the reliability was acceptable. In the D-study, reliability was maintained at 0.7 or higher if there were more than 21 items/case in 8 cases and more than 18 items/case in 9 cases. However, according to the G-study, increasing the number of items nested in cases rather than the number of cases could further improve reliability. The consortium needs to maintain a case bank with various items to implement a reliable blueprinting combination for the OSCE.
Keywords: Students, Medical; Educational measurement; Reproducibility of Results; Republic of Korea; Psychometrics
Share :
Facebook Twitter Linked In Google+
METRICS Graph View
  • 0 Crossref
  • 0 Scopus
  • 218 View
  • 48 Download
We recommend
Related articles in JEEHP

Empirical analysis comparing the tele-objective structured clinical examination and the in-person assessment in Australia2021 ;18

Potential of feedback during objective structured clinical examination to evoke an emotional response in medical students in Canada2020 January;17

No observed effect of a student-led mock objective structured clinical examination on subsequent performance scores in medical students in Canada2019 ;16

Comparison of the effects of simulated patient clinical skill training and student roleplay on objective structured clinical examination performance among medical students in Australia2019 ;16

Sequential Objective Structured Clinical Examination based on item response theory in Iran2017 ;14

Editorial Office
Institute of Medical Education, College of Medicine, Hallym Unversity, Hallymdaehak-gil 1, Chuncheon 24252, Korea
TEL: +82-33-248-2652   

Copyright © 2022 by Korea Health Personnel Licensing Examination Institute.

Developed in M2PI

Close layer
prev next