1School of Medicine, The University of New South Wales, Kensington, Australia
2Office of Medical Education, University of New South Wales, Sydney, Australia
3Centre for Medical and Health Sciences Education, Faculty of Medical and Health Sciences, University of Auckland, Auckland, New Zealand
© 2021 Korea Health Personnel Licensing Examination Institute
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Authors’ contributions
Conceptualisation: JZF, DWA, ST, BS. Data curation: BS. Formal analysis: BS. Funding acquisition: not applicable. Methodology: JZF, DWA, ST, BS. Project administration: DWA. Writing–original draft: JZF, DWA, ST, BS. Writing–review & editing: JZF, DWA, ST, BS.
Conflict of interest
Boaz Shulruf has been an associate editor of the Journal of Educational Evaluation for Health Professions since 2017, but had no role in the decision to publish this review. No other potential conflict of interest relevant to this article was reported.
Funding
None.
Data availability
Data files are available from Harvard Dataverse: https://doi.org/10.7910/DVN/0XDXKS
Dataset 1. Results of 2 objective structured clinical examination (OCSE) examinations of medical students in the University of New South Wales, Australia in 2019 and 2020.
Domain |
N |
Mean |
SD |
SEM |
t-value | Equal variance assumed | df | Sig. | Mean difference | 95% CI | Effect size | ||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
2019 | 2020 | 2019 | 2020 | 2019 | 2020 | 2019 | 2020 | ||||||||
Communication | 280 | 285 | 7.618 | 7.569 | 0.621 | 0.571 | 0.0371 | 0.0338 | 0.971 | Yes | 563.00 | 0.332 | 0.049 | -0.050 to 0.147 | -0.082 |
Clinical history & background | 280 | 285 | 7.090 | 7.147 | 0.866 | 0.691 | 0.0518 | 0.0409 | -0.864 | No | 532.48 | 0.389 | -0.057 | -0.186 to 0.072 | 0.073 |
Medical knowledge | 280 | 285 | 7.186 | 7.122 | 0.887 | 0.908 | 0.0530 | 0.0538 | 0.848 | Yes | 563.00 | 0.397 | 0.064 | -0.084 to 0.212 | -0.071 |
Physical examination | 280 | 285 | 7.104 | 7.381 | 0.847 | 0.822 | 0.0506 | 0.0487 | -3.947 | Yes | 563.00 | <0.001a) | -0.277 | -0.415 to -0.139 | 0.332 |
Interpretation | 280 | 285 | 7.058 | 7.156 | 0.822 | 0.796 | 0.0491 | 0.0472 | -1.445 | Yes | 563.00 | 0.149 | 0-.098 | -0.232 to 0.035 | 0.121 |
Diagnosis & management | 280 | 285 | 7.292 | 7.394 | 0.786 | 0.819 | 0.047 | 0.0485 | -1.514 | Yes | 563.00 | 0.131 | -0.102 | -0.235 to 0.030 | 0.127 |
Discipline |
N |
Mean |
SD |
SEM |
t-value | Equal variance assumed | df | Sig. | Mean difference | 95% CI | Effect size | ||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
2019 | 2020 | 2019 | 2020 | 2019 | 2020 | 2019 | 2020 | ||||||||
Medicine | 280 | 285 | 7.275 | 7.492 | 0.768 | 0.921 | 0.0459 | 0.0546 | -3.036 | No | 548.43 | 0.003a) | -0.216 | -0.357 to -0.076 | 0.257 |
Surgery | 280 | 285 | 7.376 | 7.354 | 0.851 | 0.934 | 0.0509 | 0.0553 | 0.298 | Yes | 563.00 | 0.766 | 0.022 | -0.125 to 0.170 | -0.025 |
Psychiatry | 280 | 285 | 7.344 | 7.227 | 0.921 | 0.925 | 0.0550 | 0.0548 | 1.505 | Yes | 563.00 | 0.133 | 0.117 | -0.036 to 0.269 | -0.127 |
Emergency medicine | 280 | 285 | 7.032 | 7.422 | 1.172 | 0.975 | 0.0701 | 0.0578 | -4.304 | Yes | 563.00 | <0.001b) | -0.390 | -0.568 to -0.212 | 0.363 |
Obstetrics & gynecology | 280 | 285 | 7.440 | 7.521 | 1.056 | 1.012 | 0.0631 | 0.0600 | -0.934 | Yes | 563.00 | 0.351 | -0.081 | -0.252 to 0.090 | 0.078 |
Primary care | 280 | 285 | 6.936 | 6.985 | 1.125 | 0.951 | 0.0672 | 0.0564 | -0.565 | No | 544.63 | 0.572 | -0.050 | -0.222 to 0.123 | 0.047 |
Paediatrics | 280 | 285 | 7.501 | 7.438 | 0.958 | 0.793 | 0.0573 | 0.0470 | 0.858 | No | 540.44 | 0.391 | 0.064 | -0.082 to 0.209 | -0.072 |
Method of the examination | In person | TeleOSCE |
---|---|---|
Planning a relevant physical examination | Formulated according to the station requirements, the information gleaned from history and observable features of the patient | Formulated according to the station requirements, the information gleaned from history and observable features of the patient |
Performing the physical examination | Performed on the patient | Steps listed for examiner |
Extracting clinical information | Interpreted from steps of the clinical examination | Clinical information is provided by the examiner to the student upon request |
Formulating a diagnosis | Extrapolated from the clinical information garnered from previous steps | Extrapolated from the clinical information garnered from previous steps |
Domain | N |
Mean |
SD |
SEM |
t-value | Equal variance assumed | df | Sig. | Mean difference | 95% CI | Effect size | ||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
2019 | 2020 | 2019 | 2020 | 2019 | 2020 | 2019 | 2020 | ||||||||
Communication | 280 | 285 | 7.618 | 7.569 | 0.621 | 0.571 | 0.0371 | 0.0338 | 0.971 | Yes | 563.00 | 0.332 | 0.049 | -0.050 to 0.147 | -0.082 |
Clinical history & background | 280 | 285 | 7.090 | 7.147 | 0.866 | 0.691 | 0.0518 | 0.0409 | -0.864 | No | 532.48 | 0.389 | -0.057 | -0.186 to 0.072 | 0.073 |
Medical knowledge | 280 | 285 | 7.186 | 7.122 | 0.887 | 0.908 | 0.0530 | 0.0538 | 0.848 | Yes | 563.00 | 0.397 | 0.064 | -0.084 to 0.212 | -0.071 |
Physical examination | 280 | 285 | 7.104 | 7.381 | 0.847 | 0.822 | 0.0506 | 0.0487 | -3.947 | Yes | 563.00 | <0.001 |
-0.277 | -0.415 to -0.139 | 0.332 |
Interpretation | 280 | 285 | 7.058 | 7.156 | 0.822 | 0.796 | 0.0491 | 0.0472 | -1.445 | Yes | 563.00 | 0.149 | 0-.098 | -0.232 to 0.035 | 0.121 |
Diagnosis & management | 280 | 285 | 7.292 | 7.394 | 0.786 | 0.819 | 0.047 | 0.0485 | -1.514 | Yes | 563.00 | 0.131 | -0.102 | -0.235 to 0.030 | 0.127 |
Discipline | N |
Mean |
SD |
SEM |
t-value | Equal variance assumed | df | Sig. | Mean difference | 95% CI | Effect size | ||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
2019 | 2020 | 2019 | 2020 | 2019 | 2020 | 2019 | 2020 | ||||||||
Medicine | 280 | 285 | 7.275 | 7.492 | 0.768 | 0.921 | 0.0459 | 0.0546 | -3.036 | No | 548.43 | 0.003 |
-0.216 | -0.357 to -0.076 | 0.257 |
Surgery | 280 | 285 | 7.376 | 7.354 | 0.851 | 0.934 | 0.0509 | 0.0553 | 0.298 | Yes | 563.00 | 0.766 | 0.022 | -0.125 to 0.170 | -0.025 |
Psychiatry | 280 | 285 | 7.344 | 7.227 | 0.921 | 0.925 | 0.0550 | 0.0548 | 1.505 | Yes | 563.00 | 0.133 | 0.117 | -0.036 to 0.269 | -0.127 |
Emergency medicine | 280 | 285 | 7.032 | 7.422 | 1.172 | 0.975 | 0.0701 | 0.0578 | -4.304 | Yes | 563.00 | <0.001 |
-0.390 | -0.568 to -0.212 | 0.363 |
Obstetrics & gynecology | 280 | 285 | 7.440 | 7.521 | 1.056 | 1.012 | 0.0631 | 0.0600 | -0.934 | Yes | 563.00 | 0.351 | -0.081 | -0.252 to 0.090 | 0.078 |
Primary care | 280 | 285 | 6.936 | 6.985 | 1.125 | 0.951 | 0.0672 | 0.0564 | -0.565 | No | 544.63 | 0.572 | -0.050 | -0.222 to 0.123 | 0.047 |
Paediatrics | 280 | 285 | 7.501 | 7.438 | 0.958 | 0.793 | 0.0573 | 0.0470 | 0.858 | No | 540.44 | 0.391 | 0.064 | -0.082 to 0.209 | -0.072 |
TeleOSCE, tele-objective structured clinical examination.
SD, standard deviation; SEM, standard error of the mean; df, degrees of freedom; Sig., significance; CI, confidence interval. Significant at P<0.001 after correction for alpha inflation (Bonferroni correction).
SD, standard deviation; SEM, standard error of the mean; df, degrees of freedom; Sig., significance; CI, confidence interval. Significant at P<0.05 after correction for alpha inflation (Bonferroni correction). Significant at P<0.01 after correction for alpha inflation (Bonferroni correction).