Journal of Educational Evaluation for Health Professions

Review

Performance of large language models in medical licensing examinations: a systematic review and meta-analysis: Haniyeh Nouri, Abdollah Mahdavi, Ali Abedi, Alireza Mohammadnia, Mahnaz Hamedan, Masoud Amanzadeh; J Educ Eval Health Prof. 2025;22:36. Published online November 18, 2025; DOI: https://doi.org/10.3352/jeehp.2025.22.36

2,707 View
231 Download
3 Web of Science
4 Crossref

Purpose
This study systematically evaluates and compares the performance of large language models (LLMs) in answering medical licensing examination questions. By conducting subgroup analyses based on language, question format, and model type, this meta-analysis aims to provide a comprehensive overview of LLM capabilities in medical education and clinical decision-making.
Methods
This systematic review, registered in PROSPERO and following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, searched MEDLINE (PubMed), Scopus, and Web of Science for relevant articles published up to February 1, 2025. The search strategy included Medical Subject Headings (MeSH) terms and keywords related to (“ChatGPT” OR “GPT” OR “LLM variants”) AND (“medical licensing exam*” OR “medical exam*” OR “medical education” OR “radiology exam*”). Eligible studies evaluated LLM accuracy on medical licensing examination questions. Pooled accuracy was estimated using a random-effects model, with subgroup analyses by LLM type, language, and question format. Publication bias was assessed using Egger’s regression test.
Results
This systematic review identified 2,404 studies. After removing duplicates and excluding irrelevant articles through title and abstract screening, 36 studies were included after full-text review. The pooled accuracy was 72% (95% confidence interval, 70.0% to 75.0%) with high heterogeneity (I2=99%, P<0.001). Among LLMs, GPT-4 achieved the highest accuracy (81%), followed by Bing (79%), Claude (74%), Gemini/Bard (70%), and GPT-3.5 (60%) (P=0.001). Performance differences across languages (range, 62% in Polish to 77% in German) were not statistically significant (P=0.170).
Conclusion
LLMs, particularly GPT-4, can match or exceed medical students’ examination performance and may serve as supportive educational tools. However, due to variability and the risk of errors, they should be used cautiously as complements rather than replacements for traditional learning methods.

Citations

Citations to this article as recorded by

Effective prompt design for large language models in clinical practice
Steven Callens
Acta Clinica Belgica.2026; 81(2): 118. CrossRef
Examination of Gemini's ability to answer anatomical questions: An overview
D. Chytas, G. Noussios, M.-K. Kaseta, D. Chrysikos, A.V. Vasiliadis, C. Lyrtzis, T. Troupis
Morphologie.2026; 110(369): 101118. CrossRef
Evaluación comparativa de modelos de inteligencia artificial de última generación frente a psiquiatras humanos en el examen nacional de subespecialidad en Perú: un estudio transversal
Javier A. Flores-Cohaila, Jeff Huarcaya-Victoria, Cesar Copaja-Corzo
Educación Médica.2026; 27(3): 101179. CrossRef
ChatGPT vs Claude: Scoping Review with ☸️SAIMSARA

SAIMSARA Journal.2026;[Epub] CrossRef

Research articles

Leveraging feedback mechanisms to improve the quality of objective structured clinical examinations in Singapore: an exploratory action research study: Han Ting Jillian Yeo, Dujeepa Dasharatha Samarasekera, Michael Dean; J Educ Eval Health Prof. 2025;22:28. Published online September 30, 2025; DOI: https://doi.org/10.3352/jeehp.2025.22.28

3,523 View
176 Download

Abstract PDF Supplementary Material: Purpose
Variability in examiner scoring threatens the fairness and reliability of objective structured clinical examinations (OSCEs). While examiner standardization exists, there is currently no structured, psychometric-informed, individualized feedback mechanism for examiners. This study explored the feasibility and perceived value of such a mechanism using an action research approach to co-design and iteratively refine examiner feedback reports.
Methods
Two exploratory cycles were conducted between November 2023 and June 2024 with phase 4 OSCE examiners at the Yong Loo Lin School of Medicine. In cycle 1, psychometric analyses of examiner scoring for a phase 4 OSCE informed the design of individualized reports, which were evaluated through interviews. Revisions were made to the format of the report and implemented in cycle 2, where examiner responses were again collected. Data were analyzed thematically, supported by reflective logs and field notes.
Results
Nine examiners participated in cycle 1 and 7 in cycle 2. In cycle 1, examiners highlighted challenges in interpreting complex terminology, leading to report refinements such as glossaries and visual graphs. In cycle 2, examiners demonstrated greater confidence in applying feedback, requested longitudinal reports, and shifted from initial resistance to reflective engagement. Across cycles, the reports improved credibility, neutrality, and examiner self-regulation.
Conclusion
This exploratory study suggests that psychometric-informed feedback reports can facilitate examiner reflection and transparency in OSCEs. While the findings highlight feasibility and examiner acceptance, longitudinal delivery of feedback, collection of quantitative outcome data, and larger samples are needed to establish whether such reports improve scoring consistency and assessment fairness.

Assessment of the viability of integrating virtual reality programs in practical tests for the Korean Radiological Technologists Licensing Examination: a survey study: Hye Min Park, Eun Seong Kim, Deok Mun Kwon, Pyong Kon Cho, Seoung Hwan Kim, Ki Baek Lee, Seong Hu Kim, Moon Il Bong, Won Seok Yang, Jin Eui Kim, Gi Bong Kang, Yong Su Yoon, Jung Su Kim; J Educ Eval Health Prof. 2023;20:33. Published online November 28, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.33

4,861 View
152 Download
1 Web of Science
1 Crossref

Abstract PDF Supplementary Material

Purpose
The objective of this study was to assess the feasibility of incorporating virtual reality/augmented reality (VR/AR) programs into practical tests administered as part of the Korean Radiological Technologists Licensing Examination (KRTLE). This evaluation is grounded in a comprehensive survey that targeted enrolled students in departments of radiology across the nation.
Methods
In total, 682 students from radiology departments across the nation were participants in the survey. An online survey platform was used, and the questionnaire was structured into 5 distinct sections and 27 questions. A frequency analysis for each section of the survey was conducted using IBM SPSS ver. 27.0.
Results
Direct or indirect exposure to VR/AR content was reported by 67.7% of all respondents. Furthermore, 55.4% of the respondents expressed that VR/AR could be integrated into their classes, which signified a widespread acknowledgment of VR among the students. With regards to the integration of a VR/AR or mixed reality program into the practical tests for purposes of the KRTLE, a substantial amount of the respondents (57.3%) exhibited a positive inclination and recommended its introduction.
Conclusion
The application of VR/AR programs within practical tests of the KRTLE will be used as an alternative for evaluating clinical examination procedures and validating job skills.

Citations

Citations to this article as recorded by

Understanding perception of the radiology community concerning virtual reality (VR) and augmented reality (AR) technology in radiology education
Suneet Paulson, Dwight Figueiredo, Sushant Matre
Egyptian Journal of Radiology and Nuclear Medicine.2025;[Epub] CrossRef

Is online objective structured clinical examination teaching an acceptable replacement in post-COVID-19 medical education in the United Kingdom?: a descriptive study: Vashist Motkur, Aniket Bharadwaj, Nimalesh Yogarajah; J Educ Eval Health Prof. 2022;19:30. Published online November 7, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.30

5,646 View
180 Download
6 Web of Science
6 Crossref

Abstract PDF Supplementary Material

Purpose
Coronavirus disease 2019 (COVID-19) restrictions resulted in an increased emphasis on virtual communication in medical education. This study assessed the acceptability of virtual teaching in an online objective structured clinical examination (OSCE) series and its role in future education.
Methods
Six surgical OSCE stations were designed, covering common surgical topics, with specific tasks testing data interpretation, clinical knowledge, and communication skills. These were delivered via Zoom to students who participated in student/patient/examiner role-play. Feedback was collected by asking students to compare online teaching with previous experiences of in-person teaching. Descriptive statistics were used for Likert response data, and thematic analysis for free-text items.
Results
Sixty-two students provided feedback, with 81% of respondents finding online instructions preferable to paper equivalents. Furthermore, 65% and 68% found online teaching more efficient and accessible, respectively, than in-person teaching. Only 34% found communication with each other easier online; Forty percent preferred online OSCE teaching to in-person teaching. Students also expressed feedback in positive and negative free-text comments.
Conclusion
The data suggested that generally students were unwilling for online teaching to completely replace in-person teaching. The success of online teaching was dependent on the clinical skill being addressed; some were less amenable to a virtual setting. However, online OSCE teaching could play a role alongside in-person teaching.

Citations

Citations to this article as recorded by

Perception of online learning, knowledge assessment, and clinical skills among third-year ophthalmology residents studying during the COVID-19 pandemic in Thailand
Wanicha Chuenkongkaew, Jimethat Chalermpong, Weerawat Kiddee, Prut Hanutsaha, Anita Manassakorn, Olan Suwan-apichon, Sakchai Vongkittirux, Raveewan Choontanom, Pittaya Phamonvaechavan, Tharnsook Kasemsup, Paradee Kunavisarut, Sudawadee Somboonthanakij, Su
Scientific Reports.2026;[Epub] CrossRef
The assessment of clinical competence in medical students during the COVID-19 pandemic: a scoping review
Harry McGrath, Elysha Brennan, Dominic Harmon
Irish Journal of Medical Science (1971 -).2026;[Epub] CrossRef
Evaluation of a virtual objective structured clinical examination in the metaverse (Second Life) to assess the clinical skills in emergency radiology of medical students in Spain: a cross-sectional study
Alba Virtudes Perez-Baena, Teodoro Rudolphi-Solero, Rocio Lorenzo-Alvarez, Dolores Dominguez-Pinos, Miguel Jose Ruiz-Gomez, Francisco Sendra-Portero
Journal of Educational Evaluation for Health Professions.2025; 22: 12. CrossRef
A Comparative Study of Student Perspectives on Online Versus In-Person Objective Structured Clinical Examination (OSCE) Teaching at a Medical School in London
Nimalesh Yogarajah, Aniket Bharadwaj, Amelia Snook, Vashist Motkur
Cureus.2025;[Epub] CrossRef
Feasibility and reliability of the pandemic-adapted online-onsite hybrid graduation OSCE in Japan
Satoshi Hara, Kunio Ohta, Daisuke Aono, Toshikatsu Tamai, Makoto Kurachi, Kimikazu Sugimori, Hiroshi Mihara, Hiroshi Ichimura, Yasuhiko Yamamoto, Hideki Nomura
Advances in Health Sciences Education.2024; 29(3): 949. CrossRef
Should Virtual Objective Structured Clinical Examination (OSCE) Teaching Replace or Complement Face-to-Face Teaching in the Post-COVID-19 Educational Environment: An Evaluation of an Innovative National COVID-19 Teaching Programme
Charles Gamble, Alice Oatham, Raj Parikh
Cureus.2023;[Epub] CrossRef

Comparing the cut score for the borderline group method and borderline regression method with norm-referenced standard setting in an objective structured clinical examination in medical school in Korea: Song Yi Park, Sang-Hwa Lee, Min-Jeong Kim, Ki-Hwan Ji, Ji Ho Ryu; J Educ Eval Health Prof. 2021;18:25. Published online September 27, 2021; DOI: https://doi.org/10.3352/jeehp.2021.18.25

9,602 View
339 Download
4 Web of Science
5 Crossref

Abstract PDF Supplementary Material

Purpose
Setting standards is critical in health professions. However, appropriate standard setting methods do not always apply to the set cut score in performance assessment. The aim of this study was to compare the cut score when the standard setting is changed from the norm-referenced method to the borderline group method (BGM) and borderline regression method (BRM) in an objective structured clinical examination (OSCE) in medical school.
Methods
This was an explorative study to model the implementation of the BGM and BRM. A total of 107 fourth-year medical students attended the OSCE at 7 stations for encountering standardized patients (SPs) and at 1 station for performing skills on a manikin on July 15th, 2021. Thirty-two physician examiners evaluated the performance by completing a checklist and global rating scales.
Results
The cut score of the norm-referenced method was lower than that of the BGM (P<0.01) and BRM (P<0.02). There was no significant difference in the cut score between the BGM and BRM (P=0.40). The station with the highest standard deviation and the highest proportion of the borderline group showed the largest cut score difference in standard setting methods.
Conclusion
Prefixed cut scores by the norm-referenced method without considering station contents or examinee performance can vary due to station difficulty and content, affecting the appropriateness of standard setting decisions. If there is an adequate consensus on the criteria for the borderline group, standard setting with the BRM could be applied as a practical and defensible method to determine the cut score for OSCE.

Citations

Citations to this article as recorded by

Refining competency benchmarks: a scoping review of Angoff standard-setting in dental education
Galvin Sim Siang Lin, Abdul Rauf Badrul Hisham, Muhammad Nazmi Abdul Majid, Chan Choong Foong, Ting Khee Ho, Lara T. Friedlander
BMC Oral Health.2026;[Epub] CrossRef
Standard setting methods in objective structured clinical examination (OSCE): A comparative study of five methods
Reshma Ansari, Norhafizah Ab Manan, Nur Ain Mahat, Norfaizatul Shalida Omar, Atikah Abdul Latiff, Sara Idris, Azli Shahril Othman
Journal of Medical Education Development.2024; 17(56): 87. CrossRef
Analyzing the Quality of Objective Structured Clinical Examination in Alborz University of Medical Sciences
Suleiman Ahmadi, Amin Habibi, Mitra Rahimzadeh, Shahla Bahrami
Alborz University Medical Journal.2023; 12(4): 485. CrossRef
Possibility of using the yes/no Angoff method as a substitute for the percent Angoff method for estimating the cutoff score of the Korean Medical Licensing Examination: a simulation study
Janghee Park
Journal of Educational Evaluation for Health Professions.2022; 19: 23. CrossRef
Newly appointed medical faculty members’ self-evaluation of their educational roles at the Catholic University of Korea College of Medicine in 2020 and 2021: a cross-sectional survey-based study
Sun Kim, A Ra Cho, Chul Woon Chung
Journal of Educational Evaluation for Health Professions.2021; 18: 28. CrossRef

Empirical analysis comparing the tele-objective structured clinical examination and the in-person assessment in Australia: Jonathan Zachary Felthun, Silas Taylor, Boaz Shulruf, Digby Wigram Allen; J Educ Eval Health Prof. 2021;18:23. Published online September 23, 2021; DOI: https://doi.org/10.3352/jeehp.2021.18.23

8,824 View
258 Download
4 Web of Science
5 Crossref

Abstract PDF Supplementary Material

Purpose
It aimed to compare the use of the tele-objective structured clinical examination (teleOSCE) with in-person assessment in high-stakes clinical examination so as to determine the impact of the teleOSCE on the assessment undertaken. Discussion follows regarding what skills and domains can effectively be assessed in a teleOSCE.
Methods
This study is a retrospective observational analysis. It compares the results achieved by final year medical students in their clinical examination, assessed using the teleOSCE in 2020 (n=285), with those who were examined using the traditional in-person format in 2019 (n=280). The study was undertaken at the University of New South Wales, Australia.
Results
In the domain of physical examination, students in 2020 scored 0.277 points higher than those in 2019 (mean difference=–0.277, P<0.001, effect size=0.332). Across all other domains, there was no significant difference in mean scores between 2019 and 2020.
Conclusion
The teleOSCE does not negatively impact assessment in clinical examination in all domains except physical examination. If the teleOSCE is the future of clinical skills examination, assessment of physical examination will require concomitant workplace-based assessment.

Citations

Citations to this article as recorded by

Feasibility of an online clinical assessment of competence in physiotherapy students
Brooke Flew, Lucy Chipchase, Darren Lee, Jodie A. McClelland
Physiotherapy Theory and Practice.2025; 41(3): 508. CrossRef
Navigating digital assessments in medical education: Findings from a scoping review
Chin-Siang Ang, Sakura Ito, Jennifer Cleland
Medical Teacher.2025; 47(8): 1274. CrossRef
Feasibility and reliability of the pandemic-adapted online-onsite hybrid graduation OSCE in Japan
Satoshi Hara, Kunio Ohta, Daisuke Aono, Toshikatsu Tamai, Makoto Kurachi, Kimikazu Sugimori, Hiroshi Mihara, Hiroshi Ichimura, Yasuhiko Yamamoto, Hideki Nomura
Advances in Health Sciences Education.2024; 29(3): 949. CrossRef
Radiography education in 2022 and beyond - Writing the history of the present: A narrative review
Y.X. Tay, J.P. McNulty
Radiography.2023; 29(2): 391. CrossRef
Newly appointed medical faculty members’ self-evaluation of their educational roles at the Catholic University of Korea College of Medicine in 2020 and 2021: a cross-sectional survey-based study
Sun Kim, A Ra Cho, Chul Woon Chung
Journal of Educational Evaluation for Health Professions.2021; 18: 28. CrossRef

Review

Assessment methods and the validity and reliability of measurement tools in online objective structured clinical examinations: a systematic scoping review: Jonathan Zachary Felthun, Silas Taylor, Boaz Shulruf, Digby Wigram Allen; J Educ Eval Health Prof. 2021;18:11. Published online June 1, 2021; DOI: https://doi.org/10.3352/jeehp.2021.18.11

12,659 View
495 Download
18 Web of Science
18 Crossref

Abstract PDF Supplementary Material

The coronavirus disease 2019 (COVID-19) pandemic has required educators to adapt the in-person objective structured clinical examination (OSCE) to online settings in order for it to remain a critical component of the multifaceted assessment of a student’s competency. This systematic scoping review aimed to summarize the assessment methods and validity and reliability of the measurement tools used in current online OSCE (hereafter, referred to as teleOSCE) approaches. A comprehensive literature review was undertaken following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines. Articles were eligible if they reported any form of performance assessment, in any field of healthcare, delivered in an online format. Two reviewers independently screened the results and analyzed relevant studies. Eleven articles were included in the analysis. Pre-recorded videos were used in 3 studies, while observations by remote examiners through an online platform were used in 7 studies. Acceptability as perceived by students was reported in 2 studies. This systematic scoping review identified several insights garnered from implementing teleOSCEs, the components transferable from telemedicine, and the need for systemic research to establish the ideal teleOSCE framework. TeleOSCEs may be able to improve the accessibility and reproducibility of clinical assessments and equip students with the requisite skills to effectively practice telemedicine in the future.

Citations

Citations to this article as recorded by

Intra-rater reliability of in-person versus simulated remote synchronous faculty evaluation of pharmacy student objective structured clinical examinations
Alex N. Isaacs, Jenny L. Newlon, Kimberly S. Illingworth, Alan J. Zillich, Zachary A. Weber, Jamie L. Woodyard
Currents in Pharmacy Teaching and Learning.2026; 18(2): 102520. CrossRef
Introducing a Smartphone Tele-Objective Structured Clinical Examination to Support High-Stakes Competency Decisions: A Quasi-Experimental Study and Curricular Implications
Xiaozhi Wang, Junjie Du, Binlin Luo, Liling Chen, Huanhuan Chen, Surong Jiang, Wei Sun, Lei Zhou, Lars Konge, Hua Huang, Qiang Ding
Journal of Medical Education and Curricular Development.2026;[Epub] CrossRef
Validity of pediatric index of mortality 2 scoring in a pediatric ICU setting in Ethiopia: A prospective observational study
Mikiyas G. Teferi, Bethel A. Awoke, Iyassu S. Melkie, Oghenekenu Oshare, Soliana Solomon Birhanu, Gelila Gemeda Gage, Biruk T. Mengistie, Chernet T. Mengistie, Alehilign M. Abebe
Global Pediatrics.2026; 15: 100322. CrossRef
Assessing fundamental clinical skills of osteopathic medical students
John R. Boulet, Jeanne M. Sandella, John Gimpel, Richard LaBaere
Journal of Osteopathic Medicine.2025; 125(10): 469. CrossRef
Exploring an online clinical competency assessment: an alternative to a traditional in-person assessment for internationally trained physiotherapists
Brooke Flew, Lucy Chipchase, Darren Lee, Jodie A. McClelland
BMC Medical Education.2025;[Epub] CrossRef
Structured oral clinical assessment for pharmacotherapy competencies in medical education: a study of validity and reliability analyses of seven domains
Abdul Khairul Rizki Purba, David Sontani Perdanakusuma, Arifa Mustika, Tanja Fens, Maarten Jacobus Postma
Korean Journal of Medical Education.2025; 37(3): 331. CrossRef
Feasibility and reliability of the pandemic-adapted online-onsite hybrid graduation OSCE in Japan
Satoshi Hara, Kunio Ohta, Daisuke Aono, Toshikatsu Tamai, Makoto Kurachi, Kimikazu Sugimori, Hiroshi Mihara, Hiroshi Ichimura, Yasuhiko Yamamoto, Hideki Nomura
Advances in Health Sciences Education.2024; 29(3): 949. CrossRef
A level playing field? Evaluation of the virtual Objective Structured Clinical Examination in Psychiatry and Addiction Medicine: A mixed methods study
Rebecca E Reay, Paul A Maguire, Jeffrey CL Looi
Australasian Psychiatry.2024; 32(4): 359. CrossRef
Conducting an objective structured clinical examination under COVID-restricted conditions
Andrea Gotzmann, John Boulet, Yichi Zhang, Judy McCormick, Mathieu Wojcik, Ilona Bartman, Debra Pugh
BMC Medical Education.2024;[Epub] CrossRef
The virtual Clinical Assessment of Skills and Competence: the impact and challenges of a digitised final examination
Kenny Chu, Shivanthi Sathanandan
BJPsych Bulletin.2023; 47(2): 110. CrossRef
Virtual Learning and Assessment in Rheumatology Fellowship Training: Objective Structured Clinical Examination Revisited
Rachel M. Wolfe, Faye N. Hant, Rumey C. Ishizawar, Lisa G. Criscione‐Schreiber, Beth L. Jonas, Kenneth S. O'Rourke, Marcy B. Bolster
Arthritis Care & Research.2023; 75(12): 2435. CrossRef
Innovations in assessment in health professions education during the COVID‐19 pandemic: A scoping review
Jamal Giri, Claire Stewart
The Clinical Teacher.2023;[Epub] CrossRef
Evaluation of the Utility of Online Objective Structured Clinical Examination Conducted During the COVID-19 Pandemic
Mona Arekat, Mohamed Hany Shehata, Abdelhalim Deifalla, Ahmed Al-Ansari, Archana Kumar, Mohamed Alsenbesy, Hamdi Alshenawi, Amgad El-Agroudy, Mariwan Husni, Diaa Rizk, Abdelaziz Elamin, Afif Ben Salah, Hani Atwa
Advances in Medical Education and Practice.2022; Volume 13: 407. CrossRef
Comparison of student pharmacists' performance on in-person vs. virtual OSCEs in a pre-APPE capstone course
Justine S. Gortney, Joseph P. Fava, Andrew D. Berti, Brittany Stewart
Currents in Pharmacy Teaching and Learning.2022; 14(9): 1116. CrossRef
Is online objective structured clinical examination teaching an acceptable replacement in post-COVID-19 medical education in the United Kingdom?: a descriptive study
Vashist Motkur, Aniket Bharadwaj, Nimalesh Yogarajah
Journal of Educational Evaluation for Health Professions.2022; 19: 30. CrossRef
Equal Z standard-setting method to estimate the minimum number of panelists for a medical school’s objective structured clinical examination in Taiwan: a simulation study
Ying-Ying Yang, Pin-Hsiang Huang, Ling-Yu Yang, Chia-Chang Huang, Chih-Wei Liu, Shiau-Shian Huang, Chen-Huan Chen, Fa-Yauh Lee, Shou-Yen Kao, Boaz Shulruf
Journal of Educational Evaluation for Health Professions.2022; 19: 27. CrossRef
Applying the Student Response System in the Online Dermatologic Video Curriculum on Medical Students' Interaction and Learning Outcomes during the COVID-19 Pandemic
Chih-Tsung Hung, Shao-An Fang, Feng-Cheng Liu, Chih-Hsiung Hsu, Ting-Yu Yu, Wei-Ming Wang
Indian Journal of Dermatology.2022; 67(4): 477. CrossRef
Empirical analysis comparing the tele-objective structured clinical examination and the in-person assessment in Australia
Jonathan Zachary Felthun, Silas Taylor, Boaz Shulruf, Digby Wigram Allen
Journal of Educational Evaluation for Health Professions.2021; 18: 23. CrossRef

Research articles

Malaysian pharmacy students’ perspectives on the virtual objective structured clinical examination during the coronavirus disease 2019 pandemic: Mohamed Hassan Elnaem, Muhammad Eid Akkawi, Nor Ilyani Mohamed Nazar, Norny Syafinaz Ab Rahman, Mohamad Haniki Nik Mohamed; J Educ Eval Health Prof. 2021;18:6. Published online April 12, 2021; DOI: https://doi.org/10.3352/jeehp.2021.18.6

9,812 View
340 Download
16 Web of Science
16 Crossref

Abstract PDF Supplementary Material

Purpose
This study investigated pharmacy students’ perceptions of various aspects of virtual objective structured clinical examinations (vOSCEs) conducted during the coronavirus disease 2019 pandemic in Malaysia.
Methods
This cross-sectional study involved third- and fourth-year pharmacy students at the International Islamic University Malaysia. A validated self-administered questionnaire was distributed to students who had taken a vOSCE a week before.
Results
Out of the 253 students who were approached, 231 (91.3%) completed the questionnaire. More than 75% of the participants agreed that the instructions and preparations were clear and helpful in familiarizing them with the vOSCE flow. It was found that 53.2% of the respondents were satisfied with the flow and conduct of the vOSCE. However, only approximately one-third of the respondents believed that the tasks provided in the vOSCE were more convenient, less stressful, and easier to perform than those in the conventional OSCE. Furthermore, 49.7% of the students favored not having a vOSCE in the future when conducting a conventional OSCE becomes feasible again. Internet connection was reported as a problem hindering the performance of the vOSCE by 51.9% of the participants. Students who were interested in clinical pharmacy courses were more satisfied than other students with the preparation and operation of the vOSCE, the faculty support, and the allocated time.
Conclusion
Students were satisfied with the organization and operation of the vOSCE. However, they still preferred the conventional OSCE over the vOSCE. These findings might indicate a further need to expose students to telehealthcare models.

Citations

Citations to this article as recorded by

Evaluating the impact of required attendance on examination performance and pharmacy student perceptions in an infectious diseases course
Dana R. Bowers, Alanda A. Barash, Kimberly C. McKeirnan
Currents in Pharmacy Teaching and Learning.2026; 18(1): 102487. CrossRef
Comparison of the adequacies of the OSCE and vOSCE to assess the competencies required under Brazilian medical curriculum guidelines: a multicenter study
Hervaldo Sampaio Carvalho, Maria do Carmo Barros Melo, Fabiana Maria Kakehasi, Priscila Menezes Ferri Liu, Fernanda Gontijo Minafra Silveira Santos, Ricardo Luiz de Melo Martins, Marilene Vale de Castro Monteiro, Marcelo Esteves Chaves Campos, Roberto Zon
BMC Medical Education.2025;[Epub] CrossRef
What's been trending with OSCEs in pharmacy education over the last 20 years? A bibliometric review and content analysis
Angelina S. Lim, Yeap Li Ling, Kyle J. Wilby, Vivienne Mak
Currents in Pharmacy Teaching and Learning.2024; 16(3): 212. CrossRef
Evaluation of grit and its associated factors among undergraduate pharmacy students from 14 Asian and Middle Eastern countries amid the COVID-19 pandemic
Mohamed Hassan Elnaem, Muna Barakat, Naeem Mubarak, Mohammed Salim K.T., Doaa H. Abdelaziz, Ahmed Ibrahim Fathelrahman, Abrar K. Thabit, Diana Laila Ramatillah, Ali Azeez Al-Jumaili, Nabeel Kashan Syed, Mohammed Fathelrahman Adam, Md. Sanower Hossain, Moh
Saudi Pharmaceutical Journal.2023; 31(3): 410. CrossRef
Shifting to Authentic Assessments? A Systematic Review of Student Perceptions of High-Fidelity Assessments in Pharmacy
Harjit Singh, Daniel Malone, Angelina S. Lim
American Journal of Pharmaceutical Education.2023; 87(7): 100099. CrossRef
A Telehealth Module and Virtual Objective Structured Clinical Examination of Health Literacy in Pharmacy Education
Sanah Hasan, Hamzah AlZubaidi, Subish Palaian, Muaed AlOmar, Nadir Kheir, Yassin Al Hariri, Sawsan Shanableh, Ahmed Gaili, Abby Kahaleh
American Journal of Pharmaceutical Education.2023; 87(12): 100555. CrossRef
Is It Time to Revise the Competency-Based Assessment? Objective Structured Clinical Examination and Technology Integration
Haniye Mastour, Nazanin Shamaeian Razavi
Shiraz E-Medical Journal.2023;[Epub] CrossRef
Virtual OSCE: Experience and challenges with a large cohort of pharmacy students
Hanis Hanum Zulkifly, Izzati Abdul Halim Zaki, Mahmathi Karuppannan, Zakiah Mohd Noordin
Pharmacy Education.2022; 22(1): 23. CrossRef
Students’ and Examiners’ Experiences of Their First Virtual Pharmacy Objective Structured Clinical Examination (OSCE) in Australia during the COVID-19 Pandemic
Vivienne Mak, Sunanthiny Krishnan, Sara Chuang
Healthcare.2022; 10(2): 328. CrossRef
Perceptions of Pharmacy Students on the E-Learning Strategies Adopted during the COVID-19 Pandemic: A Systematic Review
Carla Pires
Pharmacy.2022; 10(1): 31. CrossRef
Perceptions of undergraduate pharmacy students towards online assessments used during the COVID-19 pandemic in a public university in Malaysia
Usman Abubakar, A'isyah Humaira' Mohd Salehudin, Nik Afiqah Athirah Nik Mohd Asri, Nur Atiqah Mohammad Rohi, Nur Hasyimah Ramli, Nur Izzah Mohd Khairuddin, Nur Fariesya Saiful Izham, Siti Hajar Nasrullah, Auwal Adam Sa’ad
Pharmacy Education.2022; 22(1): 191. CrossRef
Evaluation of the Utility of Online Objective Structured Clinical Examination Conducted During the COVID-19 Pandemic
Mona Arekat, Mohamed Hany Shehata, Abdelhalim Deifalla, Ahmed Al-Ansari, Archana Kumar, Mohamed Alsenbesy, Hamdi Alshenawi, Amgad El-Agroudy, Mariwan Husni, Diaa Rizk, Abdelaziz Elamin, Afif Ben Salah, Hani Atwa
Advances in Medical Education and Practice.2022; Volume 13: 407. CrossRef
COVID-19-Driven Improvements and Innovations in Pharmacy Education: A Scoping Review
Jennifer Courtney, Erika Titus-Lay, Ashim Malhotra, Jeffrey Nehira, Islam Mohamed, Welly Mente, Uyen Le, Linda Buckley, Xiaodong Feng, Ruth Vinall
Pharmacy.2022; 10(3): 60. CrossRef
Supporting pharmacy students' preparation for an entry-to-practice OSCE using video cases
Michelle Flood, Judith Strawbridge, Eimear Ní Sheachnasaigh, Theo Ryan, Laura J. Sahm, Aoife Fleming, James W. Barlow
Currents in Pharmacy Teaching and Learning.2022; 14(12): 1525. CrossRef
Applying the Student Response System in the Online Dermatologic Video Curriculum on Medical Students' Interaction and Learning Outcomes during the COVID-19 Pandemic
Chih-Tsung Hung, Shao-An Fang, Feng-Cheng Liu, Chih-Hsiung Hsu, Ting-Yu Yu, Wei-Ming Wang
Indian Journal of Dermatology.2022; 67(4): 477. CrossRef
Empirical analysis comparing the tele-objective structured clinical examination and the in-person assessment in Australia
Jonathan Zachary Felthun, Silas Taylor, Boaz Shulruf, Digby Wigram Allen
Journal of Educational Evaluation for Health Professions.2021; 18: 23. CrossRef

Increased competency of registered dietitian nutritionists in physical examination skills after simulation-based education in the United States: Elizabeth MacQuillan, Jennifer Ford, Kristin Baird; J Educ Eval Health Prof. 2020;17:40. Published online December 14, 2020; DOI: https://doi.org/10.3352/jeehp.2020.17.40

8,302 View
181 Download
2 Web of Science
4 Crossref

Abstract PDF Supplementary Material

Purpose
This study aimed to translate simulation-based dietitian nutritionist education to clinical competency attainment in a group of practicing registered dietitian nutritionists (RDNs). Using a standardized instrument to measure performance on a newly-required clinical skill, the nutrition-focused physical exam (NFPE), competence was measured both before and after a simulation-based education (SBE) session.
Methods
Eighteen practicing RDNs were recruited by their employer, Spectrum Health. Following a pre-briefing session, participants completed an initial 10-minute encounter, performing NFPE on a standardized patient (SP). Next, participants completed a 90-minute SBE training session on skills within the NFPE, including hands-on practice and role play, followed by a post-training SP encounter. Video recordings of the SP encounters were scored to assess competence in 7 skill areas within the NFPE. Scores were analyzed for participants’ initial competence and change in competence.
Results
The proportions of participants with initial competence ranged from 0% to 44% across the 7 skill areas assessed. The only competency where participants initially scored in the “meets expectations” range was “approach to the patient.” When raw competence scores were assessed for changes from pre- to post-SBE training, the paired t-test indicated significant increases in all 7 competency areas following the simulation-based training (P<0.001).
Conclusion
This study showed the effectiveness of a SBE training program for increasing competence scores of practicing RDNs on a defined clinical skill.

Citations

Citations to this article as recorded by

Simulation‐Based Training for Nutrition‐Focused Physical Exams and Malnutrition Diagnosis for Rural Preceptors
Danielle M. Villano, Brenna Swanson, Nathaniel Johnson
Journal of Human Nutrition and Dietetics.2026;[Epub] CrossRef
Evaluation of Mental Health First Aid Training and Simulated Psychosis Care Role-Plays for Pharmacy Education
Tina X. Ung, Claire L. O’Reilly, Rebekah J. Moles, Jack C. Collins, Ricki Ng, Lily Pham, Bandana Saini, Jennifer A. Ong, Timothy F. Chen, Carl R. Schneider, Sarira El-Den
American Journal of Pharmaceutical Education.2024; 88(11): 101288. CrossRef
Barriers for Liver Transplant in Patients with Alcohol-Related Hepatitis
Gina Choi, Jihane N. Benhammou, Jung J. Yum, Elena G. Saab, Ankur P. Patel, Andrew J. Baird, Stephanie Aguirre, Douglas G. Farmer, Sammy Saab
Journal of Clinical and Experimental Hepatology.2022; 12(1): 13. CrossRef
Assessment of Mental Health First Aid Training and Simulated Psychosis Role-Play Exercises in Pharmacy Education
Fatou S. Ndiaye, Amina B. Diallo
Annals of Pharmacy Education, Safety, and Public Health Advocacy.2022; 2(1): 167. CrossRef

Female medical and nursing students’ knowledge, attitudes, and skills regarding breast self-examination in Oman: a comparison between pre- and post-training: Rajani Ranganath, John Muthusami, Miriam Simon, Tatiyana Mandal, Meena Anand Kukkamulla; J Educ Eval Health Prof. 2020;17:37. Published online December 1, 2020; DOI: https://doi.org/10.3352/jeehp.2020.17.37

10,290 View
353 Download
6 Web of Science
7 Crossref

Abstract PDF Supplementary Material

Purpose
Breast cancer is one of the most common cancers in women worldwide. Educational and awareness programs impact early practices of breast self-examination, resulting in the early detection of cancer and thereby decreasing mortality. The study aimed to assess the levels of knowledge and awareness of breast cancer and breast self-examination among medical and nursing students in Oman and to compare their knowledge, attitudes, and skills after a training program.
Methods
This quasi-experimental study was carried out for female 90 medical and 80 nursing students in Oman in November 2019. A pre-test questionnaire was given before the training program and a post-test questionnaire was administered after the training program. Students’ knowledge, attitude, and skills regarding breast cancer and breast self-examination were compared. Scores for skills of practicing breast self-examination were compared between lecture and activity group and lecture-only group.
Results
Pre-test and post-test data were collected from 170 female students. Significant improvements were observed in the post-test scores for students’ knowledge, attitude, and skills after the intervention (P<0.001). The mean scores for skills of practicing breast self-examination after the lecture and the activity were higher than those obtained after the lecture only (P=0.014 for medical students and P=0.016 for nursing students).
Conclusion
An educational training program on breast cancer and breast examination with an emphasis on skills can motivate participants to perform breast self-examination regularly, and may therefore help students to train other women to perform breast self-examination for the early detection of breast cancer.

Citations

Citations to this article as recorded by

Effectiveness of face to face and virtual education to promote breast self-examination based on the theory of planned behavior: a randomized controlled trial study
Halime Cheraghalizadeh, Hajar Adib-Rad, Hajar Pasha, Mohammad Chehrazi, Fatemeh Nasiri‑Amiri, Shabnam Omidvar
BMC Cancer.2025;[Epub] CrossRef
Predicting breast self-examination awareness in Sub-Saharan Africa using machine learning
Nebebe Demis Baykemagn, Meron Asmamaw Alemayehu, Tirualem Zeleke Yehuala, Agmasie Damtew Walle, Andualem Enyew Gedefaw, Abraham Keffale Mengistu
Scientific Reports.2025;[Epub] CrossRef
Breast Self-Examination: Evaluating Knowledge, Attitudes, and Practices Among Female Medical Students
Shruti Raghavan, Shraddha Mishra, Abhijit Das, Sandhya Singh
Cureus.2025;[Epub] CrossRef
Nursing Students’ Motivation, Awareness, and Knowledge of Women’s Health: A Norwegian Quasi-Experimental Study
Christine Tørris
Education Sciences.2024; 14(3): 273. CrossRef
Breast self-examination among female medical students at Damascus University: A cross-sectional study
Mohammed Alshafie, Anas Bitar, Massa Alfawal, Mhd Basheer Alameer, Dima Alhomsi, Maher Saifo
Heliyon.2024; 10(15): e35312. CrossRef
Kavram Haritası ile Verilen Kendi Kendine Meme Muayenesi Eğitiminin Hemşirelik Öğrencilerinin Sağlık İnançları ve Öz Yeterlilik Düzeylerine Etkisi
Aysun Acun, Yadigar Ordu
Black Sea Journal of Health Science.2023; 6(4): 632. CrossRef
Effectiveness of Online Peer-Assisted Learning Session in Fostering the Knowledge on Breast Cancer and Breast Self-Examination among Undergraduate Medical Students
R Ranganath, MA Simon, YA Shah, FI AlAbduwani, H Al Mubarak, FA Al-Shamsi
Journal of Nature and Science of Medicine.2023; 6(2): 71. CrossRef

Similarity of the cut score in test sets with different item amounts using the modified Angoff, modified Ebel, and Hofstee standard-setting methods for the Korean Medical Licensing Examination: Janghee Park, Mi Kyoung Yim, Na Jin Kim, Duck Sun Ahn, Young-Min Kim; J Educ Eval Health Prof. 2020;17:28. Published online October 5, 2020; DOI: https://doi.org/10.3352/jeehp.2020.17.28

10,083 View
223 Download
9 Web of Science
8 Crossref

Abstract PDF Supplementary Material

Purpose
The Korea Medical Licensing Exam (KMLE) typically contains a large number of items. The purpose of this study was to investigate whether there is a difference in the cut score between evaluating all items of the exam and evaluating only some items when conducting standard-setting.
Methods
We divided the item sets that appeared on 3 recent KMLEs for the past 3 years into 4 subsets of each year of 25% each based on their item content categories, discrimination index, and difficulty index. The entire panel of 15 members assessed all the items (360 items, 100%) of the year 2017. In split-half set 1, each item set contained 184 (51%) items of year 2018 and each set from split-half set 2 contained 182 (51%) items of the year 2019 using the same method. We used the modified Angoff, modified Ebel, and Hofstee methods in the standard-setting process.
Results
Less than a 1% cut score difference was observed when the same method was used to stratify item subsets containing 25%, 51%, or 100% of the entire set. When rating fewer items, higher rater reliability was observed.
Conclusion
When the entire item set was divided into equivalent subsets, assessing the exam using a portion of the item set (90 out of 360 items) yielded similar cut scores to those derived using the entire item set. There was a higher correlation between panelists’ individual assessments and the overall assessments.

Citations

Citations to this article as recorded by

Standard setting for dental knowledge tests: reproducibility of the modified Angoff and Ebel method across judges
Ting Khee Ho, Noor Lide Abu Kassim, Lucy O’Malley, Reza Vahid Roudsari
BMC Medical Education.2025;[Epub] CrossRef
Angoff methods in standard setting in health professional education: a systematic review and meta-analysis
Kannan Sridharan, Gowri Sivaramakrishnan
BMC Medical Education.2025;[Epub] CrossRef
Application of computer-based testing in the Korean Medical Licensing Examination, the emergence of the metaverse in medical education, journal metrics and statistics, and appreciation to reviewers and volunteers
Sun Huh
Journal of Educational Evaluation for Health Professions.2022; 19: 2. CrossRef
Possibility of using the yes/no Angoff method as a substitute for the percent Angoff method for estimating the cutoff score of the Korean Medical Licensing Examination: a simulation study
Janghee Park
Journal of Educational Evaluation for Health Professions.2022; 19: 23. CrossRef
Equal Z standard-setting method to estimate the minimum number of panelists for a medical school’s objective structured clinical examination in Taiwan: a simulation study
Ying-Ying Yang, Pin-Hsiang Huang, Ling-Yu Yang, Chia-Chang Huang, Chih-Wei Liu, Shiau-Shian Huang, Chen-Huan Chen, Fa-Yauh Lee, Shou-Yen Kao, Boaz Shulruf
Journal of Educational Evaluation for Health Professions.2022; 19: 27. CrossRef
Possibility of independent use of the yes/no Angoff and Hofstee methods for the standard setting of the Korean Medical Licensing Examination written test: a descriptive study
Do-Hwan Kim, Ye Ji Kang, Hoon-Ki Park
Journal of Educational Evaluation for Health Professions.2022; 19: 33. CrossRef
Presidential address: Quarantine guidelines to protect examinees from coronavirus disease 2019, clinical skills examination for dental licensing, and computer-based testing for medical, dental, and oriental medicine licensing
Yoon-Seong Lee
Journal of Educational Evaluation for Health Professions.2021; 18: 1. CrossRef
Comparing the cut score for the borderline group method and borderline regression method with norm-referenced standard setting in an objective structured clinical examination in medical school in Korea
Song Yi Park, Sang-Hwa Lee, Min-Jeong Kim, Ki-Hwan Ji, Ji Ho Ryu
Journal of Educational Evaluation for Health Professions.2021; 18: 25. CrossRef

Correlation between physician assistant students’ performance score of history taking and physical exam documentation and scores of Graduate Record Examination, clinical year grade point average, and score of Physician Assistant National Certifying Exam in the United States: Sara Lolar, Jamie McQueen, Sara Maher; J Educ Eval Health Prof. 2020;17:16. Published online May 27, 2020; DOI: https://doi.org/10.3352/jeehp.2020.17.16

9,551 View
169 Download
1 Web of Science
6 Crossref

Abstract PDF Supplementary Material

Purpose
Learning to perform and document patient history taking and physical exam (H&P) entails a major component of the first year academic education of physician assistant (PA) students at Wayne State University, USA. The H&P is summative of multiple aspects of PA education, and students must master communication with patients and other health care providers. The objectives of this study were first, to determine if there was a correlation between scores on the Graduate Record Examination (GRE) component testing and scores on graded H&Ps. The second objective was to identify a correlation between proficiency with H&P documentation and academic and clinical year grade point average (GPA) and Physician Assistant National Certifying Exam (PANCE) score.
Methods
Subjects included 147 PA students from Wayne State University from 2014–2016. PA students visited local hospitals or outpatient clinics during the academic year to perform and document patient H&Ps. Correlation between the H&P mean scores and GRE component scores, GPAs, and PANCE scores were analyzed.
Results
The subjects were 26.5 years-old (+6.5) and 111 females (75.5%). There was no correlation between the GRE component score and the H&P mean score. The H&P score was positively correlated with GPA 1 (r=0.512, P<0.001), with GPA 2 (r=0.425, P<0.001) and with PANCE score (r=0.448, P<0.001).
Conclusion
PA student skill with H&P documentation was positively related to academic performance score during PA school and achievement score on the PANCE at Wayne State University, USA.

Citations

Citations to this article as recorded by

Graduate Record Examination Removal From Admissions and Physician Assistant/Associate Student Diversity
Bettie Coplan, Anthony Miller, Tamara Ritsema, Kirsten Bonnin, Morgan Luck, Jane McDaniel, Curt Bay, Angela Kiselyk, Jonathan Kilstrom, Michelle DiBaise
The Journal of Physician Assistant Education.2026; 37(1): 19. CrossRef
Entrustable Professional Activities for Newly Graduated Physician Associates: Validation Through a Modified Delphi Study
David J. Bunnell
The Journal of Physician Assistant Education.2026;[Epub] CrossRef
Association Between Physician Assistant Program Characteristics and Student Diversity
Ryan D. White, Bettie Coplan, Brenda Quincy, Adrian Banning, Chris Gillette, Matthew Wright, Michele Toussaint, Nina Multak, Mary Warner
The Journal of Physician Assistant Education.2025; 36(3): e253. CrossRef
Evaluating the role of GRE and students’ success in the didactic phase of PA school
Jeanetta Yuan, Benjamin Rispoli, Harrynauth Persaud
SN Social Sciences.2025;[Epub] CrossRef
History-taking level and its influencing factors among nursing undergraduates based on the virtual standardized patient testing results: Cross sectional study
Jingrong Du, Xiaowen Zhu, Juan Wang, Jing Zheng, Xiaomin Zhang, Ziwen Wang, Kun Li
Nurse Education Today.2022; 111: 105312. CrossRef
A Decline in Black and Dermatology Physician Assistants
Jameka McElroy-Brooklyn, Cynthia Faires Griffith
The Journal of Physician Assistant Education.2022; 33(4): 275. CrossRef

Brief Report

Potential of feedback during objective structured clinical examination to evoke an emotional response in medical students in Canada: Dalia Limor Karol, Debra Pugh; J Educ Eval Health Prof. 2020;17:5. Published online February 18, 2020; DOI: https://doi.org/10.3352/jeehp.2020.17.5

10,019 View
177 Download
4 Web of Science
4 Crossref

Abstract PDF Supplementary Material

Feedback has been shown to be an important driver for learning. However, many factors, such as the emotional reactions feedback evokes, may impact its effect. This study aimed to explore medical students’ perspectives on the verbal feedback they receive during an objective structured clinical examination (OSCE); their emotional reaction to this; and its impact on their subsequent performance. To do this, medical students enrolled at 4 Canadian medical schools were invited to complete a web-based survey regarding their experiences. One hundred and fifty-eight participants completed the survey. Twenty-nine percent of respondents asserted that they had experienced emotional reactions to verbal feedback received in an OSCE setting. The most common emotional responses reported were embarrassment and anxiousness. Some students (n=20) reported that the feedback they received negatively impacted subsequent OSCE performance. This study demonstrates that feedback provided during an OSCE can evoke an emotional response in students and potentially impact subsequent performance.

Citations

Citations to this article as recorded by

Helping students bridge their cognitive competence gap: Effectiveness of a faculty development workshop on ‘giving feedback’. A mixed methods study
JYOTSNA AGARWAL, VIKRAMJEET SINGH, MANISH KUMAR SINGH, THOMAS V. CHACKO
The National Medical Journal of India.2026; 39: 108. CrossRef
Faculty Perceptions of Feedback in Objective Structured Clinical Examinations (OSCEs)
Zainab Abdullah, Lubna Kashif, Marina Khan, Syeda Sanaa Fatima, Asya Tauqir, Saima Manzoor
Pakistan Journal of Health Sciences.2026; : 118. CrossRef
Memory, credibility and insight: How video-based feedback promotes deeper reflection and learning in objective structured clinical exams
Alexandra Makrides, Peter Yeates
Medical Teacher.2022; 44(6): 664. CrossRef
Objective structured clinical examination in fundamentals of nursing and obstetric care as method of verification and assessing the degree of achievement of learning outcomes
Lucyna Sochocka, Teresa Niechwiadowicz-Czapka, Mariola Wojtal, Monika Przestrzelska, Iwona Kiersnowska, Katarzyna Szwamel
Pielegniarstwo XXI wieku / Nursing in the 21st Century.2021; 20(3): 190. CrossRef

Research articles

Dreyfus scale-based feedback increased medical students’ satisfaction with the complex cluster part of a interviewing and physical examination course and improved skills readiness in Taiwan: Shiau-Shian Huang, Chia-Chang Huang, Ying-Ying Yang, Shuu-Jiun Wang, Boaz Shulruf, Chen-Huan Chen; J Educ Eval Health Prof. 2019;16:30. Published online October 11, 2019; DOI: https://doi.org/10.3352/jeehp.2019.16.30

12,826 View
176 Download
1 Web of Science

Abstract PDF Supplementary Material: Purpose
In contrast to the core part of the clinical interviewing and physical examination (PE) skills course, corresponding to the basic, head-to-toe, and thoracic systems, learners need structured feedback in the cluster part of the course, which includes the abdominal, neuromuscular, and musculoskeletal systems. This study evaluated the effects of using Dreyfus scale-based feedback, which has elements of continuous professional development, instead of Likert scale-based feedback in the cluster part of training in Taiwan.
Methods
Instructors and final-year medical students in the 2015–2016 classes of National Yang-Ming University, Taiwan comprised the regular cohort, whereas those in the 2017–2018 classes formed the intervention cohort. In the intervention cohort, Dreyfus scale-based feedback, rather than Likert scale-based feedback, was used in the cluster part of the course.
Results
In the cluster part of the course in the regular cohort, pre-trained standardized patients rated the class climate as poor, and students expressed low satisfaction with the instructors and course and low self-assessed readiness. In comparison with the regular cohort, improved end-of-course group objective structured clinical examination scores after the cluster part were noted in the intervention cohort. In other words, the implementation of Dreyfus scale-based feedback in the intervention cohort for the cluster part improved the deficit in this section of the course.
Conclusion
The implementation of Dreyfus scale-based feedback helped instructors to create a good class climate in the cluster part of the clinical interviewing and PE skills course. Simultaneously, this new intervention achieved the goal of promoting medical students’ readiness for interviewing, PE, and self-directed learning.

Mismatch between the proposed ability concepts of the Graduate Record Examination and the critical thinking skills of physical therapy applicants suggested by an expert panel in the United States: Emily Shannon Hughes; J Educ Eval Health Prof. 2019;16:24. Published online August 27, 2019; DOI: https://doi.org/10.3352/jeehp.2019.16.24

13,595 View
204 Download
4 Web of Science
5 Crossref

Abstract PDF Supplementary Material

Purpose
The Graduate Record Examination (GRE) is a general examination predictive of success in US-based graduate programs. Used to assess students’ written, mathematical, and critical thinking (CT) skills, the GRE is utilized for admission to approximately 85% of US physical therapist education (PTE) programs. The purpose of this study was to assess whether the CT skills measured by the GRE match those deemed by an expert panel as the most important to assess for PTE program acceptance.
Methods
Using a modified E-Delphi approach, a 3-phase survey was distributed over 8 weeks to a panel consisting of licensed US physical therapists with expertise on CT and PTE program directors. The CT skills isolated by the expert panel, based on Facione’s Delphi report, were compared to the CT skills assessed by the GRE.
Results
The CT skills supported by the Delphi report and chosen by the expert panel for assessment prior to acceptance into US PTE programs included clarifying meaning, categorization, and analyzing arguments. Only clarifying meaning matched the CT skills from the GRE.
Conclusion
The GRE is a test for general admission to graduate programs, lacking context related to healthcare or physical therapy. The current study fails to support the GRE as an assessment tool of CT for admission to PTE programs. A context-based admission test evaluating the CT skills identified in this study should be developed for use in the admission process to predict which students will complete US PTE programs and pass the licensure exam.

Citations

Citations to this article as recorded by

Psychometric evaluation of the Spanish version of the physiotherapy critical thinking in clinical practice questionnaire
Beatriz Hernández-Méndez, Àlex Ginés-Puertas, Javier Jerez-Roig, Joan-Daniel Martí-Romeu, David Cámara-Menoyo, Jordi Cuartero-Archs, Mercedes Piqueras-Céspedes, María Llaberia-Marcual, Esperanza Zuriguel-Pérez
Clinical Rehabilitation.2025; 39(10): 1355. CrossRef
Evaluating the role of GRE and students’ success in the didactic phase of PA school
Jeanetta Yuan, Benjamin Rispoli, Harrynauth Persaud
SN Social Sciences.2025;[Epub] CrossRef
The Doctor of Physical Therapy Admission Test (DPTAT): A Vision for a Primary Care, Doctoring Profession
Nathan J. Savage
Medical Science Educator.2025; 35(6): 2803. CrossRef
Diversifying the Physical Therapist Workforce Through Holistic Hiring, Admissions, and Retention Processes
Nipaporn Somyoo, Kimberly Varnado, Eder A Garavito, Janet Kneiss
Physical Therapy.2024;[Epub] CrossRef
Correlation between physician assistant students’ performance score of history taking and physical exam documentation and scores of Graduate Record Examination, clinical year grade point average, and score of Physician Assistant National Certifying Exam i
Sara Lolar, Jamie McQueen, Sara Maher
Journal of Educational Evaluation for Health Professions.2020; 17: 16. CrossRef

Brief report

No observed effect of a student-led mock objective structured clinical examination on subsequent performance scores in medical students in Canada: Lorenzo Madrazo, Claire Bo Lee, Meghan McConnell, Karima Khamisa, Debra Pugh; J Educ Eval Health Prof. 2019;16:14. Published online May 27, 2019; DOI: https://doi.org/10.3352/jeehp.2019.16.14

17,015 View
219 Download
11 Web of Science
10 Crossref

Abstract PDF Supplementary Material

Student-led peer-assisted mock objective structured clinical examinations (MOSCEs) have been used in various settings to help students prepare for subsequent higher-stakes, faculty-run OSCEs. MOSCE participants generally valued feedback from peers and reported benefits to learning. Our study investigated whether participation in a peer-assisted MOSCE affected subsequent OSCE performance. To determine whether mean OSCE scores differed depending on whether medical students participated in the MOSCE, we conducted a between-subjects analysis of variance, with cohort (2016 vs. 2017) and MOSCE participation (MOSCE vs. no MOSCE) as independent variables and the mean OSCE score as the dependent variable. Participation in the MOSCE had no influence on mean OSCE scores (P=0.19). There was a significant correlation between mean MOSCE scores and mean OSCE scores (Pearson r=0.52, P<0.001). Although previous studies described self-reported benefits from participation in student-led MOSCEs, it was not associated with objective benefits in this study.

Citations

Citations to this article as recorded by

Empowering medical students: Peer-Led OSCE reduces anxiety and may enhance test performance
Leonardo Mateus de Lima, Maria Helena Favarato, Iolanda Fátima Lopes Calvo Tibério, Lorenzo Faggioni
PLOS One.2026; 21(2): e0340407. CrossRef
Impact of familiarity with the format of the exam on performance in the OSCE of undergraduate medical students – an interventional study
Hannes Neuwirt, Iris E. Eder, Philipp Gauckler, Lena Horvath, Stefan Koeck, Maria Noflatscher, Benedikt Schaefer, Anja Simeon, Verena Petzer, Wolfgang M. Prodinger, Christoph Berendonk
BMC Medical Education.2024;[Epub] CrossRef
Perceived and actual value of Student‐led Objective Structured Clinical Examinations
Brandon Stretton, Adam Montagu, Aline Kunnel, Jenni Louise, Nathan Behrendt, Joshua Kovoor, Stephen Bacchi, Josephine Thomas, Ellen Davies
The Clinical Teacher.2024;[Epub] CrossRef
Improving preparation for pharmacy entry-to-practice OSCE using a participatory action research
Catherine Huneault, Philippe Haeberli, Alexandra Mühle, Philippe Laurent, Jérôme Berger
Currents in Pharmacy Teaching and Learning.2024; 16(11): 102152. CrossRef
Benefits of semiology taught using near-peer tutoring are sustainable
Benjamin Gripay, Thomas André, Marie De Laval, Brice Peneau, Alexandre Secourgeon, Nicolas Lerolle, Cédric Annweiler, Grégoire Justeau, Laurent Connan, Ludovic Martin, Loïc Bière
BMC Medical Education.2022;[Epub] CrossRef
Identification des facteurs associés à la réussite aux examens cliniques objectifs et structurés dans la faculté de médecine de Rouen
M. Leclercq, M. Vannier, Y. Benhamou, A. Liard, V. Gilard, I. Auquit-Auckbur, H. Levesque, L. Sibert, P. Schneider
La Revue de Médecine Interne.2022; 43(5): 278. CrossRef
Evaluation of the Experience of Peer-led Mock Objective Structured Practical Examination for First- and Second-year Medical Students
Faisal Alsaif, Lamia Alkuwaiz, Mohammed Alhumud, Reem Idris, Lina Neel, Mansour Aljabry, Mona Soliman
Advances in Medical Education and Practice.2022; Volume 13: 987. CrossRef
The use of a formative OSCE to prepare emergency medicine residents for summative OSCEs: a mixed-methods cohort study
Magdalene Hui Min Lee, Dong Haur Phua, Kenneth Wei Jian Heng
International Journal of Emergency Medicine.2021;[Epub] CrossRef
Tutor–Student Partnership in Practice OSCE to Enhance Medical Education
Eve Cosker, Valentin Favier, Patrice Gallet, Francis Raphael, Emmanuelle Moussier, Louise Tyvaert, Marc Braun, Eva Feigerlova
Medical Science Educator.2021; 31(6): 1803. CrossRef
Peers as OSCE assessors for junior medical students – a review of routine use: a mixed methods study
Simon Schwill, Johanna Fahrbach-Veeser, Andreas Moeltner, Christiane Eicher, Sonia Kurczyk, David Pfisterer, Joachim Szecsenyi, Svetla Loukanova
BMC Medical Education.2020;[Epub] CrossRef

Research articles

Medical students’ thought process while solving problems in 3 different types of clinical assessments in Korea: clinical performance examination, multimedia case-based assessment, and modified essay question: Sejin Kim, Ikseon Choi, Bo Young Yoon, Min Jeong Kwon, Seok-jin Choi, Sang Hyun Kim, Jong-Tae Lee, Byoung Doo Rhee; J Educ Eval Health Prof. 2019;16:10. Published online May 9, 2019; DOI: https://doi.org/10.3352/jeehp.2019.16.10

19,511 View
297 Download
2 Web of Science
5 Crossref

Abstract PDF Supplementary Material

Purpose
This study aimed to explore students’ cognitive patterns while solving clinical problems in 3 different types of assessments—clinical performance examination (CPX), multimedia case-based assessment (CBA), and modified essay question (MEQ)—and thereby to understand how different types of assessments stimulate different patterns of thinking.
Methods
A total of 6 test-performance cases from 2 fourth-year medical students were used in this cross-case study. Data were collected through one-on-one interviews using a stimulated recall protocol where students were shown videos of themselves taking each assessment and asked to elaborate on what they were thinking. The unit of analysis was the smallest phrases or sentences in the participants’ narratives that represented meaningful cognitive occurrences. The narrative data were reorganized chronologically and then analyzed according to the hypothetico-deductive reasoning framework for clinical reasoning.
Results
Both participants demonstrated similar proportional frequencies of clinical reasoning patterns on the same clinical assessments. The results also revealed that the three different assessment types may stimulate different patterns of clinical reasoning. For example, the CPX strongly promoted the participants’ reasoning related to inquiry strategy, while the MEQ strongly promoted hypothesis generation. Similarly, data analysis and synthesis by the participants were more strongly stimulated by the CBA than by the other assessment types.
Conclusion
This study found that different assessment designs stimulated different patterns of thinking during problem-solving. This finding can contribute to the search for ways to improve current clinical assessments. Importantly, the research method used in this study can be utilized as an alternative way to examine the validity of clinical assessments.

Citations

Citations to this article as recorded by

Designing a Faculty Development Program on Electronic Question Banks: Schedule, Evaluation, and Sustainability Plan
Saurabh RamBihariLal Shrivastava, Rachmadya Nur Hidayah
Journal of the Scientific Society.2025; 52(2): 181. CrossRef
Evaluation of modified essay questions (MEQs) as an assessment tool in third-year medical students’ modular summative assessment
Ayman Elsamanoudy, Mohamed Shehata, Amer Almarabheh, Zienab Alrefaie
BMC Medical Education.2024;[Epub] CrossRef
Future directions of online learning environment design at medical schools: a transition towards a post-pandemic context
Sejin Kim
Kosin Medical Journal.2023; 38(1): 12. CrossRef
Clinical Reasoning Training based on the analysis of clinical case using a virtual environment
Sandra Elena Lisperguer Soto, María Soledad Calvo, Gabriela Paz Urrejola Contreras, Miguel Ángel Pérez Lizama
Educación Médica.2021; 22(3): 139. CrossRef
Newly appointed medical faculty members’ self-evaluation of their educational roles at the Catholic University of Korea College of Medicine in 2020 and 2021: a cross-sectional survey-based study
Sun Kim, A Ra Cho, Chul Woon Chung
Journal of Educational Evaluation for Health Professions.2021; 18: 28. CrossRef

Linear programming method to construct equated item sets for the implementation of periodical computer-based testing for the Korean Medical Licensing Examination: Dong Gi Seo, Myeong Gi Kim, Na Hui Kim, Hye Sook Shin, Hyun Jung Kim; J Educ Eval Health Prof. 2018;15:26. Published online October 18, 2018; DOI: https://doi.org/10.3352/jeehp.2018.15.26

23,385 View
294 Download
3 Web of Science
3 Crossref

Abstract PDF Supplementary Material

Purpose
This study aimed to identify the best way of developing equivalent item sets and to propose a stable and effective management plan for periodical licensing examinations.
Methods
Five pre-equated item sets were developed based on the predicted correct answer rate of each item using linear programming. These pre-equated item sets were compared to the ones that were developed with a random item selection method based on the actual correct answer rate (ACAR) and difficulty from item response theory (IRT). The results with and without common items were also compared in the same way. ACAR and the IRT difficulty were used to determine whether there was a significant difference between the pre-equating conditions.
Results
There was a statistically significant difference in IRT difficulty among the results from different pre-equated conditions. The predicted correct answer rate was divided using 2 or 3 difficulty categories, and the ACAR and IRT difficulty parameters of the 5 item sets were equally constructed. Comparing the item set conditions with and without common items, including common items did not make a significant contribution to the equating of the 5 item sets.
Conclusion
This study suggested that the linear programming method is applicable to construct equated-item sets that reflect each content area. The suggested best method to construct equated item sets is to divide the predicted correct answer rate using 2 or 3 difficulty categories, regardless of common items. If pre-equated item sets are required to construct a test based on the actual data, several methods should be considered by simulation studies to determine which is optimal before administering a real test.

Citations

Citations to this article as recorded by

An evaluation of the undergraduate medicine licensure examinations development and implementation practices: lessons from Ethiopia with global implications
Shewatatek Gedamu Wonde, Tefera Tadesse, Stefan K. Schauber
BMC Medical Education.2026;[Epub] CrossRef
Application of computer-based testing in the Korean Medical Licensing Examination, the emergence of the metaverse in medical education, journal metrics and statistics, and appreciation to reviewers and volunteers
Sun Huh
Journal of Educational Evaluation for Health Professions.2022; 19: 2. CrossRef
Reading Comprehension Tests for Children: Test Equating and Specific Age-Interval Reports
Patrícia Silva Lúcio, Fausto Coutinho Lourenço, Hugo Cogo-Moreira, Deborah Bandalos, Carolina Alves Ferreira de Carvalho, Adriana de Souza Batista Kida, Clara Regina Brandão de Ávila
Frontiers in Psychology.2021;[Epub] CrossRef

Educational/faculty development material

The reach of Spanish-language YouTube videos on physical examinations made by undergraduate medical students: José M. Ramos-Rincón, Isabel Belinchón-Romero, Francisco Sánchez-Ferrer, Guillermo Martínez-de la Torre, Meggan Harris, Javier Sánchez-Fernández; J Educ Eval Health Prof. 2017;14:31. Published online December 19, 2017; DOI: https://doi.org/10.3352/jeehp.2017.14.31

36,656 View
230 Download
3 Web of Science
4 Crossref

Abstract PDF Supplementary Material

This study was conducted to evaluate the performance and reach of YouTube videos on physical examinations made by Spanish university students. We analyzed performance metrics for 4 videos on physical examinations in Spanish that were created by medical students at Miguel Hernández University (Elche, Spain) and are available on YouTube, on the following topics: the head and neck (7:30), the cardiovascular system (7:38), the respiratory system (13:54), and the abdomen (11:10). We used the Analytics application offered by the YouTube platform to analyze the reach of the videos from the upload date (February 17, 2015) to July 28, 2017 (2 years, 5 months, and 11 days). The total number of views, length of watch-time, and the mean view duration for the 4 videos were, respectively: 164,403 views (mean, 41,101 views; range, 12,389 to 94,573 views), 425,888 minutes (mean, 106,472 minutes; range, 37,889 to 172,840 minutes), and 2:56 minutes (range, 1:49 to 4:03 minutes). Mexico was the most frequent playback location, followed by Spain, Colombia, and Venezuela. Uruguay, Ecuador, Mexico, and Puerto Rico had the most views per 100,000 population. Spanish-language tutorials are an alternative tool for teaching physical examination skills to students whose first language is not English. The videos were especially popular in Uruguay, Ecuador, and Mexico.

Citations

Citations to this article as recorded by

Free Open Access Medical Education (FOAMed) use in medical students: a literature review
Jacqueline Morgan
BMC Medical Education.2025;[Epub] CrossRef
Prevalence, Effectiveness, and Usefulness of Using YouTube among Medical Students in Clinical Years/Physical Examination
Mohammad Sharayah, Fadi Al Hadidi, Jehad Feras AlSamhori, Abdulsalam Ebdah, Ola Abbas Diwan, Ali Wa’el Alna’san, Abdallah Sharayah, Qutaiba Abu Quba, Abdel Rahman Ahmad Alsabi
JAPA Academy Journal.2025; 3(1): 52. CrossRef
Fizyoterapi ve rehabilitasyon öğrencilerinin Web 2.0 araçlarına yönelik davranışlarının incelenmesi: Hacettepe Üniversitesi örneği
Fatma Ayvat, Mert Doğan, Ender Ayvat, Özge Onursal Kılınç, Gülşah Sütçü, Muhammed Kılınç, Sibel Aksu Yıldırım
Journal of Exercise Therapy and Rehabilitation.2024; 11(3): 198. CrossRef
Student video production within health professions education: A scoping review
Qian Liu, Susan Geertshuis, Tehmina Gladman, Rebecca Grainger
Medical Education Online.2022;[Epub] CrossRef

Research articles

Perception survey on the introduction of clinical performance examination as part of the national nursing licensing examination in Korea: Su Jin Shin, Yeong Kyeong Kim, Soon-Rim Suh, Duk Yoo Jung, Yunju Kim, Mi Kyoung Yim; J Educ Eval Health Prof. 2017;14:26. Published online October 25, 2017; DOI: https://doi.org/10.3352/jeehp.2017.14.26

35,201 View
310 Download
2 Web of Science
5 Crossref

Search