Journal of Educational Evaluation for Health Professions

Educational/Faculty development material

Radiotorax.es: a web-based tool for formative self-assessment in chest X-ray interpretation: Verónica Illescas-Megías, Jorge Manuel Maqueda-Pérez, Dolores Domínguez-Pinos, Teodoro Rudolphi Solero, Francisco Sendra-Portero; J Educ Eval Health Prof. 2025;22:17. Published online June 9, 2025; DOI: https://doi.org/10.3352/jeehp.2025.22.17

3,858 View
167 Download

Abstract PDF Supplementary Material: Radiotorax.es is a free, non-profit web-based tool designed to support formative self-assessment in chest X-ray interpretation. This article presents its structure, educational applications, and usage data from 11 years of continuous operation. Users complete interpretation rounds of 20 clinical cases, compare their reports with expert evaluations, and conduct a structured self-assessment. From 2011 to 2022, 14,389 users registered, and 7,726 completed at least one session. Most were medical students (75.8%), followed by residents (15.2%) and practicing physicians (9.0%). The platform has been integrated into undergraduate medical curricula and used in various educational contexts, including tutorials, peer and expert review, and longitudinal tracking. Its flexible design supports self-directed learning, instructor-guided use, and multicenter research. As a freely accessible resource based on real clinical cases, Radiotorax.es provides a scalable, realistic, and well-received training environment that promotes diagnostic skill development, reflection, and educational innovation in radiology education.

Research article

Evaluation of a virtual objective structured clinical examination in the metaverse (Second Life) to assess the clinical skills in emergency radiology of medical students in Spain: a cross-sectional study: Alba Virtudes Perez-Baena, Teodoro Rudolphi-Solero, Rocio Lorenzo-Alvarez, Dolores Dominguez-Pinos, Miguel Jose Ruiz-Gomez, Francisco Sendra-Portero; J Educ Eval Health Prof. 2025;22:12. Published online April 21, 2025; DOI: https://doi.org/10.3352/jeehp.2025.22.12

3,631 View
260 Download
1 Crossref

Purpose
The objective structured clinical examination (OSCE) is an effective but resource-intensive tool for assessing clinical competence. This study hypothesized that implementing a virtual OSCE in the Second Life (SL) platform in the metaverse as a cost-effective alternative will effectively assess and enhance clinical skills in emergency radiology while being feasible and well-received. The aim was to evaluate a virtual radiology OSCE in SL as a formative assessment, focusing on feasibility, educational impact, and students’ perceptions.
Methods
Two virtual 6-station OSCE rooms dedicated to emergency radiology were developed in SL. Sixth-year medical students completed the OSCE during a 1-hour session in 2022–2023, followed by feedback including a correction checklist, individual scores, and group comparisons. Students completed a questionnaire with Likert-scale questions, a 10-point rating, and open-ended comments. Quantitative data were analyzed using the Student t-test and the Mann-Whitney U test, and qualitative data through thematic analysis.
Results
In total, 163 students participated, achieving mean scores of 5.1±1.4 and 4.9±1.3 (out of 10) in the 2 virtual OSCE rooms, respectively (P=0.287). One hundred seventeen students evaluated the OSCE, praising the teaching staff (9.3±1.0), project organization (8.8±1.2), OSCE environment (8.7±1.5), training usefulness (8.6±1.5), and formative self-assessment (8.5±1.4). Likert-scale questions and students’ open-ended comments highlighted the virtual environment’s attractiveness, case selection, self-evaluation usefulness, project excellence, and training impact. Technical difficulties were reported by 13 students (8%).
Conclusion
This study demonstrated the feasibility of incorporating formative OSCEs in SL as a useful teaching tool for undergraduate radiology education, which was cost-effective and highly valued by students.

Citations

Citations to this article as recorded by

Effectiveness of VR and traditional training in medical education for mass casualty management: an OSCE-based randomized controlled trial
Zhe Li, Wan Chen, Guozheng Qiu, Lei Shi, Yutao Tang, Xibin Xu, Sanshan Zhu, Liwen Lyu
BMC Medical Education.2026;[Epub] CrossRef

Educational/Faculty development material

The performance of ChatGPT-4.0o in medical imaging evaluation: a cross-sectional study: Elio Stefan Arruzza, Carla Marie Evangelista, Minh Chau; J Educ Eval Health Prof. 2024;21:29. Published online October 31, 2024; DOI: https://doi.org/10.3352/jeehp.2024.21.29

5,347 View
306 Download
9 Web of Science
11 Crossref

Abstract PDF Supplementary Material

This study investigated the performance of ChatGPT-4.0o in evaluating the quality of positioning in radiographic images. Thirty radiographs depicting a variety of knee, elbow, ankle, hand, pelvis, and shoulder projections were produced using anthropomorphic phantoms and uploaded to ChatGPT-4.0o. The model was prompted to provide a solution to identify any positioning errors with justification and offer improvements. A panel of radiographers assessed the solutions for radiographic quality based on established positioning criteria, with a grading scale of 1–5. In only 20% of projections, ChatGPT-4.0o correctly recognized all errors with justifications and offered correct suggestions for improvement. The most commonly occurring score was 3 (9 cases, 30%), wherein the model recognized at least 1 specific error and provided a correct improvement. The mean score was 2.9. Overall, low accuracy was demonstrated, with most projections receiving only partially correct solutions. The findings reinforce the importance of robust radiography education and clinical experience.

Citations

Citations to this article as recorded by

Evaluating Large Language Models for Burning Mouth Syndrome Diagnosis
Takayuki Suga, Osamu Uehara, Yoshihiro Abiko, Akira Toyofuku
Journal of Pain Research.2025; Volume 18: 1387. CrossRef
Evaluating the performance of GPT-3.5, GPT-4, and GPT-4o in the Chinese National Medical Licensing Examination
Dingyuan Luo, Mengke Liu, Runyuan Yu, Yulian Liu, Wenjun Jiang, Qi Fan, Naifeng Kuang, Qiang Gao, Tao Yin, Zuncheng Zheng
Scientific Reports.2025;[Epub] CrossRef
The ‘Negotiator’: Assessing artificial intelligence (AI) interview preparation for graduate radiographers
M. Chau, E. Arruzza, C.L. Singh
Journal of Medical Imaging and Radiation Sciences.2025; 56(5): 101982. CrossRef
Transforming behavioral intention and academic performance: ChatGPT-4.0 insights through SEM, ANN, and cIPMA analysis
Fazeelat Aziz, Cai Li, Asad Ullah Khan
Information Development.2025; 41(3): 933. CrossRef
Can ChatGPT Aid in Musculoskeletal Intervention?
Mohamed Ashiq Shazahan, Saavi Reddy Pellakuru, Sonal Saran, Shashank Chapala, Sindhura Mettu, Rajesh Botchu
Journal of Clinical Interventional Radiology ISVIR.2025; 09(03): 148. CrossRef
‘Bill’: An artificial intelligence (AI) clinical scenario coach for medical radiation science education
M. Chau, G. Higgins, E. Arruzza, C.L. Singh
Radiography.2025; 31(5): 103002. CrossRef
Correlates of Trust of Generative Artificial Intelligence Tools Among Patients and Caregivers: A Review of Empirical Research
Oliver T. Nguyen, Arsalan Ahmad, Douglas A. Wiegmann
Proceedings of the Human Factors and Ergonomics Society Annual Meeting.2025; 69(1): 1656. CrossRef
The performance of ChatGPT on medical image-based assessments and implications for medical education
Xiang Yang, Wei Chen
BMC Medical Education.2025;[Epub] CrossRef
Technologies, opportunities, challenges, and future directions for integrating generative artificial intelligence into medical education: a narrative review
Junseok Kang, Jihyun Ahn
Ewha Medical Journal.2025; 48(4): e53. CrossRef
Conversational LLM Chatbot ChatGPT-4 for Colonoscopy Boston Bowel Preparation Scoring: An Artificial Intelligence-to-Head Concordance Analysis
Raffaele Pellegrino, Alessandro Federico, Antonietta Gerarda Gravina
Diagnostics.2024; 14(22): 2537. CrossRef
Effectiveness of ChatGPT-4o in developing continuing professional development plans for graduate radiographers: a descriptive study
Minh Chau, Elio Stefan Arruzza, Kelly Spuur
Journal of Educational Evaluation for Health Professions.2024; 21: 34. CrossRef

Research article

GPT-4o’s competency in answering the simulated written European Board of Interventional Radiology exam compared to a medical student and experts in Germany and its ability to generate exam items on interventional radiology: a descriptive study: Sebastian Ebel, Constantin Ehrengut, Timm Denecke, Holger Gößmann, Anne Bettina Beeskow; J Educ Eval Health Prof. 2024;21:21. Published online August 20, 2024; DOI: https://doi.org/10.3352/jeehp.2024.21.21

4,853 View
341 Download
14 Web of Science
11 Crossref

Abstract PDF Supplementary Material

Purpose
This study aimed to determine whether ChatGPT-4o, a generative artificial intelligence (AI) platform, was able to pass a simulated written European Board of Interventional Radiology (EBIR) exam and whether GPT-4o can be used to train medical students and interventional radiologists of different levels of expertise by generating exam items on interventional radiology.
Methods
GPT-4o was asked to answer 370 simulated exam items of the Cardiovascular and Interventional Radiology Society of Europe (CIRSE) for EBIR preparation (CIRSE Prep). Subsequently, GPT-4o was requested to generate exam items on interventional radiology topics at levels of difficulty suitable for medical students and the EBIR exam. Those generated items were answered by 4 participants, including a medical student, a resident, a consultant, and an EBIR holder. The correctly answered items were counted. One investigator checked the answers and items generated by GPT-4o for correctness and relevance. This work was done from April to July 2024.
Results
GPT-4o correctly answered 248 of the 370 CIRSE Prep items (67.0%). For 50 CIRSE Prep items, the medical student answered 46.0%, the resident 42.0%, the consultant 50.0%, and the EBIR holder 74.0% correctly. All participants answered 82.0% to 92.0% of the 50 GPT-4o generated items at the student level correctly. For the 50 GPT-4o items at the EBIR level, the medical student answered 32.0%, the resident 44.0%, the consultant 48.0%, and the EBIR holder 66.0% correctly. All participants could pass the GPT-4o-generated items for the student level; while the EBIR holder could pass the GPT-4o-generated items for the EBIR level. Two items (0.3%) out of 150 generated by the GPT-4o were assessed as implausible.
Conclusion
GPT-4o could pass the simulated written EBIR exam and create exam items of varying difficulty to train medical students and interventional radiologists.

Citations

Citations to this article as recorded by

Evaluating the performance of ChatGPT in patient consultation and image-based preliminary diagnosis in thyroid eye disease
Yue Wang, Shuo Yang, Chengcheng Zeng, Yingwei Xie, Ya Shen, Jian Li, Xiao Huang, Ruili Wei, Yuqing Chen
Frontiers in Medicine.2025;[Epub] CrossRef
Solving Complex Pediatric Surgical Case Studies: A Comparative Analysis of Copilot, ChatGPT-4, and Experienced Pediatric Surgeons' Performance
Richard Gnatzy, Martin Lacher, Michael Berger, Michael Boettcher, Oliver J. Deffaa, Joachim Kübler, Omid Madadi-Sanjani, Illya Martynov, Steffi Mayer, Mikko P. Pakarinen, Richard Wagner, Tomas Wester, Augusto Zani, Ophelia Aubert
European Journal of Pediatric Surgery.2025; 35(05): 382. CrossRef
Preliminary assessment of large language models’ performance in answering questions on developmental dysplasia of the hip
Shiwei Li, Jun Jiang, Xiaodong Yang
Journal of Children's Orthopaedics.2025; 19(3): 207. CrossRef
AI and Interventional Radiology: A Narrative Review of Reviews on Opportunities, Challenges, and Future Directions
Andrea Lastrucci, Nicola Iosca, Yannick Wandael, Angelo Barra, Graziano Lepri, Nevio Forini, Renzo Ricci, Vittorio Miele, Daniele Giansanti
Diagnostics.2025; 15(7): 893. CrossRef
Evaluating the performance of GPT-3.5, GPT-4, and GPT-4o in the Chinese National Medical Licensing Examination
Dingyuan Luo, Mengke Liu, Runyuan Yu, Yulian Liu, Wenjun Jiang, Qi Fan, Naifeng Kuang, Qiang Gao, Tao Yin, Zuncheng Zheng
Scientific Reports.2025;[Epub] CrossRef
Evaluating Large Language Models for Preoperative Patient Education in Superior Capsular Reconstruction: Comparative Study of Claude, GPT, and Gemini
Yukang Liu, Hua Li, Jianfeng Ouyang, Zhaowen Xue, Min Wang, Hebei He, Bin Song, Xiaofei Zheng, Wenyi Gan
JMIR Perioperative Medicine.2025; 8: e70047. CrossRef
Evaluating ChatGPT's performance across radiology subspecialties: A meta-analysis of board-style examination accuracy and variability
Dan Nguyen, Grace Hyun J. Kim, Arash Bedayat
Clinical Imaging.2025; 125: 110551. CrossRef
Performance of ChatGPT-4 on the French Board of Plastic Reconstructive and Aesthetic Surgery written exam: a descriptive study
Emma Dejean-Bouyer, Anoujat Kanlagna, François Thuau, Pierre Perrot, Ugo Lancien
Journal of Educational Evaluation for Health Professions.2025; 22: 27. CrossRef
Technologies, opportunities, challenges, and future directions for integrating generative artificial intelligence into medical education: a narrative review
Junseok Kang, Jihyun Ahn
Ewha Medical Journal.2025; 48(4): e53. CrossRef
From GPT-3.5 to GPT-4.o: A Leap in AI’s Medical Exam Performance
Markus Kipp
Information.2024; 15(9): 543. CrossRef
Performance of ChatGPT and Bard on the medical licensing examinations varies across different cultures: a comparison study
Yikai Chen, Xiujie Huang, Fangjie Yang, Haiming Lin, Haoyu Lin, Zhuoqun Zheng, Qifeng Liang, Jinhai Zhang, Xinxin Li
BMC Medical Education.2024;[Epub] CrossRef

Brief Report

Benefits of a resident-run orientation for new radiology trainees: Kara Gaetke-Udager, Katherine E. Maturen, Daniel C. Barr, Kuanwong Watcharotone, Janet E. Bailey; J Educ Eval Health Prof. 2015;12:24. Published online June 12, 2015; DOI: https://doi.org/10.3352/jeehp.2015.12.24

26,749 View
159 Download
6 Web of Science
7 Crossref

Abstract PDF

Incoming radiology residents must rapidly assimilate large amounts of technical, medical, and operational information. This can be overwhelming and contribute to anxiety. Typical introductory curricula focused on radiologic content may not address the concerns of new residents. Two consecutive classes of incoming radiology residents participated in our study. For groups A (n=11) and B (n=11), the existing introductory lectures were given by faculty. For group B, residents hosted sessions for each rotation, including round-table discussions and work area tours, with emphasis on resident roles, personnel, and workflow. With institutional review board exemption, residents were anonymously surveyed before and after the sessions regarding: awareness of responsibilities, familiarity with anatomy, and anxiety regarding each rotation on a 1-4 scale. Free-text comments were collected. Comparison was performed using Wilcoxon rank sum test. Group A reported increased role awareness (P=0.04), greater content familiarity (P<0.05), and decreased anxiety (P=0.02) in one rotation each. There were 3 of 12 rotations in group B that showed significantly increased role awareness (P range <0.01 to 0.01) and decreased anxiety (P range <0.01 to <0.05). In addition, two rotations indicated improved role awareness only (P=0.02 and P=0.04), while there were four rotations reported decreased anxiety only (P range 0.01 to 0.03). Free-text commenters preferred the resident-run portions of the sessions. In conclusion, adding role-oriented introductory sessions to existing lectures for first-year residents decreased anxiety and increased role awareness for all rotations; therefore, it is suggested that anxiety may be better addressed by role-oriented content, and resident-to-resident teaching may have benefits.

Citations

Citations to this article as recorded by

A Virtual Multi-institution Pediatric Radiology Peer Teaching Conference Improves Knowledge of Educators
Eric L. Tung, Randheer Shailam, Matthew G. Tung, Katherine Barton
Current Problems in Diagnostic Radiology.2025; 54(3): 377. CrossRef
Revolutionizing radiology education: exploring innovative teaching methods
Izzet Altun, Ozerk Turan, Omer Awan
Abdominal Radiology.2025; 50(12): 6225. CrossRef
Performance Anxiety in Radiology Education and Clinical Practice
Miriam Stock, Edward Mardakhaev, Judah Burns
Journal of the American College of Radiology.2023; 20(11): 1162. CrossRef
Developing a Resident-led First-year Radiology Resident Lecture Series
Ryan S. Dolan, David Theriot, Dexter Mendoza, Christopher Ho, Mark E. Mullins, Ryan B. Peterson
Current Problems in Diagnostic Radiology.2022; 51(4): 434. CrossRef
Effectively Incorporating Peer Teaching to Improve Radiology Education—Radiology In Training
Eric L. Tung, Laura L. Avery
Radiology.2022; 303(3): E32. CrossRef
An Introductory, Computer-Based Learning Module for Interpreting Noncontrast Head Computed Tomography
Kara Gaetke-Udager, Zachary N London, Sean Woolen, Hemant Parmar, Janet E. Bailey, Daniel C. Barr
MedEdPORTAL.2018;[Epub] CrossRef
The Use of Team-Building Activities to Build a Better Resident
James D. Ireland, Linda A. Deloney, Shyann Renfroe, Kedar Jambhekar
Current Problems in Diagnostic Radiology.2017; 46(6): 399. CrossRef

First
Prev
Page of 1
Next
Last

Search