Journal of Educational Evaluation for Health Professions

Volume 20; 2023

Editorial

Editorial policies of Journal of Educational Evaluation for Health Professions on the use of generative artificial intelligence in article writing and peer review: Sun Huh; J Educ Eval Health Prof. 2023;20:40. Published online December 31, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.40

2,963 View
180 Download
3 Web of Science
3 Crossref

PDF

Citations

Citations to this article as recorded by

Ethical guidelines for the use of generative artificial intelligence and artificial intelligence-assisted tools in scholarly publishing: a thematic analysis
Adéle da Veiga
Science Editing.2025; 12(1): 28. CrossRef
The emergence of generative artificial intelligence platforms in 2023, journal metrics, appreciation to reviewers and volunteers, and obituary
Sun Huh
Journal of Educational Evaluation for Health Professions.2024; 21: 9. CrossRef
Large language model usage guidelines in Korean medical journals: a survey using human-artificial intelligence collaboration
Sangzin Ahn
Journal of Yeungnam Medical Science.2024; 42: 14. CrossRef

Research article

Information amount, accuracy, and relevance of generative artificial intelligence platforms’ answers regarding learning objectives of medical arthropodology evaluated in English and Korean queries in December 2023: a descriptive study: Hyunju Lee, Soobin Park; J Educ Eval Health Prof. 2023;20:39. Published online December 28, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.39

3,312 View
243 Download
3 Web of Science
3 Crossref

Abstract PDF Supplementary Material

Purpose
This study assessed the performance of 6 generative artificial intelligence (AI) platforms on the learning objectives of medical arthropodology in a parasitology class in Korea. We examined the AI platforms’ performance by querying in Korean and English to determine their information amount, accuracy, and relevance in prompts in both languages.
Methods
From December 15 to 17, 2023, 6 generative AI platforms—Bard, Bing, Claude, Clova X, GPT-4, and Wrtn—were tested on 7 medical arthropodology learning objectives in English and Korean. Clova X and Wrtn are platforms from Korean companies. Responses were evaluated using specific criteria for the English and Korean queries.
Results
Bard had abundant information but was fourth in accuracy and relevance. GPT-4, with high information content, ranked first in accuracy and relevance. Clova X was 4th in amount but 2nd in accuracy and relevance. Bing provided less information, with moderate accuracy and relevance. Wrtn’s answers were short, with average accuracy and relevance. Claude AI had reasonable information, but lower accuracy and relevance. The responses in English were superior in all aspects. Clova X was notably optimized for Korean, leading in relevance.
Conclusion
In a study of 6 generative AI platforms applied to medical arthropodology, GPT-4 excelled overall, while Clova X, a Korea-based AI product, achieved 100% relevance in Korean queries, the highest among its peers. Utilizing these AI platforms in classrooms improved the authors’ self-efficacy and interest in the subject, offering a positive experience of interacting with generative AI platforms to question and receive information.

Citations

Citations to this article as recorded by

Opportunities, challenges, and future directions of large language models, including ChatGPT in medical education: a systematic scoping review
Xiaojun Xu, Yixiao Chen, Jing Miao
Journal of Educational Evaluation for Health Professions.2024; 21: 6. CrossRef
The emergence of generative artificial intelligence platforms in 2023, journal metrics, appreciation to reviewers and volunteers, and obituary
Sun Huh
Journal of Educational Evaluation for Health Professions.2024; 21: 9. CrossRef
Comparison of the Performance of ChatGPT, Claude and Bard in Support of Myopia Prevention and Control
Yan Wang, Lihua Liang, Ran Li, Yihua Wang, Changfu Hao
Journal of Multidisciplinary Healthcare.2024; Volume 17: 3917. CrossRef

Review

Application of artificial intelligence chatbots, including ChatGPT, in education, scholarly work, programming, and content generation and its prospects: a narrative review: Tae Won Kim; J Educ Eval Health Prof. 2023;20:38. Published online December 27, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.38

17,775 View
1,261 Download
27 Web of Science
26 Crossref

Abstract PDF Supplementary Material

This study aims to explore ChatGPT’s (GPT-3.5 version) functionalities, including reinforcement learning, diverse applications, and limitations. ChatGPT is an artificial intelligence (AI) chatbot powered by OpenAI’s Generative Pre-trained Transformer (GPT) model. The chatbot’s applications span education, programming, content generation, and more, demonstrating its versatility. ChatGPT can improve education by creating assignments and offering personalized feedback, as shown by its notable performance in medical exams and the United States Medical Licensing Exam. However, concerns include plagiarism, reliability, and educational disparities. It aids in various research tasks, from design to writing, and has shown proficiency in summarizing and suggesting titles. Its use in scientific writing and language translation is promising, but professional oversight is needed for accuracy and originality. It assists in programming tasks like writing code, debugging, and guiding installation and updates. It offers diverse applications, from cheering up individuals to generating creative content like essays, news articles, and business plans. Unlike search engines, ChatGPT provides interactive, generative responses and understands context, making it more akin to human conversation, in contrast to conventional search engines’ keyword-based, non-interactive nature. ChatGPT has limitations, such as potential bias, dependence on outdated data, and revenue generation challenges. Nonetheless, ChatGPT is considered to be a transformative AI tool poised to redefine the future of generative technology. In conclusion, advancements in AI, such as ChatGPT, are altering how knowledge is acquired and applied, marking a shift from search engines to creativity engines. This transformation highlights the increasing importance of AI literacy and the ability to effectively utilize AI in various domains of life.

Citations

Citations to this article as recorded by

The Development and Validation of an Artificial Intelligence Chatbot Dependence Scale
Xing Zhang, Mingyue Yin, Mingyang Zhang, Zhaoqian Li, Hansen Li
Cyberpsychology, Behavior, and Social Networking.2025; 28(2): 126. CrossRef
Readability, quality and accuracy of generative artificial intelligence chatbots for commonly asked questions about labor epidurals: a comparison of ChatGPT and Bard
D. Lee, M. Brown, J. Hammond, M. Zakowski
International Journal of Obstetric Anesthesia.2025; 61: 104317. CrossRef
ChatGPT-4 Performance on German Continuing Medical Education—Friend or Foe (Trick or Treat)? Protocol for a Randomized Controlled Trial
Christian Burisch, Abhav Bellary, Frank Breuckmann, Jan Ehlers, Serge C Thal, Timur Sellmann, Daniel Gödde
JMIR Research Protocols.2025; 14: e63887. CrossRef
The effect of incorporating large language models into the teaching on critical thinking disposition: An “AI + Constructivism Learning Theory” attempt
Peng Wang, Kexin Yin, Mingzhu Zhang, Yuanxin Zheng, Tong Zhang, Yanjun Kang, Xun Feng
Education and Information Technologies.2025;[Epub] CrossRef
The Impact of Adaptive Learning Technologies, Personalized Feedback, and Interactive AI Tools on Student Engagement: The Moderating Role of Digital Literacy
Husam Yaseen, Abdelaziz Saleh Mohammad, Najwa Ashal, Hesham Abusaimeh, Ahmad Ali, Abdel-Aziz Ahmad Sharabati
Sustainability.2025; 17(3): 1133. CrossRef
Artificial Intelligence in Nursing: New Opportunities and Challenges
Estel·la Ramírez‐Baraldes, Daniel García‐Gutiérrez, Cristina García‐Salido
European Journal of Education.2025;[Epub] CrossRef
Can ChatGPT be used as a scientific source of information on tooth extraction?
Shiori Yamamoto, Masakazu Hamada, Kyoko Nishiyama, Ayako Motoki, Yusei Fujita, Narikazu Uzawa
Journal of Oral and Maxillofacial Surgery, Medicine, and Pathology.2025;[Epub] CrossRef
Exploring artificial intelligence (AI) Chatbot usage behaviors and their association with mental health outcomes in Chinese university students
Xing Zhang, Zhaoqian Li, Mingyang Zhang, Mingyue Yin, Zhangyu Yang, Dong Gao, Hansen Li
Journal of Affective Disorders.2025; 380: 394. CrossRef
The analysis of optimization in music aesthetic education under artificial intelligence
Yixuan Peng
Scientific Reports.2025;[Epub] CrossRef
The Role of Artificial Intelligence in Computer Science Education: A Systematic Review with a Focus on Database Instruction
Alkmini Gaitantzi, Ioannis Kazanidis
Applied Sciences.2025; 15(7): 3960. CrossRef
A Bibliometric Exposition and Review on Leveraging LLMs for Programming Education
Joanah Pwanedo Amos, Oluwatosin Ahmed Amodu, Raja Azlina Raja Mahmood, Akanbi Bolakale Abdulqudus, Anies Faziehan Zakaria, Abimbola Rhoda Iyanda, Umar Ali Bukar, Zurina Mohd Hanapi
IEEE Access.2025; 13: 58364. CrossRef
Can ChatGPT be trusted as a resource for a scholarly article on treatment planning implant-supported prostheses?
Steven J. Sadowsky
The Journal of Prosthetic Dentistry.2025;[Epub] CrossRef
Use of machine translation in foreign language education
Blanka Klimova
Cogent Arts & Humanities.2025;[Epub] CrossRef
Comparison of triage performance among DRP tool, ChatGPT, and outpatient rehabilitation doctors
Yucong Zou, Ruixue Ye, Yan Gao, Jing Zhou, Yawei Li, Wenshi Chen, Fubing Zha, Yulong Wang
Scientific Reports.2025;[Epub] CrossRef
Opportunities, challenges, and future directions of large language models, including ChatGPT in medical education: a systematic scoping review
Xiaojun Xu, Yixiao Chen, Jing Miao
Journal of Educational Evaluation for Health Professions.2024; 21: 6. CrossRef
Artificial Intelligence: Fundamentals and Breakthrough Applications in Epilepsy
Wesley Kerr, Sandra Acosta, Patrick Kwan, Gregory Worrell, Mohamad A. Mikati
Epilepsy Currents.2024;[Epub] CrossRef
A Developed Graphical User Interface-Based on Different Generative Pre-trained Transformers Models
Ekrem Küçük, İpek Balıkçı Çiçek, Zeynep Küçükakçalı, Cihan Yetiş, Cemil Çolak
ODÜ Tıp Dergisi.2024; 11(1): 18. CrossRef
Art or Artifact: Evaluating the Accuracy, Appeal, and Educational Value of AI-Generated Imagery in DALL·E 3 for Illustrating Congenital Heart Diseases
Mohamad-Hani Temsah, Abdullah N. Alhuzaimi, Mohammed Almansour, Fadi Aljamaan, Khalid Alhasan, Munirah A. Batarfi, Ibraheem Altamimi, Amani Alharbi, Adel Abdulaziz Alsuhaibani, Leena Alwakeel, Abdulrahman Abdulkhaliq Alzahrani, Khaled B. Alsulaim, Amr Jam
Journal of Medical Systems.2024;[Epub] CrossRef
Authentic assessment in medical education: exploring AI integration and student-as-partners collaboration
Syeda Sadia Fatima, Nabeel Ashfaque Sheikh, Athar Osama
Postgraduate Medical Journal.2024; 100(1190): 959. CrossRef
Comparative performance analysis of large language models: ChatGPT-3.5, ChatGPT-4 and Google Gemini in glucocorticoid-induced osteoporosis
Linjian Tong, Chaoyang Zhang, Rui Liu, Jia Yang, Zhiming Sun
Journal of Orthopaedic Surgery and Research.2024;[Epub] CrossRef
Can AI-Generated Clinical Vignettes in Japanese Be Used Medically and Linguistically?
Yasutaka Yanagita, Daiki Yokokawa, Shun Uchida, Yu Li, Takanori Uehara, Masatomi Ikusaka
Journal of General Internal Medicine.2024; 39(16): 3282. CrossRef
ChatGPT vs. sleep disorder specialist responses to common sleep queries: Ratings by experts and laypeople
Jiyoung Kim, Seo-Young Lee, Jee Hyun Kim, Dong-Hyeon Shin, Eun Hye Oh, Jin A Kim, Jae Wook Cho
Sleep Health.2024; 10(6): 665. CrossRef
Technology integration into Chinese as a foreign language learning in higher education: An integrated bibliometric analysis and systematic review (2000–2024)
Binze Xu
Language Teaching Research.2024;[Epub] CrossRef
The Transformative Power of Generative Artificial Intelligence for Achieving the Sustainable Development Goal of Quality Education
Prema Nedungadi, Kai-Yu Tang, Raghu Raman
Sustainability.2024; 16(22): 9779. CrossRef
Is AI the new course creator
Sheri Conklin, Tom Dorgan, Daisyane Barreto
Discover Education.2024;[Epub] CrossRef
Emergency Medicine Assistants in the Field of Toxicology, Comparison of ChatGPT-3.5 and GEMINI Artificial Intelligence Systems
Hatice Aslı Bedel, Cihan Bedel, Fatih Selvi, Ökkeş Zortuk, Yusuf Karanci
Acta medica Lituanica.2024; 31(2): 294. CrossRef

Research articles

Negative effects on medical students’ scores for clinical performance during the COVID-19 pandemic in Taiwan: a comparative study: Eunice Jia-Shiow Yuan, Shiau-Shian Huang, Chia-An Hsu, Jiing-Feng Lirng, Tzu-Hao Li, Chia-Chang Huang, Ying-Ying Yang, Chung-Pin Li, Chen-Huan Chen; J Educ Eval Health Prof. 2023;20:37. Published online December 26, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.37

2,604 View
123 Download
2 Web of Science
2 Crossref

Abstract PDF Supplementary Material

Purpose
Coronavirus disease 2019 (COVID-19) has heavily impacted medical clinical education in Taiwan. Medical curricula have been altered to minimize exposure and limit transmission. This study investigated the effect of COVID-19 on Taiwanese medical students’ clinical performance using online standardized evaluation systems and explored the factors influencing medical education during the pandemic.
Methods
Medical students were scored from 0 to 100 based on their clinical performance from 1/1/2018 to 6/31/2021. The students were placed into pre-COVID-19 (before 2/1/2020) and midst-COVID-19 (on and after 2/1/2020) groups. Each group was further categorized into COVID-19-affected specialties (pulmonary, infectious, and emergency medicine) and other specialties. Generalized estimating equations (GEEs) were used to compare and examine the effects of relevant variables on student performance.
Results
In total, 16,944 clinical scores were obtained for COVID-19-affected specialties and other specialties. For the COVID-19-affected specialties, the midst-COVID-19 score (88.513.52) was significantly lower than the pre-COVID-19 score (90.143.55) (P<0.0001). For the other specialties, the midst-COVID-19 score (88.323.68) was also significantly lower than the pre-COVID-19 score (90.063.58) (P<0.0001). There were 1,322 students (837 males and 485 females). Male students had significantly lower scores than female students (89.333.68 vs. 89.993.66, P=0.0017). GEE analysis revealed that the COVID-19 pandemic (unstandardized beta coefficient=-1.99, standard error [SE]=0.13, P<0.0001), COVID-19-affected specialties (B=0.26, SE=0.11, P=0.0184), female students (B=1.10, SE=0.20, P<0.0001), and female attending physicians (B=-0.19, SE=0.08, P=0.0145) were independently associated with students’ scores.
Conclusion
COVID-19 negatively impacted medical students' clinical performance, regardless of their specialty. Female students outperformed male students, irrespective of the pandemic.

Citations

Citations to this article as recorded by

Using the OSCE to assess medical students’ communication and clinical reasoning during five years of restricted clinical practice
Armijo-Rivera Soledad, Zamorano Saavedra Catalina, Vicencio-Clarke Scarlett, Behrens Pérez Claudia, Pérez-Villalobos Cristhian
BMC Medical Education.2025;[Epub] CrossRef
The emergence of generative artificial intelligence platforms in 2023, journal metrics, appreciation to reviewers and volunteers, and obituary
Sun Huh
Journal of Educational Evaluation for Health Professions.2024; 21: 9. CrossRef

Use of learner-driven, formative, ad-hoc, prospective assessment of competence in physical therapist clinical education in the United States: a prospective cohort study: Carey Holleran, Jeffrey Konrad, Barbara Norton, Tamara Burlis, Steven Ambler; J Educ Eval Health Prof. 2023;20:36. Published online December 8, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.36

2,212 View
192 Download

Abstract PDF Supplementary Material: Purpose
The purpose of this project was to implement a process for learner-driven, formative, prospective, ad-hoc, entrustment assessment in Doctor of Physical Therapy clinical education. Our goals were to develop an innovative entrustment assessment tool, and then explore whether the tool detected (1) differences between learners at different stages of development and (2) differences within learners across the course of a clinical education experience. We also investigated whether there was a relationship between the number of assessments and change in performance.
Methods
A prospective, observational, cohort of clinical instructors (CIs) was recruited to perform learner-driven, formative, ad-hoc, prospective, entrustment assessments. Two entrustable professional activities (EPAs) were used: (1) gather a history and perform an examination and (2) implement and modify the plan of care, as needed. CIs provided a rating on the entrustment scale and provided narrative support for their rating.
Results
Forty-nine learners participated across 4 clinical experiences (CEs), resulting in 453 EPA learner-driven assessments. For both EPAs, statistically significant changes were detected both between learners at different stages of development and within learners across the course of a CE. Improvement within each CE was significantly related to the number of feedback opportunities.
Conclusion
The results of this pilot study provide preliminary support for the use of learner-driven, formative, ad-hoc assessments of competence based on EPAs with a novel entrustment scale. The number of formative assessments requested correlated with change on the EPA scale, suggesting that formative feedback may augment performance improvement.

Effect of a transcultural nursing course on improving the cultural competency of nursing graduate students in Korea: a before-and-after study: Kyung Eui Bae, Geum Hee Jeong; J Educ Eval Health Prof. 2023;20:35. Published online December 4, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.35

3,483 View
248 Download

Abstract PDF Supplementary Material: Purpose
This study aimed to evaluate the impact of a transcultural nursing course on enhancing the cultural competency of graduate nursing students in Korea. We hypothesized that participants’ cultural competency would significantly improve in areas such as communication, biocultural ecology and family, dietary habits, death rituals, spirituality, equity, and empowerment and intermediation after completing the course. Furthermore, we assessed the participants’ overall satisfaction with the course.
Methods
A before-and-after study was conducted with graduate nursing students at Hallym University, Chuncheon, Korea, from March to June 2023. A transcultural nursing course was developed based on Giger & Haddad’s transcultural nursing model and Purnell’s theoretical model of cultural competence. Data was collected using a cultural competence scale for registered nurses developed by Kim and his colleagues. A total of 18 students participated, and the paired t-test was employed to compare pre-and post-intervention scores.
Results
The study revealed significant improvements in all 7 categories of cultural nursing competence (P<0.01). Specifically, the mean differences in scores (pre–post) ranged from 0.74 to 1.09 across the categories. Additionally, participants expressed high satisfaction with the course, with an average score of 4.72 out of a maximum of 5.0.
Conclusion
The transcultural nursing course effectively enhanced the cultural competency of graduate nursing students. Such courses are imperative to ensure quality care for the increasing multicultural population in Korea.

Effect of motion-graphic video-based training on the performance of operating room nurse students in cataract surgery in Iran: a randomized controlled study: Behnaz Fatahi, Samira Fatahi, Sohrab Nosrati, Masood Bagheri; J Educ Eval Health Prof. 2023;20:34. Published online November 28, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.34

3,935 View
154 Download

Abstract PDF Supplementary Material: Purpose
The present study was conducted to determine the effect of motion-graphic video-based training on the performance of operating room nurse students in cataract surgery using phacoemulsification at Kermanshah University of Medical Sciences in Iran.
Methods
This was a randomized controlled study conducted among 36 students training to become operating room nurses. The control group only received routine training, and the intervention group received motion-graphic video-based training on the scrub nurse’s performance in cataract surgery in addition to the educator’s training. The performance of the students in both groups as scrub nurses was measured through a researcher-made checklist in a pre-test and a post-test.
Results
The mean scores for performance in the pre-test and post-test were 17.83 and 26.44 in the control group and 18.33 and 50.94 in the intervention group, respectively, and a significant difference was identified between the mean scores of the pre- and post-test in both groups (P=0.001). The intervention also led to a significant increase in the mean performance score in the intervention group compared to the control group (P=0.001).
Conclusion
Considering the significant difference in the performance score of the intervention group compared to the control group, motion-graphic video-based training had a positive effect on the performance of operating room nurse students, and such training can be used to improve clinical training.

Assessment of the viability of integrating virtual reality programs in practical tests for the Korean Radiological Technologists Licensing Examination: a survey study: Hye Min Park, Eun Seong Kim, Deok Mun Kwon, Pyong Kon Cho, Seoung Hwan Kim, Ki Baek Lee, Seong Hu Kim, Moon Il Bong, Won Seok Yang, Jin Eui Kim, Gi Bong Kang, Yong Su Yoon, Jung Su Kim; J Educ Eval Health Prof. 2023;20:33. Published online November 28, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.33

2,399 View
129 Download
1 Web of Science
1 Crossref

Abstract PDF Supplementary Material

Purpose
The objective of this study was to assess the feasibility of incorporating virtual reality/augmented reality (VR/AR) programs into practical tests administered as part of the Korean Radiological Technologists Licensing Examination (KRTLE). This evaluation is grounded in a comprehensive survey that targeted enrolled students in departments of radiology across the nation.
Methods
In total, 682 students from radiology departments across the nation were participants in the survey. An online survey platform was used, and the questionnaire was structured into 5 distinct sections and 27 questions. A frequency analysis for each section of the survey was conducted using IBM SPSS ver. 27.0.
Results
Direct or indirect exposure to VR/AR content was reported by 67.7% of all respondents. Furthermore, 55.4% of the respondents expressed that VR/AR could be integrated into their classes, which signified a widespread acknowledgment of VR among the students. With regards to the integration of a VR/AR or mixed reality program into the practical tests for purposes of the KRTLE, a substantial amount of the respondents (57.3%) exhibited a positive inclination and recommended its introduction.
Conclusion
The application of VR/AR programs within practical tests of the KRTLE will be used as an alternative for evaluating clinical examination procedures and validating job skills.

Citations

Citations to this article as recorded by

Understanding perception of the radiology community concerning virtual reality (VR) and augmented reality (AR) technology in radiology education
Suneet Paulson, Dwight Figueiredo, Sushant Matre
Egyptian Journal of Radiology and Nuclear Medicine.2025;[Epub] CrossRef

Brief report

ChatGPT (GPT-3.5) as an assistant tool in microbial pathogenesis studies in Sweden: a cross-sectional comparative study: Catharina Hultgren, Annica Lindkvist, Volkan Özenci, Sophie Curbo; J Educ Eval Health Prof. 2023;20:32. Published online November 22, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.32

2,496 View
151 Download
3 Web of Science
3 Crossref

Abstract PDF Supplementary Material

ChatGPT (GPT-3.5) has entered higher education and there is a need to determine how to use it effectively. This descriptive study compared the ability of GPT-3.5 and teachers to answer questions from dental students and construct detailed intended learning outcomes. When analyzed according to a Likert scale, we found that GPT-3.5 answered the questions from dental students in a similar or even more elaborate way compared to the answers that had previously been provided by a teacher. GPT-3.5 was also asked to construct detailed intended learning outcomes for a course in microbial pathogenesis, and when these were analyzed according to a Likert scale they were, to a large degree, found irrelevant. Since students are using GPT-3.5, it is important that instructors learn how to make the best use of it both to be able to advise students and to benefit from its potential.

Citations

Citations to this article as recorded by

Opportunities, challenges, and future directions of large language models, including ChatGPT in medical education: a systematic scoping review
Xiaojun Xu, Yixiao Chen, Jing Miao
Journal of Educational Evaluation for Health Professions.2024; 21: 6. CrossRef
Unlocking learning: exploring take-home examinations and viva voce examinations in microbiology education for biomedical laboratory science students
Sophie Curbo, Annica Lindkvist, Catharina Hultgren, Jorge Cervantes
Journal of Microbiology & Biology Education.2024;[Epub] CrossRef
Information amount, accuracy, and relevance of generative artificial intelligence platforms’ answers regarding learning objectives of medical arthropodology evaluated in English and Korean queries in December 2023: a descriptive study
Hyunju Lee, Soobin Park
Journal of Educational Evaluation for Health Professions.2023; 20: 39. CrossRef

Technical report

Item difficulty index, discrimination index, and reliability of the 26 health professions licensing examinations in 2022, Korea: a psychometric study: Yoon Hee Kim, Bo Hyun Kim, Joonki Kim, Bokyoung Jung, Sangyoung Bae; J Educ Eval Health Prof. 2023;20:31. Published online November 22, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.31

3,305 View
183 Download

Abstract PDF Supplementary Material: Purpose
This study presents item analysis results of the 26 health personnel licensing examinations managed by the Korea Health Personnel Licensing Examination Institute (KHPLEI) in 2022.
Methods
The item difficulty index, item discrimination index, and reliability were calculated. The item discrimination index was calculated using a discrimination index based on the upper and lower 27% rule and the item-total correlation.
Results
Out of 468,352 total examinees, 418,887 (89.4%) passed. The pass rates ranged from 27.3% for health educators level 1 to 97.1% for oriental medical doctors. Most examinations had a high average difficulty index, albeit to varying degrees, ranging from 61.3% for prosthetists and orthotists to 83.9% for care workers. The average discrimination index based on the upper and lower 27% rule ranged from 0.17 for oriental medical doctors to 0.38 for radiological technologists. The average item-total correlation ranged from 0.20 for oriental medical doctors to 0.38 for radiological technologists. The Cronbach α, as a measure of reliability, ranged from 0.872 for health educators-level 3 to 0.978 for medical technologists. The correlation coefficient between the average difficulty index and average discrimination index was -0.2452 (P=0.1557), that between the average difficulty index and the average item-total correlation was 0.3502 (P=0.0392), and that between the average discrimination index and the average item-total correlation was 0.7944 (P<0.0001).
Conclusion
This technical report presents the item analysis results and reliability of the recent examinations by the KHPLEI, demonstrating an acceptable range of difficulty index and discrimination index values, as well as good reliability.

Research articles

Performance of ChatGPT, Bard, Claude, and Bing on the Peruvian National Licensing Medical Examination: a cross-sectional study: Betzy Clariza Torres-Zegarra, Wagner Rios-Garcia, Alvaro Micael Ñaña-Cordova, Karen Fatima Arteaga-Cisneros, Xiomara Cristina Benavente Chalco, Marina Atena Bustamante Ordoñez, Carlos Jesus Gutierrez Rios, Carlos Alberto Ramos Godoy, Kristell Luisa Teresa Panta Quezada, Jesus Daniel Gutierrez-Arratia, Javier Alejandro Flores-Cohaila; J Educ Eval Health Prof. 2023;20:30. Published online November 20, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.30

4,013 View
242 Download
20 Web of Science
27 Crossref

Abstract PDF Supplementary Material

Purpose
We aimed to describe the performance and evaluate the educational value of justifications provided by artificial intelligence chatbots, including GPT-3.5, GPT-4, Bard, Claude, and Bing, on the Peruvian National Medical Licensing Examination (P-NLME).
Methods
This was a cross-sectional analytical study. On July 25, 2023, each multiple-choice question (MCQ) from the P-NLME was entered into each chatbot (GPT-3, GPT-4, Bing, Bard, and Claude) 3 times. Then, 4 medical educators categorized the MCQs in terms of medical area, item type, and whether the MCQ required Peru-specific knowledge. They assessed the educational value of the justifications from the 2 top performers (GPT-4 and Bing).
Results
GPT-4 scored 86.7% and Bing scored 82.2%, followed by Bard and Claude, and the historical performance of Peruvian examinees was 55%. Among the factors associated with correct answers, only MCQs that required Peru-specific knowledge had lower odds (odds ratio, 0.23; 95% confidence interval, 0.09–0.61), whereas the remaining factors showed no associations. In assessing the educational value of justifications provided by GPT-4 and Bing, neither showed any significant differences in certainty, usefulness, or potential use in the classroom.
Conclusion
Among chatbots, GPT-4 and Bing were the top performers, with Bing performing better at Peru-specific MCQs. Moreover, the educational value of justifications provided by the GPT-4 and Bing could be deemed appropriate. However, it is essential to start addressing the educational value of these chatbots, rather than merely their performance on examinations.

Citations

Citations to this article as recorded by

PICOT questions and search strategies formulation: A novel approach using artificial intelligence automation
Lucija Gosak, Gregor Štiglic, Lisiane Pruinelli, Dominika Vrbnjak
Journal of Nursing Scholarship.2025; 57(1): 5. CrossRef
Using large language models (ChatGPT, Copilot, PaLM, Bard, and Gemini) in Gross Anatomy course: Comparative analysis
Volodymyr Mavrych, Paul Ganguly, Olena Bolgova
Clinical Anatomy.2025; 38(2): 200. CrossRef
Capable exam-taker and question-generator: the dual role of generative AI in medical education assessment
Yihong Qiu, Chang Liu
Global Medical Education.2025;[Epub] CrossRef
Comparison of artificial intelligence systems in answering prosthodontics questions from the dental specialty exam in Turkey
Busra Tosun, Zeynep Sen Yilmaz
Journal of Dental Sciences.2025;[Epub] CrossRef
Benchmarking LLM chatbots’ oncological knowledge with the Turkish Society of Medical Oncology’s annual board examination questions
Efe Cem Erdat, Engin Eren Kavak
BMC Cancer.2025;[Epub] CrossRef
Evaluating the Performance of Large Language Models in Anatomy Education Advancing Anatomy Learning with ChatGPT-4o
Fatma Ok, Burak Karip, Fulya Temizsoy Korkmaz
European Journal of Therapeutics.2025; 31(1): 35. CrossRef
Large Language Models in Biochemistry Education: Comparative Evaluation of Performance
Olena Bolgova, Inna Shypilova, Volodymyr Mavrych
JMIR Medical Education.2025; 11: e67244. CrossRef
Attributional patterns toward students with and without learning disabilities: Artificial intelligence models vs. trainee teachers
Inbar Levkovich, Eyal Rabin, Rania Hussein Farraj, Zohar Elyoseph
Research in Developmental Disabilities.2025; 160: 104970. CrossRef
The double-edged sword of generative AI: surpassing an expert or a deceptive “false friend”?
Franziska C.S. Altorfer, Michael J. Kelly, Fedan Avrumova, Varun Rohatgi, Jiaqi Zhu, Christopher M. Bono, Darren R. Lebl
The Spine Journal.2025;[Epub] CrossRef
Claude, ChatGPT, Copilot, and Gemini performance versus students in different topics of neuroscience
Volodymyr Mavrych, Ahmed Yaqinuddin, Olena Bolgova
Advances in Physiology Education.2025; 49(2): 430. CrossRef
Large Language Models Take on the AAMC Situational Judgment Test: Evaluating Dilemma-Based Scenarios
Angelo Cadiente, Jamie Chen, Lora J. Kasselman, Bryan Pilkington
International Journal of Artificial Intelligence in Education.2025;[Epub] CrossRef
Validation of a generative artificial intelligence tool for the critical appraisal of articles on the epidemiology of mental health: Its application in the Middle East and North Africa
Cheima Moussa, Sarah Altayyar, Marion Vergonjeanne, Thibaut Gelle, Pierre-Marie Preux
Journal of Epidemiology and Population Health.2025; 73(2): 202990. CrossRef
Artificial intelligence (AI) performance on pharmacy skills laboratory course assignments
Vivian Do, Krista L. Donohoe, Apryl N. Peddi, Eleanor Carr, Christina Kim, Virginia Mele, Dhruv Patel, Alexis N. Crawford
Currents in Pharmacy Teaching and Learning.2025; 17(7): 102367. CrossRef
Performance of GPT-4V in Answering the Japanese Otolaryngology Board Certification Examination Questions: Evaluation Study
Masao Noda, Takayoshi Ueno, Ryota Koshu, Yuji Takaso, Mari Dias Shimada, Chizu Saito, Hisashi Sugimoto, Hiroaki Fushiki, Makoto Ito, Akihiro Nomura, Tomokazu Yoshizaki
JMIR Medical Education.2024; 10: e57054. CrossRef
Response to Letter to the Editor re: “Artificial Intelligence Versus Expert Plastic Surgeon: Comparative Study Shows ChatGPT ‘Wins' Rhinoplasty Consultations: Should We Be Worried? [1]” by Durairaj et al
Kay Durairaj, Omer Baker
Facial Plastic Surgery & Aesthetic Medicine.2024; 26(3): 276. CrossRef
Opportunities, challenges, and future directions of large language models, including ChatGPT in medical education: a systematic scoping review
Xiaojun Xu, Yixiao Chen, Jing Miao
Journal of Educational Evaluation for Health Professions.2024; 21: 6. CrossRef
Performance of ChatGPT Across Different Versions in Medical Licensing Examinations Worldwide: Systematic Review and Meta-Analysis
Mingxin Liu, Tsuyoshi Okuhara, XinYi Chang, Ritsuko Shirabe, Yuriko Nishiie, Hiroko Okada, Takahiro Kiuchi
Journal of Medical Internet Research.2024; 26: e60807. CrossRef
Comparative accuracy of ChatGPT-4, Microsoft Copilot and Google Gemini in the Italian entrance test for healthcare sciences degrees: a cross-sectional study
Giacomo Rossettini, Lia Rodeghiero, Federica Corradi, Chad Cook, Paolo Pillastrini, Andrea Turolla, Greta Castellini, Stefania Chiappinotto, Silvia Gianola, Alvisa Palese
BMC Medical Education.2024;[Epub] CrossRef
Evaluating the competency of ChatGPT in MRCP Part 1 and a systematic literature review of its capabilities in postgraduate medical assessments
Oliver Vij, Henry Calver, Nikki Myall, Mrinalini Dey, Koushan Kouranloo, Thiago P. Fernandes
PLOS ONE.2024; 19(7): e0307372. CrossRef
Large Language Models in Pediatric Education: Current Uses and Future Potential
Srinivasan Suresh, Sanghamitra M. Misra
Pediatrics.2024;[Epub] CrossRef
Comparison of the Performance of ChatGPT, Claude and Bard in Support of Myopia Prevention and Control
Yan Wang, Lihua Liang, Ran Li, Yihua Wang, Changfu Hao
Journal of Multidisciplinary Healthcare.2024; Volume 17: 3917. CrossRef
Evaluating Large Language Models in Dental Anesthesiology: A Comparative Analysis of ChatGPT-4, Claude 3 Opus, and Gemini 1.0 on the Japanese Dental Society of Anesthesiology Board Certification Exam
Misaki Fujimoto, Hidetaka Kuroda, Tomomi Katayama, Atsuki Yamaguchi, Norika Katagiri, Keita Kagawa, Shota Tsukimoto, Akito Nakano, Uno Imaizumi, Aiji Sato-Boku, Naotaka Kishimoto, Tomoki Itamiya, Kanta Kido, Takuro Sanuki
Cureus.2024;[Epub] CrossRef
Dermatological Knowledge and Image Analysis Performance of Large Language Models Based on Specialty Certificate Examination in Dermatology
Ka Siu Fan, Ka Hay Fan
Dermato.2024; 4(4): 124. CrossRef
ChatGPT and Other Large Language Models in Medical Education — Scoping Literature Review
Alexandra Aster, Matthias Carl Laupichler, Tamina Rockwell-Kollmann, Gilda Masala, Ebru Bala, Tobias Raupach
Medical Science Educator.2024; 35(1): 555. CrossRef
Performance of ChatGPT and Bard on the medical licensing examinations varies across different cultures: a comparison study
Yikai Chen, Xiujie Huang, Fangjie Yang, Haiming Lin, Haoyu Lin, Zhuoqun Zheng, Qifeng Liang, Jinhai Zhang, Xinxin Li
BMC Medical Education.2024;[Epub] CrossRef
Temporal Relationship Between Internet Search Volumes for Diarrhea Synonyms Using ChatGPT and Emergency Department Visits for Diarrhea-Related Symptoms in South Korea (Preprint)
Jinsoo Kim, Ansun Jeong, Juseong Jin, Sangjun Lee, Do Kyoon Yoon, Soyeoun Kim
Journal of Medical Internet Research.2024;[Epub] CrossRef
Information amount, accuracy, and relevance of generative artificial intelligence platforms’ answers regarding learning objectives of medical arthropodology evaluated in English and Korean queries in December 2023: a descriptive study
Hyunju Lee, Soobin Park
Journal of Educational Evaluation for Health Professions.2023; 20: 39. CrossRef

Medical students’ patterns of using ChatGPT as a feedback tool and perceptions of ChatGPT in a Leadership and Communication course in Korea: a cross-sectional study: Janghee Park; J Educ Eval Health Prof. 2023;20:29. Published online November 10, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.29

4,137 View
270 Download
10 Web of Science
12 Crossref

Abstract PDF Supplementary Material

Purpose
This study aimed to analyze patterns of using ChatGPT before and after group activities and to explore medical students’ perceptions of ChatGPT as a feedback tool in the classroom.
Methods
The study included 99 2nd-year pre-medical students who participated in a “Leadership and Communication” course from March to June 2023. Students engaged in both individual and group activities related to negotiation strategies. ChatGPT was used to provide feedback on their solutions. A survey was administered to assess students’ perceptions of ChatGPT’s feedback, its use in the classroom, and the strengths and challenges of ChatGPT from May 17 to 19, 2023.
Results
The students responded by indicating that ChatGPT’s feedback was helpful, and revised and resubmitted their group answers in various ways after receiving feedback. The majority of respondents expressed agreement with the use of ChatGPT during class. The most common response concerning the appropriate context of using ChatGPT’s feedback was “after the first round of discussion, for revisions.” There was a significant difference in satisfaction with ChatGPT’s feedback, including correctness, usefulness, and ethics, depending on whether or not ChatGPT was used during class, but there was no significant difference according to gender or whether students had previous experience with ChatGPT. The strongest advantages were “providing answers to questions” and “summarizing information,” and the worst disadvantage was “producing information without supporting evidence.”
Conclusion
The students were aware of the advantages and disadvantages of ChatGPT, and they had a positive attitude toward using ChatGPT in the classroom.

Citations

Citations to this article as recorded by

Higher education students’ perceptions of ChatGPT: A global study of early reactions
Dejan Ravšelj, Damijana Keržič, Nina Tomaževič, Lan Umek, Nejc Brezovar, Noorminshah A. Iahad, Ali Abdulla Abdulla, Anait Akopyan, Magdalena Waleska Aldana Segura, Jehan AlHumaid, Mohamed Farouk Allam, Maria Alló, Raphael Papa Kweku Andoh, Octavian Andron
PLOS ONE.2025; 20(2): e0315011. CrossRef
Generative AI in Otolaryngology Residency Personal Statement Writing: A Mixed‐Methods Analysis
Jacob G. J. Wihlidal, Nikolaus E. Wolter, Evan J. Propst, Vincent Lin, Michael Au, Shaunak Amin, Jennifer M. Siu
The Laryngoscope.2025;[Epub] CrossRef
Applications of Artificial Intelligence for Nonpsychomotor Skills Training in Health Professions Education: A Scoping Review
Kenya A. Costa-Dookhan, Zachary Adirim, Marta Maslej, Kayle Donner, Terri Rodak, Sophie Soklaridis, Sanjeev Sockalingam, Anupam Thakur
Academic Medicine.2025; 100(5): 635. CrossRef
Opportunities, challenges, and future directions of large language models, including ChatGPT in medical education: a systematic scoping review
Xiaojun Xu, Yixiao Chen, Jing Miao
Journal of Educational Evaluation for Health Professions.2024; 21: 6. CrossRef
Embracing ChatGPT for Medical Education: Exploring Its Impact on Doctors and Medical Students
Yijun Wu, Yue Zheng, Baijie Feng, Yuqi Yang, Kai Kang, Ailin Zhao
JMIR Medical Education.2024; 10: e52483. CrossRef
Integration of ChatGPT Into a Course for Medical Students: Explorative Study on Teaching Scenarios, Students’ Perception, and Applications
Anita V Thomae, Claudia M Witt, Jürgen Barth
JMIR Medical Education.2024; 10: e50545. CrossRef
A cross sectional investigation of ChatGPT-like large language models application among medical students in China
Guixia Pan, Jing Ni
BMC Medical Education.2024;[Epub] CrossRef
A Pilot Study of Medical Student Opinions on Large Language Models
Alan Y Xu, Vincent S Piranio, Skye Speakman, Chelsea D Rosen, Sally Lu, Chris Lamprecht, Robert E Medina, Maisha Corrielus, Ian T Griffin, Corinne E Chatham, Nicolas J Abchee, Daniel Stribling, Phuong B Huynh, Heather Harrell, Benjamin Shickel, Meghan Bre
Cureus.2024;[Epub] CrossRef
The intent of ChatGPT usage and its robustness in medical proficiency exams: a systematic review
Tatiana Chaiban, Zeinab Nahle, Ghaith Assi, Michelle Cherfane
Discover Education.2024;[Epub] CrossRef
Feasibility of a Randomised Controlled Trial of Large Artificial Intelligence-Based Linguistic Models for Clinical Reasoning Training of Physical Therapy Students. A Pilot Study (Preprint)
Raúl Ferrer Peña, Silvia Di Bonaventura, Alberto Pérez González, Alfredo Lerín Calvo
JMIR Formative Research.2024;[Epub] CrossRef
ChatGPT and Clinical Training: Perception, Concerns, and Practice of Pharm-D Students
Mohammed Zawiah, Fahmi Al-Ashwal, Lobna Gharaibeh, Rana Abu Farha, Karem Alzoubi, Khawla Abu Hammour, Qutaiba A Qasim, Fahd Abrah
Journal of Multidisciplinary Healthcare.2023; Volume 16: 4099. CrossRef
Information amount, accuracy, and relevance of generative artificial intelligence platforms’ answers regarding learning objectives of medical arthropodology evaluated in English and Korean queries in December 2023: a descriptive study
Hyunju Lee, Soobin Park
Journal of Educational Evaluation for Health Professions.2023; 20: 39. CrossRef

Efficacy and limitations of ChatGPT as a biostatistical problem-solving tool in medical education in Serbia: a descriptive study: Aleksandra Ignjatović, Lazar Stevanović; J Educ Eval Health Prof. 2023;20:28. Published online October 16, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.28

5,076 View
239 Download
13 Web of Science
13 Crossref

Abstract PDF Supplementary Material

Purpose
This study aimed to assess the performance of ChatGPT (GPT-3.5 and GPT-4) as a study tool in solving biostatistical problems and to identify any potential drawbacks that might arise from using ChatGPT in medical education, particularly in solving practical biostatistical problems.
Methods
ChatGPT was tested to evaluate its ability to solve biostatistical problems from the Handbook of Medical Statistics by Peacock and Peacock in this descriptive study. Tables from the problems were transformed into textual questions. Ten biostatistical problems were randomly chosen and used as text-based input for conversation with ChatGPT (versions 3.5 and 4).
Results
GPT-3.5 solved 5 practical problems in the first attempt, related to categorical data, cross-sectional study, measuring reliability, probability properties, and the t-test. GPT-3.5 failed to provide correct answers regarding analysis of variance, the chi-square test, and sample size within 3 attempts. GPT-4 also solved a task related to the confidence interval in the first attempt and solved all questions within 3 attempts, with precise guidance and monitoring.
Conclusion
The assessment of both versions of ChatGPT performance in 10 biostatistical problems revealed that GPT-3.5 and 4’s performance was below average, with correct response rates of 5 and 6 out of 10 on the first attempt. GPT-4 succeeded in providing all correct answers within 3 attempts. These findings indicate that students must be aware that this tool, even when providing and calculating different statistical analyses, can be wrong, and they should be aware of ChatGPT’s limitations and be careful when incorporating this model into medical education.

Citations

Citations to this article as recorded by

From statistics to deep learning: Using large language models in psychiatric research
Yining Hua, Andrew Beam, Lori B. Chibnik, John Torous
International Journal of Methods in Psychiatric Research.2025;[Epub] CrossRef
Assessing the Current Limitations of Large Language Models in Advancing Health Care Education
JaeYong Kim, Bathri Narayan Vajravelu
JMIR Formative Research.2025; 9: e51319. CrossRef
ChatGPT for Univariate Statistics: Validation of AI-Assisted Data Analysis in Healthcare Research
Michael R Ruta, Tony Gaidici, Chase Irwin, Jonathan Lifshitz
Journal of Medical Internet Research.2025; 27: e63550. CrossRef
Can Generative AI and ChatGPT Outperform Humans on Cognitive-Demanding Problem-Solving Tasks in Science?
Xiaoming Zhai, Matthew Nyaaba, Wenchao Ma
Science & Education.2024;[Epub] CrossRef
Opportunities, challenges, and future directions of large language models, including ChatGPT in medical education: a systematic scoping review
Xiaojun Xu, Yixiao Chen, Jing Miao
Journal of Educational Evaluation for Health Professions.2024; 21: 6. CrossRef
Comparing the Performance of ChatGPT-4 and Medical Students on MCQs at Varied Levels of Bloom’s Taxonomy
Ambadasu Bharatha, Nkemcho Ojeh, Ahbab Mohammad Fazle Rabbi, Michael Campbell, Kandamaran Krishnamurthy, Rhaheem Layne-Yarde, Alok Kumar, Dale Springer, Kenneth Connell, Md Anwarul Majumder
Advances in Medical Education and Practice.2024; Volume 15: 393. CrossRef
Revolutionizing Cardiology With Words: Unveiling the Impact of Large Language Models in Medical Science Writing
Abhijit Bhattaru, Naveena Yanamala, Partho P. Sengupta
Canadian Journal of Cardiology.2024; 40(10): 1950. CrossRef
ChatGPT in medicine: prospects and challenges: a review article
Songtao Tan, Xin Xin, Di Wu
International Journal of Surgery.2024; 110(6): 3701. CrossRef
In-depth analysis of ChatGPT’s performance based on specific signaling words and phrases in the question stem of 2377 USMLE step 1 style questions
Leonard Knoedler, Samuel Knoedler, Cosima C. Hoch, Lukas Prantl, Konstantin Frank, Laura Soiderer, Sebastian Cotofana, Amir H. Dorafshar, Thilo Schenck, Felix Vollbach, Giuseppe Sofo, Michael Alfertshofer
Scientific Reports.2024;[Epub] CrossRef
Evaluating the quality of responses generated by ChatGPT
Danimir Mandić, Gordana Miščević, Ljiljana Bujišić
Metodicka praksa.2024; 27(1): 5. CrossRef
A Comparative Evaluation of Statistical Product and Service Solutions (SPSS) and ChatGPT-4 in Statistical Analyses
Al Imran Shahrul, Alizae Marny F Syed Mohamed
Cureus.2024;[Epub] CrossRef
ChatGPT and Other Large Language Models in Medical Education — Scoping Literature Review
Alexandra Aster, Matthias Carl Laupichler, Tamina Rockwell-Kollmann, Gilda Masala, Ebru Bala, Tobias Raupach
Medical Science Educator.2024;[Epub] CrossRef
Exploring the potential of large language models for integration into an academic statistical consulting service–the EXPOLS study protocol
Urs Alexander Fichtner, Jochen Knaus, Erika Graf, Georg Koch, Jörg Sahlmann, Dominikus Stelzer, Martin Wolkewitz, Harald Binder, Susanne Weber, Bekalu Tadesse Moges
PLOS ONE.2024; 19(12): e0308375. CrossRef

Mentorship and self-efficacy are associated with lower burnout in physical therapists in the United States: a cross-sectional survey study: Matthew Pugliese, Jean-Michel Brismée, Brad Allen, Sean Riley, Justin Tammany, Paul Mintken; J Educ Eval Health Prof. 2023;20:27. Published online September 27, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.27

7,441 View
492 Download
5 Web of Science
7 Crossref

Abstract PDF Supplementary Material

Purpose
This study investigated the prevalence of burnout in physical therapists in the United States and the relationships between burnout and education, mentorship, and self-efficacy.
Methods
This was a cross-sectional survey study. An electronic survey was distributed to practicing physical therapists across the United States over a 6-week period from December 2020 to January 2021. The survey was completed by 2,813 physical therapists from all states. The majority were female (68.72%), White or Caucasian (80.13%), and employed full-time (77.14%). Respondents completed questions on demographics, education, mentorship, self-efficacy, and burnout. The Burnout Clinical Subtypes Questionnaire 12 (BCSQ-12) and self-reports were used to quantify burnout, and the General Self-Efficacy Scale (GSES) was used to measure self-efficacy. Descriptive and inferential analyses were performed.
Results
Respondents from home health (median BCSQ-12=42.00) and skilled nursing facility settings (median BCSQ-12=42.00) displayed the highest burnout scores. Burnout was significantly lower among those who provided formal mentorship (median BCSQ-12=39.00, P=0.0001) compared to no mentorship (median BCSQ-12=41.00). Respondents who received formal mentorship (median BCSQ-12=38.00, P=0.0028) displayed significantly lower burnout than those who received no mentorship (median BCSQ-12=41.00). A moderate negative correlation (rho=-0.49) was observed between the GSES and burnout scores. A strong positive correlation was found between self-reported burnout status and burnout scores (rrb=0.61).
Conclusion
Burnout is prevalent in the physical therapy profession, as almost half of respondents (49.34%) reported burnout. Providing or receiving mentorship and higher self-efficacy were associated with lower burnout. Organizations should consider measuring burnout levels, investing in mentorship programs, and implementing strategies to improve self-efficacy.

Citations

Citations to this article as recorded by

Wellness and Stress Management Practices Among Healthcare Professionals and Health Professional Students
Asli C. Yalim, Katherine Daly, Monica Bailey, Denise Kay, Xiang Zhu, Mohammed Patel, Laurie C. Neely, Desiree A. Díaz, Denyi M. Canario Asencio, Karla Rosario, Melissa Cowan, Magdalena Pasarica
American Journal of Health Promotion.2025; 39(2): 204. CrossRef
Final results of the National Oncology Mentorship Program 2023 and its impact on burnout and professional fulfilment
Udit Nindra, Gowri Shivasabesan, Rhiannon Mellor, Weng Ng, Wei Chua, Deme Karikios, Bethan Richards, Jia Liu
Internal Medicine Journal.2025; 55(2): 233. CrossRef
Incidence of Shared Clinical Instruction in Physical Therapy Clinical Education in the United States
Nicki Silberman, Lori Hochman, Jaya Rachwani
Journal of Physical Therapy Education.2025;[Epub] CrossRef
Assessing and Managing Prevalence of Burnout in Inpatient Occupational and Physical Therapists in an Acute Care Setting
Kelly Murphy, Maria Tucker, Laura Cataldo, Sarah Sergeant, Bill Kuklinski, Jordan Billings, Katrina Jancsy
Journal of Acute Care Physical Therapy.2025; 16(2): 39. CrossRef
Benefits, Challenges, and Strategies for Implementing Shared Clinical Instruction in the United States: A Qualitative Study
Nicki Silberman, Lori Hochman, Vicki LaFay, Jennifer Cunningham
Journal of Physical Therapy Education.2025;[Epub] CrossRef
Interprofessional education to support alcohol use screening and future team-based management of stress-related disorders in vulnerable populations
Taylor Fitzpatrick-Schmidt, Scott Edwards
Frontiers in Education.2024;[Epub] CrossRef
Prevalence of Stress and Burnout in Physical Therapist Clinical Instructors
Ryan J. Pontiff, Peggy Gleeson, Katy Mitchell, Rupal M. Patel
Journal of Physical Therapy Education.2024;[Epub] CrossRef

Development and validation of the student ratings in clinical teaching scale in Australia: a methodological study: Pin-Hsiang Huang, Anthony John O’Sullivan, Boaz Shulruf; J Educ Eval Health Prof. 2023;20:26. Published online September 5, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.26

2,190 View
170 Download
1 Web of Science

Abstract PDF Supplementary Material: Purpose
This study aimed to devise a valid measurement for assessing clinical students’ perceptions of teaching practices.
Methods
A new tool was developed based on a meta-analysis encompassing effective clinical teaching-learning factors. Seventy-nine items were generated using a frequency (never to always) scale. The tool was applied to the University of New South Wales year 2, 3, and 6 medical students. Exploratory and confirmatory factor analysis (exploratory factor analysis [EFA] and confirmatory factor analysis [CFA], respectively) were conducted to establish the tool’s construct validity and goodness of fit, and Cronbach’s α was used for reliability.
Results
In total, 352 students (44.2%) completed the questionnaire. The EFA identified student-centered learning, problem-solving learning, self-directed learning, and visual technology (reliability, 0.77 to 0.89). CFA showed acceptable goodness of fit (chi-square P<0.01, comparative fit index=0.930 and Tucker-Lewis index=0.917, root mean square error of approximation=0.069, standardized root mean square residual=0.06).
Conclusion
The established tool—Student Ratings in Clinical Teaching (STRICT)—is a valid and reliable tool that demonstrates how students perceive clinical teaching efficacy. STRICT measures the frequency of teaching practices to mitigate the biases of acquiescence and social desirability. Clinical teachers may use the tool to adapt their teaching practices with more active learning activities and to utilize visual technology to facilitate clinical learning efficacy. Clinical educators may apply STRICT to assess how these teaching practices are implemented in current clinical settings.

First
Prev
Page of 3
Next
Last

Previous issues