Journal of Educational Evaluation for Health Professions

Research articles

The effect of strengthening nurse practitioners’ competency in occupational health services for agricultural workers exposed to pesticides in primary care units, Thailand: a before-and-after study: Napamon Pumsopa, Ann Jirapongsuwan, Surintorn Kalampakorn, Sukhontha Siri; J Educ Eval Health Prof. 2025;22:14. Published online April 21, 2025; DOI: https://doi.org/10.3352/jeehp.2025.22.14 [Epub ahead of print]

99 View
28 Download

Abstract PDF: Purpose
This study aimed to evaluate the effect of the Strengthening Nurse Practitioners’ Competency in Occupational Health Service (SNPCOHS) program. It was hypothesized that nurse practitioners (NPs) participating in the program would demonstrate increased competency in providing occupational health services to agricultural workers exposed to pesticides in primary care units (PCUs) compared to their baseline competency and to a comparison group.
Methods
A quasi-experimental study was conducted between August and December 2023. The 4-week intervention included five hours of an e-learning program, three hours of online discussion, and two hours dedicated to completing an assignment. The program was evaluated at three time points: pre-intervention, post-intervention (week 4), and follow-up (week 8). Sixty NPs volunteered to participate, with 30 in the experimental group and 30 in the comparison group. Data on demographics, professional attributes, knowledge, skills, and perceived self-efficacy were collected using self-administered questionnaires via Google Forms. Data analysis involved descriptive statistics, independent t-tests, and repeated measures ANOVA.
Results
The experimental group demonstrated significantly higher mean scores in professional attributes, knowledge, skills, and perceived self-efficacy in providing occupational health services to agricultural workers exposed to pesticides compared to the comparison group at both week 4 and week 8 post-intervention.
Conclusion
The SNPCOHS program is well-suited for self-directed learning for nurses in primary care units, supporting effective occupational health service delivery. It should be disseminated and supported as an e-learning resource for nurse practitioners in PCUs.

A nationwide survey on the curriculum and educational resources related to the Clinical Skills Test of the Korean Medical Licensing Examination: a cross-sectional descriptive study: Eun-Kyung Chung, Seok Hoon Kang, Do-Hoon Kim, MinJeong Kim, Ji-Hyun Seo, Keunmi Lee, Eui-Ryoung Han; J Educ Eval Health Prof. 2025;22:11. Published online March 13, 2025; DOI: https://doi.org/10.3352/jeehp.2025.22.11

645 View
99 Download

Abstract PDF Supplementary Material: Purpose
The revised Clinical Skills Test (CST) of the Korean Medical Licensing Exam aims to provide a better assessment of physicians’ clinical competence and ability to interact with patients. This study examined the impact of the revised CST on medical education curricula and resources nationwide, while also identifying areas for improvement within the revised CST.
Methods
This study surveyed faculty responsible for clinical clerkships at 40 medical schools throughout Korea to evaluate the status and changes in clinical skills education, assessment, and resources related to the CST. The researchers distributed the survey via email through regional consortia between December 7, 2023 and January 19, 2024.
Results
Nearly all schools implemented preliminary student–patient encounters during core clinical rotations. Schools primarily conducted clinical skills assessments in the third and fourth years, with a simplified form introduced in the first and second years. Remedial education was conducted through various methods, including one-on-one feedback from faculty after the assessment. All schools established clinical skills centers and made ongoing improvements. Faculty members did not perceive the CST revisions as significantly altering clinical clerkship or skills assessments. They suggested several improvements, including assessing patient records to improve accuracy and increasing the objectivity of standardized patient assessments to ensure fairness.
Conclusion
During the CST, students’ involvement in patient encounters and clinical skills education increased, improving the assessment and feedback processes for clinical skills within the curriculum. To enhance students’ clinical competencies and readiness, strengthening the validity and reliability of the CST is essential.

Inter-rater reliability and content validity of the measurement tool for portfolio assessments used in the Introduction to Clinical Medicine course at Ewha Womans University College of Medicine: a methodological study: Dong-Mi Yoo, Jae Jin Han; J Educ Eval Health Prof. 2024;21:39. Published online December 10, 2024; DOI: https://doi.org/10.3352/jeehp.2024.21.39

868 View
165 Download

Abstract PDF Supplementary Material: Purpose
This study aimed to examine the reliability and validity of a measurement tool for portfolio assessments in medical education. Specifically, it investigated scoring consistency among raters and assessment criteria appropriateness according to an expert panel.
Methods
A cross-sectional observational study was conducted from September to December 2018 for the Introduction to Clinical Medicine course at the Ewha Womans University College of Medicine. Data were collected for 5 randomly selected portfolios scored by a gold-standard rater and 6 trained raters. An expert panel assessed the validity of 12 assessment items using the content validity index (CVI). Statistical analysis included Pearson correlation coefficients for rater alignment, the intraclass correlation coefficient (ICC) for inter-rater reliability, and the CVI for item-level validity.
Results
Rater 1 had the highest Pearson correlation (0.8916) with the gold-standard rater, while Rater 5 had the lowest (0.4203). The ICC for all raters was 0.3821, improving to 0.4415 after excluding Raters 1 and 5, indicating a 15.6% reliability increase. All assessment items met the CVI threshold of ≥0.75, with some achieving a perfect score (CVI=1.0). However, items like “sources” and “level and degree of performance” showed lower validity (CVI=0.72).
Conclusion
The present measurement tool for portfolio assessments demonstrated moderate reliability and strong validity, supporting its use as a credible tool. For a more reliable portfolio assessment, more faculty training is needed.

Revised evaluation objectives of the Korean Dentist Clinical Skill Test: a survey study and focus group interviews: Jae-Hoon Kim, Young J Kim, Deuk-Sang Ma, Se-Hee Park, Ahran Pae, June-Sung Shim, Il-Hyung Yang, Ui-Won Jung, Byung-Joon Choi, Yang-Hyun Chun; J Educ Eval Health Prof. 2024;21:11. Published online May 30, 2024; DOI: https://doi.org/10.3352/jeehp.2024.21.11

1,464 View
242 Download
1 Crossref

Abstract PDF Supplementary Material

Purpose
This study aimed to propose a revision of the evaluation objectives of the Korean Dentist Clinical Skill Test by analyzing the opinions of those involved in the examination after a review of those objectives.
Methods
The clinical skill test objectives were reviewed based on the national-level dental practitioner competencies, dental school educational competencies, and the third dental practitioner job analysis. Current and former examinees were surveyed about their perceptions of the evaluation objectives. The validity of 22 evaluation objectives and overlapping perceptions based on area of specialty were surveyed on a 5-point Likert scale by professors who participated in the clinical skill test and dental school faculty members. Additionally, focus group interviews were conducted with experts on the examination.
Results
It was necessary to consider including competency assessments for “emergency rescue skills” and “planning and performing prosthetic treatment.” There were no significant differences between current and former examinees in their perceptions of the clinical skill test’s objectives. The professors who participated in the examination and dental school faculty members recognized that most of the objectives were valid. However, some responses stated that “oromaxillofacial cranial nerve examination,” “temporomandibular disorder palpation test,” and “space management for primary and mixed dentition” were unfeasible evaluation objectives and overlapped with dental specialty areas.
Conclusion
When revising the Korean Dentist Clinical Skill Test’s objectives, it is advisable to consider incorporating competency assessments related to “emergency rescue skills” and “planning and performing prosthetic treatment.”

Citations

Citations to this article as recorded by

Establishment of a tooth preparation competency cohort at Dankook University Dental School
Hojin Moon
Journal of Korean Dental Association.2025; 63(2): 42. CrossRef

Implementation strategy for introducing a clinical skills examination to the Korean Oriental Medicine Licensing Examination: a mixed-method modified Delphi study: Chan-Young Kwon, Sanghoon Lee, Min Hwangbo, Chungsik Cho, Sangwoo Shin, Dong-Hyeon Kim, Aram Jeong, Hye-Yoon Lee; J Educ Eval Health Prof. 2023;20:23. Published online July 17, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.23

2,744 View
161 Download

Abstract PDF Supplementary Material: Purpose
This study investigated the validity of introducing a clinical skills examination (CSE) to the Korean Oriental Medicine Licensing Examination through a mixed-method modified Delphi study.
Methods
A 3-round Delphi study was conducted between September and November 2022. The expert panel comprised 21 oriental medicine education experts who were officially recommended by relevant institutions and organizations. The questionnaires included potential content for the CSE and a detailed implementation strategy. Subcommittees were formed to discuss concerns around the introduction of the CSE, which were collected as open-ended questions. In this study, a 66.7% or greater agreement rate was defined as achieving a consensus.
Results
The expert panel’s evaluation of the proposed clinical presentations and basic clinical skills suggested their priorities. Of the 10 items investigated for building a detailed implementation strategy for the introduction of the CSE to the Korean Oriental Medicine Licensing Examination, a consensus was achieved on 9. However, the agreement rate on the timing of the introduction of the CSE was low. Concerns around 4 clinical topics were discussed in the subcommittees, and potential solutions were proposed.
Conclusion
This study offers preliminary data and raises some concerns that can be used as a reference while discussing the introduction of the CSE to the Korean Oriental Medicine Licensing Examination.

Enhancement of the technical and non-technical skills of nurse anesthesia students using the Anesthetic List Management Assessment Tool in Iran: a quasi-experimental study: Ali Khalafi, Maedeh Kordnejad, Vahid Saidkhani; J Educ Eval Health Prof. 2023;20:19. Published online June 16, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.19

2,230 View
107 Download

Abstract PDF Supplementary Material: Purpose
This study investigated the effect of evaluations based on the Anesthetic List Management Assessment Tool (ALMAT) form on improving the technical and non-technical skills of final-year nurse anesthesia students at Ahvaz Jundishapur University of Medical Sciences (AJUMS).
Methods
This was a semi-experimental study with a pre-test and post-test design. It included 45 final-year nurse anesthesia students of AJUMS and lasted for 3 months. The technical and non-technical skills of the intervention group were assessed at 4 university hospitals using formative-feedback evaluation based on the ALMAT form, from induction of anesthesia until reaching mastery and independence. Finally, the students’ degree of improvement in technical and non-technical skills was compared between the intervention and control groups. Statistical tests (the independent t-test, paired t-test, and Mann-Whitney test) were used to analyze the data.
Results
The rate of improvement in post-test scores of technical skills was significantly higher in the intervention group than in the control group (P˂0.0001). Similarly, the students in the intervention group received significantly higher post-test scores for non-technical skills than the students in the control group (P˂0.0001).
Conclusion
The findings of this study showed that the use of ALMAT as a formative-feedback evaluation method to evaluate technical and non-technical skills had a significant effect on improving these skills and was effective in helping students learn and reach mastery and independence.

Improvement of the clinical skills of nurse anesthesia students using mini-clinical evaluation exercises in Iran: a randomized controlled study: Ali Khalafi, Yasamin Sharbatdar, Nasrin Khajeali, Mohammad Hosein Haghighizadeh, Mahshid Vaziri; J Educ Eval Health Prof. 2023;20:12. Published online April 6, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.12

3,902 View
145 Download
4 Web of Science
6 Crossref

Abstract PDF Supplementary Material

Purpose
The present study aimed to investigate the effect of a mini-clinical evaluation exercise (CEX) assessment on improving the clinical skills of nurse anesthesia students at Ahvaz Jundishapur University of Medical Sciences, Ahvaz, Iran.
Methods
This study started on November 1, 2022, and ended on December 1, 2022. It was conducted among 50 nurse anesthesia students divided into intervention and control groups. The intervention group’s clinical skills were evaluated 4 times using the mini-CEX method. In contrast, the same skills were evaluated in the control group based on the conventional method—that is, general supervision by the instructor during the internship and a summative evaluation based on a checklist at the end of the course. The intervention group students also filled out a questionnaire to measure their satisfaction with the mini-CEX method.
Results
The mean score of the students in both the control and intervention groups increased significantly on the post-test (P<0.0001), but the improvement in the scores of the intervention group was significantly greater compared with the control group (P<0.0001). The overall mean score for satisfaction in the intervention group was 76.3 out of a maximum of 95.
Conclusion
The findings of this study showed that using mini-CEX as a formative evaluation method to evaluate clinical skills had a significant effect on the improvement of nurse anesthesia students’ clinical skills, and they had a very favorable opinion about this evaluation method.

Citations

Citations to this article as recorded by

Exploring the clinical practice training program for Master of Nursing Specialist in anesthesia: A qualitative study
Huihui Hu, Yanxin Gu, Yi Yang, Rui Gao, Peishuang Wang, Fang Zhou
Nurse Education Today.2025; 147: 106577. CrossRef
Psychometric testing of anesthesia nursing competence scale (AnestComp)
Samira Mahmoudi, Akram Yazdani, Fatemeh Hasanshiri
Perioperative Care and Operating Room Management.2024; 34: 100368. CrossRef
Application of flipped classroom teaching method based on ADDIE concept in clinical teaching for neurology residents
Juan Zhang, Hong Chen, Xie Wang, Xiaofeng Huang, Daojun Xie
BMC Medical Education.2024;[Epub] CrossRef
Impactos do Mini-Cex no ensino-aprendizagem da saúde: uma revisão integrativa
João Henrique Anizio de Farias, Draenne Micarla dos Santos Silva, Clédson Calixto de Oliveira, Elzenir Pereira de Oliveira Almeida
Revista de Gestão e Secretariado.2024; 15(9): e4150. CrossRef
Comparing Satisfaction of Undergraduate Nursing Students`: Mini-CEX vs CIM in Assessing Clinical Competence
Somia Saghir, Anny Ashiq Ali, Kashif Khan, Uzma Bibi, Shafaat Ullah, Rafi Ullah, Zaifullah Khan, Tahir Khan
Pakistan Journal of Health Sciences.2023; : 134. CrossRef
Enhancement of the technical and non-technical skills of nurse anesthesia students using the Anesthetic List Management Assessment Tool in Iran: a quasi-experimental study
Ali Khalafi, Maedeh Kordnejad, Vahid Saidkhani
Journal of Educational Evaluation for Health Professions.2023; 20: 19. CrossRef

Effect of a smartphone-based online electronic logbook to evaluate the clinical skills of nurse anesthesia students in Iran: a randomized controlled study: Ali Khalafi, Nahid Jamshidi, Nasrin Khajeali, Saeed Ghanbari; J Educ Eval Health Prof. 2023;20:10. Published online March 31, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.10

3,170 View
146 Download

Abstract PDF Supplementary Material: Purpose
This study was conducted to evaluate a smartphone-based online electronic logbook used to assess the clinical skills of nurse anesthesia students in Iran.
Methods
This randomized controlled study was conducted after tool development at Ahvaz Jundishapur University of Medical Sciences in Ahvaz, Iran from January 2022 to December 2022. The online electronic logbook involved in this study was an Android-compatible application used to evaluate the clinical skills of nurse anesthesia students. In the implementation phase, the online electronic logbook was piloted for 3 months in anesthesia training in comparison with a paper logbook. For this purpose, 49 second- and third-year anesthesia nursing students selected using the census method were assigned to intervention (online electronic logbook) and control (paper logbook) groups. The online electronic logbook and paper logbook were compared in terms of student satisfaction and learning outcomes.
Results
A total of 39 students participated in the study. The mean satisfaction score of the intervention group was significantly higher than that of the control group (P=0.027). The mean score of learning outcomes was also significantly higher for the intervention than the control group (P=0.028).
Conclusion
Smartphone technology can provide a platform for improving the evaluation of the clinical skills of nursing anesthesia students, leading to increased satisfaction and improved learning outcomes.

Physical therapy students’ perception of their ability of clinical and clinical decision-making skills enhanced after simulation-based learning courses in the United States: a repeated measures design: Fabian Bizama, Mansoor Alameri, Kristy Jean Demers, Derrick Ferguson Campbell; J Educ Eval Health Prof. 2022;19:34. Published online December 19, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.34

4,190 View
268 Download
2 Web of Science
6 Crossref

Abstract PDF Supplementary Material

Purpose
It aimed to investigate physical therapy students’ perception of their ability of clinical and clinical decision-making skills after a simulation-based learning course in the United States.
Methods
Survey questionnaires were administered to voluntary participants, including 44 second and third-year physical therapy students of the University of St. Augustine for Health Sciences during 2021–2022. Thirty-six questionnaire items consisted of 4 demographic items, 1 general evaluation, 21 test items for clinical decision-making skills, and 4 clinical skill items. Descriptive and inferential statistics evaluated differences in students’ perception of their ability in clinical decision-making and clinical skills, pre- and post-simulation, and post-first clinical experience during 2021–2022.
Results
Friedman test revealed a significant increase from pre- to post-simulation in perception of the ability of clinical and clinical decision-making skills total tool score (P<0.001), clinical decision-making 21-item score (P<0.001), and clinical skills score (P<0.001). No significant differences were found between post-simulation and post-first clinical experience. Post-hoc tests indicated a significant difference between pre-simulation and post-simulation (P<0.001) and between pre-simulation and post-first clinical experience (P<0.001). Forty-three students (97.6%) either strongly agreed (59.1%) or agreed (38.5%) that simulation was a valuable learning experience.
Conclusion
The above findings suggest that simulation-based learning helped students begin their first clinical experience with enhanced clinical and clinical decision-making skills.

Citations

Citations to this article as recorded by

An Educational Escape Room's Influence on Physical Therapy Students' Perception of Clinical Reasoning Development: A Qualitative Study
Jessica L. Sullivan, Karen E. H. Grossnickle, Elizabeth S. Moore, Briyana Morrell
Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare.2025;[Epub] CrossRef
Clinical reasoning competence of finnish physiotherapy students - cross-sectional study
Kristiina Makkonen, Hilkka Korpi, Tuulikki Sjögren
European Journal of Physiotherapy.2025; : 1. CrossRef
Physiotherapists' training in oncology rehabilitation from entry‐level to advanced education: A qualitative study
Gianluca Bertoni, Valentina Conti, Marco Testa, Ilaria Coppola, Stefania Costi, Simone Battista
Physiotherapy Research International.2024;[Epub] CrossRef
Physiotherapy education: Simulation-based learning on biopsychosocial training in chronic pain – A mixed-methods study
D. Barranco-i-Reixachs, C. Bravo, A. Orio, F. Rubí-Carnacea
Fisioterapia.2024; 46(6): 325. CrossRef
Simulación clínica mediada por tecnología: un escenario didáctico a partir de recursos para la formación de los profesionales en rehabilitación
Cyndi Yacira Meneses Castaño, Isabel Jimenez Becerra, Paola Teresa Penagos Gomez
Educación Médica.2023; 24(4): 100810. CrossRef
Self-Efficacy with Telehealth Examination: the Doctor of Physical Therapy Student Perspective
Derrick F. Campbell, Jean-Michel Brismee, Brad Allen, Troy Hooper, Manuel A. Domenech, Kathleen J. Manella
Philippine Journal of Physical Therapy.2023; 2(2): 12. CrossRef

Is online objective structured clinical examination teaching an acceptable replacement in post-COVID-19 medical education in the United Kingdom?: a descriptive study: Vashist Motkur, Aniket Bharadwaj, Nimalesh Yogarajah; J Educ Eval Health Prof. 2022;19:30. Published online November 7, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.30

2,906 View
155 Download
2 Web of Science
2 Crossref

Abstract PDF Supplementary Material

Purpose
Coronavirus disease 2019 (COVID-19) restrictions resulted in an increased emphasis on virtual communication in medical education. This study assessed the acceptability of virtual teaching in an online objective structured clinical examination (OSCE) series and its role in future education.
Methods
Six surgical OSCE stations were designed, covering common surgical topics, with specific tasks testing data interpretation, clinical knowledge, and communication skills. These were delivered via Zoom to students who participated in student/patient/examiner role-play. Feedback was collected by asking students to compare online teaching with previous experiences of in-person teaching. Descriptive statistics were used for Likert response data, and thematic analysis for free-text items.
Results
Sixty-two students provided feedback, with 81% of respondents finding online instructions preferable to paper equivalents. Furthermore, 65% and 68% found online teaching more efficient and accessible, respectively, than in-person teaching. Only 34% found communication with each other easier online; Forty percent preferred online OSCE teaching to in-person teaching. Students also expressed feedback in positive and negative free-text comments.
Conclusion
The data suggested that generally students were unwilling for online teaching to completely replace in-person teaching. The success of online teaching was dependent on the clinical skill being addressed; some were less amenable to a virtual setting. However, online OSCE teaching could play a role alongside in-person teaching.

Citations

Citations to this article as recorded by

Feasibility and reliability of the pandemic-adapted online-onsite hybrid graduation OSCE in Japan
Satoshi Hara, Kunio Ohta, Daisuke Aono, Toshikatsu Tamai, Makoto Kurachi, Kimikazu Sugimori, Hiroshi Mihara, Hiroshi Ichimura, Yasuhiko Yamamoto, Hideki Nomura
Advances in Health Sciences Education.2024; 29(3): 949. CrossRef
Should Virtual Objective Structured Clinical Examination (OSCE) Teaching Replace or Complement Face-to-Face Teaching in the Post-COVID-19 Educational Environment: An Evaluation of an Innovative National COVID-19 Teaching Programme
Charles Gamble, Alice Oatham, Raj Parikh
Cureus.2023;[Epub] CrossRef

Effect of a forensic nursing virtual education course on knowledge and clinical decision-making of master’s nursing students in Iran: a non-equivalent control group pre- and post-test study: Zeynab Firuzi, Mitra Sedghi Sabet, Fateme Jafaraghaee, Hedayat Jafari, Ehsan Kazemnezhad Leyli, Samad Karkhah, Mohammad Javad Ghazanfari; J Educ Eval Health Prof. 2022;19:20. Published online August 25, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.20

3,657 View
309 Download
1 Web of Science
1 Crossref

Abstract PDF Supplementary Material

Purpose
Forensic nursing is a specialty in the nursing profession based on legal procedures. This study aimed to assess the effect of a forensic nursing virtual education course on knowledge and clinical decision-making among master’s nursing students.
Methods
In a quasi-experimental study with a pre- and post-test, 106 master’s nursing students at Guilan (n=65) and Mazandaran (n=41) Universities of Medical Sciences, Iran were enrolled. Data were collected using census sampling from March to April 2021. Participants in the intervention group received a forensic nursing virtual education course in three 90-minute sessions for 2 days.
Results
A total of 88 out of 106 master’s nursing students were enrolled in this study. The mean post-education score for knowledge in the intervention group was significantly higher than in the control group (12.52 vs. 7.67, P<0.001). The mean post-education score for clinical decision-making in the intervention group was significantly higher than in the control group (16.96 vs. 13.64, P<0.001).
Conclusion
The level of knowledge and clinical decision-making of master’s nursing students regarding forensic evidence improved after the forensic nursing virtual education course in the intervention group compared to the control group. Nursing managers and policymakers can develop appropriate strategies to improve the knowledge and clinical decision-making of nursing students by using forensic nursing education courses in the curricula of nursing programs, especially in postgraduate education as an elective or mandatory course.

Citations

Citations to this article as recorded by

Knowledge and attitude of university nursing students towards forensic nursing and their influencing factors: a mixed-methods study
Ebtsam Aly Abou Hashish
Egyptian Journal of Forensic Sciences.2024;[Epub] CrossRef

Comparison between residents with a 6-year medical program and a 7-year medical program in terms of objective structured clinical examination performance in postgraduate year training in Taiwan: a 2-group pre- and post-test non-synchronized study: Ya-Ting Chang, Ying-Ying Yang, Chung-Pin Li, Chen-Huan Chen; J Educ Eval Health Prof. 2022;19:13. Published online June 24, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.13

3,256 View
223 Download
1 Crossref

Abstract PDF Supplementary Material

Purpose
In 2013, medical schools in Taiwan implemented a 6-year medical program that replaced the previous 7-year medical education program. The postgraduate year (PGY) program was also extended from 1 year to 2 years. The new program is characterized by diversified teaching, integration of medical skills, a system-oriented curriculum, and the implementation of primary care and clinical thinking training. The purpose of this study was to examine whether postgraduate residents who learned under the new program have better patient care skills than those who learned under the previous program.
Methods
Of 101 residents in the PGY program at Taipei Veterans General Hospital, 78 were trained in the 6-year program, while 23 were trained in the 7-year program. During the PGY training, 2 objective structured clinical examinations (OSCEs) were used to evaluate clinical reasoning, communication skills, and procedural skills at the beginning of the training and after 11 months of training, respectively. The scores of each OSCE and the rate of improvement of the pre- and post-tests were analyzed.
Results
Residents trained in the new program scored higher on clinical reasoning (P<0.001) and the total scores of the 3 tested skills (P=0.019) on the pre-test. In terms of improvement, residents educated in the previous system improved more in clinical reasoning than those educated in the new education system.
Conclusion
The new medical education program, which emphasizes clinical thinking, improved residents’ clinical skills. The PGY program was effective in improving the clinical performance of residents who were educated in the previous system.

Citations

Citations to this article as recorded by

Addressing Burnout Problems among Postgraduate Trainee Doctors
Lien-Chung Wei
Taiwanese Journal of Psychiatry.2023; 37(2): 96. CrossRef

Comparing the cut score for the borderline group method and borderline regression method with norm-referenced standard setting in an objective structured clinical examination in medical school in Korea: Song Yi Park, Sang-Hwa Lee, Min-Jeong Kim, Ki-Hwan Ji, Ji Ho Ryu; J Educ Eval Health Prof. 2021;18:25. Published online September 27, 2021; DOI: https://doi.org/10.3352/jeehp.2021.18.25

6,915 View
323 Download
3 Web of Science
4 Crossref

Abstract PDF Supplementary Material

Purpose
Setting standards is critical in health professions. However, appropriate standard setting methods do not always apply to the set cut score in performance assessment. The aim of this study was to compare the cut score when the standard setting is changed from the norm-referenced method to the borderline group method (BGM) and borderline regression method (BRM) in an objective structured clinical examination (OSCE) in medical school.
Methods
This was an explorative study to model the implementation of the BGM and BRM. A total of 107 fourth-year medical students attended the OSCE at 7 stations for encountering standardized patients (SPs) and at 1 station for performing skills on a manikin on July 15th, 2021. Thirty-two physician examiners evaluated the performance by completing a checklist and global rating scales.
Results
The cut score of the norm-referenced method was lower than that of the BGM (P<0.01) and BRM (P<0.02). There was no significant difference in the cut score between the BGM and BRM (P=0.40). The station with the highest standard deviation and the highest proportion of the borderline group showed the largest cut score difference in standard setting methods.
Conclusion
Prefixed cut scores by the norm-referenced method without considering station contents or examinee performance can vary due to station difficulty and content, affecting the appropriateness of standard setting decisions. If there is an adequate consensus on the criteria for the borderline group, standard setting with the BRM could be applied as a practical and defensible method to determine the cut score for OSCE.

Citations

Citations to this article as recorded by

Standard setting methods in objective structured clinical examination (OSCE): A comparative study of five methods
Reshma Ansari, Norhafizah Ab Manan, Nur Ain Mahat, Norfaizatul Shalida Omar, Atikah Abdul Latiff, Sara Idris, Azli Shahril Othman
Journal of Medical Education Development.2024; 17(56): 87. CrossRef
Analyzing the Quality of Objective Structured Clinical Examination in Alborz University of Medical Sciences
Suleiman Ahmadi, Amin Habibi, Mitra Rahimzadeh, Shahla Bahrami
Alborz University Medical Journal.2023; 12(4): 485. CrossRef
Possibility of using the yes/no Angoff method as a substitute for the percent Angoff method for estimating the cutoff score of the Korean Medical Licensing Examination: a simulation study
Janghee Park
Journal of Educational Evaluation for Health Professions.2022; 19: 23. CrossRef
Newly appointed medical faculty members’ self-evaluation of their educational roles at the Catholic University of Korea College of Medicine in 2020 and 2021: a cross-sectional survey-based study
Sun Kim, A Ra Cho, Chul Woon Chung
Journal of Educational Evaluation for Health Professions.2021; 18: 28. CrossRef

Empirical analysis comparing the tele-objective structured clinical examination and the in-person assessment in Australia: Jonathan Zachary Felthun, Silas Taylor, Boaz Shulruf, Digby Wigram Allen; J Educ Eval Health Prof. 2021;18:23. Published online September 23, 2021; DOI: https://doi.org/10.3352/jeehp.2021.18.23

6,101 View
241 Download
4 Web of Science
5 Crossref

Abstract PDF Supplementary Material

Purpose
It aimed to compare the use of the tele-objective structured clinical examination (teleOSCE) with in-person assessment in high-stakes clinical examination so as to determine the impact of the teleOSCE on the assessment undertaken. Discussion follows regarding what skills and domains can effectively be assessed in a teleOSCE.
Methods
This study is a retrospective observational analysis. It compares the results achieved by final year medical students in their clinical examination, assessed using the teleOSCE in 2020 (n=285), with those who were examined using the traditional in-person format in 2019 (n=280). The study was undertaken at the University of New South Wales, Australia.
Results
In the domain of physical examination, students in 2020 scored 0.277 points higher than those in 2019 (mean difference=–0.277, P<0.001, effect size=0.332). Across all other domains, there was no significant difference in mean scores between 2019 and 2020.
Conclusion
The teleOSCE does not negatively impact assessment in clinical examination in all domains except physical examination. If the teleOSCE is the future of clinical skills examination, assessment of physical examination will require concomitant workplace-based assessment.

Citations

Citations to this article as recorded by

Feasibility of an online clinical assessment of competence in physiotherapy students
Brooke Flew, Lucy Chipchase, Darren Lee, Jodie A. McClelland
Physiotherapy Theory and Practice.2025; 41(3): 508. CrossRef
Feasibility and reliability of the pandemic-adapted online-onsite hybrid graduation OSCE in Japan
Satoshi Hara, Kunio Ohta, Daisuke Aono, Toshikatsu Tamai, Makoto Kurachi, Kimikazu Sugimori, Hiroshi Mihara, Hiroshi Ichimura, Yasuhiko Yamamoto, Hideki Nomura
Advances in Health Sciences Education.2024; 29(3): 949. CrossRef
Navigating digital assessments in medical education: Findings from a scoping review
Chin-Siang Ang, Sakura Ito, Jennifer Cleland
Medical Teacher.2024; : 1. CrossRef
Radiography education in 2022 and beyond - Writing the history of the present: A narrative review
Y.X. Tay, J.P. McNulty
Radiography.2023; 29(2): 391. CrossRef
Newly appointed medical faculty members’ self-evaluation of their educational roles at the Catholic University of Korea College of Medicine in 2020 and 2021: a cross-sectional survey-based study
Sun Kim, A Ra Cho, Chul Woon Chung
Journal of Educational Evaluation for Health Professions.2021; 18: 28. CrossRef

Brief report

Feasibility of clinical performance assessment of medical students on a virtual sub-internship in the United States: John Woller, Sean Tackett, Ariella Apfel, Janet Record, Danelle Cayea, Shannon Walker, Amit Pahwa; J Educ Eval Health Prof. 2021;18:12. Published online June 22, 2021; DOI: https://doi.org/10.3352/jeehp.2021.18.12

5,841 View
311 Download
1 Web of Science
1 Crossref

Abstract PDF Supplementary Material

We aimed to determine whether it was feasible to assess medical students as they completed a virtual sub-internship. Six students (out of 31 who completed an in-person sub-internship) participated in a 2-week virtual sub-internship, caring for patients remotely. Residents and attendings assessed those 6 students in 15 domains using the same assessment measures from the in-person sub-internship. Raters marked “unable to assess” in 75/390 responses (19%) for the virtual sub-internship versus 88/3,405 (2.6%) for the in-person sub-internship (P=0.01), most frequently for the virtual sub-internship in the domains of the physical examination (21, 81%), rapport with patients (18, 69%), and compassion (11, 42%). Students received complete assessments in most areas. Scores were higher for the in-person than the virtual sub-internship (4.67 vs. 4.45, P<0.01) for students who completed both. Students uniformly rated the virtual clerkship positively. Students can be assessed in many domains in the context of a virtual sub-internship.

Citations

Citations to this article as recorded by

Association of Virtual Away Rotations With Residency Applicant Outcomes in Otolaryngology
Nicholas R. Lenze, William J. Benjamin, Lauren A. Bohm, Marc C. Thorne, Michael J. Brenner, Angela P. Mihalic, Robbi A. Kupfer
OTO Open.2023;[Epub] CrossRef

First
Prev
Page of 3
Next
Last

Search