Journal of Educational Evaluation for Health Professions

Research articles

Assessment of the viability of integrating virtual reality programs in practical tests for the Korean Radiological Technologists Licensing Examination: a survey study: Hye Min Park, Eun Seong Kim, Deok Mun Kwon, Pyong Kon Cho, Seoung Hwan Kim, Ki Baek Lee, Seong Hu Kim, Moon Il Bong, Won Seok Yang, Jin Eui Kim, Gi Bong Kang, Yong Su Yoon, Jung Su Kim; J Educ Eval Health Prof. 2023;20:33. Published online November 28, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.33

959 View
89 Download

Abstract PDF Supplementary Material: Purpose
The objective of this study was to assess the feasibility of incorporating virtual reality/augmented reality (VR/AR) programs into practical tests administered as part of the Korean Radiological Technologists Licensing Examination (KRTLE). This evaluation is grounded in a comprehensive survey that targeted enrolled students in departments of radiology across the nation.
Methods
In total, 682 students from radiology departments across the nation were participants in the survey. An online survey platform was used, and the questionnaire was structured into 5 distinct sections and 27 questions. A frequency analysis for each section of the survey was conducted using IBM SPSS ver. 27.0.
Results
Direct or indirect exposure to VR/AR content was reported by 67.7% of all respondents. Furthermore, 55.4% of the respondents expressed that VR/AR could be integrated into their classes, which signified a widespread acknowledgment of VR among the students. With regards to the integration of a VR/AR or mixed reality program into the practical tests for purposes of the KRTLE, a substantial amount of the respondents (57.3%) exhibited a positive inclination and recommended its introduction.
Conclusion
The application of VR/AR programs within practical tests of the KRTLE will be used as an alternative for evaluating clinical examination procedures and validating job skills.

Is online objective structured clinical examination teaching an acceptable replacement in post-COVID-19 medical education in the United Kingdom?: a descriptive study: Vashist Motkur, Aniket Bharadwaj, Nimalesh Yogarajah; J Educ Eval Health Prof. 2022;19:30. Published online November 7, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.30

1,633 View
129 Download
2 Web of Science
2 Crossref

Purpose
Coronavirus disease 2019 (COVID-19) restrictions resulted in an increased emphasis on virtual communication in medical education. This study assessed the acceptability of virtual teaching in an online objective structured clinical examination (OSCE) series and its role in future education.
Methods
Six surgical OSCE stations were designed, covering common surgical topics, with specific tasks testing data interpretation, clinical knowledge, and communication skills. These were delivered via Zoom to students who participated in student/patient/examiner role-play. Feedback was collected by asking students to compare online teaching with previous experiences of in-person teaching. Descriptive statistics were used for Likert response data, and thematic analysis for free-text items.
Results
Sixty-two students provided feedback, with 81% of respondents finding online instructions preferable to paper equivalents. Furthermore, 65% and 68% found online teaching more efficient and accessible, respectively, than in-person teaching. Only 34% found communication with each other easier online; Forty percent preferred online OSCE teaching to in-person teaching. Students also expressed feedback in positive and negative free-text comments.
Conclusion
The data suggested that generally students were unwilling for online teaching to completely replace in-person teaching. The success of online teaching was dependent on the clinical skill being addressed; some were less amenable to a virtual setting. However, online OSCE teaching could play a role alongside in-person teaching.

Citations

Citations to this article as recorded by

Feasibility and reliability of the pandemic-adapted online-onsite hybrid graduation OSCE in Japan
Satoshi Hara, Kunio Ohta, Daisuke Aono, Toshikatsu Tamai, Makoto Kurachi, Kimikazu Sugimori, Hiroshi Mihara, Hiroshi Ichimura, Yasuhiko Yamamoto, Hideki Nomura
Advances in Health Sciences Education.2023;[Epub] CrossRef
Should Virtual Objective Structured Clinical Examination (OSCE) Teaching Replace or Complement Face-to-Face Teaching in the Post-COVID-19 Educational Environment: An Evaluation of an Innovative National COVID-19 Teaching Programme
Charles Gamble, Alice Oatham, Raj Parikh
Cureus.2023;[Epub] CrossRef

Comparing the cut score for the borderline group method and borderline regression method with norm-referenced standard setting in an objective structured clinical examination in medical school in Korea: Song Yi Park, Sang-Hwa Lee, Min-Jeong Kim, Ki-Hwan Ji, Ji Ho Ryu; J Educ Eval Health Prof. 2021;18:25. Published online September 27, 2021; DOI: https://doi.org/10.3352/jeehp.2021.18.25

5,340 View
295 Download
2 Web of Science
3 Crossref

Abstract PDF Supplementary Material

Purpose
Setting standards is critical in health professions. However, appropriate standard setting methods do not always apply to the set cut score in performance assessment. The aim of this study was to compare the cut score when the standard setting is changed from the norm-referenced method to the borderline group method (BGM) and borderline regression method (BRM) in an objective structured clinical examination (OSCE) in medical school.
Methods
This was an explorative study to model the implementation of the BGM and BRM. A total of 107 fourth-year medical students attended the OSCE at 7 stations for encountering standardized patients (SPs) and at 1 station for performing skills on a manikin on July 15th, 2021. Thirty-two physician examiners evaluated the performance by completing a checklist and global rating scales.
Results
The cut score of the norm-referenced method was lower than that of the BGM (P<0.01) and BRM (P<0.02). There was no significant difference in the cut score between the BGM and BRM (P=0.40). The station with the highest standard deviation and the highest proportion of the borderline group showed the largest cut score difference in standard setting methods.
Conclusion
Prefixed cut scores by the norm-referenced method without considering station contents or examinee performance can vary due to station difficulty and content, affecting the appropriateness of standard setting decisions. If there is an adequate consensus on the criteria for the borderline group, standard setting with the BRM could be applied as a practical and defensible method to determine the cut score for OSCE.

Citations

Citations to this article as recorded by

Analyzing the Quality of Objective Structured Clinical Examination in Alborz University of Medical Sciences
Suleiman Ahmadi, Amin Habibi, Mitra Rahimzadeh, Shahla Bahrami
Alborz University Medical Journal.2023; 12(4): 485. CrossRef
Possibility of using the yes/no Angoff method as a substitute for the percent Angoff method for estimating the cutoff score of the Korean Medical Licensing Examination: a simulation study
Janghee Park
Journal of Educational Evaluation for Health Professions.2022; 19: 23. CrossRef
Newly appointed medical faculty members’ self-evaluation of their educational roles at the Catholic University of Korea College of Medicine in 2020 and 2021: a cross-sectional survey-based study
Sun Kim, A Ra Cho, Chul Woon Chung
Journal of Educational Evaluation for Health Professions.2021; 18: 28. CrossRef

Empirical analysis comparing the tele-objective structured clinical examination and the in-person assessment in Australia: Jonathan Zachary Felthun, Silas Taylor, Boaz Shulruf, Digby Wigram Allen; J Educ Eval Health Prof. 2021;18:23. Published online September 23, 2021; DOI: https://doi.org/10.3352/jeehp.2021.18.23

4,783 View
224 Download
2 Web of Science
4 Crossref

Abstract PDF Supplementary Material

Purpose
It aimed to compare the use of the tele-objective structured clinical examination (teleOSCE) with in-person assessment in high-stakes clinical examination so as to determine the impact of the teleOSCE on the assessment undertaken. Discussion follows regarding what skills and domains can effectively be assessed in a teleOSCE.
Methods
This study is a retrospective observational analysis. It compares the results achieved by final year medical students in their clinical examination, assessed using the teleOSCE in 2020 (n=285), with those who were examined using the traditional in-person format in 2019 (n=280). The study was undertaken at the University of New South Wales, Australia.
Results
In the domain of physical examination, students in 2020 scored 0.277 points higher than those in 2019 (mean difference=–0.277, P<0.001, effect size=0.332). Across all other domains, there was no significant difference in mean scores between 2019 and 2020.
Conclusion
The teleOSCE does not negatively impact assessment in clinical examination in all domains except physical examination. If the teleOSCE is the future of clinical skills examination, assessment of physical examination will require concomitant workplace-based assessment.

Citations

Citations to this article as recorded by

Feasibility of an online clinical assessment of competence in physiotherapy students
Brooke Flew, Lucy Chipchase, Darren Lee, Jodie A. McClelland
Physiotherapy Theory and Practice.2024; : 1. CrossRef
Radiography education in 2022 and beyond - Writing the history of the present: A narrative review
Y.X. Tay, J.P. McNulty
Radiography.2023; 29(2): 391. CrossRef
Feasibility and reliability of the pandemic-adapted online-onsite hybrid graduation OSCE in Japan
Satoshi Hara, Kunio Ohta, Daisuke Aono, Toshikatsu Tamai, Makoto Kurachi, Kimikazu Sugimori, Hiroshi Mihara, Hiroshi Ichimura, Yasuhiko Yamamoto, Hideki Nomura
Advances in Health Sciences Education.2023;[Epub] CrossRef
Newly appointed medical faculty members’ self-evaluation of their educational roles at the Catholic University of Korea College of Medicine in 2020 and 2021: a cross-sectional survey-based study
Sun Kim, A Ra Cho, Chul Woon Chung
Journal of Educational Evaluation for Health Professions.2021; 18: 28. CrossRef

Review

Assessment methods and the validity and reliability of measurement tools in online objective structured clinical examinations: a systematic scoping review: Jonathan Zachary Felthun, Silas Taylor, Boaz Shulruf, Digby Wigram Allen; J Educ Eval Health Prof. 2021;18:11. Published online June 1, 2021; DOI: https://doi.org/10.3352/jeehp.2021.18.11

6,553 View
392 Download
10 Web of Science
9 Crossref

Abstract PDF Supplementary Material

The coronavirus disease 2019 (COVID-19) pandemic has required educators to adapt the in-person objective structured clinical examination (OSCE) to online settings in order for it to remain a critical component of the multifaceted assessment of a student’s competency. This systematic scoping review aimed to summarize the assessment methods and validity and reliability of the measurement tools used in current online OSCE (hereafter, referred to as teleOSCE) approaches. A comprehensive literature review was undertaken following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines. Articles were eligible if they reported any form of performance assessment, in any field of healthcare, delivered in an online format. Two reviewers independently screened the results and analyzed relevant studies. Eleven articles were included in the analysis. Pre-recorded videos were used in 3 studies, while observations by remote examiners through an online platform were used in 7 studies. Acceptability as perceived by students was reported in 2 studies. This systematic scoping review identified several insights garnered from implementing teleOSCEs, the components transferable from telemedicine, and the need for systemic research to establish the ideal teleOSCE framework. TeleOSCEs may be able to improve the accessibility and reproducibility of clinical assessments and equip students with the requisite skills to effectively practice telemedicine in the future.

Citations

Citations to this article as recorded by

The virtual Clinical Assessment of Skills and Competence: the impact and challenges of a digitised final examination
Kenny Chu, Shivanthi Sathanandan
BJPsych Bulletin.2023; 47(2): 110. CrossRef
Virtual Learning and Assessment in Rheumatology Fellowship Training: Objective Structured Clinical Examination Revisited
Rachel M. Wolfe, Faye N. Hant, Rumey C. Ishizawar, Lisa G. Criscione‐Schreiber, Beth L. Jonas, Kenneth S. O'Rourke, Marcy B. Bolster
Arthritis Care & Research.2023; 75(12): 2435. CrossRef
Feasibility and reliability of the pandemic-adapted online-onsite hybrid graduation OSCE in Japan
Satoshi Hara, Kunio Ohta, Daisuke Aono, Toshikatsu Tamai, Makoto Kurachi, Kimikazu Sugimori, Hiroshi Mihara, Hiroshi Ichimura, Yasuhiko Yamamoto, Hideki Nomura
Advances in Health Sciences Education.2023;[Epub] CrossRef
Innovations in assessment in health professions education during the COVID‐19 pandemic: A scoping review
Jamal Giri, Claire Stewart
The Clinical Teacher.2023;[Epub] CrossRef
Evaluation of the Utility of Online Objective Structured Clinical Examination Conducted During the COVID-19 Pandemic
Mona Arekat, Mohamed Hany Shehata, Abdelhalim Deifalla, Ahmed Al-Ansari, Archana Kumar, Mohamed Alsenbesy, Hamdi Alshenawi, Amgad El-Agroudy, Mariwan Husni, Diaa Rizk, Abdelaziz Elamin, Afif Ben Salah, Hani Atwa
Advances in Medical Education and Practice.2022; Volume 13: 407. CrossRef
Comparison of student pharmacists' performance on in-person vs. virtual OSCEs in a pre-APPE capstone course
Justine S. Gortney, Joseph P. Fava, Andrew D. Berti, Brittany Stewart
Currents in Pharmacy Teaching and Learning.2022; 14(9): 1116. CrossRef
Is online objective structured clinical examination teaching an acceptable replacement in post-COVID-19 medical education in the United Kingdom?: a descriptive study
Vashist Motkur, Aniket Bharadwaj, Nimalesh Yogarajah
Journal of Educational Evaluation for Health Professions.2022; 19: 30. CrossRef
Equal Z standard-setting method to estimate the minimum number of panelists for a medical school’s objective structured clinical examination in Taiwan: a simulation study
Ying-Ying Yang, Pin-Hsiang Huang, Ling-Yu Yang, Chia-Chang Huang, Chih-Wei Liu, Shiau-Shian Huang, Chen-Huan Chen, Fa-Yauh Lee, Shou-Yen Kao, Boaz Shulruf
Journal of Educational Evaluation for Health Professions.2022; 19: 27. CrossRef
Empirical analysis comparing the tele-objective structured clinical examination and the in-person assessment in Australia
Jonathan Zachary Felthun, Silas Taylor, Boaz Shulruf, Digby Wigram Allen
Journal of Educational Evaluation for Health Professions.2021; 18: 23. CrossRef

Research articles

Malaysian pharmacy students’ perspectives on the virtual objective structured clinical examination during the coronavirus disease 2019 pandemic: Mohamed Hassan Elnaem, Muhammad Eid Akkawi, Nor Ilyani Mohamed Nazar, Norny Syafinaz Ab Rahman, Mohamad Haniki Nik Mohamed; J Educ Eval Health Prof. 2021;18:6. Published online April 12, 2021; DOI: https://doi.org/10.3352/jeehp.2021.18.6

6,488 View
326 Download
12 Web of Science
13 Crossref

Abstract PDF Supplementary Material

Purpose
This study investigated pharmacy students’ perceptions of various aspects of virtual objective structured clinical examinations (vOSCEs) conducted during the coronavirus disease 2019 pandemic in Malaysia.
Methods
This cross-sectional study involved third- and fourth-year pharmacy students at the International Islamic University Malaysia. A validated self-administered questionnaire was distributed to students who had taken a vOSCE a week before.
Results
Out of the 253 students who were approached, 231 (91.3%) completed the questionnaire. More than 75% of the participants agreed that the instructions and preparations were clear and helpful in familiarizing them with the vOSCE flow. It was found that 53.2% of the respondents were satisfied with the flow and conduct of the vOSCE. However, only approximately one-third of the respondents believed that the tasks provided in the vOSCE were more convenient, less stressful, and easier to perform than those in the conventional OSCE. Furthermore, 49.7% of the students favored not having a vOSCE in the future when conducting a conventional OSCE becomes feasible again. Internet connection was reported as a problem hindering the performance of the vOSCE by 51.9% of the participants. Students who were interested in clinical pharmacy courses were more satisfied than other students with the preparation and operation of the vOSCE, the faculty support, and the allocated time.
Conclusion
Students were satisfied with the organization and operation of the vOSCE. However, they still preferred the conventional OSCE over the vOSCE. These findings might indicate a further need to expose students to telehealthcare models.

Citations

Citations to this article as recorded by

What's been trending with OSCEs in pharmacy education over the last 20 years? A bibliometric review and content analysis
Angelina S. Lim, Yeap Li Ling, Kyle J. Wilby, Vivienne Mak
Currents in Pharmacy Teaching and Learning.2024; 16(3): 212. CrossRef
Evaluation of grit and its associated factors among undergraduate pharmacy students from 14 Asian and Middle Eastern countries amid the COVID-19 pandemic
Mohamed Hassan Elnaem, Muna Barakat, Naeem Mubarak, Mohammed Salim K.T., Doaa H. Abdelaziz, Ahmed Ibrahim Fathelrahman, Abrar K. Thabit, Diana Laila Ramatillah, Ali Azeez Al-Jumaili, Nabeel Kashan Syed, Mohammed Fathelrahman Adam, Md. Sanower Hossain, Moh
Saudi Pharmaceutical Journal.2023; 31(3): 410. CrossRef
Shifting to Authentic Assessments? A Systematic Review of Student Perceptions of High-Fidelity Assessments in Pharmacy
Harjit Singh, Daniel Malone, Angelina S. Lim
American Journal of Pharmaceutical Education.2023; 87(7): 100099. CrossRef
A Telehealth Module and Virtual Objective Structured Clinical Examination of Health Literacy in Pharmacy Education
Sanah Hasan, Hamzah AlZubaidi, Subish Palaian, Muaed AlOmar, Nadir Kheir, Yassin Al Hariri, Sawsan Shanableh, Ahmed Gaili, Abby Kahaleh
American Journal of Pharmaceutical Education.2023; 87(12): 100555. CrossRef
Is It Time to Revise the Competency-Based Assessment? Objective Structured Clinical Examination and Technology Integration
Haniye Mastour, Nazanin Shamaeian Razavi
Shiraz E-Medical Journal.2023;[Epub] CrossRef
Virtual OSCE: Experience and challenges with a large cohort of pharmacy students
Hanis Hanum Zulkifly, Izzati Abdul Halim Zaki, Mahmathi Karuppannan, Zakiah Mohd Noordin
Pharmacy Education.2022; 22(1): 23. CrossRef
Students’ and Examiners’ Experiences of Their First Virtual Pharmacy Objective Structured Clinical Examination (OSCE) in Australia during the COVID-19 Pandemic
Vivienne Mak, Sunanthiny Krishnan, Sara Chuang
Healthcare.2022; 10(2): 328. CrossRef
Perceptions of Pharmacy Students on the E-Learning Strategies Adopted during the COVID-19 Pandemic: A Systematic Review
Carla Pires
Pharmacy.2022; 10(1): 31. CrossRef
Perceptions of undergraduate pharmacy students towards online assessments used during the COVID-19 pandemic in a public university in Malaysia
Usman Abubakar, A'isyah Humaira' Mohd Salehudin, Nik Afiqah Athirah Nik Mohd Asri, Nur Atiqah Mohammad Rohi, Nur Hasyimah Ramli, Nur Izzah Mohd Khairuddin, Nur Fariesya Saiful Izham, Siti Hajar Nasrullah, Auwal Adam Sa’ad
Pharmacy Education.2022; 22(1): 191. CrossRef
Evaluation of the Utility of Online Objective Structured Clinical Examination Conducted During the COVID-19 Pandemic
Mona Arekat, Mohamed Hany Shehata, Abdelhalim Deifalla, Ahmed Al-Ansari, Archana Kumar, Mohamed Alsenbesy, Hamdi Alshenawi, Amgad El-Agroudy, Mariwan Husni, Diaa Rizk, Abdelaziz Elamin, Afif Ben Salah, Hani Atwa
Advances in Medical Education and Practice.2022; Volume 13: 407. CrossRef
COVID-19-Driven Improvements and Innovations in Pharmacy Education: A Scoping Review
Jennifer Courtney, Erika Titus-Lay, Ashim Malhotra, Jeffrey Nehira, Islam Mohamed, Welly Mente, Uyen Le, Linda Buckley, Xiaodong Feng, Ruth Vinall
Pharmacy.2022; 10(3): 60. CrossRef
Supporting pharmacy students' preparation for an entry-to-practice OSCE using video cases
Michelle Flood, Judith Strawbridge, Eimear Ní Sheachnasaigh, Theo Ryan, Laura J. Sahm, Aoife Fleming, James W. Barlow
Currents in Pharmacy Teaching and Learning.2022; 14(12): 1525. CrossRef
Empirical analysis comparing the tele-objective structured clinical examination and the in-person assessment in Australia
Jonathan Zachary Felthun, Silas Taylor, Boaz Shulruf, Digby Wigram Allen
Journal of Educational Evaluation for Health Professions.2021; 18: 23. CrossRef

Increased competency of registered dietitian nutritionists in physical examination skills after simulation-based education in the United States: Elizabeth MacQuillan, Jennifer Ford, Kristin Baird; J Educ Eval Health Prof. 2020;17:40. Published online December 14, 2020; DOI: https://doi.org/10.3352/jeehp.2020.17.40

4,460 View
144 Download
1 Web of Science
1 Crossref

Abstract PDF Supplementary Material

Purpose
This study aimed to translate simulation-based dietitian nutritionist education to clinical competency attainment in a group of practicing registered dietitian nutritionists (RDNs). Using a standardized instrument to measure performance on a newly-required clinical skill, the nutrition-focused physical exam (NFPE), competence was measured both before and after a simulation-based education (SBE) session.
Methods
Eighteen practicing RDNs were recruited by their employer, Spectrum Health. Following a pre-briefing session, participants completed an initial 10-minute encounter, performing NFPE on a standardized patient (SP). Next, participants completed a 90-minute SBE training session on skills within the NFPE, including hands-on practice and role play, followed by a post-training SP encounter. Video recordings of the SP encounters were scored to assess competence in 7 skill areas within the NFPE. Scores were analyzed for participants’ initial competence and change in competence.
Results
The proportions of participants with initial competence ranged from 0% to 44% across the 7 skill areas assessed. The only competency where participants initially scored in the “meets expectations” range was “approach to the patient.” When raw competence scores were assessed for changes from pre- to post-SBE training, the paired t-test indicated significant increases in all 7 competency areas following the simulation-based training (P<0.001).
Conclusion
This study showed the effectiveness of a SBE training program for increasing competence scores of practicing RDNs on a defined clinical skill.

Citations

Citations to this article as recorded by

Barriers for Liver Transplant in Patients with Alcohol-Related Hepatitis
Gina Choi, Jihane N. Benhammou, Jung J. Yum, Elena G. Saab, Ankur P. Patel, Andrew J. Baird, Stephanie Aguirre, Douglas G. Farmer, Sammy Saab
Journal of Clinical and Experimental Hepatology.2022; 12(1): 13. CrossRef

Female medical and nursing students’ knowledge, attitudes, and skills regarding breast self-examination in Oman: a comparison between pre- and post-training: Rajani Ranganath, John Muthusami, Miriam Simon, Tatiyana Mandal, Meena Anand Kukkamulla; J Educ Eval Health Prof. 2020;17:37. Published online December 1, 2020; DOI: https://doi.org/10.3352/jeehp.2020.17.37

5,599 View
262 Download
1 Web of Science
2 Crossref

Abstract PDF Supplementary Material

Purpose
Breast cancer is one of the most common cancers in women worldwide. Educational and awareness programs impact early practices of breast self-examination, resulting in the early detection of cancer and thereby decreasing mortality. The study aimed to assess the levels of knowledge and awareness of breast cancer and breast self-examination among medical and nursing students in Oman and to compare their knowledge, attitudes, and skills after a training program.
Methods
This quasi-experimental study was carried out for female 90 medical and 80 nursing students in Oman in November 2019. A pre-test questionnaire was given before the training program and a post-test questionnaire was administered after the training program. Students’ knowledge, attitude, and skills regarding breast cancer and breast self-examination were compared. Scores for skills of practicing breast self-examination were compared between lecture and activity group and lecture-only group.
Results
Pre-test and post-test data were collected from 170 female students. Significant improvements were observed in the post-test scores for students’ knowledge, attitude, and skills after the intervention (P<0.001). The mean scores for skills of practicing breast self-examination after the lecture and the activity were higher than those obtained after the lecture only (P=0.014 for medical students and P=0.016 for nursing students).
Conclusion
An educational training program on breast cancer and breast examination with an emphasis on skills can motivate participants to perform breast self-examination regularly, and may therefore help students to train other women to perform breast self-examination for the early detection of breast cancer.

Citations

Citations to this article as recorded by

Nursing Students’ Motivation, Awareness, and Knowledge of Women’s Health: A Norwegian Quasi-Experimental Study
Christine Tørris
Education Sciences.2024; 14(3): 273. CrossRef
Kavram Haritası ile Verilen Kendi Kendine Meme Muayenesi Eğitiminin Hemşirelik Öğrencilerinin Sağlık İnançları ve Öz Yeterlilik Düzeylerine Etkisi
Aysun ACUN, Yadigar ORDU
Black Sea Journal of Health Science.2023; 6(4): 632. CrossRef

Similarity of the cut score in test sets with different item amounts using the modified Angoff, modified Ebel, and Hofstee standard-setting methods for the Korean Medical Licensing Examination: Janghee Park, Mi Kyoung Yim, Na Jin Kim, Duck Sun Ahn, Young-Min Kim; J Educ Eval Health Prof. 2020;17:28. Published online October 5, 2020; DOI: https://doi.org/10.3352/jeehp.2020.17.28

6,175 View
188 Download
7 Web of Science
6 Crossref

Abstract PDF Supplementary Material

Purpose
The Korea Medical Licensing Exam (KMLE) typically contains a large number of items. The purpose of this study was to investigate whether there is a difference in the cut score between evaluating all items of the exam and evaluating only some items when conducting standard-setting.
Methods
We divided the item sets that appeared on 3 recent KMLEs for the past 3 years into 4 subsets of each year of 25% each based on their item content categories, discrimination index, and difficulty index. The entire panel of 15 members assessed all the items (360 items, 100%) of the year 2017. In split-half set 1, each item set contained 184 (51%) items of year 2018 and each set from split-half set 2 contained 182 (51%) items of the year 2019 using the same method. We used the modified Angoff, modified Ebel, and Hofstee methods in the standard-setting process.
Results
Less than a 1% cut score difference was observed when the same method was used to stratify item subsets containing 25%, 51%, or 100% of the entire set. When rating fewer items, higher rater reliability was observed.
Conclusion
When the entire item set was divided into equivalent subsets, assessing the exam using a portion of the item set (90 out of 360 items) yielded similar cut scores to those derived using the entire item set. There was a higher correlation between panelists’ individual assessments and the overall assessments.

Citations

Citations to this article as recorded by

Application of computer-based testing in the Korean Medical Licensing Examination, the emergence of the metaverse in medical education, journal metrics and statistics, and appreciation to reviewers and volunteers
Sun Huh
Journal of Educational Evaluation for Health Professions.2022; 19: 2. CrossRef
Possibility of using the yes/no Angoff method as a substitute for the percent Angoff method for estimating the cutoff score of the Korean Medical Licensing Examination: a simulation study
Janghee Park
Journal of Educational Evaluation for Health Professions.2022; 19: 23. CrossRef
Equal Z standard-setting method to estimate the minimum number of panelists for a medical school’s objective structured clinical examination in Taiwan: a simulation study
Ying-Ying Yang, Pin-Hsiang Huang, Ling-Yu Yang, Chia-Chang Huang, Chih-Wei Liu, Shiau-Shian Huang, Chen-Huan Chen, Fa-Yauh Lee, Shou-Yen Kao, Boaz Shulruf
Journal of Educational Evaluation for Health Professions.2022; 19: 27. CrossRef
Possibility of independent use of the yes/no Angoff and Hofstee methods for the standard setting of the Korean Medical Licensing Examination written test: a descriptive study
Do-Hwan Kim, Ye Ji Kang, Hoon-Ki Park
Journal of Educational Evaluation for Health Professions.2022; 19: 33. CrossRef
Presidential address: Quarantine guidelines to protect examinees from coronavirus disease 2019, clinical skills examination for dental licensing, and computer-based testing for medical, dental, and oriental medicine licensing
Yoon-Seong Lee
Journal of Educational Evaluation for Health Professions.2021; 18: 1. CrossRef
Comparing the cut score for the borderline group method and borderline regression method with norm-referenced standard setting in an objective structured clinical examination in medical school in Korea
Song Yi Park, Sang-Hwa Lee, Min-Jeong Kim, Ki-Hwan Ji, Ji Ho Ryu
Journal of Educational Evaluation for Health Professions.2021; 18: 25. CrossRef

Correlation between physician assistant students’ performance score of history taking and physical exam documentation and scores of Graduate Record Examination, clinical year grade point average, and score of Physician Assistant National Certifying Exam in the United States: Sara Lolar, Jamie McQueen, Sara Maher; J Educ Eval Health Prof. 2020;17:16. Published online May 27, 2020; DOI: https://doi.org/10.3352/jeehp.2020.17.16

6,385 View
151 Download
1 Web of Science
2 Crossref

Abstract PDF Supplementary Material

Purpose
Learning to perform and document patient history taking and physical exam (H&P) entails a major component of the first year academic education of physician assistant (PA) students at Wayne State University, USA. The H&P is summative of multiple aspects of PA education, and students must master communication with patients and other health care providers. The objectives of this study were first, to determine if there was a correlation between scores on the Graduate Record Examination (GRE) component testing and scores on graded H&Ps. The second objective was to identify a correlation between proficiency with H&P documentation and academic and clinical year grade point average (GPA) and Physician Assistant National Certifying Exam (PANCE) score.
Methods
Subjects included 147 PA students from Wayne State University from 2014–2016. PA students visited local hospitals or outpatient clinics during the academic year to perform and document patient H&Ps. Correlation between the H&P mean scores and GRE component scores, GPAs, and PANCE scores were analyzed.
Results
The subjects were 26.5 years-old (+6.5) and 111 females (75.5%). There was no correlation between the GRE component score and the H&P mean score. The H&P score was positively correlated with GPA 1 (r=0.512, P<0.001), with GPA 2 (r=0.425, P<0.001) and with PANCE score (r=0.448, P<0.001).
Conclusion
PA student skill with H&P documentation was positively related to academic performance score during PA school and achievement score on the PANCE at Wayne State University, USA.

Citations

Citations to this article as recorded by

History-taking level and its influencing factors among nursing undergraduates based on the virtual standardized patient testing results: Cross sectional study
Jingrong Du, Xiaowen Zhu, Juan Wang, Jing Zheng, Xiaomin Zhang, Ziwen Wang, Kun Li
Nurse Education Today.2022; 111: 105312. CrossRef
A Decline in Black and Dermatology Physician Assistants
Jameka McElroy-Brooklyn, Cynthia Faires Griffith
Journal of Physician Assistant Education.2022; 33(4): 275. CrossRef

Brief Report

Potential of feedback during objective structured clinical examination to evoke an emotional response in medical students in Canada: Dalia Limor Karol, Debra Pugh; J Educ Eval Health Prof. 2020;17:5. Published online February 18, 2020; DOI: https://doi.org/10.3352/jeehp.2020.17.5

6,549 View
164 Download
3 Web of Science
2 Crossref

Abstract PDF Supplementary Material

Feedback has been shown to be an important driver for learning. However, many factors, such as the emotional reactions feedback evokes, may impact its effect. This study aimed to explore medical students’ perspectives on the verbal feedback they receive during an objective structured clinical examination (OSCE); their emotional reaction to this; and its impact on their subsequent performance. To do this, medical students enrolled at 4 Canadian medical schools were invited to complete a web-based survey regarding their experiences. One hundred and fifty-eight participants completed the survey. Twenty-nine percent of respondents asserted that they had experienced emotional reactions to verbal feedback received in an OSCE setting. The most common emotional responses reported were embarrassment and anxiousness. Some students (n=20) reported that the feedback they received negatively impacted subsequent OSCE performance. This study demonstrates that feedback provided during an OSCE can evoke an emotional response in students and potentially impact subsequent performance.

Citations

Citations to this article as recorded by

Memory, credibility and insight: How video-based feedback promotes deeper reflection and learning in objective structured clinical exams
Alexandra Makrides, Peter Yeates
Medical Teacher.2022; 44(6): 664. CrossRef
Objective structured clinical examination in fundamentals of nursing and obstetric care as method of verification and assessing the degree of achievement of learning outcomes
Lucyna Sochocka, Teresa Niechwiadowicz-Czapka, Mariola Wojtal, Monika Przestrzelska, Iwona Kiersnowska, Katarzyna Szwamel
Pielegniarstwo XXI wieku / Nursing in the 21st Century.2021; 20(3): 190. CrossRef

Research articles

Dreyfus scale-based feedback increased medical students’ satisfaction with the complex cluster part of a interviewing and physical examination course and improved skills readiness in Taiwan: Shiau-Shian Huang, Chia-Chang Huang, Ying-Ying Yang, Shuu-Jiun Wang, Boaz Shulruf, Chen-Huan Chen; J Educ Eval Health Prof. 2019;16:30. Published online October 11, 2019; DOI: https://doi.org/10.3352/jeehp.2019.16.30

10,259 View
126 Download
1 Web of Science

Abstract PDF Supplementary Material: Purpose
In contrast to the core part of the clinical interviewing and physical examination (PE) skills course, corresponding to the basic, head-to-toe, and thoracic systems, learners need structured feedback in the cluster part of the course, which includes the abdominal, neuromuscular, and musculoskeletal systems. This study evaluated the effects of using Dreyfus scale-based feedback, which has elements of continuous professional development, instead of Likert scale-based feedback in the cluster part of training in Taiwan.
Methods
Instructors and final-year medical students in the 2015–2016 classes of National Yang-Ming University, Taiwan comprised the regular cohort, whereas those in the 2017–2018 classes formed the intervention cohort. In the intervention cohort, Dreyfus scale-based feedback, rather than Likert scale-based feedback, was used in the cluster part of the course.
Results
In the cluster part of the course in the regular cohort, pre-trained standardized patients rated the class climate as poor, and students expressed low satisfaction with the instructors and course and low self-assessed readiness. In comparison with the regular cohort, improved end-of-course group objective structured clinical examination scores after the cluster part were noted in the intervention cohort. In other words, the implementation of Dreyfus scale-based feedback in the intervention cohort for the cluster part improved the deficit in this section of the course.
Conclusion
The implementation of Dreyfus scale-based feedback helped instructors to create a good class climate in the cluster part of the clinical interviewing and PE skills course. Simultaneously, this new intervention achieved the goal of promoting medical students’ readiness for interviewing, PE, and self-directed learning.

Mismatch between the proposed ability concepts of the Graduate Record Examination and the critical thinking skills of physical therapy applicants suggested by an expert panel in the United States: Emily Shannon Hughes; J Educ Eval Health Prof. 2019;16:24. Published online August 27, 2019; DOI: https://doi.org/10.3352/jeehp.2019.16.24

10,736 View
183 Download
1 Web of Science
1 Crossref

Abstract PDF Supplementary Material

Purpose
The Graduate Record Examination (GRE) is a general examination predictive of success in US-based graduate programs. Used to assess students’ written, mathematical, and critical thinking (CT) skills, the GRE is utilized for admission to approximately 85% of US physical therapist education (PTE) programs. The purpose of this study was to assess whether the CT skills measured by the GRE match those deemed by an expert panel as the most important to assess for PTE program acceptance.
Methods
Using a modified E-Delphi approach, a 3-phase survey was distributed over 8 weeks to a panel consisting of licensed US physical therapists with expertise on CT and PTE program directors. The CT skills isolated by the expert panel, based on Facione’s Delphi report, were compared to the CT skills assessed by the GRE.
Results
The CT skills supported by the Delphi report and chosen by the expert panel for assessment prior to acceptance into US PTE programs included clarifying meaning, categorization, and analyzing arguments. Only clarifying meaning matched the CT skills from the GRE.
Conclusion
The GRE is a test for general admission to graduate programs, lacking context related to healthcare or physical therapy. The current study fails to support the GRE as an assessment tool of CT for admission to PTE programs. A context-based admission test evaluating the CT skills identified in this study should be developed for use in the admission process to predict which students will complete US PTE programs and pass the licensure exam.

Citations

Citations to this article as recorded by

Correlation between physician assistant students’ performance score of history taking and physical exam documentation and scores of Graduate Record Examination, clinical year grade point average, and score of Physician Assistant National Certifying Exam i
Sara Lolar, Jamie McQueen, Sara Maher
Journal of Educational Evaluation for Health Professions.2020; 17: 16. CrossRef

Brief report

No observed effect of a student-led mock objective structured clinical examination on subsequent performance scores in medical students in Canada: Lorenzo Madrazo, Claire Bo Lee, Meghan McConnell, Karima Khamisa, Debra Pugh; J Educ Eval Health Prof. 2019;16:14. Published online May 27, 2019; DOI: https://doi.org/10.3352/jeehp.2019.16.14

14,116 View
203 Download
9 Web of Science
8 Crossref

Abstract PDF Supplementary Material

Student-led peer-assisted mock objective structured clinical examinations (MOSCEs) have been used in various settings to help students prepare for subsequent higher-stakes, faculty-run OSCEs. MOSCE participants generally valued feedback from peers and reported benefits to learning. Our study investigated whether participation in a peer-assisted MOSCE affected subsequent OSCE performance. To determine whether mean OSCE scores differed depending on whether medical students participated in the MOSCE, we conducted a between-subjects analysis of variance, with cohort (2016 vs. 2017) and MOSCE participation (MOSCE vs. no MOSCE) as independent variables and the mean OSCE score as the dependent variable. Participation in the MOSCE had no influence on mean OSCE scores (P=0.19). There was a significant correlation between mean MOSCE scores and mean OSCE scores (Pearson r=0.52, P<0.001). Although previous studies described self-reported benefits from participation in student-led MOSCEs, it was not associated with objective benefits in this study.

Citations

Citations to this article as recorded by

Impact of familiarity with the format of the exam on performance in the OSCE of undergraduate medical students – an interventional study
Hannes Neuwirt, Iris E. Eder, Philipp Gauckler, Lena Horvath, Stefan Koeck, Maria Noflatscher, Benedikt Schaefer, Anja Simeon, Verena Petzer, Wolfgang M. Prodinger, Christoph Berendonk
BMC Medical Education.2024;[Epub] CrossRef
Perceived and actual value of Student‐led Objective Structured Clinical Examinations
Brandon Stretton, Adam Montagu, Aline Kunnel, Jenni Louise, Nathan Behrendt, Joshua Kovoor, Stephen Bacchi, Josephine Thomas, Ellen Davies
The Clinical Teacher.2024;[Epub] CrossRef
Benefits of semiology taught using near-peer tutoring are sustainable
Benjamin Gripay, Thomas André, Marie De Laval, Brice Peneau, Alexandre Secourgeon, Nicolas Lerolle, Cédric Annweiler, Grégoire Justeau, Laurent Connan, Ludovic Martin, Loïc Bière
BMC Medical Education.2022;[Epub] CrossRef
Identification des facteurs associés à la réussite aux examens cliniques objectifs et structurés dans la faculté de médecine de Rouen
M. Leclercq, M. Vannier, Y. Benhamou, A. Liard, V. Gilard, I. Auquit-Auckbur, H. Levesque, L. Sibert, P. Schneider
La Revue de Médecine Interne.2022; 43(5): 278. CrossRef
Evaluation of the Experience of Peer-led Mock Objective Structured Practical Examination for First- and Second-year Medical Students
Faisal Alsaif, Lamia Alkuwaiz, Mohammed Alhumud, Reem Idris, Lina Neel, Mansour Aljabry, Mona Soliman
Advances in Medical Education and Practice.2022; Volume 13: 987. CrossRef
The use of a formative OSCE to prepare emergency medicine residents for summative OSCEs: a mixed-methods cohort study
Magdalene Hui Min Lee, Dong Haur Phua, Kenneth Wei Jian Heng
International Journal of Emergency Medicine.2021;[Epub] CrossRef
Tutor–Student Partnership in Practice OSCE to Enhance Medical Education
Eve Cosker, Valentin Favier, Patrice Gallet, Francis Raphael, Emmanuelle Moussier, Louise Tyvaert, Marc Braun, Eva Feigerlova
Medical Science Educator.2021; 31(6): 1803. CrossRef
Peers as OSCE assessors for junior medical students – a review of routine use: a mixed methods study
Simon Schwill, Johanna Fahrbach-Veeser, Andreas Moeltner, Christiane Eicher, Sonia Kurczyk, David Pfisterer, Joachim Szecsenyi, Svetla Loukanova
BMC Medical Education.2020;[Epub] CrossRef

Research article

Medical students’ thought process while solving problems in 3 different types of clinical assessments in Korea: clinical performance examination, multimedia case-based assessment, and modified essay question: Sejin Kim, Ikseon Choi, Bo Young Yoon, Min Jeong Kwon, Seok-jin Choi, Sang Hyun Kim, Jong-Tae Lee, Byoung Doo Rhee; J Educ Eval Health Prof. 2019;16:10. Published online May 9, 2019; DOI: https://doi.org/10.3352/jeehp.2019.16.10

16,393 View
271 Download
3 Crossref

Abstract PDF Supplementary Material

Purpose
This study aimed to explore students’ cognitive patterns while solving clinical problems in 3 different types of assessments—clinical performance examination (CPX), multimedia case-based assessment (CBA), and modified essay question (MEQ)—and thereby to understand how different types of assessments stimulate different patterns of thinking.
Methods
A total of 6 test-performance cases from 2 fourth-year medical students were used in this cross-case study. Data were collected through one-on-one interviews using a stimulated recall protocol where students were shown videos of themselves taking each assessment and asked to elaborate on what they were thinking. The unit of analysis was the smallest phrases or sentences in the participants’ narratives that represented meaningful cognitive occurrences. The narrative data were reorganized chronologically and then analyzed according to the hypothetico-deductive reasoning framework for clinical reasoning.
Results
Both participants demonstrated similar proportional frequencies of clinical reasoning patterns on the same clinical assessments. The results also revealed that the three different assessment types may stimulate different patterns of clinical reasoning. For example, the CPX strongly promoted the participants’ reasoning related to inquiry strategy, while the MEQ strongly promoted hypothesis generation. Similarly, data analysis and synthesis by the participants were more strongly stimulated by the CBA than by the other assessment types.
Conclusion
This study found that different assessment designs stimulated different patterns of thinking during problem-solving. This finding can contribute to the search for ways to improve current clinical assessments. Importantly, the research method used in this study can be utilized as an alternative way to examine the validity of clinical assessments.

Citations

Citations to this article as recorded by

Future directions of online learning environment design at medical schools: a transition towards a post-pandemic context
Sejin Kim
Kosin Medical Journal.2023; 38(1): 12. CrossRef
Clinical Reasoning Training based on the analysis of clinical case using a virtual environment
Sandra Elena Lisperguer Soto, María Soledad Calvo, Gabriela Paz Urrejola Contreras, Miguel Ángel Pérez Lizama
Educación Médica.2021; 22(3): 139. CrossRef
Newly appointed medical faculty members’ self-evaluation of their educational roles at the Catholic University of Korea College of Medicine in 2020 and 2021: a cross-sectional survey-based study
Sun Kim, A Ra Cho, Chul Woon Chung
Journal of Educational Evaluation for Health Professions.2021; 18: 28. CrossRef

First
Prev
Page of 3
Next
Last

Search