Journal of Educational Evaluation for Health Professions

Research articles

Use of learner-driven, formative, ad-hoc, prospective assessment of competence in physical therapist clinical education in the United States: a prospective cohort study: Carey Holleran, Jeffrey Konrad, Barbara Norton, Tamara Burlis, Steven Ambler; J Educ Eval Health Prof. 2023;20:36. Published online December 8, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.36; Funded: Washington University School of Medicine in St. Louis

681 View
109 Download

Abstract PDF Supplementary Material: Purpose
The purpose of this project was to implement a process for learner-driven, formative, prospective, ad-hoc, entrustment assessment in Doctor of Physical Therapy clinical education. Our goals were to develop an innovative entrustment assessment tool, and then explore whether the tool detected (1) differences between learners at different stages of development and (2) differences within learners across the course of a clinical education experience. We also investigated whether there was a relationship between the number of assessments and change in performance.
Methods
A prospective, observational, cohort of clinical instructors (CIs) was recruited to perform learner-driven, formative, ad-hoc, prospective, entrustment assessments. Two entrustable professional activities (EPAs) were used: (1) gather a history and perform an examination and (2) implement and modify the plan of care, as needed. CIs provided a rating on the entrustment scale and provided narrative support for their rating.
Results
Forty-nine learners participated across 4 clinical experiences (CEs), resulting in 453 EPA learner-driven assessments. For both EPAs, statistically significant changes were detected both between learners at different stages of development and within learners across the course of a CE. Improvement within each CE was significantly related to the number of feedback opportunities.
Conclusion
The results of this pilot study provide preliminary support for the use of learner-driven, formative, ad-hoc assessments of competence based on EPAs with a novel entrustment scale. The number of formative assessments requested correlated with change on the EPA scale, suggesting that formative feedback may augment performance improvement.

Negative effects on medical students’ scores for clinical performance during the COVID-19 pandemic in Taiwan: a comparative study: Eunice Jia-Shiow Yuan, Shiau-Shian Huang, Chia-An Hsu, Jiing-Feng Lirng, Tzu-Hao Li, Chia-Chang Huang, Ying-Ying Yang, Chung-Pin Li, Chen-Huan Chen; J Educ Eval Health Prof. 2023;20:37. Published online December 26, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.37; Funded: Taipei Veterans General Hospital, Ministry of Education, National Science and Technology Council

727 View
70 Download

Abstract PDF Supplementary Material: Purpose
Coronavirus disease 2019 (COVID-19) has heavily impacted medical clinical education in Taiwan. Medical curricula have been altered to minimize exposure and limit transmission. This study investigated the effect of COVID-19 on Taiwanese medical students’ clinical performance using online standardized evaluation systems and explored the factors influencing medical education during the pandemic.
Methods
Medical students were scored from 0 to 100 based on their clinical performance from 1/1/2018 to 6/31/2021. The students were placed into pre-COVID-19 (before 2/1/2020) and midst-COVID-19 (on and after 2/1/2020) groups. Each group was further categorized into COVID-19-affected specialties (pulmonary, infectious, and emergency medicine) and other specialties. Generalized estimating equations (GEEs) were used to compare and examine the effects of relevant variables on student performance.
Results
In total, 16,944 clinical scores were obtained for COVID-19-affected specialties and other specialties. For the COVID-19-affected specialties, the midst-COVID-19 score (88.513.52) was significantly lower than the pre-COVID-19 score (90.143.55) (P<0.0001). For the other specialties, the midst-COVID-19 score (88.323.68) was also significantly lower than the pre-COVID-19 score (90.063.58) (P<0.0001). There were 1,322 students (837 males and 485 females). Male students had significantly lower scores than female students (89.333.68 vs. 89.993.66, P=0.0017). GEE analysis revealed that the COVID-19 pandemic (unstandardized beta coefficient=-1.99, standard error [SE]=0.13, P<0.0001), COVID-19-affected specialties (B=0.26, SE=0.11, P=0.0184), female students (B=1.10, SE=0.20, P<0.0001), and female attending physicians (B=-0.19, SE=0.08, P=0.0145) were independently associated with students’ scores.
Conclusion
COVID-19 negatively impacted medical students' clinical performance, regardless of their specialty. Female students outperformed male students, irrespective of the pandemic.

Review

Application of artificial intelligence chatbots, including ChatGPT, in education, scholarly work, programming, and content generation and its prospects: a narrative review: Tae Won Kim; J Educ Eval Health Prof. 2023;20:38. Published online December 27, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.38; Funded: Korea Communications Development Fund

1,534 View
289 Download
1 Web of Science
2 Crossref

Abstract PDF Supplementary Material

This study aims to explore ChatGPT’s (GPT-3.5 version) functionalities, including reinforcement learning, diverse applications, and limitations. ChatGPT is an artificial intelligence (AI) chatbot powered by OpenAI’s Generative Pre-trained Transformer (GPT) model. The chatbot’s applications span education, programming, content generation, and more, demonstrating its versatility. ChatGPT can improve education by creating assignments and offering personalized feedback, as shown by its notable performance in medical exams and the United States Medical Licensing Exam. However, concerns include plagiarism, reliability, and educational disparities. It aids in various research tasks, from design to writing, and has shown proficiency in summarizing and suggesting titles. Its use in scientific writing and language translation is promising, but professional oversight is needed for accuracy and originality. It assists in programming tasks like writing code, debugging, and guiding installation and updates. It offers diverse applications, from cheering up individuals to generating creative content like essays, news articles, and business plans. Unlike search engines, ChatGPT provides interactive, generative responses and understands context, making it more akin to human conversation, in contrast to conventional search engines’ keyword-based, non-interactive nature. ChatGPT has limitations, such as potential bias, dependence on outdated data, and revenue generation challenges. Nonetheless, ChatGPT is considered to be a transformative AI tool poised to redefine the future of generative technology. In conclusion, advancements in AI, such as ChatGPT, are altering how knowledge is acquired and applied, marking a shift from search engines to creativity engines. This transformation highlights the increasing importance of AI literacy and the ability to effectively utilize AI in various domains of life.

Citations

Citations to this article as recorded by

Opportunities, challenges, and future directions of large language models, including ChatGPT in medical education: a systematic scoping review
Xiaojun Xu, Yixiao Chen, Jing Miao
Journal of Educational Evaluation for Health Professions.2024; 21: 6. CrossRef
Artificial Intelligence: Fundamentals and Breakthrough Applications in Epilepsy
Wesley Kerr, Sandra Acosta, Patrick Kwan, Gregory Worrell, Mohamad A. Mikati
Epilepsy Currents.2024;[Epub] CrossRef

Research articles

Information amount, accuracy, and relevance of generative artificial intelligence platforms’ answers regarding learning objectives of medical arthropodology evaluated in English and Korean queries in December 2023: a descriptive study: Hyunju Lee, Soobin Park; J Educ Eval Health Prof. 2023;20:39. Published online December 28, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.39; Funded: College of Medicine, Hallym University

1,039 View
138 Download
1 Crossref

Abstract PDF Supplementary Material

Purpose
This study assessed the performance of 6 generative artificial intelligence (AI) platforms on the learning objectives of medical arthropodology in a parasitology class in Korea. We examined the AI platforms’ performance by querying in Korean and English to determine their information amount, accuracy, and relevance in prompts in both languages.
Methods
From December 15 to 17, 2023, 6 generative AI platforms—Bard, Bing, Claude, Clova X, GPT-4, and Wrtn—were tested on 7 medical arthropodology learning objectives in English and Korean. Clova X and Wrtn are platforms from Korean companies. Responses were evaluated using specific criteria for the English and Korean queries.
Results
Bard had abundant information but was fourth in accuracy and relevance. GPT-4, with high information content, ranked first in accuracy and relevance. Clova X was 4th in amount but 2nd in accuracy and relevance. Bing provided less information, with moderate accuracy and relevance. Wrtn’s answers were short, with average accuracy and relevance. Claude AI had reasonable information, but lower accuracy and relevance. The responses in English were superior in all aspects. Clova X was notably optimized for Korean, leading in relevance.
Conclusion
In a study of 6 generative AI platforms applied to medical arthropodology, GPT-4 excelled overall, while Clova X, a Korea-based AI product, achieved 100% relevance in Korean queries, the highest among its peers. Utilizing these AI platforms in classrooms improved the authors’ self-efficacy and interest in the subject, offering a positive experience of interacting with generative AI platforms to question and receive information.

Citations

Citations to this article as recorded by

Opportunities, challenges, and future directions of large language models, including ChatGPT in medical education: a systematic scoping review
Xiaojun Xu, Yixiao Chen, Jing Miao
Journal of Educational Evaluation for Health Professions.2024; 21: 6. CrossRef

Obstetrics and gynecology residents’ satisfaction and self-confidence after an anal sphincter injury simulation-based workshop in Indonesia: a pre- and post-intervention comparison study: Riska Wahyuningtyas, Eighty Mardiyan Kurniawati, Budi Utomo, Gatut Hardianto, Hari Paraton, Tri Hastono, Djoko Kuswanto; J Educ Eval Health Prof. 2022;19:4. Published online February 14, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.4; Funded: Universitas Airlangga

4,738 View
443 Download
1 Web of Science
1 Crossref

Abstract PDF Supplementary Material

Purpose
Obstetric anal sphincter injury is one of the most common complications during delivery. Simulation models with manikins can be used as an effective medical learning method to improve students’ abilities before encountering patients. The present study aimed to describe the development of an anal sphincter injury model and to assess residents’ satisfaction and self-confidence after a perineal repair workshop with an anal sphincter injury simulator in Indonesia.
Methods
This was a cross-sectional study with evaluation of outcomes before and after the workshop. We created a silicone-latex simulation anal sphincter injury model. Then, we validated this simulation and used it as a simulation model for the workshop. We asked residents about their satisfaction with repairing anal sphincter injuries using a simulation model and residents’ self-confidence when practicing anal sphincter injury repair.
Results
All residents felt the simulation-based workshop was valuable (100%). Most of the scores for the similarity of the simulation model were good (about 8 out of maximum 10). The self-assessment of confidence was measured before and after the workshop. Overall self-confidence increased significantly after the workshop in identifying the external sphincter ani (EAS) (P=0.031), suturing the anal mucosa (P=0.001), suturing the internal sphincter ani (P=0.001), suturing the EAS (P<0.001), and evaluating the sphincter ani tone (P=0.016).
Conclusion
The anal sphincter injury simulator improved residents’ self-confidence in identifying the EAS, suturing the anal mucosa, suturing the internal sphincter ani, suturing the EAS, and evaluating sphincter ani tone.

Citations

Citations to this article as recorded by

Objective structured assessment of medical students’ technical skills in second-degree perineal laceration repair with sponge model-based training
Gregor Leonhard Olmes, Merle Doerk, Erich-Franz Solomayer, Meletios P. Nigdelis, Romina-Marina Sima, Bashar Haj Hamoud
Archives of Gynecology and Obstetrics.2023;[Epub] CrossRef

Medical residents and attending physicians’ perceptions of feedback and teaching in the United States: a qualitative study: Madeleine Matthiesen, Michael S. Kelly, Kristina Dzara, Arabella Simpkin Begin; J Educ Eval Health Prof. 2022;19:9. Published online April 26, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.9; Funded: Massachusetts General Hospital Center, National Institutes of Health

8,253 View
355 Download
1 Web of Science
1 Crossref

Abstract PDF Supplementary Material

Purpose
Residents and attendings agree on the importance of feedback to resident education. However, while faculty report providing frequent feedback, residents often do not perceive receiving it, particularly in the context of teaching. Given the nuanced differences between feedback and teaching, we aimed to explore resident and attending perceptions of feedback and teaching in the clinical setting.
Methods
We conducted a qualitative study of internal medicine residents and attendings from December 2018 through March 2019 at the Massachusetts General Hospital to investigate perceptions of feedback in the inpatient clinical setting. Residents and faculty were recruited to participate in focus groups. Data were analyzed using thematic analysis to explore perspectives and barriers to feedback provision and identification.
Results
Five focus groups included 33 total participants in 3 attending (n=20) and 2 resident (n=13) groups. Thematic analysis of focus group transcripts identified 7 themes which organized into 3 thematic categories: (1) disentangling feedback and teaching, (2) delivering high-quality feedback, and (3) experiencing feedback in the group setting. Residents and attendings highlighted important themes in discriminating feedback from teaching. They indicated that while feedback is reactive in response to an action or behavior, teaching is proactive and oriented toward future endeavors.
Conclusion
Confusion between the critical concepts of teaching and feedback may be minimized by allowing them to each have their intended impact, either in response to prior events or aimed toward those yet to take place.

Citations

Citations to this article as recorded by

Resident Assessment of Clinician Educators According to Core ACGME Competencies
Bailey A. Pope, Patricia A. Carney, Mary C. Brooks, Doug R. Rice, Ashly A. Albright, Stephanie A. C. Halvorson
Journal of General Internal Medicine.2024; 39(3): 377. CrossRef

Is it possible to introduce an interview to the Korean Medical Licensing Examination to assess professional attributes?: a survey-based observational study: Seung-Joo Na, HyeRin Roh, Kyung Hee Chun, Kyung Hye Park, Do-Hwan Kim; J Educ Eval Health Prof. 2022;19:10. Published online May 10, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.10; Funded: Korea Health Personnel Licensing Examination

2,971 View
285 Download

Abstract PDF Supplementary Material: Purpose
This study aimsed to gather opinions from medical educators on the possibility of introducing an interview to the Korean Medical Licensing Examination (KMLE) to assess professional attributes. Specifically following topics were dealt with: the appropriate timing and tool to assess unprofessional conduct; ; the possiblity of prevention of unprofessional conduct by introducing an interview to the KMLE; and the possibility of implementation of an interview to the KMLE.
Methods
A cross-sectional study approach based on a survey questionnaire was adopted. We analyzed 104 pieces of news about doctors’ unprofessional conduct to determine the deficient professional attributes. We derived 24 items of unprofessional conduct and developed the questionnaire and surveyed 250 members of the Korean Society of Medical Education 2 times. Descriptive statistics, cross-tabulation analysis, and Fisher’s exact test were applied to the responses. The answers to the open-ended questions were analyzed using conventional content analysis.
Results
In the survey, 49 members (19.6%) responded. Out of 49, 24 (49.5%) responded in the 2nd survey. To assess unprofessional conduct, there was no dominant timing among basic medical education (BME), KMLE, and continuing professional development (CPD). There was no overwhelming assessment tool among written examination, objective structured clinical examination, practice observation, and interview. Response rates of “impossible” (49.0%) and “possible” (42.9%) suggested an interview of the KMLE prevented unprofessional conduct. In terms of implementation, “impossible” (50.0%) was selected more often than “possible” (33.3%).
Conclusion
Professional attributes should be assessed by various tools over the period from BME to CPD. Hence, it may be impossible to introduce an interview to assess professional attributes to the KMLE, and a system is needed such as self-regulation by the professional body rather than licensing examination.

Simulation-based training using a novel Surabaya hysterectomy mannequin following video demonstration to improve abdominal hysterectomy skills of obstetrics and gynecology residents during the COVID-19 pandemic in Indonesia: a pre- and post-intervention study: Dara Dasawulansari Syamsuri, Brahmana Askandar Tjokroprawiro, Eighty Mardiyan Kurniawati, Budi Utomo, Djoko Kuswanto; J Educ Eval Health Prof. 2022;19:11. Published online May 17, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.11; Funded: Universitas Airlangga

5,214 View
305 Download
2 Crossref

Abstract PDF Supplementary Material

Purpose
During the coronavirus disease 2019 (COVID-19) pandemic, the number of abdominal hysterectomy procedures decreased in Indonesia. The existing commercial abdominal hysterectomy simulation model is expensive and difficult to reuse. This study compared residents’ abdominal hysterectomy skills after simulation-based training using the Surabaya hysterectomy mannequin following a video demonstration.
Methods
We randomized 3rd- and 4th-year obstetrics and gynecology residents to a video-based group (group 1), a simulation-based group (group 2), and a combination group (group 3). Abdominal hysterectomy skills were compared between before and after the educational intervention. The pre- and post-tests were scored by blinded experts using the validated Objective Structured Assessment of Technical Skills (OSATS) and Global Rating Scale (GRS).
Results
A total of 33 residents were included in the pre- and post-tests. The OSATS and GRS mean differences after the intervention were higher in group 3 than in groups 1 and 2 (OSATS: 4.64 [95% CI, 2.90–6.37] vs. 2.55 [95% CI, 2.19–2.90] vs. 3.82 [95% CI, 2.41–5.22], P=0.047; GRS: 10.00 [95% CI, 7.01–12.99] vs. 5.18 [95% CI, 3.99–6.38] vs. 7.18 [95% CI, 6.11–8.26], P=0.006). The 3rd-year residents in group 3 had greater mean differences in OSATS and GRS scores than the 4th-year residents (OSATS: 5.67 [95% CI, 2.88–8.46]; GRS: 12.83 [95% CI, 8.61–17.05] vs. OSATS: 3.40 [95% CI, 0.83–5.97]; GRS: 5.67 [95% CI, 2.80–8.54]).
Conclusion
Simulation-based training using the Surabaya hysterectomy mannequin following video demonstration can be a bridge to learning about abdominal hysterectomy for residents who had less surgical experience during the COVID-19 pandemic.

Citations

Citations to this article as recorded by

Improving the quality of care and patient safety in oncology, the contribution of simulation-based training: A scoping review
Nadia Al Wachami, Mohamed Chahboune, Ibtissam Youlyouz-marfak, Mohamed Reda Mesradi, Hajar Lemriss, Abderraouf Hilali
International Journal of Nursing Sciences.2024;[Epub] CrossRef
Effect of midwife competence training in primary healthcare facilities on obstetric risk early detection
Ai Nur Zannah, Yuningsih Yuningsih, Asri Iman Sari, Eka Afdi Septiyono
Healthcare in Low-resource Settings.2024;[Epub] CrossRef

Comparison between residents with a 6-year medical program and a 7-year medical program in terms of objective structured clinical examination performance in postgraduate year training in Taiwan: a 2-group pre- and post-test non-synchronized study: Ya-Ting Chang, Ying-Ying Yang, Chung-Pin Li, Chen-Huan Chen; J Educ Eval Health Prof. 2022;19:13. Published online June 24, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.13; Funded: Taipei Veterans General Hospital, Ministry of Education, Ministry of Science and Technology, Taiwan

2,253 View
205 Download

Abstract PDF Supplementary Material: Purpose
In 2013, medical schools in Taiwan implemented a 6-year medical program that replaced the previous 7-year medical education program. The postgraduate year (PGY) program was also extended from 1 year to 2 years. The new program is characterized by diversified teaching, integration of medical skills, a system-oriented curriculum, and the implementation of primary care and clinical thinking training. The purpose of this study was to examine whether postgraduate residents who learned under the new program have better patient care skills than those who learned under the previous program.
Methods
Of 101 residents in the PGY program at Taipei Veterans General Hospital, 78 were trained in the 6-year program, while 23 were trained in the 7-year program. During the PGY training, 2 objective structured clinical examinations (OSCEs) were used to evaluate clinical reasoning, communication skills, and procedural skills at the beginning of the training and after 11 months of training, respectively. The scores of each OSCE and the rate of improvement of the pre- and post-tests were analyzed.
Results
Residents trained in the new program scored higher on clinical reasoning (P<0.001) and the total scores of the 3 tested skills (P=0.019) on the pre-test. In terms of improvement, residents educated in the previous system improved more in clinical reasoning than those educated in the new education system.
Conclusion
The new medical education program, which emphasizes clinical thinking, improved residents’ clinical skills. The PGY program was effective in improving the clinical performance of residents who were educated in the previous system.

Effect of online education on the knowledge on, attitudes towards, and skills in patient safety for nursing students in Korea: a mixed-methods study: Dan Bi Cho, Won Lee, So Yoon Kim, Sungkyoung Choi; J Educ Eval Health Prof. 2022;19:14. Published online June 30, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.14; Funded: Ministry of Health and Welfare, Korea Health Industry Development Institute

3,134 View
368 Download
1 Web of Science
1 Crossref

Abstract PDF Supplementary Material

Purpose
The purpose of this study was to evaluate the impact of synchronous online education on the patient safety competency (knowledge, attitudes, and skills) of nursing students in Korea and to explore what they thought about this educational method.
Methods
A single-group pre-and post-test design and summative content analysis were implemented. On November 14th, 2020, 110 nursing students completed synchronous online patient safety education. Patient safety competency was measured before and after the intervention using the revised Patient Safety Competency Self-Evaluation tool. The descriptive statistics, paired t-test, and Wilcoxon signed-rank test were used to analyze the data. Students also expressed their opinions about this education based on open-ended questions.
Results
All the patient safety competency scores significantly increased after intervention. A summative content analysis of the open-ended questions yielded 5 categories: educational materials, education level, education time, interaction, and educational media.
Conclusion
This study found that synchronous online patient safety education improves nursing students’ knowledge on, attitudes towards, and skills in patient safety. Nursing students also expressed a variety of positive aspects of the online education method. To improve the efficacy of synchronous online patient safety education, there is a need for further empirical studies on the appropriate class duration and difficulty of the content. It is essential to find a way to combine online education with various learning activities.

Citations

Citations to this article as recorded by

Effects of the small private online course combined with simulation-based training in a patient safety education program among nursing students: A quasi-experimental study
Hui Li, Wenzhong Zhao, Bo Li, Yanhong Li
International Journal of Nursing Sciences.2023; 10(4): 555. CrossRef

Educational/Faculty development material

Environmental management education using immersive virtual reality in asthmatic children in Korea: a randomized controlled study (secondary publication): Seung Hyun Kim, Sang Hyun Park, Insoon Kang, Yuyoung Song, Jaehoon Lim, Wonsuck Yoon, Young Yoo; J Educ Eval Health Prof. 2022;19:15. Published online July 11, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.15; Funded: Environmental Health Center for Childhood Asthma, Ministry of Environmental, Allergy Immunology Center

7,665 View
255 Download
2 Web of Science
2 Crossref

Abstract PDF Supplementary Material

Awareness of environmental control is considered a significant influence on the performance of asthma self-management behaviors, which are involved in maintaining effective asthma control. This study aimed to investigate whether immersive virtual reality (VR) education is effective in environmental control education for asthmatic children in Korea. Thirty asthmatic children aged 9 to 13 years with aeroallergen sensitization were enrolled. Environmental control education for asthmatic participants was performed using immersive VR (VR group) or conventional leaflets provided by asthma specialists (control group). Five questionnaires, on awareness of environmental control, memory, assessment of intent to act, a satisfaction test, and an Asthma Control Test (ACT), were used to estimate the effects of education. The scores for awareness of environmental control, memory, and intent to act significantly increased after education in both groups, and the scores remained high until 4 weeks after education. Both groups’ ACT scores were consistently high before and 4 weeks after education. Satisfaction scores were very high in the VR group. The increased scores in awareness of environmental control and intent to act indicate that the environmental control education using VR is worthy of attention as an effective educational tool for asthma management. Further developed techniques, including active environmental interventions by participants in VR, could be applied to effective asthma management.

Citations

Citations to this article as recorded by

A Systematic Review of Randomized Controlled Trials on Virtual Reality Application in Pediatric Patients
Ashish Varma, Waqar M Naqvi, Salima Mulla, Samana Syed, Sumit Thakur, Sakshi P Arora, Anuj R Varma, Smruti Besekar
Cureus.2022;[Epub] CrossRef
Medical students’ self-assessed efficacy and satisfaction with training on endotracheal intubation and central venous catheterization with smart glasses in Taiwan: a non-equivalent control-group pre- and post-test study
Yu-Fan Lin, Chien-Ying Wang, Yen-Hsun Huang, Sheng-Min Lin, Ying-Ying Yang
Journal of Educational Evaluation for Health Professions.2022; 19: 25. CrossRef

Technical report

Development of examination objectives based on nursing competency for the Korean Nursing Licensing Examination: a validity study: Sujin Shin, Gwang Suk Kim, Jun-Ah Song, Inyoung Lee; J Educ Eval Health Prof. 2022;19:19. Published online August 22, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.19; Funded: Korea Health Personnel Licensing Examination Institute

2,039 View
199 Download
3 Web of Science
3 Crossref

Abstract PDF Supplementary Material

Purpose
This study aimed to develop the examination objectives based on nursing competency of the Korean Nursing Licensing Examination.
Methods
This is a validity study to develop the examination objectives based on nursing competency. Data were collected in December 2021. We reviewed the literature related to changing nurse roles and on the learning objectives for the Korea Medical Licensing Examination and other health personnel licensing examinations. Thereafter, we created a draft of the nursing problems list for examination objectives based on the literature review, and the content validity was evaluated by experts. A final draft of the examination objectives is presented and discussed.
Results
A total of 4 domains, 12 classes, and 85 nursing problems for the Korean Nursing Liscensing Examination were developed. They included the essentials of objectives, related factors, evaluation goals, related activity statements, related clients, related settings, and specific outcomes.
Conclusion
This study developed a draft of the examination objectives based on clinical competency that were related to the clinical situations of nurses and comprised appropriate test items for the licensing examination. Above results may be able to provide fundamental data for item development that reflects future nursing practices.

Citations

Citations to this article as recorded by

A validity study of COMLEX-USA Level 3 with the new test design
Xia Mao, John R. Boulet, Jeanne M. Sandella, Michael F. Oliverio, Larissa Smith
Journal of Osteopathic Medicine.2024;[Epub] CrossRef
A Survey on Perceptions of the Direction of Korean Medicine Education and National Licensing Examination
Han-Byul Cho, Won-Suk Sung, Jiseong Hong, Yeonseok Kang, Eun-Jung Kim
Healthcare.2023; 11(12): 1685. CrossRef
Suggestion for item allocation to 8 nursing activity categories of the Korean Nursing Licensing Examination: a survey-based descriptive study
Kyunghee Kim, So Young Kang, Younhee Kang, Youngran Kweon, Hyunjung Kim, Youngshin Song, Juyeon Cho, Mi-Young Choi, Hyun Su Lee
Journal of Educational Evaluation for Health Professions.2023; 20: 18. CrossRef

Research article

Possibility of using the yes/no Angoff method as a substitute for the percent Angoff method for estimating the cutoff score of the Korean Medical Licensing Examination: a simulation study: Janghee Park; J Educ Eval Health Prof. 2022;19:23. Published online August 31, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.23; Funded: Soonchunhyang University Research Fund

2,466 View
163 Download
2 Web of Science
2 Crossref

Abstract PDF Supplementary Material

Purpose
The percent Angoff (PA) method has been recommended as a reliable method to set the cutoff score instead of a fixed cut point of 60% in the Korean Medical Licensing Examination (KMLE). The yes/no Angoff (YNA) method, which is easy for panelists to judge, can be considered as an alternative because the KMLE has many items to evaluate. This study aimed to compare the cutoff score and the reliability depending on whether the PA or the YNA standard-setting method was used in the KMLE.
Methods
The materials were the open-access PA data of the KMLE. The PA data were converted to YNA data in 5 categories, in which the probabilities for a “yes” decision by panelists were 50%, 60%, 70%, 80%, and 90%. SPSS for descriptive analysis and G-string for generalizability theory were used to present the results.
Results
The PA method and the YNA method counting 60% as “yes,” estimated similar cutoff scores. Those cutoff scores were deemed acceptable based on the results of the Hofstee method. The highest reliability coefficients estimated by the generalizability test were from the PA method and the YNA method, with probabilities of 70%, 80%, 60%, and 50% for deciding “yes,” in descending order. The panelist’s specialty was the main cause of the error variance. The error size was similar regardless of the standard-setting method.
Conclusion
The above results showed that the PA method was more reliable than the YNA method in estimating the cutoff score of the KMLE. However, the YNA method with a 60% probability for deciding “yes” also can be used as a substitute for the PA method in estimating the cutoff score of the KMLE.

Citations

Citations to this article as recorded by

Issues in the 3rd year of the COVID-19 pandemic, including computer-based testing, study design, ChatGPT, journal metrics, and appreciation to reviewers
Sun Huh
Journal of Educational Evaluation for Health Professions.2023; 20: 5. CrossRef
Possibility of independent use of the yes/no Angoff and Hofstee methods for the standard setting of the Korean Medical Licensing Examination written test: a descriptive study
Do-Hwan Kim, Ye Ji Kang, Hoon-Ki Park
Journal of Educational Evaluation for Health Professions.2022; 19: 33. CrossRef

Brief report

Initial steps for integrating academic electronic health records into clinical curricula of physical and occupational therapy in the United States: a survey-based observational study: Stephen Burrows, Lola Halperin, Eric Nemec, Wendy Romney; J Educ Eval Health Prof. 2022;19:24. Published online September 2, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.24; Funded: Sacred Heart University

2,903 View
193 Download

Abstract PDF Supplementary Material: Training programs must be designed to prepare physical and occupational therapy students to use electronic health records (EHR) and interprofessional collaboration. This report aims to describe physical and occupational therapy students’ perceptions of integrating an academic EHR (AEHR) in their problem-based learning (PBL) curricula in the College of Health Professions, Sacred Heart University, Fairfield, Connecticut, the United States. A paper-based case approach to PBL was adapted by creating patient cases in an AEHR. Students were asked to complete chart reviews and review provider notes to enhance their learning. An online survey was conducted to determine their perceptions of using AEHR from May 2014 to August 2015. Eighty-five students completed the survey, and 88.1% felt that using an AEHR was needed, and 82.4% felt that the additional notes enhanced their understanding of the interdisciplinary team. However, 83.5% reported the AEHR system increased the time needed to extract meaningful information. Incorporating an AEHR into curricula is essential to ensure students are adequately prepared for future patient interactions.

Research article

Medical students’ self-assessed efficacy and satisfaction with training on endotracheal intubation and central venous catheterization with smart glasses in Taiwan: a non-equivalent control-group pre- and post-test study: Yu-Fan Lin, Chien-Ying Wang, Yen-Hsun Huang, Sheng-Min Lin, Ying-Ying Yang; J Educ Eval Health Prof. 2022;19:25. Published online September 2, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.25; Funded: Taipei Veterans General Hospital, Ministry of Education, Ministry of Science and Technology, Taiwan

2,975 View
227 Download
1 Web of Science
1 Crossref

Abstract PDF Supplementary Material

Purpose
Endotracheal intubation and central venous catheterization are essential procedures in clinical practice. Simulation-based technology such as smart glasses has been used to facilitate medical students’ training on these procedures. We investigated medical students’ self-assessed efficacy and satisfaction regarding the practice and training of these procedures with smart glasses in Taiwan.
Methods
This observational study enrolled 145 medical students in the 5th and 6th years participating in clerkships at Taipei Veterans General Hospital between October 2020 and December 2021. Students were divided into the smart glasses or the control group and received training at a workshop. The primary outcomes included students’ pre- and post-intervention scores for self-assessed efficacy and satisfaction with the training tool, instructor’s teaching, and the workshop.
Results
The pre-intervention scores for self-assessed efficacy of 5th- and 6th-year medical students in endotracheal intubation and central venous catheterization procedures showed no significant difference. The post-intervention score of self-assessed efficacy in the smart glasses group was better than that of the control group. Moreover, 6th-year medical students in the smart glasses group showed higher satisfaction with the training tool, instructor’s teaching, and workshop than those in the control group.
Conclusion
Smart glasses served as a suitable simulation tool for endotracheal intubation and central venous catheterization procedures training in medical students. Medical students practicing with smart glasses showed improved self-assessed efficacy and higher satisfaction with training, especially for procedural steps in a space-limited field. Simulation training on procedural skills with smart glasses in 5th-year medical students may be adjusted to improve their satisfaction.

Citations

Citations to this article as recorded by

The use of smart glasses in nursing education: A scoping review
Charlotte Romare, Lisa Skär
Nurse Education in Practice.2023; 73: 103824. CrossRef

First
Prev
Page of 9
Next
Last

Funded articles