Skip Navigation
Skip to contents

JEEHP : Journal of Educational Evaluation for Health Professions

OPEN ACCESS
SEARCH
Search

New issue

Page Path
HOME > Browse articles > New issue
32 New issue
Filter
Filter
Article category
Keywords
Authors
Funded articles
Volume 19; 2022
Prev issue Next issue

Brief report
Self-directed learning quotient and common learning types of pre-medical students in Korea by the Multi-Dimensional Learning Strategy Test 2nd edition: a descriptive study
Sun Kim, A Ra Cho, Chul Woon Chung
J Educ Eval Health Prof. 2022;19:32.   Published online November 28, 2022
DOI: https://doi.org/10.3352/jeehp.2022.19.32
  • 114 View
  • 28 Download
AbstractAbstract PDFSupplementary Material
This study aimed to find the self-directed learning quotient and common learning types of pre-medical students through the confirmation of 4 characteristics of learning strategies, including personality, motivation, emotion, and behavior. The response data were collected from 277 out of 294 target first-year pre-medical students from 2019 to 2021, using the Multi-Dimensional Learning Strategy Test 2nd edition. The most common learning type was a self-directed type (44.0%), stagnant type (33.9%), latent type (14.4%), and conscientiousness type (7.6%). The self-directed learning index was high (29.2%), moderate (24.6%), somewhat high (21.7%), somewhat low (14.4%), and low (10.1%). This study confirmed that many students lacked self-directed learning capabilities for learning strategies. In addition, it was found that the difficulties experienced by each student were different, and the variables resulting in difficulties were also diverse. It may provide insights into how to develop programs that can help students increase their self-directed learning capability.
Research articles
Medical student selection process enhanced by improving selection algorithms and changing the focus of interviews in Australia: a descriptive study
Boaz Shulruf, Gary Mayer Velan, Sean Edward Kennedy
J Educ Eval Health Prof. 2022;19:31.   Published online November 28, 2022
DOI: https://doi.org/10.3352/jeehp.2022.19.31
  • 93 View
  • 15 Download
AbstractAbstract PDFSupplementary Material
Purpose
The study investigates the efficacy of new features introduced to the selection process for medical school at the University of New South Wales, Australia: (1) considering the relative ranks rather than scores of the Undergraduate Medicine and Health Sciences Admission Test and Australian Tertiary Admission Rank; (2) structured interview focusing on interpersonal interaction and concerns should the applicants become students; and (3) embracing interviewers’ diverse perspectives.
Methods
Data from 5 cohorts of students were analyzed, comparing outcomes of the second year in the medicine program of 4 cohorts of the old selection process and 1 of the new process. The main analysis comprised multiple linear regression models for predicting academic, clinical, and professional outcomes, by section tools and demographic variables.
Results
Selection interview marks from the new interview (512 applicants, 2 interviewers each) were analyzed for inter-rater reliability, which identified a high level of agreement (kappa=0.639). No such analysis was possible for the old interview since it required interviewers to reach a consensus. Multivariate linear regression models utilizing outcomes for 5 cohorts (N=905) revealed that the new selection process was much more effective in predicting academic and clinical achievement in the program (R2=9.4%–17.8% vs. R2=1.5%–8.4%).
Conclusion
The results suggest that the medical student selection process can be significantly enhanced by employing a non-compensatory selection algorithm; and using a structured interview focusing on interpersonal interaction and concerns should the applicants become students; as well as embracing interviewers’ diverse perspectives.
Is online objective structured clinical examination teaching an acceptable replacement in post-COVID-19 medical education in the United Kingdom?: a descriptive study
Vashist Motkur, Aniket Bharadwaj, Nimalesh Yogarajah
J Educ Eval Health Prof. 2022;19:30.   Published online November 7, 2022
DOI: https://doi.org/10.3352/jeehp.2022.19.30
  • 255 View
  • 57 Download
AbstractAbstract PDFSupplementary Material
Purpose
Coronavirus disease 2019 (COVID-19) restrictions resulted in an increased emphasis on virtual communication in medical education. This study assessed the acceptability of virtual teaching in an online objective structured clinical examination (OSCE) series and its role in future education.
Methods
Six surgical OSCE stations were designed, covering common surgical topics, with specific tasks testing data interpretation, clinical knowledge, and communication skills. These were delivered via Zoom to students who participated in student/patient/examiner role-play. Feedback was collected by asking students to compare online teaching with previous experiences of in-person teaching. Descriptive statistics were used for Likert response data, and thematic analysis for free-text items.
Results
Sixty-two students provided feedback, with 81% of respondents finding online instructions preferable to paper equivalents. Furthermore, 65% and 68% found online teaching more efficient and accessible, respectively, than in-person teaching. Only 34% found communication with each other easier online. 40% preferred online OSCE teaching to in-person teaching. Students also expressed feedback in positive and negative free-text comments.
Conclusion
The data suggested that generally students were unwilling for online teaching to completely replace in-person teaching. The success of online teaching was dependent on the clinical skill being addressed; some were less amenable to a virtual setting. However, online OSCE teaching could play a role alongside in-person teaching.
Enhanced numeracy skills following team-based learning in United States pharmacy students: a longitudinal cohort study
Rob Edwin Carpenter, Leanne Coyne, Dave Silberman, Jody Kyoto Takemoto
J Educ Eval Health Prof. 2022;19:29.   Published online October 27, 2022
DOI: https://doi.org/10.3352/jeehp.2022.19.29
  • 262 View
  • 62 Download
AbstractAbstract PDF
Purpose
The literature suggests that the ability to numerate cannot be fully understood without accounting for the social context in which mathematical activity is represented. Team-based learning (TBL) is an andragogical approach with theoretical links to sociocultural and community-of-practice learning. This study aimed to quantitatively explore the impact of TBL instruction on numeracy development in 2 cohorts of pharmacy students and identify the impact of TBL instruction on numeracy development from a social perspective for healthcare education.
Methods
Two cohorts of students were administered the Health Science Reasoning Test-Numeracy (HSRT-N) before beginning pharmacy school. Two years after using TBL as the primary method of instruction, both comprehensive and domain data from the HSRT-N were analyzed.
Results
In total, 163 pharmacy student scores met the inclusion criteria. The students’ numeracy skills measured by HSRT-N improved after 2 years of TBL instruction.
Conclusion
Numeracy was the most significantly improved HSRT-N domain in pharmacy students following two years of TBL instruction. Although a closer examination of numeracy development in TBL is warranted, initial data suggest that TBL instruction may be an adequate proxy for advancing numeracy in a cohort of pharmacy students. TBL may encourage a social practice of mathematics to improve pharmacy students’ ability to numerate critically.
Factors affecting nursing and health technician students' satisfaction with distance learning during the COVID-19 pandemic in Morocco: a descriptive study
Aziz Naciri, Mohamed Radid, Abderrahmane Achbani, Mohamed Amine Baba, Ahmed Kharbach, Ghizlane Chemsi
J Educ Eval Health Prof. 2022;19:28.   Published online October 17, 2022
DOI: https://doi.org/10.3352/jeehp.2022.19.28
  • 483 View
  • 104 Download
AbstractAbstract PDF
Purpose
Distance learning describes any learning based on the use of new multimedia technologies and the internet to allow students to acquire new knowledge and skills at a distance. This study aimed to determine satisfaction levels with distance learning and associated factors among nursing and health technician students during the coronavirus disease 2019 pandemic in Morocco.
Methods
An descriptive study was conducted between April and June 2022 among nursing and health technician students using a self-administered instrument. The student satisfaction questionnaire consists of 24 questions categorized into 6 subscales: instructor, technology, course setup, interaction, outcomes, and overall satisfaction. It was based on a 5-point Likert scale, ranging from 1 (strongly disagree) to 5 (strongly agree). Univariate and multivariate logistic regression analyses were conducted to identify factors associated with student satisfaction during distance learning.
Results
A total of 330 students participated in this study, and 176 students (53.3%) were satisfied with the distance learning activities. A mean score higher than 2.8 out of 5 was obtained for all subscales. Multiple regression analysis showed that students’ year of study (adjusted odds ratio [aOR], 2.34; 95% confidence interval [CI], 1.28–4.27) and internet quality (aOR, 0.47; 95% CI, 0.29–0.77) were the significant factors associated with students’ satisfaction during distance learning.
Conclusion
This study highlights the satisfaction level of students and factors that influenced it during distance learning. A thorough understanding of student satisfaction with digital environments will contribute to the successful implementation of distance learning devices in nursing.
Equal Z standard-setting method to estimate the minimum number of panelists for a medical school’s objective structured clinical examination in Taiwan: a simulation study  
Ying-Ying Yang, Pin-Hsiang Huang, Ling-Yu Yang, Chia-Chang Huang, Chih-Wei Liu, Shiau-Shian Huang, Chen-Huan Chen, Fa-Yauh Lee, Shou-Yen Kao, Boaz Shulruf
J Educ Eval Health Prof. 2022;19:27.   Published online October 17, 2022
DOI: https://doi.org/10.3352/jeehp.2022.19.27
  • 487 View
  • 61 Download
AbstractAbstract PDFSupplementary Material
Purpose
Undertaking a standard-setting exercise is a common method for setting pass/fail cut scores for high-stakes examinations. The recently introduced equal Z standard-setting method (EZ method) has been found to be a valid and effective alternative for the commonly used Angoff and Hofstee methods and their variants. The current study aims to estimate the minimum number of panelists required for obtaining acceptable and reliable cut scores using the EZ method.
Methods
The primary data were extracted from 31 panelists who used the EZ method for setting cut scores for a 12-station of medical school’s final objective structured clinical examination (OSCE) in Taiwan. For this study, a new data set composed of 1,000 random samples of different panel sizes, ranging from 5 to 25 panelists, was established and analyzed. Analysis of variance was performed to measure the differences in the cut scores set by the sampled groups, across all sizes within each station.
Results
On average, a panel of 10 experts or more yielded cut scores with confidence more than or equal to 90% and 15 experts yielded cut scores with confidence more than or equal to 95%. No significant differences in cut scores associated with panel size were identified for panels of 5 or more experts.
Conclusion
The EZ method was found to be valid and feasible. Less than an hour was required for 12 panelists to assess 12 OSCE stations. Calculating the cut scores required only basic statistical skills.
Acceptability of the 8-case objective structured clinical examination of medical students in Korea using generalizability theory: a reliability study  
Song Yi Park, Sang-Hwa Lee, Min-Jeong Kim, Ki-Hwan Ji, Ji Ho Ryu
J Educ Eval Health Prof. 2022;19:26.   Published online September 8, 2022
DOI: https://doi.org/10.3352/jeehp.2022.19.26
  • 536 View
  • 154 Download
AbstractAbstract PDFSupplementary Material
Purpose
This study investigated whether the reliability was acceptable when the number of cases in the objective structured clinical examination (OSCE) decreased from 12 to 8 using generalizability theory (GT).
Methods
This psychometric study analyzed the OSCE data of 439 fourth-year medical students conducted in the Busan and Gyeongnam areas of South Korea from July 12 to 15, 2021. The generalizability study (G-study) considered 3 facets—students (p), cases (c), and items (i)—and designed the analysis as p×(i:c) due to items being nested in a case. The acceptable generalizability (G) coefficient was set to 0.70. The G-study and decision study (D-study) were performed using G String IV ver. 6.3.8 (Papawork, Hamilton, ON, Canada).
Results
All G coefficients except for July 14 (0.69) were above 0.70. The major sources of variance components (VCs) were items nested in cases (i:c), from 51.34% to 57.70%, and residual error (pi:c), from 39.55% to 43.26%. The proportion of VCs in cases was negligible, ranging from 0% to 2.03%.
Conclusion
The case numbers decreased in the 2021 Busan and Gyeongnam OSCE. However, the reliability was acceptable. In the D-study, reliability was maintained at 0.70 or higher if there were more than 21 items/case in 8 cases and more than 18 items/case in 9 cases. However, according to the G-study, increasing the number of items nested in cases rather than the number of cases could further improve reliability. The consortium needs to maintain a case bank with various items to implement a reliable blueprinting combination for the OSCE.
Medical students’ self-assessed efficacy and satisfaction with training on endotracheal intubation and central venous catheterization with smart glasses in Taiwan: a non-equivalent control-group pre- and post-test study  
Yu-Fan Lin, Chien-Ying Wang, Yen-Hsun Huang, Sheng-Min Lin, Ying-Ying Yang
J Educ Eval Health Prof. 2022;19:25.   Published online September 2, 2022
DOI: https://doi.org/10.3352/jeehp.2022.19.25
  • 613 View
  • 156 Download
AbstractAbstract PDFSupplementary Material
Purpose
Endotracheal intubation and central venous catheterization are essential procedures in clinical practice. Simulation-based technology such as smart glasses has been used to facilitate medical students’ training on these procedures. We investigated medical students’ self-assessed efficacy and satisfaction regarding the practice and training of these procedures with smart glasses in Taiwan.
Methods
This observational study enrolled 145 medical students in the 5th and 6th years participating in clerkships at Taipei Veterans General Hospital between October 2020 and December 2021. Students were divided into the smart glasses or the control group and received training at a workshop. The primary outcomes included students’ pre- and post-intervention scores for self-assessed efficacy and satisfaction with the training tool, instructor’s teaching, and the workshop.
Results
The pre-intervention scores for self-assessed efficacy of 5th- and 6th-year medical students in endotracheal intubation and central venous catheterization procedures showed no significant difference. The post-intervention score of self-assessed efficacy in the smart glasses group was better than that of the control group. Moreover, 6th-year medical students in the smart glasses group showed higher satisfaction with the training tool, instructor’s teaching, and workshop than those in the control group.
Conclusion
Smart glasses served as a suitable simulation tool for endotracheal intubation and central venous catheterization procedures training in medical students. Medical students practicing with smart glasses showed improved self-assessed efficacy and higher satisfaction with training, especially for procedural steps in a space-limited field. Simulation training on procedural skills with smart glasses in 5th-year medical students may be adjusted to improve their satisfaction.
Brief report
Initial steps for integrating academic electronic health records into clinical curricula of physical and occupational therapy in the United States: a survey-based observational study  
Stephen Burrows, Lola Halperin, Eric Nemec, Wendy Romney
J Educ Eval Health Prof. 2022;19:24.   Published online September 2, 2022
DOI: https://doi.org/10.3352/jeehp.2022.19.24
  • 837 View
  • 146 Download
AbstractAbstract PDFSupplementary Material
Training programs must be designed to prepare physical and occupational therapy students to use electronic health records (EHR) and interprofessional collaboration. This report aims to describe physical and occupational therapy students’ perceptions of integrating an academic EHR (AEHR) in their problem-based learning (PBL) curricula in the College of Health Professions, Sacred Heart University, Fairfield, Connecticut, the United States. A paper-based case approach to PBL was adapted by creating patient cases in an AEHR. Students were asked to complete chart reviews and review provider notes to enhance their learning. An online survey was conducted to determine their perceptions of using AEHR from May 2014 to August 2015. Eighty-five students completed the survey, and 88.1% felt that using an AEHR was needed, and 82.4% felt that the additional notes enhanced their understanding of the interdisciplinary team. However, 83.5% reported the AEHR system increased the time needed to extract meaningful information. Incorporating an AEHR into curricula is essential to ensure students are adequately prepared for future patient interactions.
Research articles
Possibility of using the yes/no Angoff method as a substitute for the percent Angoff method for estimating the cutoff score of the Korean Medical Licensing Examination: a simulation study
Janghee Park
J Educ Eval Health Prof. 2022;19:23.   Published online August 31, 2022
DOI: https://doi.org/10.3352/jeehp.2022.19.23
  • 731 View
  • 116 Download
AbstractAbstract PDFSupplementary Material
Purpose
The percent Angoff (PA) method has been recommended as a reliable method to set the cutoff score instead of a fixed cut point of 60% in the Korean Medical Licensing Examination (KMLE). The yes/no Angoff (YNA) method, which is easy for panelists to judge, can be considered as an alternative because the KMLE has many items to evaluate. This study aimed to compare the cutoff score and the reliability depending on whether the PA or the YNA standard-setting method was used in the KMLE.
Methods
The materials were the open-access PA data of the KMLE. The PA data were converted to YNA data in 5 categories, in which the probabilities for a “yes” decision by panelists were 50%, 60%, 70%, 80%, and 90%. SPSS for descriptive analysis and G-string for generalizability theory were used to present the results.
Results
The PA method and the YNA method counting 60% as “yes,” estimated similar cutoff scores. Those cutoff scores were deemed acceptable based on the results of the Hofstee method. The highest reliability coefficients estimated by the generalizability test were from the PA method and the YNA method, with probabilities of 70%, 80%, 60%, and 50% for deciding “yes,” in descending order. The panelist’s specialty was the main cause of the error variance. The error size was similar regardless of the standard-setting method.
Conclusion
The above results showed that the PA method was more reliable than the YNA method in estimating the cutoff score of the KMLE. However, the YNA method with a 60% probability for deciding “yes” also can be used as a substitute for the PA method in estimating the cutoff score of the KMLE.
Self-control as an important factor affecting the online learning readiness of Vietnamese medical and health students during the COVID-19 pandemic: a network analysis  
Minh Tu Nguyen, Binh Thang Tran, Thanh Gia Nguyen, Minh Tri Phan, Thi Thu Tham Luong, Dinh Duong Le
J Educ Eval Health Prof. 2022;19:22.   Published online August 25, 2022
DOI: https://doi.org/10.3352/jeehp.2022.19.22
  • 1,244 View
  • 143 Download
AbstractAbstract PDFSupplementary Material
Purpose
The current study aimed to use network analysis to investigate medical and health students’ readiness for online learning during the coronavirus disease 2019 (COVID-19) pandemic at the University of Medicine and Pharmacy, Hue University.
Methods
A questionnaire survey on the students’ readiness for online learning was performed using a Google Form from May 13 to June 22, 2021. In total, 1,377 completed responses were eligible for analysis out of 1,411 participants. The network structure was estimated for readiness scales with 6 factors: computer skills, internet skills, online communication, motivation, self-control, and self-learning. Data were fitted using a Gaussian graphical model with the extended Bayesian information criterion.
Results
In 1,377 students, a network structure was identified with 6 nodes and no isolated nodes. The top 3 partial correlations were similar in networks for the overall sample and subgroups of gender and grade levels. The self-control node was the strongest for the connection to others, with the highest nodal strength. The change of nodal strength was greatest in online communication for both gender and grade levels. The correlation stability coefficient for nodal strength was achieved for all networks.
Conclusion
These findings indicated that self-control was the most important factor in students’ readiness network structures for online learning. Therefore, self-control needs to be encouraged during online learning to improve the effectiveness of achieving online learning outcomes for students.
Content validity test of a safety checklist for simulated participants in simulation-based education in the United Kingdom: a methodological study
Matthew Bradley
J Educ Eval Health Prof. 2022;19:21.   Published online August 25, 2022
DOI: https://doi.org/10.3352/jeehp.2022.19.21
  • 609 View
  • 117 Download
AbstractAbstract PDFSupplementary Material
Purpose
Simulation training is an ever-growing means of healthcare education and often involves simulated participants (SPs), commonly known as actors. Simulation-based education (SBE) can sometimes endanger SPs, and as such we have created a safety checklist for them to follow. This study describes how we developed the checklist through a quality improvement project, and then evaluated feedback responses to assess whether SPs felt our checklist was safe.
Methods
The checklist was provided to SPs working in an acute trust simulation service when delivering multidisciplinary SBE over 4 months. Using multiple plan–do–study–act cycles, the checklist was refined by reflecting on SP feedback to ensure that the standards of the safe simulation were met. We collected 21 responses from September to December 2021 after SPs completed an SBE event.
Results
The responses showed that 100% of SPs felt safe during SBE when using our checklist. The average “confidence in safety” rating before using the checklist was 6.8/10, which increased significantly to 9.2/10 after using the checklist (P<0.0005). The checklist was refined throughout the 4 months and implemented in adult and pediatric SBE as a standard operating procedure.
Conclusion
We recommend using our safety checklist as a standard operating procedure to improve the confidence and safety of SPs during safe and effective simulations.
Effect of a forensic nursing virtual education course on knowledge and clinical decision-making of master’s nursing students in Iran: a non-equivalent control group pre- and post-test study
Zeynab Firuzi, Mitra Sedghi Sabet, Fateme Jafaraghaee, Hedayat Jafari, Ehsan Kazemnezhad Leyli, Samad Karkhah, Mohammad Javad Ghazanfari
J Educ Eval Health Prof. 2022;19:20.   Published online August 25, 2022
DOI: https://doi.org/10.3352/jeehp.2022.19.20
  • 730 View
  • 139 Download
AbstractAbstract PDFSupplementary Material
Purpose
Forensic nursing is a specialty in the nursing profession based on legal procedures. This study aimed to assess the effect of a forensic nursing virtual education course on knowledge and clinical decision-making among master’s nursing students.
Methods
In a quasi-experimental study with a pre- and post-test, 106 master’s nursing students at Guilan (n=65) and Mazandaran (n=41) Universities of Medical Sciences, Iran were enrolled. Data were collected using census sampling from March to April 2021. Participants in the intervention group received a forensic nursing virtual education course in three 90-minute sessions for 2 days.
Results
A total of 88 out of 106 master’s nursing students were enrolled in this study. The mean post-education score for knowledge in the intervention group was significantly higher than in the control group (12.52 vs. 7.67, P<0.001). The mean post-education score for clinical decision-making in the intervention group was significantly higher than in the control group (16.96 vs. 13.64, P<0.001).
Conclusion
The level of knowledge and clinical decision-making of master’s nursing students regarding forensic evidence improved after the forensic nursing virtual education course in the intervention group compared to the control group. Nursing managers and policymakers can develop appropriate strategies to improve the knowledge and clinical decision-making of nursing students by using forensic nursing education courses in the curricula of nursing programs, especially in postgraduate education as an elective or mandatory course.
Technical report
Development of examination objectives based on nursing competency for the Korean Nursing Licensing Examination: a validity study  
Sujin Shin, Gwang Suk Kim, Jun-Ah Song, Inyoung Lee
J Educ Eval Health Prof. 2022;19:19.   Published online August 22, 2022
DOI: https://doi.org/10.3352/jeehp.2022.19.19
  • 653 View
  • 146 Download
AbstractAbstract PDFSupplementary Material
Purpose
This study aimed to develop the examination objectives based on nursing competency of the Korean Nursing Licensing Examination.
Methods
This is a validity study to develop the examination objectives based on nursing competency. Data were collected in December 2021. We reviewed the literature related to changing nurse roles and on the learning objectives for the Korea Medical Licensing Examination and other health personnel licensing examinations. Thereafter, we created a draft of the nursing problems list for examination objectives based on the literature review, and the content validity was evaluated by experts. A final draft of the examination objectives is presented and discussed.
Results
A total of 4 domains, 12 classes, and 85 nursing problems for the Korean Nursing Liscensing Examination were developed. They included the essentials of objectives, related factors, evaluation goals, related activity statements, related clients, related settings, and specific outcomes.
Conclusion
This study developed a draft of the examination objectives based on clinical competency that were related to the clinical situations of nurses and comprised appropriate test items for the licensing examination. Above results may be able to provide fundamental data for item development that reflects future nursing practices.
Case report
Successful pilot application of multi-attribute utility analysis concepts in evaluating academic-clinical partnerships in the United States: a case report  
Sara Elizabeth North, Amanda Nicole Sharp
J Educ Eval Health Prof. 2022;19:18.   Published online August 19, 2022
DOI: https://doi.org/10.3352/jeehp.2022.19.18
  • 713 View
  • 109 Download
AbstractAbstract PDFSupplementary Material
Strong partnerships between academic health professions programs and clinical practice settings, termed academic-clinical partnerships, are essential in providing quality clinical training experiences. However, the literature does not operationalize a model by which an academic program may identify priority attributes and evaluate its partnerships. This study aimed to develop a values-based academic-clinical partnership evaluation approach, rooted in methodologies from the field of evaluation and implemented in the context of an academic Doctor of Physical Therapy clinical education program. The authors developed a semi-quantitative evaluation approach incorporating concepts from multi-attribute utility analysis (MAUA) that enabled consistent, values-based partnership evaluation. Data-informed actions led to improved overall partnership effectiveness. Pilot outcomes support the feasibility and desirability of moving toward MAUA as a potential methodological framework. Further research may lead to the development of a standardized process for any academic health profession program to perform a values-based evaluation of their academic-clinical partnerships to guide decision-making.

JEEHP : Journal of Educational Evaluation for Health Professions