Journal of Educational Evaluation for Health Professions

Research articles

Improving item pool utilization for health professions examinations under variable-length computerized adaptive testing designs: a shadow-test approach: Hwanggyu Lim, Kyung (Chris) Tyek Han; J Educ Eval Health Prof. 2025;22:35. Published online November 3, 2025; DOI: https://doi.org/10.3352/jeehp.2025.22.35

1,939 View
165 Download

Abstract PDF Supplementary Material: Purpose
The shadow-test approach to computerized adaptive testing (CAT) ensures content validity in health professions examinations but may suffer from poor item pool utilization in variable-length designs, increasing operational costs and security risks. This study aimed to address this challenge by developing algorithms that enhance the sustainability of shadow CAT in variable-length design.
Methods
A simulation study was conducted to evaluate 3 proposed modifications of the α-stratification method designed to improve item pool utilization. These methods, which integrated randomesque selection and multiple-form strategies, were compared with 2 baseline algorithms within a variable-length shadow CAT framework. Performance was assessed in terms of measurement precision, pool utilization, and test efficiency.
Results
The proposed modifications significantly outperformed the baseline methods across all measures of item pool utilization and exposure control. The most effective method (Modification 2) reduced the proportion of unused items from 35.6% to 5.0% and produced more uniform item exposure rates. These substantial gains in operational sustainability were achieved while maintaining measurement precision comparable to the baseline methods.
Conclusion
The proposed algorithms effectively mitigate poor item pool utilization in shadow CAT under variable-length design. This enhanced framework provides a robust, secure, and sustainable solution for high-stakes adaptive assessments in the health professions that remain content-valid, precise, and operationally efficient.

Evaluation of a virtual objective structured clinical examination in the metaverse (Second Life) to assess the clinical skills in emergency radiology of medical students in Spain: a cross-sectional study: Alba Virtudes Perez-Baena, Teodoro Rudolphi-Solero, Rocio Lorenzo-Alvarez, Dolores Dominguez-Pinos, Miguel Jose Ruiz-Gomez, Francisco Sendra-Portero; J Educ Eval Health Prof. 2025;22:12. Published online April 21, 2025; DOI: https://doi.org/10.3352/jeehp.2025.22.12

3,632 View
260 Download
1 Crossref

Purpose
The objective structured clinical examination (OSCE) is an effective but resource-intensive tool for assessing clinical competence. This study hypothesized that implementing a virtual OSCE in the Second Life (SL) platform in the metaverse as a cost-effective alternative will effectively assess and enhance clinical skills in emergency radiology while being feasible and well-received. The aim was to evaluate a virtual radiology OSCE in SL as a formative assessment, focusing on feasibility, educational impact, and students’ perceptions.
Methods
Two virtual 6-station OSCE rooms dedicated to emergency radiology were developed in SL. Sixth-year medical students completed the OSCE during a 1-hour session in 2022–2023, followed by feedback including a correction checklist, individual scores, and group comparisons. Students completed a questionnaire with Likert-scale questions, a 10-point rating, and open-ended comments. Quantitative data were analyzed using the Student t-test and the Mann-Whitney U test, and qualitative data through thematic analysis.
Results
In total, 163 students participated, achieving mean scores of 5.1±1.4 and 4.9±1.3 (out of 10) in the 2 virtual OSCE rooms, respectively (P=0.287). One hundred seventeen students evaluated the OSCE, praising the teaching staff (9.3±1.0), project organization (8.8±1.2), OSCE environment (8.7±1.5), training usefulness (8.6±1.5), and formative self-assessment (8.5±1.4). Likert-scale questions and students’ open-ended comments highlighted the virtual environment’s attractiveness, case selection, self-evaluation usefulness, project excellence, and training impact. Technical difficulties were reported by 13 students (8%).
Conclusion
This study demonstrated the feasibility of incorporating formative OSCEs in SL as a useful teaching tool for undergraduate radiology education, which was cost-effective and highly valued by students.

Citations

Citations to this article as recorded by

Effectiveness of VR and traditional training in medical education for mass casualty management: an OSCE-based randomized controlled trial
Zhe Li, Wan Chen, Guozheng Qiu, Lei Shi, Yutao Tang, Xibin Xu, Sanshan Zhu, Liwen Lyu
BMC Medical Education.2026;[Epub] CrossRef

Review

Immersive simulation in nursing and midwifery education: a systematic review: Lahoucine Ben Yahya, Aziz Naciri, Mohamed Radid, Ghizlane Chemsi; J Educ Eval Health Prof. 2024;21:19. Published online August 8, 2024; DOI: https://doi.org/10.3352/jeehp.2024.21.19

14,602 View
707 Download
7 Web of Science
13 Crossref

Abstract PDF Supplementary Material

Purpose
Immersive simulation is an innovative training approach in health education that enhances student learning. This study examined its impact on engagement, motivation, and academic performance in nursing and midwifery students.
Methods
A comprehensive systematic search was meticulously conducted in 4 reputable databases—Scopus, PubMed, Web of Science, and Science Direct—following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. The research protocol was pre-registered in the PROSPERO registry, ensuring transparency and rigor. The quality of the included studies was assessed using the Medical Education Research Study Quality Instrument.
Results
Out of 90 identified studies, 11 were included in the present review, involving 1,090 participants. Four out of 5 studies observed high post-test engagement scores in the intervention groups. Additionally, 5 out of 6 studies that evaluated motivation found higher post-test motivational scores in the intervention groups than in control groups using traditional approaches. Furthermore, among the 8 out of 11 studies that evaluated academic performance during immersive simulation training, 5 reported significant differences (P<0.001) in favor of the students in the intervention groups.
Conclusion
Immersive simulation, as demonstrated by this study, has a significant potential to enhance student engagement, motivation, and academic performance, surpassing traditional teaching methods. This potential underscores the urgent need for future research in various contexts to better integrate this innovative educational approach into nursing and midwifery education curricula, inspiring hope for improved teaching methods.

Citations

Citations to this article as recorded by

A Review of Reviews on Virtual Reality in Educational Context: Constraints, Implications, and Research Agendas
Yuchun Zhong, Chunqi Li, Juming Jiang, Luke Kutszik Fryer, Alex Shum
Journal of Educational Computing Research.2026; 64(2): 439. CrossRef
Digital Interventions for Palliative Care Education for Nursing Students: A Systematic Review
Abdulelah Alanazi, Gary Mitchell, Fadwa Naji Al Halaiqa, Fadi Khraim, Stephanie Craig
Nursing Reports.2026; 16(1): 16. CrossRef
Artificial intelligence self-efficacy and attitudes among nursing students: a multicenter network analysis of educational stratification
Qin Zeng, Jun Zhu, Yuji Wang, Shaoyu Su, Yan Huang
BMC Medical Education.2026;[Epub] CrossRef
The power of moulage: Teaching with wound moulage simulations in nursing education
Romaine Meichtry, Rabea Krings, Alina Klein, Monika Droz, Claudia Schlegel
Teaching and Learning in Nursing.2025; 20(3): e745. CrossRef
NursingXR: Advancing Nursing Education Through Virtual Reality-Based Training
Mohammad F. Obeid, Ahmed Ewais, Mohammad R. Asia
Applied Sciences.2025; 15(6): 2949. CrossRef
Simulation-Based Education and Nursing Student Learning Motivations: A Scoping Review
Keisuke Nojima, Ryosuke Yoshida, Seiichi Sato, Tomohide Fukuda, Kanae Kakinuma
Cureus.2025;[Epub] CrossRef
Realidad virtual y simulación clínica en la formación de enfermería: impacto en la educación y el desarrollo de habilidades clínicas
Mario Roberto Sate, María Eugenia Gonzalez, Carmen Graciela Mezacapo, Pablo Andrés Salgado, Gloria Ester Rodríguez
LATAM Revista Latinoamericana de Ciencias Sociales y Humanidades.2025;[Epub] CrossRef
The effect of immersive simulation-based learning on an anatomy program in nursing education: a quasi-experimental study
Lahoucine Ben Yahya, Mohamed Radid, Mohamed El Yaagoubi, Lahcen Elmoumou, Otmane Abouri, Aziz Naciri, Ghizlane Chemsi
Korean Journal of Medical Education.2025; 37(3): 281. CrossRef
Mixed reality versus manikins in basic life support simulation-based training for medical students in France: the mixed reality non-inferiority randomized controlled trial
Sofia Barlocco De La Vega, Evelyne Guerif-Dubreucq, Jebrane Bouaoud, Myriam Awad, Léonard Mathon, Agathe Beauvais, Thomas Olivier, Pierre-Clément Thiébaud, Anne-Laure Philippon
Journal of Educational Evaluation for Health Professions.2025; 22: 15. CrossRef
Examining jigsaw, metaverse, and classic learning methods' impact on nursing students' self-efficacy in pediatric drug administration: a mixed-methods study
Şerife Tutar, Hande Özgörü, Faruk Durna, Yurdagül Şahin
Journal of Health Sciences and Medicine.2025; 8(6): 1065. CrossRef
Simulation-Based Learning in Nursing Education: Current Evidence and Future Directions
Firas Khraisat, Ghada M. Khrais
Inquisiva Open.2025;[Epub] CrossRef
AI-guided interview simulation to improve employability and reduce anxiety in final-year nursing and midwifery students: A quasi-experimental study
Seda Sarıköse, Tuba Sengul, Betül Uncu, Holly Kirkland-Kyhn, Nurten Kaya
Clinical Simulation in Nursing.2025; 109: 101852. CrossRef
Application of Virtual Reality, Artificial Intelligence, and Other Innovative Technologies in Healthcare Education (Nursing and Midwifery Specialties): Challenges and Strategies
Galya Georgieva-Tsaneva, Ivanichka Serbezova, Silvia Beloeva
Education Sciences.2024; 15(1): 11. CrossRef

Research article

Equal Z standard-setting method to estimate the minimum number of panelists for a medical school’s objective structured clinical examination in Taiwan: a simulation study: Ying-Ying Yang, Pin-Hsiang Huang, Ling-Yu Yang, Chia-Chang Huang, Chih-Wei Liu, Shiau-Shian Huang, Chen-Huan Chen, Fa-Yauh Lee, Shou-Yen Kao, Boaz Shulruf; J Educ Eval Health Prof. 2022;19:27. Published online October 17, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.27

3,834 View
143 Download

Abstract PDF Supplementary Material: Purpose
Undertaking a standard-setting exercise is a common method for setting pass/fail cut scores for high-stakes examinations. The recently introduced equal Z standard-setting method (EZ method) has been found to be a valid and effective alternative for the commonly used Angoff and Hofstee methods and their variants. The current study aims to estimate the minimum number of panelists required for obtaining acceptable and reliable cut scores using the EZ method.
Methods
The primary data were extracted from 31 panelists who used the EZ method for setting cut scores for a 12-station of medical school’s final objective structured clinical examination (OSCE) in Taiwan. For this study, a new data set composed of 1,000 random samples of different panel sizes, ranging from 5 to 25 panelists, was established and analyzed. Analysis of variance was performed to measure the differences in the cut scores set by the sampled groups, across all sizes within each station.
Results
On average, a panel of 10 experts or more yielded cut scores with confidence more than or equal to 90% and 15 experts yielded cut scores with confidence more than or equal to 95%. No significant differences in cut scores associated with panel size were identified for panels of 5 or more experts.
Conclusion
The EZ method was found to be valid and feasible. Less than an hour was required for 12 panelists to assess 12 OSCE stations. Calculating the cut scores required only basic statistical skills.

Software report

Integration of computer-simulated practical exercises into undergraduate medical pharmacology education at Mulungushi University, Zambia: Christian Chinyere Ezeala; J Educ Eval Health Prof. 2020;17:8. Published online February 24, 2020; DOI: https://doi.org/10.3352/jeehp.2020.17.8

10,920 View
259 Download
4 Web of Science
5 Crossref

Abstract PDF

Purpose
This study was conducted to determine whether a computer simulation of practical exercises in undergraduate medical pharmacology led to the realization of the intended learning outcomes.
Methods
The study was a descriptive analysis of laboratory classes carried out using computer simulation programs. Five programs were used to teach practical pharmacology to undergraduate medical students at the Mulungushi University School of Medicine and Health Sciences. The study period was January 2018 to December 2019. The computer programs included a pharmacokinetics simulator (CyberPatient), organ bath simulator (OBSim), AutonomiCAL for simulating autonomic pharmacology, and Virtual Cat and Virtual Rat (RatCVS) for simulating cardiovascular pharmacology. Students utilized these programs during their pharmacology laboratory classes, wrote reports, and answered relevant clinical questions.
Results
The 5 programs provided easy and precise platforms for students to explore concepts and demonstrate knowledge of pharmacokinetics, pharmacodynamics, autonomic and cardiovascular pharmacology, and their clinical applications.
Conclusion
The programs were effective learning tools. Students’ learning was easily assessed based on their laboratory reports. Although the computer programs met medical students’ learning needs, wet laboratory exercises are also needed to meet the needs of students who require practical laboratory skills.

Citations

Citations to this article as recorded by

Review of the Simulators Used in Pharmacology Education and Statistical Models When Creating the Simulators
Toshiaki Ara, Hiroyuki Kitamura
Applied Biosciences.2025; 4(1): 6. CrossRef
Virtual rat web: A versatile simulation tool for pharmacology education in a variety of settings
Shiko Okabe, Taichiro Goto, Daisuke Hirayama, Yuhei Nishimura
European Journal of Pharmacology.2025; 997: 177618. CrossRef
The Impact of Learning Technologies on the Learning Environment of Medical Students in Africa: A Review of the Current Literature
Daniel Thomas, Soo Young Baik, Olivia Beazer, Faraz Sharif, Muhammad Raheel
Cureus.2025;[Epub] CrossRef
Quality and impact of pharmacology digital simulation education on pre-registration healthcare students: A systematic literature review
Sharad Rayamajhi, Alison Machin, Cathal Breen, Gdiom Gebreheat, Ruth Paterson
Nurse Education Today.2024; 140: 106295. CrossRef
Simulation as a Tool to Illustrate Clinical Pharmacology Concepts to Healthcare Program Learners
Liza Barbarello Andrews, Les Barta
Current Pharmacology Reports.2020; 6(4): 182. CrossRef

Research Article

Learning gain of pharmacy students after introducing guided inquiry learning with computer simulation in a pharmacology class in Fiji: Christian C Ezeala, Arnold A Ram, Napolioni Vulakouvaki; J Educ Eval Health Prof. 2013;10:9. Published online December 23, 2012; DOI: https://doi.org/10.3352/jeehp.2013.10.9

31,473 View
164 Download
12 Crossref

PDF

Citations

Citations to this article as recorded by

A systematic review on the use of virtual patient and computer-based simulation for experiential pharmacy education
Chamipa Phanudulkitti, Surangkana Puengrung, Rittnarong Meepong, Kathryn Vanderboll, Karen Bell Farris, Sarah E. Vordenberg
Exploratory Research in Clinical and Social Pharmacy.2023; 11: 100316. CrossRef
Simulation-Based Education Implementation in Pharmacy Curriculum: A Review of the Current Status
Ghazwa B Korayem, Omar A Alshaya, Sawsan M Kurdi, Lina I Alnajjar, Aisha F Badr, Amjaad Alfahed, Ameera Cluntun
Advances in Medical Education and Practice.2022; Volume 13: 649. CrossRef
Impact of the educational technology use in undergraduate pharmacy teaching and learning – A systematic review
Chooi Yeng Lee, Shaun Wen Huey Lee
Pharmacy Education.2021; 21: 159. CrossRef
Integration of computer-simulated practical exercises into undergraduate medical pharmacology education at Mulungushi University, Zambia
Christian Chinyere Ezeala
Journal of Educational Evaluation for Health Professions.2020; 17: 8. CrossRef
Simulation as a Tool to Illustrate Clinical Pharmacology Concepts to Healthcare Program Learners
Liza Barbarello Andrews, Les Barta
Current Pharmacology Reports.2020; 6(4): 182. CrossRef
Assessing the effectiveness of an online dental pharmacology course
Miguel A. Morales‐Pérez, Alba R. Muñoz‐Gómez, Gabriela Argumedo, José F. Gómez‐Clavel
Journal of Dental Education.2020; 84(8): 902. CrossRef
A Review of Research on Process Oriented Guided Inquiry Learning: Implications for Research and Practice
Jon-Marc G. Rodriguez, Kevin H. Hunter, Leah J. Scharlott, Nicole M. Becker
Journal of Chemical Education.2020; 97(10): 3506. CrossRef
Pharmacists’ Knowledge and Practice of Issues Related to Using Psychotropic Medication in Elderly People in Ethiopia: A Prospective Cross-Sectional Study
Gashaw Binega Mekonnen, Alemante Tafese Beyna
BioMed Research International.2020; 2020: 1. CrossRef
Examining the effectiveness of guided inquiry with problem-solving process and cognitive function training in a high school chemistry course
Niwat Tornee, Tassanee Bunterm, Kerry Lee, Supaporn Muchimapura
Pedagogies: An International Journal.2019; 14(2): 126. CrossRef
Curriculum integration of virtual patients
Karen Dahri, Kimberley MacNeil, Fong Chan, Emilie Lamoureux, Mattie Bakker, Katherine Seto, Janice Yeung
Currents in Pharmacy Teaching and Learning.2019; 11(12): 1309. CrossRef
Palestinian pharmacists’ knowledge of issues related to using psychotropic medications in older people: a cross-sectional study
Ramzi Shawahna, Mais Khaskiyyi, Hadeel Abdo, Yasmen Msarwe, Rania Odeh, Souad Salame
Journal of Educational Evaluation for Health Professions.2017; 14: 8. CrossRef
Peritoneal Dialysis University for Surgeons: A Peritoneal Access Training Program
John H. Crabtree, Todd Penner, Sean W. Armstrong, John Burkart
Peritoneal Dialysis International: Journal of the International Society for Peritoneal Dialysis.2016; 36(2): 177. CrossRef

First
Prev
Page of 1
Next
Last

Search