Journal of Educational Evaluation for Health Professions

Research articles

Reliability of a workplace-based assessment for the United States general surgical trainees’ intraoperative performance using multivariate generalizability theory: a psychometric study: Ting Sun, Stella Yun Kim, Brigitte Kristin Smith, Yoon Soo Park; J Educ Eval Health Prof. 2024;21:26. Published online September 24, 2024; DOI: https://doi.org/10.3352/jeehp.2024.21.26

1,236 View
204 Download
1 Web of Science
1 Crossref

Purpose
The System for Improving and Measuring Procedure Learning (SIMPL), a smartphone-based operative assessment application, was developed to assess the intraoperative performance of surgical residents. This study aims to examine the reliability of the SIMPL assessment and determine the optimal number of procedures for a reliable assessment.
Methods
In this retrospective observational study, we analyzed data collected between 2015 and 2023 from 4,616 residents across 94 General Surgery Residency programs in the United States that utilized the SIMPL smartphone application. We employed multivariate generalizability theory and initially conducted generalizability studies to estimate the variance components associated with procedures. We then performed decision studies to estimate the reliability coefficient and the minimum number of procedures required for a reproducible assessment.
Results
We estimated that the reliability of the assessment of surgical trainees’ intraoperative autonomy and performance using SIMPL exceeded 0.70. Additionally, the optimal number of procedures required for a reproducible assessment was 10, 17, 15, and 17 for postgraduate year (PGY) 2, PGY 3, PGY 4, and PGY 5, respectively. Notably, the study highlighted that the assessment of residents in their senior years necessitated a larger number of procedures compared to those in their junior years.
Conclusion
The study demonstrated that the SIMPL assessment is reliably effective for evaluating the intraoperative performance of surgical trainees. Adjusting the number of procedures based on the trainees’ training stage enhances the assessment process’s accuracy and effectiveness.

Citations

Citations to this article as recorded by

General Surgery Residents Competence and Autonomy in Core Vascular Surgery Procedures
Taylor M. Carter, M. Libby Weaver, Ting Sun, Brigitte Smith
Journal of Surgical Education.2025; 82(3): 103415. CrossRef

Effect of a smartphone-based online electronic logbook to evaluate the clinical skills of nurse anesthesia students in Iran: a randomized controlled study: Ali Khalafi, Nahid Jamshidi, Nasrin Khajeali, Saeed Ghanbari; J Educ Eval Health Prof. 2023;20:10. Published online March 31, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.10

3,170 View
146 Download

Abstract PDF Supplementary Material: Purpose
This study was conducted to evaluate a smartphone-based online electronic logbook used to assess the clinical skills of nurse anesthesia students in Iran.
Methods
This randomized controlled study was conducted after tool development at Ahvaz Jundishapur University of Medical Sciences in Ahvaz, Iran from January 2022 to December 2022. The online electronic logbook involved in this study was an Android-compatible application used to evaluate the clinical skills of nurse anesthesia students. In the implementation phase, the online electronic logbook was piloted for 3 months in anesthesia training in comparison with a paper logbook. For this purpose, 49 second- and third-year anesthesia nursing students selected using the census method were assigned to intervention (online electronic logbook) and control (paper logbook) groups. The online electronic logbook and paper logbook were compared in terms of student satisfaction and learning outcomes.
Results
A total of 39 students participated in the study. The mean satisfaction score of the intervention group was significantly higher than that of the control group (P=0.027). The mean score of learning outcomes was also significantly higher for the intervention than the control group (P=0.028).
Conclusion
Smartphone technology can provide a platform for improving the evaluation of the clinical skills of nursing anesthesia students, leading to increased satisfaction and improved learning outcomes.

No difference in factual or conceptual recall comprehension for tablet, laptop, and handwritten note-taking by medical students in the United States: a survey-based observational study: Warren Wiechmann, Robert Edwards, Cheyenne Low, Alisa Wray, Megan Boysen-Osborn, Shannon Toohey; J Educ Eval Health Prof. 2022;19:8. Published online April 26, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.8

14,347 View
604 Download
2 Web of Science
2 Crossref

Abstract PDF Supplementary Material

Purpose
Technological advances are changing how students approach learning. The traditional note-taking methods of longhand writing have been supplemented and replaced by tablets, smartphones, and laptop note-taking. It has been theorized that writing notes by hand requires more complex cognitive processes and may lead to better retention. However, few studies have investigated the use of tablet-based note-taking, which allows the incorporation of typing, drawing, highlights, and media. We therefore sought to confirm the hypothesis that tablet-based note-taking would lead to equivalent or better recall as compared to written note-taking.
Methods
We allocated 68 students into longhand, laptop, or tablet note-taking groups, and they watched and took notes on a presentation on which they were assessed for factual and conceptual recall. A second short distractor video was shown, followed by a 30-minute assessment at the University of California, Irvine campus, over a single day period in August 2018. Notes were analyzed for content, supplemental drawings, and other media sources.
Results
No significant difference was found in the factual or conceptual recall scores for tablet, laptop, and handwritten note-taking (P=0.61). The median word count was 131.5 for tablets, 121.0 for handwriting, and 297.0 for laptops (P=0.01). The tablet group had the highest presence of drawing, highlighting, and other media/tools.
Conclusion
In light of conflicting research regarding the best note-taking method, our study showed that longhand note-taking is not superior to tablet or laptop note-taking. This suggests students should be encouraged to pick the note-taking method that appeals most to them. In the future, traditional note-taking may be replaced or supplemented with digital technologies that provide similar efficacy with more convenience.

Citations

Citations to this article as recorded by

Typed Versus Handwritten Lecture Notes and College Student Achievement: A Meta-Analysis
Abraham E. Flanigan, Jordan Wheeler, Tiphaine Colliot, Junrong Lu, Kenneth A. Kiewra
Educational Psychology Review.2024;[Epub] CrossRef
Evaluating the use of note-taking app by Japanese resident physicians: A nationwide cross-sectional study (Preprint)
Taiju Miyagami, Yuji Nishizaki, Taro Shimizu, Yu Yamamoto, Kiyoshi Shikino, Koshi Kataoka, Masanori Nojima, Gautam. A Deshpande, Toshio Naito, Yasuharu Tokuda
JMIR Formative Research.2023;[Epub] CrossRef

Case report

Dental students’ learning attitudes and perceptions of YouTube as a lecture video hosting platform in a flipped classroom in Korea: Chang Wan Seo, A Ra Cho, Jung Chul Park, Hag Yeon Cho, Sun Kim; J Educ Eval Health Prof. 2018;15:24. Published online October 11, 2018; DOI: https://doi.org/10.3352/jeehp.2018.15.24

29,222 View
393 Download
17 Web of Science
18 Crossref

Abstract PDF Supplementary Material

Purpose
The aim of this study was to confirm the applicability of YouTube as a delivery platform of lecture videos for dental students and to assess their learning attitudes towards the flipped classroom model.
Methods
Learning experiences after using the YouTube platform to deliver preliminary video lectures in a flipped classroom were assessed by 69 second-year students (52 males, 17 females) at Dankook University College of Dentistry, Korea, who attended periodontology lectures during 2 consecutive semesters of the 2016 academic year. The instructor uploaded the lecture videos to YouTube before each class. At the end of the second semester, the students were surveyed using a questionnaire devised by the authors.
Results
Of the students, 53 (76.8%) always watched the lecture before the class, 48 (69.6%) used their smartphones, and 66 (95.7%) stated that they watched the lectures at home. The majority of the students replied that the video lectures were easier to understand than face to face lectures (82.6%) and that they would like to view the videos again after graduation (73.9%).
Conclusion
Our results indicate that YouTube is an applicable platform to deliver video lectures and to expose students to increased learning opportunities.

Citations

Citations to this article as recorded by

Social Media in Oral Health Education: A Scoping Review
Zuzanna Apel, Nathalia Carolina Fernandes Fagundes, Nazlee Sharmin, Usama Nassar, Gordon Gow, Dominic Apel, Arnaldo Perez
European Journal of Dental Education.2025; 29(1): 50. CrossRef
Does forced-shift to online learning affect university brand image in South Korea? Role of perceived harm and international students’ learning engagement
Umer Zaman, Murat Aktan, Hasnan Baber, Shahid Nawaz
Journal of Marketing for Higher Education.2024; 34(1): 390. CrossRef
YouTube as a source of information about rubber dam: quality and content analysis
Gülsen Kiraz, Arzu Kaya Mumcu, Safa Kurnaz
Restorative Dentistry & Endodontics.2024;[Epub] CrossRef
Use of social media by dental students: A comparative study
Rand Al-Obaidi
Clinical Epidemiology and Global Health.2024; 26: 101559. CrossRef
Structure, governance and delivery of specialist training programs in periodontology and implant dentistry
Moshe Goldstein, Nikolaos Donos, Wim Teughels, Nikolaos Gkranias, Andy Temmerman, Jan Derks, Bahar Eren Kuru, Maria Clotilde Carra, Ana Belen Castro, Xanthippi Dereka, Christel Dekeyser, David Herrera, Katleen Vandamme, Elena Calciolari
Journal of Clinical Periodontology.2024; 51(S27): 55. CrossRef
Evaluating video‐based lectures on YouTube for dental education
Ryan T. Gross, Nare Ghaltakhchyan, Eleanor M. Nanney, Tate H. Jackson, Christopher A. Wiesen, Paul Mihas, Adam M. Persky, Sylvia A. Frazier‐Bowers, Laura A. Jacox
Orthodontics & Craniofacial Research.2023; 26(S1): 210. CrossRef
Learning of paediatric dentistry with the flipped classroom model
Nuria E. Gallardo, Antonia M. Caleya, Maria Esperanza Sánchez, Gonzalo Feijóo
European Journal of Dental Education.2022; 26(2): 302. CrossRef
Effects of Video Length on a Flipped English Classroom
Zhonggen Yu, Mingle Gao
Sage Open.2022;[Epub] CrossRef
An Evaluation of the Usefulness of YouTube® Videos on Crown Preparation
Syed Rashid Habib, Aleshba Saba Khan, Mohsin Ali, Essam Abdulla Abutheraa, Ahmad khaled alkhrayef, Faisal Jibrin Aljibrin, Nawaf Saad Almutairi, Ammar A. Siddiqui
BioMed Research International.2022;[Epub] CrossRef
Perceptions of Students on Distance Education and E-Learning in Dentistry Education: Challenges and Opportunities
Ayşe TORAMAN, Ebru SAĞLAM, Serhat KÖSEOĞLU
Journal of Biotechnology and Strategic Health Research.2022; 6(2): 101. CrossRef
Social Media as a Learning Tool for the Budding Periodontist
Riddhi Awasthi, Balaji Manohar, S Vinay, Santosh Kumar
Advances in Human Biology.2022; 12(3): 286. CrossRef
YouTube and Education: A Scoping Review
Abdulhadi Shoufan, Fatma Mohamed
IEEE Access.2022; 10: 125576. CrossRef
Uso de la plataforma YouTube® por los estudiantes de odontología: Revisión de alcance
María Luján Méndez Bauer, Stella de los Angeles Bauer Walter
Universitas Odontologica.2022;[Epub] CrossRef
Social media as a learning tool: Dental students’ perspectives
Mona T. Rajeh, Shahinaz N. Sembawa, Afnan A. Nassar, Seba A. Al Hebshi, Khalid T. Aboalshamat, Mohammed K. Badri
Journal of Dental Education.2021; 85(4): 513. CrossRef
Social Media Usage among Dental Undergraduate Students—A Comparative Study
Eswara Uma, Pentti Nieminen, Shani Ann Mani, Jacob John, Emilia Haapanen, Marja-Liisa Laitala, Olli-Pekka Lappalainen, Eby Varghase, Ankita Arora, Kanwardeep Kaur
Healthcare.2021; 9(11): 1408. CrossRef
Flipped Classroom Experiences in Clinical Dentistry – A Strategic Mini-Review
Abdullah Aljabr
The Open Dentistry Journal.2021; 15(1): 717. CrossRef
Newly appointed medical faculty members’ self-evaluation of their educational roles at the Catholic University of Korea College of Medicine in 2020 and 2021: a cross-sectional survey-based study
Sun Kim, A Ra Cho, Chul Woon Chung
Journal of Educational Evaluation for Health Professions.2021; 18: 28. CrossRef
Attitudes toward Social Media among Practicing Dentists and Dental Students in Clinical Years in Saudi Arabia
Khalid Aboalshamat, Sharifah Alkiyadi, Sarah Alsaleh, Rana Reda, Sharifa Alkhaldi, Arwa Badeeb, Najwa Gabb
The Open Dentistry Journal.2019; 13(1): 143. CrossRef

Research Article

Smartphone-based evaluations of clinical placements—a useful complement to web-based evaluation tools: Jesper Hessius, Jakob Johansson; J Educ Eval Health Prof. 2015;12:55. Published online November 30, 2015; DOI: https://doi.org/10.3352/jeehp.2015.12.55

28,040 View
144 Download
2 Web of Science
5 Crossref

Abstract PDF

Purpose
Web-based questionnaires are currently the standard method for course evaluations. The high rate of smartphone adoption in Sweden makes possible a range of new uses, including course evaluation. This study examines the potential advantages and disadvantages of using a smartphone app as a complement to web-based course evaluation systems.
Methods
An iPhone app for course evaluations was developed and interfaced to an existing web-based tool. Evaluations submitted using the app were compared with those submitted using the web between August 2012 and June 2013, at the Faculty of Medicine at Uppsala University, Sweden.
Results
At the time of the study, 49% of the students were judged to own iPhones. Over the course of the study, 3,340 evaluations were submitted, of which 22.8% were submitted using the app. The median of mean scores in the submitted evaluations was 4.50 for the app (with an interquartile range of 3.70-5.20) and 4.60 (3.70-5.20) for the web (P= 0.24). The proportion of evaluations that included a free-text comment was 50.5% for the app and 49.9% for the web (P= 0.80).
Conclusion
An app introduced as a complement to a web-based course evaluation system met with rapid adoption. We found no difference in the frequency of free-text comments or in the evaluation scores. Apps appear to be promising tools for course evaluations. web-based course evaluation system met with rapid adoption. We found no difference in the frequency of free-text comments or in the evaluation scores. Apps appear to be promising tools for course evaluations.

Citations

Citations to this article as recorded by

Practical tips for starting a successful national postgraduate course
Magnus Sundbom
MedEdPublish.2024; 13: 26. CrossRef
Practical tips for starting a successful national postgraduate course
Magnus Sundbom
MedEdPublish.2024; 13: 26. CrossRef
Practical tips for starting a successful national postgraduate course
Magnus Sundbom
MedEdPublish.2023; 13: 26. CrossRef
Practical tips for starting a successful national postgraduate course
Magnus Sundbom
MedEdPublish.2023; 13: 26. CrossRef
Enhancing emergency care in low-income countries using mobile technology-based training tools
Hilary Edgcombe, Chris Paton, Mike English
Archives of Disease in Childhood.2016; 101(12): 1149. CrossRef

First
Prev
Page of 1
Next
Last

Search