Journal of Educational Evaluation for Health Professions

Research articles

Development and validity evidence for the resident-led large group teaching assessment instrument in the United States: a methodological study: Ariel Shana Frey-Vogel, Kristina Dzara, Kimberly Anne Gifford, Yoon Soo Park, Justin Berk, Allison Heinly, Darcy Wolcott, Daniel Adam Hall, Shannon Elliott Scott-Vernaglia, Katherine Anne Sparger, Erica Ye-pyng Chung; J Educ Eval Health Prof. 2024;21:3. Published online February 23, 2024; DOI: https://doi.org/10.3352/jeehp.2024.21.3

344 View
103 Download

Abstract PDF Supplementary Material: Purpose
Despite educational mandates to assess resident teaching competence, limited instruments with validity evidence exist for this purpose. Existing instruments do not allow faculty to assess resident-led teaching in a large group format or whether teaching was interactive. This study gathers validity evidence on the use of the Resident-led Large Group Teaching Assessment Instrument (Relate), an instrument used by faculty to assess resident teaching competency. Relate comprises 23 behaviors divided into 6 elements: learning environment, goals and objectives, content of talk, promotion of understanding and retention, session management, and closure.
Methods
Messick’s unified validity framework was used for this study. Investigators used video recordings of resident-led teaching from 3 pediatric residency programs to develop Relate and a rater guidebook. Faculty were trained on instrument use through frame-of-reference training. Resident teaching at all sites was video-recorded during 2018–2019. Two trained faculty raters assessed each video. Descriptive statistics on performance were obtained. Validity evidence sources include: rater training effect (response process), reliability and variability (internal structure), and impact on Milestones assessment (relations to other variables).
Results
Forty-eight videos, from 16 residents, were analyzed. Rater training improved inter-rater reliability from 0.04 to 0.64. The Φ-coefficient reliability was 0.50. There was a significant correlation between overall Relate performance and the pediatric teaching Milestone (r=0.34, P=0.019).
Conclusion
Relate provides validity evidence with sufficient reliability to measure resident-led large-group teaching competence.

Medical residents and attending physicians’ perceptions of feedback and teaching in the United States: a qualitative study: Madeleine Matthiesen, Michael S. Kelly, Kristina Dzara, Arabella Simpkin Begin; J Educ Eval Health Prof. 2022;19:9. Published online April 26, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.9

8,244 View
354 Download
1 Web of Science
1 Crossref

Purpose
Residents and attendings agree on the importance of feedback to resident education. However, while faculty report providing frequent feedback, residents often do not perceive receiving it, particularly in the context of teaching. Given the nuanced differences between feedback and teaching, we aimed to explore resident and attending perceptions of feedback and teaching in the clinical setting.
Methods
We conducted a qualitative study of internal medicine residents and attendings from December 2018 through March 2019 at the Massachusetts General Hospital to investigate perceptions of feedback in the inpatient clinical setting. Residents and faculty were recruited to participate in focus groups. Data were analyzed using thematic analysis to explore perspectives and barriers to feedback provision and identification.
Results
Five focus groups included 33 total participants in 3 attending (n=20) and 2 resident (n=13) groups. Thematic analysis of focus group transcripts identified 7 themes which organized into 3 thematic categories: (1) disentangling feedback and teaching, (2) delivering high-quality feedback, and (3) experiencing feedback in the group setting. Residents and attendings highlighted important themes in discriminating feedback from teaching. They indicated that while feedback is reactive in response to an action or behavior, teaching is proactive and oriented toward future endeavors.
Conclusion
Confusion between the critical concepts of teaching and feedback may be minimized by allowing them to each have their intended impact, either in response to prior events or aimed toward those yet to take place.

Citations

Citations to this article as recorded by

Resident Assessment of Clinician Educators According to Core ACGME Competencies
Bailey A. Pope, Patricia A. Carney, Mary C. Brooks, Doug R. Rice, Ashly A. Albright, Stephanie A. C. Halvorson
Journal of General Internal Medicine.2024; 39(3): 377. CrossRef

Trainees’ perceptions and expectations of formal academic mentoring during the COVID-19 pandemic in Indonesian cardiology residency programs: Sunu Budhi Raharjo, Rita Mustika, Aida Lydia, Mefri Yanni, Heru Sulastomo, Rahma Tsania Zhuhra, Celly Anantaria Atmadikoesoemah; J Educ Eval Health Prof. 2021;18:19. Published online August 9, 2021; DOI: https://doi.org/10.3352/jeehp.2021.18.19

4,763 View
316 Download
1 Web of Science
1 Crossref

Abstract PDF Supplementary Material

Purpose
During medical residency programs, physicians develop their professional identities as specialists and encounter high expectations in terms of achieving competencies. The responsibilities of medical trainees include caring for patients, balancing work with personal life, and weathering stress, depression, and burnout. Formal academic mentoring programs strive to ease these burdens. The coronavirus disease 2019 (COVID-19) pandemic has altered the trainee–academic mentor relationship, and solutions are needed to address these challenges. The present study aimed to evaluate the formal academic mentoring process through trainees’ perceptions and expectations of formal mentoring programs during COVID-19 in Indonesian cardiology residency programs.
Methods
This cross-sectional study used a self-administered online questionnaire to capture trainees’ perceptions and expectations regarding academic mentoring programs in 3 cardiology residency programs in Indonesia from October to November 2020. The questionnaire was developed before data collection. Perceptions of the existing mentoring programs were compared with expectations.
Results
Responses were gathered from 169 out of 174 residents (response rate, 97.3%). Most trainees reported having direct contact with COVID-19 patients (88.82%). They stated that changes had taken place in the mode and frequency of communication with their academic advisors during the pandemic. Significant differences were found between trainees’ perceptions of the existing mentoring programs and their expectations for academic mentoring programs (P<0.001).
Conclusion
Despite the challenges of interacting with their academic mentors, trainees still perceived academic mentors as a vital resource. Study programs need to consider trainees’ expectations when designing academic mentoring programs.

Citations

Citations to this article as recorded by

Impact of the COVID-19 pandemic on career intention amongst undergraduate medical students: a single-centre cross-sectional study conducted in Hubei Province
Xue-lin Wang, Ming-xiu Liu, Shuai Peng, Lei Yang, Chen Lu, Shi-cong Shou, Jian-ru Wang, Jun-yi Sun, Jia-qi Wang, Yan Hu, Jun Zhao, Peng Duan
BMC Medical Education.2022;[Epub] CrossRef

Changes in the working conditions and learning environment of medical residents after the enactment of the Medical Resident Act in Korea in 2015: a national 4-year longitudinal study: Sangho Sohn, Yeonjoo Seo, Yunsik Jeong, Seungwoo Lee, Jeesun Lee, Kyung Ju Lee; J Educ Eval Health Prof. 2021;18:7. Published online April 20, 2021; DOI: https://doi.org/10.3352/jeehp.2021.18.7

5,942 View
290 Download
5 Crossref

Abstract PDF Supplementary Material

Purpose
In 2015, the South Korean government legislated the Act for the Improvement of Training Conditions and Status of Medical Residents (Medical Resident Act). This study investigated changes in the working and learning environment pre- and post-implementation of the Medical Resident Act in 2017, as well as changes in training conditions by year post-implementation.
Methods
An annual cross-sectional voluntary survey was conducted by the Korean Intern Resident Association (KIRA) between 2016 and 2019. The learning and working environment, including extended shift length, rest time, learning goals, and job satisfaction, were compared by institution type, training year, and specialty.
Results
Of the 55,727 enrollees in the KIRA, 15,029 trainees took the survey, and the number of survey participants increased year by year (from 2,984 in 2016 to 4,700 in 2019). Overall working hours tended to decrease; however, interns worked the most (114 hours in 2016, 88 hours in 2019; P<0.001). Having 10 hours or more of break time has gradually become more common (P<0.001). Lunch breaks per week decreased from 5 in 2017 to 4 in 2019 (P<0.001). Trainees’ sense of educational deprivation due to physician assistants increased from 17.5% in 2016 to 25.6% in 2018 (P<0.001). Awareness of tasks and program/work achievement goals increased from 29.2% in 2016 to 58.3% in 2018 (P<0.001). Satisfaction with the learning environment increased over time, whereas satisfaction with working conditions varied.
Conclusion
The Medical Resident Act has brought promising changes to the training of medical residents in Korea, as well as their satisfaction with the training environment.

Citations

Citations to this article as recorded by

Developing and Establishing a Wound Dressing Team: Experience and Recommendations
Sik Namgoong, Seunghee Baik, Seung-Kyu Han, Ji-Won Son, Jae-Yeon Kim
Journal of Korean Medical Science.2023;[Epub] CrossRef
The effects of resident work hours on well‐being, performance, and education: A review from a Japanese perspective
Kazuya Nagasaki, Hiroyuki Kobayashi
Journal of General and Family Medicine.2023; 24(6): 323. CrossRef
Developing prompts from large language model for extracting clinical information from pathology and ultrasound reports in breast cancer
Hyeon Seok Choi, Jun Yeong Song, Kyung Hwan Shin, Ji Hyun Chang, Bum-Sup Jang
Radiation Oncology Journal.2023; 41(3): 209. CrossRef
Shortening shift’s length—Should we ask the residents if this is what they want?
Yehuda Hershkovitz, Adi Rasco, Orna Tal, David C. Mohr
PLOS ONE.2022; 17(8): e0272548. CrossRef
The Number of Monthly Night Shift Days and Depression Were Associated with an Increased Risk of Excessive Daytime Sleepiness in Emergency Physicians in South Korea
Song Yi Park, Hyung Min Lee, Jiyoung Kim
Behavioral Sciences.2022; 12(8): 279. CrossRef

Training in statistical analysis reduces the framing effect among medical students and residents in Argentina: Raúl Alfredo Borracci, Eduardo Benigno Arribalzaga, Jorge Thierer; J Educ Eval Health Prof. 2020;17:25. Published online September 1, 2020; DOI: https://doi.org/10.3352/jeehp.2020.17.25

4,495 View
129 Download
1 Web of Science
1 Crossref

Abstract PDF Supplementary Material

Purpose
The framing effect refers to a phenomenon wherein, when the same problem is presented using different representations of information, people make significant changes in their decisions. This study aimed to explore whether the framing effect could be reduced in medical students and residents by teaching them the statistical concepts of effect size, probability, and sampling for use in the medical decision-making process.
Methods
Ninety-five second-year medical students and 100 second-year medical residents of Austral University and Buenos Aires University, Argentina were invited to participate in the study between March and June 2017. A questionnaire was developed to assess the different types of framing effects in medical situations. After an initial administration of the survey, students and residents were taught statistical concepts including effect size, probability, and sampling during 2 individual independent official biostatistics courses. After these interventions, the same questionnaire was randomly administered again, and pre- and post-intervention outcomes were compared among students and residents.
Results
Almost every type of framing effect was reproduced either in the students or in the residents. After teaching medical students and residents the analytical process behind statistical concepts, a significant reduction in sample-size, risky-choice, pseudo-certainty, number-size, attribute, goal, and probabilistic formulation framing effects was observed.
Conclusion
The decision-making of medical students and residents in simulated medical situations may be affected by different frame descriptions, and these framing effects can be partially reduced by training individuals in probability analysis and statistical sampling methods.

Citations

Citations to this article as recorded by

Numeracy Education for Health Care Providers: A Scoping Review
Casey Goldstein, Nicole Woods, Rebecca MacKinnon, Rouhi Fazelzad, Bhajan Gill, Meredith Elana Giuliani, Tina Papadakos, Qinge Wei, Janet Papadakos
Journal of Continuing Education in the Health Professions.2024; 44(1): 35. CrossRef

Can incoming United States pediatric interns be entrusted with the essential communication skills of informed consent?: Nicholas Sevey, Michelle Barratt, Emma Omoruyi; J Educ Eval Health Prof. 2020;17:18. Published online June 29, 2020; DOI: https://doi.org/10.3352/jeehp.2020.17.18

4,617 View
126 Download

Abstract PDF Supplementary Material: Purpose
According to the entrustable professional activities (EPA) for entering residency by the Association of American Medical Colleges, incoming residents are expected to independently obtain informed consent for procedures they are likely to perform. This requires residents to not only inform their patients but to ensure comprehension of that information. We assessed the communication skills demonstrated by 372 incoming pediatric interns between 2007 and 2018 at the University of Texas Health Science Center at Houston, obtaining informed consent for a lumbar puncture.
Methods
During a simulated case in which interns were tasked with obtaining informed consent for a lumbar puncture, a standardized patient evaluated interns by rating 7 communication-based survey items using 5-point Likert scale from “poor” to “excellent.” We then converted the scale to a numerical system and calculated intern proficiency scores (sum of ratings for each resident) and average item performance (average item rating across all interns).
Results
Interns received an average rating of 21.6 per 28 maximum score, of which 227 interns (61.0%) achieved proficiency by scoring 21 or better. Notable differences were observed when comparing groups before and after EPA implementation (76.97% vs. 47.0% proficient, respectively). Item-level analysis showed that interns struggled most to conduct the encounter in a warm and friendly manner and encourage patients to ask questions (average ratings of 2.97/4 and 2.98/4, respectively). Interns excelled at treating the patient with respect and actively listening to questions (average ratings of 3.16, each). Both average intern proficiency scores and each average item ratings were significantly lower following EPA implementation (P<0.001).
Conclusion
Interns demonstrated moderate proficiency in communicating informed consent, though clear opportunities for improvement exist such as demonstrating warmth and encouraging questions.

Use of graded responsibility and common entrustment considerations among United States emergency medicine residency programs: Jason Lai, Benjamin Holden Schnapp, David Simon Tillman, Mary Westergaard, Jamie Hess, Aaron Kraut; J Educ Eval Health Prof. 2020;17:11. Published online April 20, 2020; DOI: https://doi.org/10.3352/jeehp.2020.17.11

5,548 View
95 Download
1 Web of Science
2 Crossref

Abstract PDF Supplementary Material

Purpose
The Accreditation Council for Graduate Medical Education (ACGME) requires all residency programs to provide increasing autonomy as residents progress through training, known as graded responsibility. However, there is little guidance on how to implement graded responsibility in practice and a paucity of literature on how it is currently implemented in emergency medicine (EM). We sought to determine how EM residency programs apply graded responsibility across a variety of activities and to identify which considerations are important in affording additional responsibilities to trainees.
Methods
We conducted a cross-sectional study of EM residency programs using a 23-question survey that was distributed by email to 162 ACGME-accredited EM program directors. Seven different domains of practice were queried.
Results
We received 91 responses (56.2% response rate) to the survey. Among all domains of practice except for managing critically ill medical patients, the use of graded responsibility exceeded 50% of surveyed programs. When graded responsibility was applied, post-graduate year (PGY) level was ranked an “extremely important” or “very important” consideration between 80.9% and 100.0% of the time.
Conclusion
The majority of EM residency programs are implementing graded responsibility within most domains of practice. When decisions are made surrounding graded responsibility, programs still rely heavily on the time-based model of PGY level to determine advancement.

Citations

Citations to this article as recorded by

Do you see what I see?: exploring trends in organizational culture perceptions across residency programs
Jennifer H. Chen, Paula Costa, Aimee Gardner
Global Surgical Education - Journal of the Association for Surgical Education.2024;[Epub] CrossRef
Guiding Fellows to Independent Practice
Maybelle Kou, Aline Baghdassarian, Kajal Khanna, Nazreen Jamal, Michele Carney, Daniel M. Fein, In Kim, Melissa L. Langhan, Jerri A. Rose, Noel S. Zuckerbraun, Cindy G. Roskind
Pediatric Emergency Care.2022; 38(10): 517. CrossRef

Effects of virtual reality training on decreasing the rates of needlestick or sharp injury in new-coming medical and nursing interns in Taiwan: Szu-Hsien Wu, Chia-Chang Huang, Shiau-Shian Huang, Ying-Ying Yang, Chih-Wei Liu, Boaz Shulruf, Chen-Huan Chen; J Educ Eval Health Prof. 2020;17:1. Published online January 20, 2020; DOI: https://doi.org/10.3352/jeehp.2020.17.1

10,992 View
411 Download
33 Web of Science
30 Crossref

Abstract PDF Supplementary Material

Purpose
Senior nursing and medical interns’ lack of familiarity and confidence with respect to practicing universal precaution for the prevention of occupational needlestick or sharp injuries may harm themselves. Trainees’ self-reported needlestick or sharp injury rate was known to be especially high during the first 2 months of internship in Taiwan. This prospective cohort study aimed to assess the effect of newly developed virtual reality (VR) game, which uses Gagne’s learning model to improve universal precaution for needlestick or sharp injury prevention and decrease the rates of needle stick or sharp injuries in new-coming medical and nursing interns in Taiwan.
Methods
From 2017 to 2019, the VR system was developed and applied in training of 59 new-coming nursing and 50 medical interns. Occupational needlestick or sharp injury prevention was sought to be achieved through a game of right and wrong choices for safe or unsafe universal precaution behaviors.
Results
In comparison with medical interns, a higher proportion of nursing interns had past experiences of deep occupational needlestick or sharp injury. Before VR training, the familiarity and confidence for needlestick or sharp injury prevention were higher among nursing interns than medical interns. Trainees with past experiences of deep needlestick or sharp injury exhibited better performance on the accuracy rate and time needed to complete 20 decisions than those without past experiences in VR practice. All trainees showed an improved performance after VR training. A high proportion of trainees reported that the VR-based training significantly decreased their anxiety about needlestick or sharp injury prevention.
Conclusion
This self-developed VR game system using Gagne’s flow improved universal precaution for needlestick or sharp injury prevention and reduced the needlestick or sharp injury rates in the first 2 months of nursing and medical internship.

Citations

Citations to this article as recorded by

Impact of an immersive virtual reality simulator education program on nursing students' intravenous injection administration: A mixed methods study
Yi-Ya Chang, Li-Fen Chao, Wen Chang, Chih-Ming Lin, Yi-Hsing Lee, Abigail Latimer, Misook L. Chung
Nurse Education Today.2024; 132: 106002. CrossRef
Positive Aspects and Potential Drawbacks of Implementing Digital Teaching/Learning Scenarios in Health Professions Using Nursing Education as an Example: A Research Report from Germany
Lydia Pfeifer, Sophia Fries, Alexander Stirner, Lisa Nagel, Christian Cohnen, Leona Aschentrup, Marleen Schönbeck, Annette Nauerth, Patrizia Raschper, Tim Herzig, Kamil J. Wrona
Nursing Reports.2024; 14(1): 468. CrossRef
Combatting the occurrence of needle-stick injuries in a medical school: why is it still an issue?
Franca Keicher, Janina Zirkel, Tobias Leutritz, Sarah König
BMC Medical Education.2024;[Epub] CrossRef
Virtual Reality Head-Mounted Displays in Medical Education
Thomas Tursø-Finnich, Rune Overgaard Jensen, Lasse X. Jensen, Lars Konge, Ebbe Thinggaard
Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare.2023; 18(1): 42. CrossRef
Review of literature: Knowledge and practice of standard precautions by nursing student and teaching techniques used in training
Stephanie Bouget Mohammedi, Caroline Landelle
American Journal of Infection Control.2023; 51(5): 574. CrossRef
Mild simulator sickness can alter heart rate variability, mental workload, and learning outcomes in a 360° virtual reality application for medical education: a post hoc analysis of a randomized controlled trial
Li-Jen Hsin, Yi-Ping Chao, Hai-Hua Chuang, Terry B. J. Kuo, Cheryl C. H. Yang, Chung-Guei Huang, Chung-Jan Kang, Wan-Ni Lin, Tuan-Jen Fang, Hsueh-Yu Li, Li-Ang Lee
Virtual Reality.2023; 27(4): 3345. CrossRef
Exploring Key Factors Influencing Nursing Students’ Cognitive Load and Willingness to Serve Older Adults: Cross-sectional Descriptive Correlational Study
Pei-Lun Hsieh, Yu-Rung Wang, Tien-Chi Huang
JMIR Serious Games.2023; 11: e43203. CrossRef
The Development and Pilot Testing of Immersive Virtual Reality Simulation Training for Prelicensure Nursing Students: A Quasi-Experimental Study
Bethany Cieslowski, Tanya Haas, Kyeung Mi Oh, Kathleen Chang, Cheryl A. Oetjen
Clinical Simulation in Nursing.2023; 77: 6. CrossRef
Metaverse applied to musculoskeletal pathology: Orthoverse and Rehabverse
Juan M. Román-Belmonte, E. Carlos Rodríguez-Merchán, Hortensia De la Corte-Rodríguez
Postgraduate Medicine.2023; 135(5): 440. CrossRef
Evaluating the effectiveness of virtual reality for safety-relevant training: a systematic review
Hans Stefan, Michael Mortimer, Ben Horan
Virtual Reality.2023; 27(4): 2839. CrossRef
Virtual reality simulation for nursing education: effectiveness and feasibility
Debra Kiegaldie, Louise Shaw
BMC Nursing.2023;[Epub] CrossRef
Digital Transformation in Nursing Education: A Systematic Review on Computer-Aided Nursing Education Pedagogies, Recent Advancements and Outlook on the Post-COVID-19 Era
Nevena Kostadinova Dicheva, Ikram Ur Rehman, Aamir Anwar, Moustafa M. Nasralla, Laden Husamaldin, Sama Aleshaiker
IEEE Access.2023; 11: 135659. CrossRef
Assessment of preclinical learning using virtual reality based education for nursing students
Sofía Martín Rodríguez, Marcial F Parrilla, Víctor Álvarez-García, Juan Ramón Pérez-Pérez, María Del Mar Fernández-Álvarez, Rubén Martín-Payo
Educational Media International.2023; 60(3-4): 209. CrossRef
How, for Whom, and in Which Contexts or Conditions Augmented and Virtual Reality Training Works in Upskilling Health Care Workers: Realist Synthesis
Norina Gasteiger, Sabine N van der Veer, Paul Wilson, Dawn Dowding
JMIR Serious Games.2022; 10(1): e31644. CrossRef
Application of computer-based testing in the Korean Medical Licensing Examination, the emergence of the metaverse in medical education, journal metrics and statistics, and appreciation to reviewers and volunteers
Sun Huh
Journal of Educational Evaluation for Health Professions.2022; 19: 2. CrossRef
A Systematic Literature Review of Virtual, Augmented, and Mixed Reality Game Applications in Healthcare
Yu Fu, Yan Hu, Veronica Sundstedt
ACM Transactions on Computing for Healthcare.2022; 3(2): 1. CrossRef
Effectiveness of Immersive Virtual Reality in Nursing Education
Jeeyae Choi, C Elise Thompson, Jeungok Choi, Colette B. Waddill, Soyoung Choi
Nurse Educator.2022; 47(3): E57. CrossRef
Systematic Bibliometric Analysis of Research Hotspots and Trends on the Application of Virtual Reality in Nursing
Junqiang Zhao, Yi Lu, Fujun Zhou, Ruping Mao, Fangqin Fei
Frontiers in Public Health.2022;[Epub] CrossRef
Development and Pilot Testing of a Non-stress Test Virtual Reality Simulator
Li-Fen Chao, Tsuey-Yuan Huang, Debra K. Moser, Fen-Fang Chung, Ying-Tung Lau, Xaviera Xiao
CIN: Computers, Informatics, Nursing.2022; 40(6): 357. CrossRef
The Effect of a Child Model on Breath-Sounds Examination Skills and Satisfaction on Nursing Students
Silpthai Thamruangrit, Sermsri Santati, Jumpee Granger, Dongruethai Buadong, Jatuporn Thongsri
Healthcare.2022; 10(7): 1165. CrossRef
Theories informing technology enhanced learning in nursing and midwifery education: A systematic review and typological classification
Siobhan O'Connor, Stephanie Kennedy, Yajing Wang, Amna Ali, Samantha Cooke, Richard G. Booth
Nurse Education Today.2022; 118: 105518. CrossRef
Tendência temporal dos acidentes por exposição percutânea em um hospital público no Brasil, 2007-2019
Renan Sallazar Ferreira Pereira, Cecília Angelita dos Santos, Adriano Marçal Pimenta
Revista Brasileira de Enfermagem.2022;[Epub] CrossRef
Temporal trend of accidents due to percutaneous exposure in a public hospital in Brazil, 2007-2019
Renan Sallazar Ferreira Pereira, Cecília Angelita dos Santos, Adriano Marçal Pimenta
Revista Brasileira de Enfermagem.2022;[Epub] CrossRef
Influence of Technology in Supporting Quality and Safety in Nursing Education
Gerry Altmiller, Loraine Hopkins Pepe
Nursing Clinics of North America.2022; 57(4): 551. CrossRef
Virtual reality simulations in nurse education: A systematic mapping review
Christian Plotzky, Ulrike Lindwedel, Michaela Sorber, Barbara Loessl, Peter König, Christophe Kunze, Christiane Kugler, Michael Meng
Nurse Education Today.2021; 101: 104868. CrossRef
Virtual Reality in Healthcare Skills Training: The Effects of Presence on Acceptance and Increase of Knowledge
Christian Plotzky, Ulrike Lindwedel, Alexander Bejan, Peter König, Christophe Kunze
i-com.2021; 20(1): 73. CrossRef
A systematic review of immersive virtual reality for industrial skills training
Unnikrishnan Radhakrishnan, Konstantinos Koumaditis, Francesco Chinello
Behaviour & Information Technology.2021; 40(12): 1310. CrossRef
No difference in learning outcomes and usability between using controllers and hand tracking during a virtual reality endotracheal intubation training for medical students in Thailand
Chaowanan Khundam, Naparat Sukkriang, Frédéric Noël
Journal of Educational Evaluation for Health Professions.2021; 18: 22. CrossRef
How to train health personnel to protect themselves from SARS-CoV-2 (novel coronavirus) infection when caring for a patient or suspected case
Sun Huh
Journal of Educational Evaluation for Health Professions.2020; 17: 10. CrossRef
Knowledge, attitude and behavior towards COVID-19 among the Turkish healthcare workers
Yasin Uzuntarla, Sumeyra Ceyhan
Journal of Clinical Medicine of Kazakhstan.2020; 6(60): 93. CrossRef

Ophthalmology training and competency levels in caring for patients with ophthalmic complaints among United States internal medicine, emergency medicine, and family medicine residents: Christopher Daniel Gelston, Jennifer Landrigan Patnaik; J Educ Eval Health Prof. 2019;16:25. Published online August 29, 2019; DOI: https://doi.org/10.3352/jeehp.2019.16.25

10,946 View
149 Download
6 Web of Science
12 Crossref

Abstract PDF Supplementary Material

Purpose
To evaluate ophthalmic educational training and confidence in caring for patients with ophthalmic complaints among internal, emergency, and family medicine residents in the United States.
Methods
A 41-item cross-sectional survey was sent to the directors of 529 internal medicine, 237 emergency medicine, and 629 family medicine residency programs, who distributed it to residents in those programs. The survey included the number of ophthalmic education hours residents received. Respondents were asked to rate their confidence in performing an ophthalmic exam and treating patients with ocular conditions using a 5-point Likert scale ranging from “not confident” to “very confident.”
Results
In total, 92.5% of internal medicine, 66.8% of emergency medicine, and 74.5% of family medicine residents received less than 10 hours of ophthalmic education during residency. Most respondents (internal medicine, 59.1%; emergency medicine, 76.0%; family medicine, 65.7%) reported that patients with ocular complaints constituted 1%–5% of visits. Mean±standard deviation confidence levels in performing an eye exam and treating patients with ophthalmic conditions were highest in emergency medicine residency programs (2.9±0.7), followed by family medicine (2.3±0.6) and internal medicine (2.2±0.6). A higher reported number of ophthalmic education hours in residency was associated with greater confidence among emergency (P<0.001), family (P<0.001), and internal (P=0.005) medicine residents.
Conclusion
Internal, emergency, and family medicine residents receive limited ophthalmic education, as reflected by their overall low confidence levels in performing an ophthalmic exam and treating patients with ocular complaints. An increase in ophthalmic educational hours during their residencies is recommended to improve upon this knowledge gap.

Citations

Citations to this article as recorded by

Evaluating patient flow through an emergency ophthalmology consult service in a tertiary care academic centre in Quebec
Emmanuel Issa Nassrallah, Zoya Chaudhry, Georges Nassrallah, Zainab Khan
Canadian Journal of Ophthalmology.2024; 59(2): e111. CrossRef
Performance of ChatGPT in Diagnosis of Corneal Eye Diseases
Mohammad Delsoz, Yeganeh Madadi, Hina Raja, Wuqaas M. Munir, Brendan Tamm, Shiva Mehravaran, Mohammad Soleimani, Ali Djalilian, Siamak Yousefi
Cornea.2024; 43(5): 664. CrossRef
Generative adversarial network-based deep learning approach in classification of retinal conditions with optical coherence tomography images
Ling-Chun Sun, Shu-I. Pao, Ke-Hao Huang, Chih-Yuan Wei, Ke-Feng Lin, Ping-Nan Chen
Graefe's Archive for Clinical and Experimental Ophthalmology.2023; 261(5): 1399. CrossRef
Competency and Level of Confidence of Emergency Residents When Dealing with Ocular Emergency Cases in Saudi Arabia
Faisal Almishali, Adhwaa Allihyani, Norah Albdaya, Ghidaa Gosty, Lujain Marghlani, Mohannad Alqurashi
International Medical Education.2023; 2(2): 113. CrossRef
Creating ophthalmology experiences in undergraduate medical education: pilot of a cased-based learning ophthalmology tool
Jessica H. Tran, Emma Loebel, Mark Edouard, Thomas Quehl, Erin Walsh, Robin Ginsburg, Tameisha Frempong, Douglas Fredrick, Laura K. Stein, Michael G. Fara, Samira S. Farouk, Nisha Chadha
BMC Medical Education.2023;[Epub] CrossRef
Evaluation of a STAT MRI Protocol for Emergent Ophthalmology Patients
Alison B. Gibbons, Peng Huang, Matthew Sklar, Philip Kim, Amanda D. Henderson
Journal of Neuro-Ophthalmology.2023;[Epub] CrossRef
Sebaceous Carcinoma Masquerading As Orbital Cellulitis
Vignesh Ramachandran, Gayane Tumyan, Asad Loya, Kristina Treat, Ivan Vrcek
Cureus.2022;[Epub] CrossRef
A Third-Year Medical School Ophthalmology Curriculum for a Longitudinal Integrated Clerkship Model
A. Itzam Marin, Helio Neves da Silva, Hongan Chen, Nihaal Mehta, Linh K. Nguyen, Jeffrey R. SooHoo, Jennifer E. Adams, Jasleen K. Singh
Journal of Academic Ophthalmology.2022; 14(02): e209. CrossRef
A Porcine-Based Task Trainer for the Instruction of Seidel’s Test and Ocular Foreign Body Removal
Nickolas E Srica, Ryan Walsh, Joseph Sikon, Lawrence Stack
Cureus.2022;[Epub] CrossRef
A Systematic Analysis of the Impact of an Ambulatory Ophthalmology Urgent Care Clinic
Sally S.E. Park, Rohin Vij, Jeff Wu, Bryan Zarrin, Jee-Young Moon, Jason Oliveira, Jeffrey S. Schultz, Anurag Shrivastava
Journal of Academic Ophthalmology.2022; 14(02): e229. CrossRef
Factors Influencing Choice of Medical Specialty among Ophthalmology and Non-Ophthalmology Residency Applicants
David Cui, Andreas M. Wingert, Ingrid U. Scott
Journal of Academic Ophthalmology.2021; 13(02): e129. CrossRef
Recent trends in medical journals’ data sharing policies and statements of data availability
Sun Huh
Archives of Plastic Surgery.2019; 46(06): 493. CrossRef

Research Articles

Correlation of burnout syndrome with specific coping strategies, behaviors, and spiritual attitudes among interns at Yale University, New Haven, USA: Benjamin R. Doolittle, Donna M. Windish; J Educ Eval Health Prof. 2015;12:41. Published online August 1, 2015; DOI: https://doi.org/10.3352/jeehp.2015.12.41

28,792 View
221 Download
29 Web of Science
31 Crossref

Abstract PDF

Purpose
This study aimed to determine the correlation of burnout syndrome with specific coping strategies, behaviors, and spiritual attitudes among interns in internal medicine, primary care, and internal medicine/pediatrics residency programs at two institutions. Methods: Intern physicians completed anonymous voluntary surveys prior to starting the internship in June 2009 and in the middle of the internship in February 2010. Three validated survey instruments were used to explore burnout, coping, and spiritual attitudes: the Maslach Burnout Inventory, the COPE Inventory, and the Hatch Spiritual Involvement and Beliefs Scale. The interns were in programs at the Yale University School of Medicine and a Yale-affiliated community hospital, New Haven, Connecticut, USA. Results: The prevalence of self-identified burnout prior to starting the internship was 1/66 (1.5%) in June 2009, increasing to 10/53 (18.9%) in February 2010 (P<0.0001). From June 2009 to February 2010, the prevalence of high emotional exhaustion increased from 30/66 (45.5%) to 45/53 (84.9%) (P<0.0001), and that of high depersonalization increased from 42/66 (63.6%) to 45/53 (84.9%) (P=0.01). Interns who employed the strategies of acceptance and active coping were less likely to experience emotional exhaustion and depersonalization (P<0.05). Perceptions of high personal accomplishment was 75.5% and was positively correlated with total scores on the Hatch Spiritual Involvement and Beliefs Scale, as well as the internal/fluid and existential/meditative domains of that instrument. Specific behaviors did not impact burnout. Conclusion: Burnout increased during the intern year. Acceptance, active coping, and spirituality were correlated with less burnout. Specific behaviors were not correlated with burnout domains.

Citations

Citations to this article as recorded by

How do they cope? A national cross-sectional study of coping in hospital doctors in Ireland
Ailbhe Doherty, Lucia Prihodova, Gillian Walsh, Blánaid Hayes
BMJ Open.2024; 14(1): e076218. CrossRef
Longitudinal Study of Emotional Intelligence, Well-being, and Burnout of Surgical and Medical Residents
Tarik Wasfie, Heather Kirkpatrick, Kimberly Barber, Jennifer R. Hella, Tara Anderson, Mark Vogel
The American Surgeon™.2023; 89(7): 3077. CrossRef
Systematic review of the relationship between burn-out and spiritual health in doctors
Ishbel Orla Whitehead, Suzanne Moffatt, Stephanie Warwick, Gemma F Spiers, Tafadzwa Patience Kunonga, Eugene Tang, Barbara Hanratty
BMJ Open.2023; 13(8): e068402. CrossRef
COMPARISON OF THE LEVEL OF BURNOUT AMONG THE ACADEMIC AND CLINICAL PHYSICAL THERAPISTS
Mamoona Tasleem Afzal, Zainab Tariq, Syeda Aiman Batool, Muhammad Shakir Khan, Saad Tariq, Mubara Rehman
Pakistan Journal of Rehabilitation.2023; 12(2): 78. CrossRef
Burnout: exploring the differences between U.S. and international medical graduates
Joan E. St. Onge, Heidi Allespach, Yvonne Diaz, Alexandria Poitier, Leonardo Tamariz, Charles Paidas, Ana Palacio
BMC Medical Education.2022;[Epub] CrossRef
Coping strategies adopted by medical residents in dealing with work-related stress: a mixed-methods study
Shamaila Manzoor, Madiha Sajjad, Idrees Anwar, Aisha Rafi
BMC Medical Education.2022;[Epub] CrossRef
Thriving in Residency: a Qualitative Study
Joshua H. Hyman, Benjamin Doolittle
Journal of General Internal Medicine.2022; 37(9): 2173. CrossRef
A national study of burnout and spiritual health in UK general practitioners during the COVID-19 pandemic
Ishbel Orla Whitehead, Suzanne Moffatt, Carol Jagger, Barbara Hanratty, Zhuo Chen
PLOS ONE.2022; 17(11): e0276739. CrossRef
Coping with the Practice of Medicine: Religion, Spirituality, and Other Personal Strategies
Cindy Schmidt, Marissa Roffler
Journal of Religion and Health.2021; 60(3): 2092. CrossRef
Specialty differences in resident resilience and burnout - A national survey
Cristina Nituica, Oana Alina Bota, John Blebea
The American Journal of Surgery.2021; 222(2): 319. CrossRef
Addressing the physician burnout epidemic with resilience curricula in medical education: a systematic review
Chanhee Seo, Mario Corrado, Karine Fournier, Tayler Bailey, Kay-Anne Haykal
BMC Medical Education.2021;[Epub] CrossRef
Thriving among Primary Care Physicians: a Qualitative Study
Katherine Ann Gielissen, Emily Pinto Taylor, David Vermette, Benjamin Doolittle
Journal of General Internal Medicine.2021; 36(12): 3759. CrossRef
Spirituality and religion in residents and inter-relationships with clinical practice and residency training: a scoping review
Hsin Han Elisha Chow, Qian Hui Chew, Kang Sim
BMJ Open.2021; 11(5): e044321. CrossRef
The prevalence of burnout among interns in Riyadh, Saudi Arabia, and its relation to engaging in unethical behaviors
Shatha Ali, Khaldoun Marwa, Malak AlRasheedi, Asma AlSuheel, Mariam Nabila, Madiha Khan
Middle East Current Psychiatry.2021;[Epub] CrossRef
Systematic review of medical education on spirituality
John Wenham, Megan Best, David W. Kissane
Internal Medicine Journal.2021; 51(11): 1781. CrossRef
Burnout syndrome in dentists. Literature review
F.Yu. Daurova, O.A. Ivashkina, M.K. Makeeva
Stomatology for All / International Dental review.2021; (4(97)): 24. CrossRef
Training on mind-body skills: Feasibility and effects on physician mindfulness, compassion, and associated effects on stress, burnout, and clinical outcomes
Michelle C. Nguyen, Steven G. Gabbe, Kathi J. Kemper, John D. Mahan, Jennifer S. Cheavens, Susan D. Moffatt-Bruce
The Journal of Positive Psychology.2020; 15(2): 194. CrossRef
Burnout in Canadian Radiology Residency: A National Assessment of Prevalence and Underlying Contributory Factors
Craig Ferguson, Gavin Low, Gillian Shiau
Canadian Association of Radiologists Journal.2020; 71(1): 40. CrossRef
Evidence-Based Interventions that Promote Resident Wellness from the Council of Emergency Residency Directors
Melissa Parsons, John Baillitz, Arlene Chung, Alexandra Mannix, Nicole Battaglioli, Michelle Clinton, Michael Gottlieb
Western Journal of Emergency Medicine.2020; 21(2): 412. CrossRef
Clinician Wellness During the COVID-19 Pandemic: Extraordinary Times and Unusual Challenges for the Allergist/Immunologist
Priya Bansal, Theresa A. Bingemann, Matthew Greenhawt, Giselle Mosnaim, Anil Nanda, John Oppenheimer, Hemant Sharma, David Stukus, Marcus Shaker
The Journal of Allergy and Clinical Immunology: In Practice.2020; 8(6): 1781. CrossRef
Burnout in Pediatric Critical Care Medicine Fellows*
Markita L. Suttle, Margaret A. Chase, William C. Sasser, Melissa Moore-Clingenpeel, Tensing Maa, Jason A. Werner, Meredith F. Bone, Donald L. Boyer, Karen J. Marcdante, Katherine E. Mason, Megan E. McCabe, Richard B. Mink, Felice Su, David A. Turner
Critical Care Medicine.2020; 48(6): 872. CrossRef
Quality and Safety in Healthcare, Part LXIII
Jay A. Harolds
Clinical Nuclear Medicine.2020; 45(11): 860. CrossRef
Employee burnout and positive dimensions of well-being: A latent workplace spirituality profile analysis
Laura Dal Corso, Alessandro De Carlo, Francesca Carluccio, Daiana Colledani, Alessandra Falco, Stefan Hoefer
PLOS ONE.2020; 15(11): e0242267. CrossRef
Associations Between Systemic Quality of Life and Burnout Among French Canadian Workers
Maxime Vachon, Marie Papineau, Gilles Dupuis, Pasquale Roberge
Social Indicators Research.2019; 142(3): 1193. CrossRef
Cultivating a way of being and doing: Individual strategies for physician well-being and resilience
Albina Gogo, Amanda Osta, Hilary McClafferty, Deborah T. Rana
Current Problems in Pediatric and Adolescent Health Care.2019; 49(12): 100663. CrossRef
Spiritual Care in the Intensive Care Unit: A Narrative Review
Jim Q. Ho, Christopher D. Nguyen, Richard Lopes, Stephen C. Ezeji-Okoye, Ware G. Kuschner
Journal of Intensive Care Medicine.2018; 33(5): 279. CrossRef
Associations between social skills and burnout dimensions in medical residents
Karina PEREIRA-LIMA, Sonia Regina LOUREIRO
Estudos de Psicologia (Campinas).2017; 34(2): 281. CrossRef
Burnout: Prevalence and Associated Factors Among Radiology Residents in New England With Comparison Against United States Resident Physicians in Other Specialties
Jeffrey P. Guenette, Stacy E. Smith
American Journal of Roentgenology.2017; 209(1): 136. CrossRef
Will the year 2016 augur well for better patient safety and health of residents in Korea according to the enactment of the Act for improving the resident training environment and enhancing resident’s status?
Sun Huh
Journal of Educational Evaluation for Health Professions.2016; 13: 2. CrossRef
A 10-Year Longitudinal Study of Effects of a Multifaceted Residency Spiritual Care Curriculum: Clinical Ability, Professional Formation, End of Life, and Culture
Gowri Anandarajah, Janet Roseman, Danny Lee, Nupur Dhandhania
Journal of Pain and Symptom Management.2016; 52(6): 859. CrossRef
ROLE OF SPIRITUALITY AS A WAY OF COPING FROM BURNOUT IN MEDICAL STUDENTS OF A TERTIARY CARE INSTITUTE IN INDIA
Sagar Shrikant Chiddarwar, Deepika Abhainath Singh
Journal of Evidence Based Medicine and Healthcare.2016; 3(20): 836-. CrossRef

Evaluation of hospital-learning environment for pediatric residency in eastern region of Saudi Arabia: Waleed H. BuAli, Abdul Sattar Khan, Mohammad Hussain Al-Qahtani, Shaikha aldossary; J Educ Eval Health Prof. 2015;12:14. Published online April 18, 2015; DOI: https://doi.org/10.3352/jeehp.2015.12.14

31,090 View
179 Download
11 Web of Science
13 Crossref

Abstract PDF

Purpose
No study had been conducted to assess the hospitals’ environment for learning purposes in multicenter sites in Saudi Arabia. It aims to evaluate the environment of hospitals for learning purposes of pediatric residents. Methods: We applied Postgraduate Hospital Educational Environment Measure (PHEEM) to measure the learning environment at six teaching hospitals in the Eastern Region of Saudi Arabia from September to December 2013. Results: The number of respondents was 104 (86.7%) out of 120 residents and 37 females and 67 male residents have responded. The residents’ response scored 100 out of 160 maximum score in rating of PHEEM that showed overall learning environment is favorable for training. There were some items in the social support domain suggesting improvements. There was no significant difference between male and female residents. There was a difference among the participant teaching hospitals (p<0.05). Conclusion: The result pointed an overall positive rating. Individual item scores suggested that their social life during residency could be uninspiring. They have the low satisfactory level and they feel racism, and sexual discrimination. Therefore, there is still a room for improvement.

Citations

Citations to this article as recorded by

Examining the Psychometric Properties of Post-graduate Hospital Educational Environment Measure for Saudi Pediatric Trainees: Principal Component Analysis
Khouloud Abdulrhman Alsofyani, Saud Bahaidarah, Abdulaziz Boker
Iranian Journal of Pediatrics.2024;[Epub] CrossRef
Scoping review of the application of the Postgraduate Hospital Educational Environment Measure (PHEEM) in medical residency
Patrícia Lofêgo Gonçalves, Ana Paula Moscon Marçal, Renata de Almeida França, Vania dos Santos Nunes Nogueira
Revista Brasileira de Educação Médica.2024;[Epub] CrossRef
Revisão de escopo da aplicação do Postgraduate Hospital Educational Environment Measure (PHEEM) na residência médica
Patrícia Lofêgo Gonçalves, Ana Paula Moscon Marçal, Renata de Almeida França, Vania dos Santos Nunes Nogueira
Revista Brasileira de Educação Médica.2024;[Epub] CrossRef
Measuring the Learning Orientation Fostered by Pediatric Residency Programs With the Graduate Medical Education Learning Environment Inventory Instrument
Jonathan G. Sawicki, Boyd F. Richards, Alan Schwartz, Dorene Balmer
Academic Pediatrics.2023; 23(6): 1288. CrossRef
State of well-being among residents in a tertiary center in Riyadh, Saudi Arabia
Fatimah Saeed AlAhmari, Alaa Aloqail, Shahad Almansour, Mohammad Bagha
BMC Medical Education.2023;[Epub] CrossRef
Trainees’ perceptions on learning environment based on the level of training in a pediatric training program in Indonesia
Rina Triasih, Felisia Ang, Weda Kusuma, Gandes Retni Rahayu
Paediatrica Indonesiana.2022; 62(4): 249. CrossRef
Evaluación del ambiente educacional en la residencia de Clínica Pediátrica en un Hospital de Referencia Provincial
Myriam Lucrecia Medina, Marcelo Gabriel Medina, Nélida Teresita Gauna, Luciana Molfino, Luis Antonio Merino
Educación Médica.2020; 21(1): 24. CrossRef
Moroccan residents’ perceptions of the hospital learning environment measured with the French version of the Postgraduate Hospital Educational Environment Measure
Hajar Berrani, Redouane Abouqal, Amal Thimou Izgua
Journal of Educational Evaluation for Health Professions.2020; 17: 4. CrossRef
Evaluating the educational environment in a residency programme in Singapore: can we help reduce burnout rates?
AML Ong, WWS Fong, AKW Chan, GC Phua, CK Tham
Singapore Medical Journal.2020; 61(9): 476. CrossRef
Sudanese paediatric residents perception towards training environment in Sudan Medical Specialisation Board, 2020
Amani Baldo, Mustafa Elnimeiri, Huda Haroun, Emuntasir Salah
Sudanese Journal of Paediatrics.2020; : 126. CrossRef
Learning environment, stress and coping in psychiatry residents within a national training program: a longitudinal study
Qian Hui Chew, Eric Holmboe, Kang Sim
Perspectives on Medical Education.2019; 8(3): 160. CrossRef
Using the Postgraduate Hospital Educational Environment Measure to Identify Areas for Improvement in a Singaporean Residency Program
Andrew Ming-Liang Ong, Warren Weng-Seng Fong, Adrian Kwok-Wai Chan, Ghee-Chee Phua, Chee-Kian Tham
Journal of Graduate Medical Education.2019; 11(4s): 73. CrossRef
Adoption and correlates of Postgraduate Hospital Educational Environment Measure (PHEEM) in the evaluation of learning environments – A systematic review
Christopher Yi Wen Chan, Min Yi Sum, Wee Shiong Lim, Nicholas Wuen Ming Chew, Dujeepa D. Samarasekera, Kang Sim
Medical Teacher.2016; 38(12): 1248. CrossRef

Health-related quality of life and happiness within an internal medicine residency training program: a longitudinal follow-up study: Abhasnee Sobhonslidsuk, Ammarin Thakkinstian, Patchareeya Satitpornkul; J Educ Eval Health Prof. 2015;12:3. Published online February 24, 2015; DOI: https://doi.org/10.3352/jeehp.2015.12.3

31,775 View
161 Download
3 Web of Science
2 Crossref

Abstract PDF

Purpose
While undergoing a hospital residency training program, residents often suffer anxiety and stress. This study aims to evaluate the change in health-related quality of life and happiness among internal medicine residents, and identify prognostic factors. Methods: Thirty-eight residents in the Ramathibodi Hospital internal medicine training program completed the World Health Organization Quality of Life-BREF and happiness Measures questionnaires at three time points: commencement, day 100, and the end of the second year of training. Confidence, expectations, anxiety, and general health were rated. Analyses were performed with mixed linear regression. Results: Financial problems were reported for 16 residents (42.1%). At baseline, most residents had moderate-to-very high confidence, expectations, and general health but also moderate-to-very high anxiety. The health-related quality of life score was highest in the social domain followed by the environmental, psychological, and physical domains. Their psychological, physical, social, and environmental scores significantly decreased after enrollment. Their happiness and general health scores were significantly reduced after enrollment. The training program duration was negatively associated with all domains. Residents with greater confidence had higher health-related quality of life scores in the physical, psychological, and environmental domains. Moreover, their general health was positively associated with the social and environmental domains. Conclusion: A reduction in health-related quality of life and happiness under the internal medicine residency program is reported. High confidence and good physical health may counterbalance the decline in health-related quality of life and happiness.

Citations

Citations to this article as recorded by

Specialty impact on residents’ perceived quality of life, stress, and job satisfaction: a comparative study
Bo Young Kim, Inah Yoon, Seong John Han, Suk-Kyung Hong, Sehoon Choi, Hyo-Jin Kwon, Eun Key Kim
Annals of Surgical Treatment and Research.2023; 105(4): 188. CrossRef
The Relationship between Personality Traits with Depressive Symptoms and Suicidal Ideation among Medical Students: A Cross-Sectional Study at One Medical School in Germany
Winnie Chow, Jan Schmidtke, Adrian Loerbroks, Thomas Muth, Peter Angerer
International Journal of Environmental Research and Public Health.2018; 15(7): 1462. CrossRef

First
Prev
Page of 1
Next
Last

Search