-
Development and validity evidence for the resident-led large group teaching assessment instrument in the United States: a methodological study
-
Ariel Shana Frey-Vogel
, Kristina Dzara , Kimberly Anne Gifford , Yoon Soo Park , Justin Berk , Allison Heinly , Darcy Wolcott , Daniel Adam Hall , Shannon Elliott Scott-Vernaglia , Katherine Anne Sparger , Erica Ye-pyng Chung
-
J Educ Eval Health Prof. 2024;21:3. Published online February 23, 2024
-
DOI: https://doi.org/10.3352/jeehp.2024.21.3
-
-
Abstract
PDF Supplementary Material
- Purpose
Despite educational mandates to assess resident teaching competence, limited instruments with validity evidence exist for this purpose. Existing instruments do not allow faculty to assess resident-led teaching in a large group format or whether teaching was interactive. This study gathers validity evidence on the use of the Resident-led Large Group Teaching Assessment Instrument (Relate), an instrument used by faculty to assess resident teaching competency. Relate comprises 23 behaviors divided into 6 elements: learning environment, goals and objectives, content of talk, promotion of understanding and retention, session management, and closure.
Methods Messick’s unified validity framework was used for this study. Investigators used video recordings of resident-led teaching from 3 pediatric residency programs to develop Relate and a rater guidebook. Faculty were trained on instrument use through frame-of-reference training. Resident teaching at all sites was video-recorded during 2018–2019. Two trained faculty raters assessed each video. Descriptive statistics on performance were obtained. Validity evidence sources include: rater training effect (response process), reliability and variability (internal structure), and impact on Milestones assessment (relations to other variables).
Results Forty-eight videos, from 16 residents, were analyzed. Rater training improved inter-rater reliability from 0.04 to 0.64. The Φ-coefficient reliability was 0.50. There was a significant correlation between overall Relate performance and the pediatric teaching Milestone (r=0.34, P=0.019).
Conclusion Relate provides validity evidence with sufficient reliability to measure resident-led large-group teaching competence.
-
Medical residents and attending physicians’ perceptions of feedback and teaching in the United States: a qualitative study
-
Madeleine Matthiesen
, Michael S. Kelly , Kristina Dzara , Arabella Simpkin Begin
-
J Educ Eval Health Prof. 2022;19:9. Published online April 26, 2022
-
DOI: https://doi.org/10.3352/jeehp.2022.19.9
-
-
9,545
View
-
389
Download
-
3
Web of Science
-
3
Crossref
-
Abstract
PDF Supplementary Material
- Purpose
Residents and attendings agree on the importance of feedback to resident education. However, while faculty report providing frequent feedback, residents often do not perceive receiving it, particularly in the context of teaching. Given the nuanced differences between feedback and teaching, we aimed to explore resident and attending perceptions of feedback and teaching in the clinical setting.
Methods We conducted a qualitative study of internal medicine residents and attendings from December 2018 through March 2019 at the Massachusetts General Hospital to investigate perceptions of feedback in the inpatient clinical setting. Residents and faculty were recruited to participate in focus groups. Data were analyzed using thematic analysis to explore perspectives and barriers to feedback provision and identification.
Results Five focus groups included 33 total participants in 3 attending (n=20) and 2 resident (n=13) groups. Thematic analysis of focus group transcripts identified 7 themes which organized into 3 thematic categories: (1) disentangling feedback and teaching, (2) delivering high-quality feedback, and (3) experiencing feedback in the group setting. Residents and attendings highlighted important themes in discriminating feedback from teaching. They indicated that while feedback is reactive in response to an action or behavior, teaching is proactive and oriented toward future endeavors.
Conclusion Confusion between the critical concepts of teaching and feedback may be minimized by allowing them to each have their intended impact, either in response to prior events or aimed toward those yet to take place.
-
Citations
Citations to this article as recorded by 
- Resident Assessment of Clinician Educators According to Core ACGME Competencies
Bailey A. Pope, Patricia A. Carney, Mary C. Brooks, Doug R. Rice, Ashly A. Albright, Stephanie A. C. Halvorson Journal of General Internal Medicine.2024; 39(3): 377. CrossRef - Radiologist preferences for faculty development initiatives to improve resident feedback in the era of competency-based medical education
Laura Wong, Ethan Sacoransky, Wilma Hopman, Omar Islam, Andrew D. Chung, Benjamin Y. M. Kwan Medical Education Online.2024;[Epub] CrossRef - Through the Looking Glass: Comparing Hospitalists' and Internal Medicine Residents' Perceptions of Feedback
Andrew V Raikhel, Helene Starks, Gabrielle Berger, Jeffrey Redinger Cureus.2024;[Epub] CrossRef
|