Increased competency of registered dietitian nutritionists in physical examination skills after simulation-based education in the United States

Background/rationale

Simulation-based education (SBE) in professional training programs has expanded in recent years [1,2]. The foundational components of SBE that set it apart from other pedagogical techniques are the inclusion of pre- and de-briefing discussions between facilitators and learners, establishment of a safe space for learners to practice and make mistakes, and the inclusion of realistic patient case scenarios in the learning environment [1,2]. Despite the more widespread inclusion of SBE in both educational and clinical learning environments, little research has established guidelines for how to quantify the effects of SBE in terms of changes in clinical competence of learners or improved patient care and safety [1-4]. Assessment of clinical skills competence is also a current need for dietetics educators, due to the recent development of a competency-based future education model of dietetics education [5]. An example of a clinical skill recently made mandatory for all registered dietitian nutritionists (RDNs) and graduates of dietetics education programs to demonstrate is the nutrition-focused physical exam (NFPE) [6].

The NFPE has been established as a key tool of dietetics practitioners for the identification of malnutrition and micronutrient imbalances in adult and pediatric patients [7]. As of 2017, the Accreditation Council on Education in Nutrition and Dietetics has required accredited dietetics programs to provide instruction in the NFPE [6]. However, recent studies of practicing RDNs showed that many RDNs reported feeling unprepared to include the NFPE in their own clinical practice [8]. As a result, health care systems that employ RDNs are seeking training opportunities to increase utilization of this valuable clinical skill [8,9]. As a physical examination requiring physical contact with patients, a training utilizing the simulated environment and actual persons portraying patient cases (standardized patients, SPs) is an ideal pedagogical method for teaching and practice of the NFPE skill.

Objectives

This study examined a group of practicing RDNs to identify changes in competence scores for clinical skills after participating in a 90-minute SBE session. Further, this study obtained a snapshot of existing competence levels of practicing RDNs for the NFPE, a required clinical skill. The research questions were as follows: first, what is the current rate of competence in the NFPE skill among practicing clinical RDNs?; second, are years of clinical experience, frequency of performing NFPE, and the education level of RDNs related with their initial competence scores in the NFPE?; third, does simulation-based training increase practicing clinicians’ competence in specific skills (e.g., NFPE)?; and fourth, what variables (experience, frequency of performing NFPE, and education level) are associated with the greatest gains in competence from before to after simulation-based training?

Ethics statement

Ethical approval was obtained from the Grand Valley State University Human Subjects Institutional Review Board as exempt research (#18-220-H).

Study design

This pre-and post-intervention comparison study included RDNs employed at Spectrum Health. This article was described according to the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) statement, available from: https://www.strobe-statement.org/.

Setting

This study took place in a university simulation center setting, specifically the SP suite. This suite consisted of outpatient exam rooms outfitted with a video camera system that enabled the recording of patient examinations conducted in a realistic environment. Specifically, the SPs were adults employed by the simulation center and trained to portray a variety of simulated patient cases. Training and data collection occurred in 2 groups, held in January and February 2020. The same researchers conducted the SBE training and scored the video-recorded SP encounters for both groups.

Participants

The RDNs were selected for training by their employer, as part of a professional development activity offering, and selection was not based on level of experience, area of practice, or level of interest. A total of 18 RDNs participated in the study in the 2 groups (containing 10 and 8 participants, respectively).

Variables

Initially, all RDN participants were asked to complete a survey on their level of education, time since dietetic registration, and frequency of performing NFPE in the course of their typical practice. The remaining variables analyzed in this study were performance scores on each of the 7 competencies that make up the overall NFPE skill.

Data sources/measurement

Bias

To reduce the potential for confirmation bias in scoring by the researchers, the raters were blinded to whether the participant videos they scored were from the pre- or post-training time points. The 2 patient cases portrayed and the skills performed in both the pre- and post-training encounters were identical; each participant saw one case pre-training and the other case post-training, but the same 2 cases were used in both sessions. Additionally, videos were marked with a code number (not names) for participants, and researchers did not have access to the code list linking data from the survey of experience level and area of practice until after all videos were scored.

Study size

A total of 18 RDNs participated in this study. Each RDN completed both a pre- and a post-training SP encounter in which they performed the NFPE skill. The G*Power tool (Heinrich-Heine-Universität Düsseldorf, Düsseldorf, Germany; http://www.gpower.hhu.de/) was used to estimate the sample size for the paired t-test. The a priori G*Power test indicated that the required sample size was 16 based on a 2-tailed test, an effect size of 0.9, alpha error of 0.05, and a power of 0.9. The actual power was 0.915.

Statistical methods

Variables such as time in practice, highest educational degree earned, and frequency of using NFPE in practice were considered in the analyses in order to detect any effects on competence. Statistical analyses conducted in IBM SPSS ver. 25.0 (IBM Corp., Armonk, NY, USA) included: first, descriptive statistics of the subject population; second, the percent of RDNs competent at the initial time point; third, the change in competence scores following remediation training, with mean competence scores compared using the paired t-test; and fourth, analysis of variance (ANOVA) to determine whether the analyzed variables (length of time in practice, education level, and area of practice) were associated with significant differences in mean competence scores from the initial assessment to the post-training re-assessment. For analyses, scores were considered both on the 9-point scale and categorically, in 3 levels. Categories were considered to better indicate the clinically relevant groups of “below expectations” (scores of 1–3), “meets expectations” (scores of 4–6), and “exceeds expectations” (scores of 7–9). For a binary analysis of initial competence, scores within the category of “meets” or “exceeds” expectations were considered as “competent.” These categorical outcome analyses were included in order to identify differences in NFPE skills that would be relevant and meaningful in the professional health care setting.

Participants

The subjects included 18 RDNs employed at Spectrum Health, a private health system in the Midwest region of the United States. All 18 participants attended the 90-minute training and completed both pre- and post-training NFPE assessments. The raw data file is available in ^{Dataset 1,} and ².

Descriptive data

The mean number of years in practice as an RDN for the participants was 9.6 years (standard deviation=7.7). Most participants (n=13, 72.2%) had earned a bachelor’s degree as their highest earned degree. The largest category reported for the average number of times performing NFPE in a week was 15 or more times per week (n=8, 44.4%), followed by 8–14 times (n=4, 22.2%). The complete results of the survey are shown in Table 1.

Table 1.

Demographic and professional characteristics of the participants (n=18)

Main results

Competence at the initial time point

When competence as a binary variable was assessed for each item, the percentage of RDNs competent at the initial time-point ranged from 0% (competency 6, abdominal exam) to 44% (competency 1, approach to the patient). The only competency where participants scored in the “meets expectations” range initially was competency 1 (approach to the patient; mean score=3.94 out of a maximum score of 9; standard deviation=1.18). The initial rates and scores for each competency are shown in Table 2.

Table 2.

Registered dietitians’ initial competence scores on the nutrition-focused physical exam (n=18)

Change in competence scores following SBE training

When competence scores on the 9-point scale were assessed for changes from pre- to post-SBE training, significant increases in competence scores were observed in all 7 competency areas. The greatest increase in competence scores among the RDN participants occurred in competency 6 (abdominal exam), with a mean increase of 3.72 points (95% CI, 3.12–4.26), followed by competency 7 (functional status assessment; mean increase, 3.31; 95% CI, 2.69–3.93). The smallest increase from before to after training occurred in competency 1 (approach to the patient; mean increase, 1.27; 95% CI, 0.73–1.81). Further investigation into the significance of differences between mean scores at the pre-training and post-training time points using the paired t-test revealed significant increases in all 7 competency areas following the simulation-based training (P<0.001). In terms of the binary distinction between “meets expectations” and “below expectations,” across all 7 competencies, on average 15 out of the 18 participants (83.3%) scored as competent following the SBE training. Complete results of the competency score change and the paired t-test are shown in Table 3.

Table 3.

Change in mean competence scores on the nutrition-focused physical exam from before to after simulation-based training, based on paired t-test analysis

Key results

Based on the competence change results (Table 3), it appears that simulation-based training was an effective method for teaching these skills, and participants achieved a mean score following the training that was within the range of competence. The simulation-based training provided in this study produced significant increases in all 7 competencies and, for the 6 competencies where participants scored “below expectations” prior to training (competencies 2–7), the post-training scores fell within the range of competence (scores between 4 and 6, equivalent to “meets expectations”).

Limitations

Limitations of this study include the fact that the researchers who developed the competency assessment tool also rated participants at the pre-/post-training time points. Although steps were taken to reduce confirmation bias, such as blinding researchers to whether videos were from a pre- or post-training time point, it is possible that subconscious rating inflation could have occurred due to the investment of the raters in the study.

Interpretation

The competencies with the lowest initial scores (competency 6, abdominal exam and competency 7, functional status assessment) were the same competencies with the greatest increase in scores after the simulation-based training. These areas are advanced-level skills that are not always included in basic instruction on performance of the NFPE [12,13]. This may be because they require specialized equipment (a stethoscope for abdominal auscultation and a dynamometer for hand grip strength measurement) not traditionally provided for the use of RDNs in clinical settings [12,13]. Therefore, the simulation-based training may have been the first time many participants were exposed to these skills.

Although the largest group of participants (n=8, 44.4%) indicated on the experience survey that they performed NFPE on average 15 or more times per week, the competence scores were initially low across all 7 skill areas. This may have originated from the simulation lab setting, which was new to all of the participants, or from participants’ unfamiliarity with some of the instrumentation provided for use (for example, the stethoscope for abdominal exam and dynamometer for functional status/grip strength).

Generalizability

The results of this study serve multiple purposes, for different stakeholders. The results will serve clinical managers and educators in the field of dietetics by providing data on competence levels of practicing RDNs in the NFPE skill. Additionally, the findings will serve to move the field of health professions education forward by translating SBE to competence gains of practicing clinicians, as opposed to assessments of learner-perceived traits, as previous studies have done [3,14].

Conclusion

This study demonstrates a method of quantifying the changes in clinical skill competence that occurred as a result of SBE training. Within the field of dietetics, other clinical skills that could be tested using SBE include feeding tube placement, such as guided nasogastric tube placement at the bedside. Future work should seek to explore potential differences in the effectiveness of SBE at competency-building in clinicians with different levels of experience and education, as well as in health professions students.