METHODS

This study was conducted in KIST Medical College (KISTMC), a newly established medical school in the private sector located in Lalitpur district of Nepal. The first batch of MBBS students was admitted in November 2008. The college is affiliated with Tribhuvan University Nepal. The university curriculum recommends the involvement of Medicine and Surgery Departments for PES training, but the methods for teaching and assessment are not well defined. KISTMC has made training more structured and involved the Departments of Internal Medicine, General Surgery, Gynaecology and Obstetrics, Orthopaedics, Otorhinolaryngology, Ophthalmology, Paediatrics, and Family Medicine for imparting training to secondyear MBBS students as a part of early clinical exposure. SPEST includes the teaching/learning of basic PES for 210 minutes (3.5 hours) once in a week for 28 weeks. Each SPEST session is divided into three sub-sessions: Tutorial (T), Demonstration (D), and Practice (P). The Tutorial element encompasses an overview by a faculty preceptor on the skills to be taught; the Demonstration involves an actual demonstration of each of the skills taught with a step-by-step description of the procedure; and Practice involves the performance/practice of each demonstrated skill by the students using a sequential description to be observed by the preceptor. At the end of the last session of training, a retro-pre-questionnaire was used for assessing the learners’ self-reported changes.

Forty-six skills were selected from many skills taught during training and included in the retro-pre-questionnaire (Table 1). The skills included in the questionnaire represented various systems taught in the different departments during the training. The retro-pre-questionnaire was modelled in line with the Dreyfus model of skill aquisition. Before use, the retro- pre-questionnaire was piloted with 10 students of a senior cohort. It was found to be reliable and valid. The objective item scores were added to obtain the overall score and the descriptive statistics for before and after the training. A paired t-test was used for evaluating the difference in the overall scores. The difference between the overall scores before and after was found to follow the normal distribution, as confirmed by the Shapiro-Wilk test. The data were analysed using SPSS ver. 15 (SPSS Inc., Chicago, IL, USA).

RESULTS

The students’ responses to the retro-pre-questionnaire before and after the structured physical examination skills training are given in Table 1. Out of the 100 students trained, 82 completed the retro-pre-questionnaires. The average skill score (maximum, 46×4=184) for before the SPEST module was 15.9 (median, 13.5) and that after was 116.5 (median, 116). The increase in scores was statistically significant (mean±SD, 100.5±23; 95% confidence interval, 95.45 to 105.59), which implied that the students perceived that they learned most of the skills after the SPEST module and that the course was effective. Further, the paired correlation was not very high (r= 0.240), but it was in the positive direction and was statistically significant (P=0.03), implying that the questionnaire before score had a low impact on the questionnaire after score.

DISCUSSION

The most widely used evaluation tool is a traditional “prethen- post” test, where participants are asked a series of questions at both the start of a program (pre-test) and then again at the end of program (post-test). This tool is believed to measure changes in participant knowledge, attitudes, or behaviours regarding whatever the program content is [3,4]. The criticism of the traditional pre/post tool has led to the use of a “retrospective pre-test” tool. This type of tool is well accepted because it is implemented at only one point in time. The theory behind designing this type of tool is that by testing what participants believe about the program content after program completion, their standard of assessing the changes in knowledge, skills, or attitudes is consistent and thus, not subject to a response shift bias [3,5,6]. Others note that the post-then-pre design reduces incomplete data sets and is convenient to administer given the time constraints that many program providers face, and is also easier for the program participants to complete [3,7,8].

If the objective is to identify how participants perceive the changes that they have made in knowledge, skills, attitudes, or behaviour, then a retro-pre method may be adequate to capture information on this type of data [3].

Our objective was to study the perceived impact of SPEST imparted to second-year undergraduate medical students. The results from the paired t-test showed that the difference between before and after the SPEST was statistically significant, which implied that the students did learn most of the skills after the implementation of the SPEST module and that the training was effective. Further, the paired correlation was not very high, but it was in the positive direction and was statistically significant, implying that the retro-pre before score had a low impact on the retro-pre after score. It is important to acknowledge that all self-reported information could be considered somewhat subjective.

In summary, because the students perceived that their level of skills improved after the training, we concluded that the retro-pre instrument is useful for assessing learners’ self-reported changes in PES after training if a large number of skills are to be assessed. However, it should be noted that although a retro-pre instrument may reveal valuable information, it is not a substitute for an objective measure or gold standard.

SUPPLEMENTARY MATERIAL

References

• 1. Program Development and Evaluation. Using the Retrospective Post then-Pre Design, Quick Tips #27 [Internet]. Madison, WI:University of Wisconsin Extension; 2005 [updated 2005 Jul 25; cited 2013 Sep 17]. Available from: http://www.uwex.edu/ces/pdande/resources/pdf/Tipsheet27.pdf
• 2. Piryani RM, Shankar PR, Thapa TP, Karki BM, Kafle RK, Khakurel MP, Bhandary S. Introduction of structured physical examination skills to second year undergraduate medical students. F1000Research 2013;2:16. http://dx.doi.org/10.12688/f1000research.2-16.v1 ArticlePubMedPMC
• 3. Colosi L, Dunifon R. What’s the difference?: “post then pre” & “pre then post” [Internet]. New York: Cornell University Cooperative Extension; 2006[cited 2013 Sep 17]. Available from:http://www.human.cornell.edu/pam/outreach/parenting/research/upload/What-s-20the-20Difference-20Post-20then-20Pre-20and-20Pre-20then-20Post.pdf
• 4. Zaidi Z, Zaidi SM, Razzaq Z, Luqman M, Moin S. Training workshops in problem-based learning: changing faculty attitudes and perceptions in a Pakistani medical college. Educ Health 2010;23:440. http://www.educationforhealth.net/text.asp?2010/23/3/440/101470 Article
• 5. Rockwell SK, Kohn H. Post-then-pre evaluation. J Ext 1989;27:2FEA5. http://www.joe.org/joe/1989summer/a5.php
• 6. Davis GA. Using retrospective pre-post questionnaire to determine program impact. J Ext 2003;41:4TOT4. http://www.joe.org/joe/2003august/tt4.php/
• 7. Raidl M, Johnson S, Gardiner K, Denham M, Spain K, Lanting R, Jayo C, Liddil A, Barron K. Use retrospective surveys to obtain complete data sets and measure impact in extension programs. J Ext 2004;42:2RIB2. http://www.joe.org/joe/2004april/rb2.php
• 8. Lamb T. The retrospective pretest: an imperfect but useful tool . Eval Exchange 2005;11:18. http://www.hfrp.org/evaluation/theevaluation-exchange/issue-archive/evaluation-methodology/theretrospective-pretest-an-imperfect-but-useful-tool

Figure & Data

References

Citations

Citations to this article as recorded by  

• How Far Has theInternational Neurourology JournalProgressed Since Its Transformation Into an English Language Journal?
  Sun Huh
  International Neurourology Journal.2014; 18(1): 3.     CrossRef