Introduction

Background

Critical thinking is described as the ability to pose a discriminating question in order to search for better ideas or to find better solutions [1]. The critical thinking process consists of collecting appropriate information, precisely evaluating the information, and using it to come to a considered conclusion. Since healthcare involves inherent uncertainties and is prone to diagnostic and management errors, improving physicians’ critical thinking could have a substantial effect on aspects of routine medical practice, such as choosing treatment plans, making an accurate diagnosis, and reducing medical errors [2]. Furthermore, critical thinking ability is strongly correlated with clinical competence and academic success [3]. Evidence shows that clinical clerkships and other forms of clinical experience provide the opportunity to enhance critical thinking through observation of more senior clinicians and gaining experience, but the impact of such experiences is insufficient [4]. As a result, medical schools are placing a major priority on assessing critical thinking, improving this ability using specialized teaching techniques, and providing sufficient educational opportunities [5]. The inadequacy of these methods has led to curricular reforms in an attempt to add critical thinking to all levels of education. Medical school instructors should be well versed in models of argument and should regularly encourage their students to engage in discussions during daily rounds, morbidity and mortality conferences, and any other teaching sites.

Despite the undeniable role of critical thinking ability in medical education, it is challenging to choose the best approach to enhance critical thinking skills. Several approaches to strengthen critical thinking skills have been developed. Investigators’ approach toward education focusing on critical thinking skills has generally reflected a viewpoint of teaching these skills as a general educational subject or as a subject integrated with specific knowledge. Students are most likely to benefit from learning critical thinking skills when they are delivered through specific educational modules within their curriculum [6]. However, the reported results in the literature are inconsistent; some researchers have found that education led to increases or decreases in critical thinking skills, while others have found that critical thinking was unchanged by a training program; therefore, it remains unclear how effectively critical thinking is being taught in medical schools. Instead of comparing medical students’ critical thinking before and after a short course, we analyzed changes in students’ ability across a 2. 5-year timeframe.

Objectives

This study aimed to evaluate the effects of implementing a curriculum integrating critical thinking on medical students’ skills. The research questions were as follows: first, is there a significant change in the medical students’ critical thinking abilities as they progress from the first week of medical school to the third year of the undergraduate medical curriculum?; second, which critical thinking skills improve as students progress through the curriculum; and third, are there differences according to gender in critical thinking before and after the program?

Methods

Ethics statement

The institutional review board of Tehran University of Medical Sciences (TUMS) approved this study (IR.TUMS.REC 1390-16263). Informed consent was obtained from participants.

Study design

This was a 1-group pre- and post-test quasi-experimental study.

Setting

The study was carried out at TUMS. In 2006, TUMS commenced the development and implementation of a newly revised curriculum for delivering undergraduate medical education. A main feature of the revised curriculum is that it focuses more attention on the integration of critical thinking programs for training and assessing medical students. We assessed medical students’ critical thinking skills as they progressed from the first week of medical school to the third year of the preclinical phase in an undergraduate medical curriculum. A pre- and post-test method was used to assess the impact of the curriculum integrating critical thinking on medical students’ critical thinking skills. Participating students were asked to complete the California Critical Thinking Skills Test (CCTST; form B) the week before their first educational session. Post-test data were collected 6 weeks after the 2 and half-year program.

Participants

All 91 medical students, who comprised 43 men (47.3%) and 48 women (52.7%) with an average age of 20±2.8 years, enrolled in the medical undergraduate curriculum at TUMS were invited to participate in the study. Sampling was done by the census method with the participation of all students enrolled in the medical undergraduate curriculum. The researchers included only data from students who completed both the pre- and post-intervention questionnaires (n=56), and excluded data from those who chose not to complete both tests. The valid response rate was 61.5%. The primary reason why some medical students did not complete the CCTST (form B) at both pre- and post-tests was absenteeism on the day of survey administration.

Measurement tool

Demographic information, including age and gender, was collected in the first part of the questionnaire. The second part assessed the critical thinking skills of medical students using the valid and reliable CCTST form B, the Persian version of which contains 5 subscales: analysis, inference, evaluation, deductive reasoning, and inductive reasoning [7]. The CCTST contains 34 multiple-choice questions with a correct answer (with scores of 0–1). The maximum scores in the 5 sections are 9 for analysis, 11 for inference, 16 for deductive reasoning, 14 for inductive reasoning, and 14 for evaluation (Supplement 2). The scope of questions encompasses semantic analysis of a single sentence to a more complex combination of critical thinking skills. In other words, answering some of the questions of the CCTST requires extracting the correct inference from a series of assumptions, evaluating the options, and providing a reasoned justification for a conclusion. The total CCTST score ranges from 0 to 34, with higher scores reflecting stronger critical thinking skills. It took students about 45 minutes to complete the test. The reliability and validity of the CCTST assessment have been reported in previous publications [7]. In a study by Khalili and Hossein Zadeh [8], the reliability coefficient of the test, determined in terms of internal consistency and using the Kuder-Richardson Formula 20, was 0.62. The reliability coefficient of the subscales after factor analysis was in the range of 0.62–0.67. In the present study, the content validity of the Persian version of the questionnaire was confirmed by 8 faculty members in the fields of cognitive psychology, philosophy, medicine, and medical education. Its reliability was calculated using the Cronbach α coefficient, which was equal to 0.83. A researcher collected data before and after the courses over a 2 and half-year curriculum. The initial survey was conducted at the beginning of the first semester. The post-test was implemented 6 weeks after the 2 and half-year program.

Interventional program

In this study, medical students participated in a critical thinking program. The experiment consisted of a critical thinking program implemented as part of the undergraduate medical curriculum of TUMS. The program consisted of 21 hands-on critical thinking skills lessons geared toward medical students in the basic sciences phase. A faculty member with expertise in the field of critical thinking ran the educational sessions, including 46 hours of training. To assist in the implementation of the intervention, at the first step a curriculum was designed consisting of a range of activities involving teaching critical thinking skills. Before pilot testing, the critical thinking program was reviewed by a team of 8 experts including a medical education specialist, a medical school instructor, a cognitive psychologist, and a medical practitioner. During the critical thinking curriculum pilot, both teachers and students expressed appreciation for the curriculum, indicating that it was meaningful and interactive. The training aimed at identifying critical thinking elements (e.g., analyzing arguments in readings), standards of scientific thinking (e.g., examining data for accuracy and precision), fallacies and cognitive errors (e.g., inquiring about availability bias), distinguishing fact versus opinion, and principles of scientific reasoning and presentation (Supplement 1).

Study size

A sample size of 53 students was calculated to represent the finite population of 91 medical students with 1 study group, a continuous primary endpoint, an anticipated mean and standard deviation of the known population of 12.48±3.23 [9], a 10% buffer against drop-out in the study group, an alpha value of 0.05, and test power of 80% using the clincalc.com online sample size calculator.

Statistical methods

Descriptive statistics were used to describe the pre- and post-test scores. The Shapiro-Wilk test of normality was performed for all the variables. Since all variables did not follow a normal distribution, we used nonparametric tests for data analysis. The Wilcoxon signed rank test was performed to compare the mean scores of the pre- and post-tests in different subscales, and the Mann-Whitney nonparametric test was used to compare the mean differences in the pre- and post-test results between male and female participants. All statistical analyses were performed using IBM SPSS ver. 20.0 (IBM Corp., Armonk, NY, USA).

Results

We assessed the critical thinking skills of 56 medical students who completed the pre- and post-tests. Raw response data is available from Dataset 1. The CCTST (form B) was administered before and after critical thinking education during a 4-semester period. The highest score in both the pre- and post-tests was for deductive skills, while the lowest mark at both time points was for analysis (Table 1). Students’ progress was calculated by subtracting the pre-test score from the post-test score. Table 1 presents the changes in the scales. Significant changes were found in the analysis (P=0.03), evaluation (P=0.04), and inductive reasoning (P<0.0001) subscales, but not in the inference (P=0.28) and deductive reasoning (P=0.42) subscales. No significant differences were found between male and female participants (P=0.77). The results are presented in Table 2.

Discussion

Key results

In this study, medical students' critical thinking skills were evaluated as they progressed through the preclinical phase of the undergraduate medical curriculum. Overall, our results showed that the cohort achieved significantly higher total critical thinking scores from entry to the third year of the preclinical phase. A comparison of mean the pre- and post-test reflection scores showed that the analysis, evaluation and inductive reasoning scores improved through the curriculum integrating critical thinking. No significant difference in the pre- or post-test scores was found based on gender.

Interpretation

The results of this study suggest that medical students have a meaningful ability to acquire critical thinking skills and that their critical thinking skills improve after delivery of direct instruction, despite relatively low scores in some critical thinking subscales. After students participated in the program, their total scores for analysis, evaluation, and inductive reasoning on the CCTST (form B) significantly improved, to a greater extent than their inference and deductive reasoning scores. Possible explanations for the statistically significant improvement include the length of the program and the explicit teaching of critical thinking. The mean scores on the pre- and post-tests of the inference and deductive reasoning subscales were lower than those of the other subscales, which may relate to the application of subscales of critical thinking skills in preclinical medical students and how they learn. Another possible explanation for this is that the medical education program at TUMS emphasized students’ capacity to analyze and engage with arguments. In other words, aspects of our program may not have supported growth in students’ hypothesis-testing and inference skills.

Comparison with previous relevant studies

Medical students’ critical thinking skills were enhanced dramatically in all subscales, except for inference and deductive reasoning. In our study, the highest scores were seen for the analysis, inductive reasoning, and evaluation subscales. Smith et al. [10] found a significant improvement in analysis skills, while scores for the induction subscale sharply decreased. Furthermore, the findings of the current study do not support those of other previous studies. Jacob [11] reported that, after using online discussion forums, critical thinking skills showed the most noticeable improvements in the inference and deduction subscales. Chen et al. [12] reported that despite a significant increase in the critical thinking score in the intervention group, only the inference domain showed a significantly higher adjusted mean score. Although these disparities in results can be partially explained by differences among studies in terms of the context, and participants, it may be interpreted that our program and environmental factors can contribute to a consequence that does not support growth of these skills. However, another possible explanation for this is that the medical education program at TUMS emphasized students’ capacity to analyze and engage with arguments. In other words, aspects of our program may not have supported growth in students’ hypothesis-testing and inference skills. In this study, no significant differences were found between the scores of male and female students. A similar study was performed at Isfahan University of Medical Sciences during 2008–2010, using the same questionnaire to assess the critical thinking skills of medical sciences students for 2 sequential semesters, and found some similar results, including the lack of a difference between genders in total scores and relatively high scores for deductive skills [13]. However, they reported no significant improvements, and in that sense, the findings of the study are quite dissimilar to those of the present study [13]. The findings of the current study are consistent with those of Aziz-Fini et al. [14] and Liu et al. [15], who found no significant relationship between nursing students’ score in critical thinking skills and their age and gender.

Limitations

There are some limitations of this study. Only pre- and post-test scores were included, which is inadequate for a full assessment of the longitudinal effect of the intervention. Furthermore, there was no comparison group in this study, which would have been necessary to determine whether the curriculum integrating critical thinking made a difference. In addition, the students knew that completing the questionnaires had no effect on their academic performance, which might have affected the results of the study. Further studies with different teaching strategies, larger sample sizes, and longer follow-up periods may help to achieve a better method to improve critical thinking skills.

Conclusion

Despite its limitations, the present study makes a meaningful contribution by indicating that teaching clinical thinking to undergraduate medical students could improve their critical thinking skills, especially in terms of analysis, inductive, and evaluation skills. However, weaker performance was found for inference and deductive skills. There was no significant difference in critical thinking scores according to gender. Integrating education on critical thinking more widely into pre-clinical undergraduate medical education could enhance the shift towards scientific thinking and reasoning among medical students.

Article information

Authors’ contributions

Conceptualization: AS, MKM, HM. Data curation: AS, MKM, TB. Formal analysis: MT. Funding acquisition: none. Methodology: AS, MKM, FM. Project administration: AS. Visualization: none. Writing–original draft: MKM, MT. Writing–review & editing: AS, MKM, MT, HM, TB, FM.

Conflict of interest

No potential conflict of interest relevant to this article was reported.

Funding

None.

Data availability

Data files are available from Harvard Dataverse: https://doi.org/10.7910/DVN/9DZWRL

Dataset 1. Response data from 56 medical students.

jeehp-18-14-dataset1.xlsx

Acknowledgments

Supplementary materials

References

• 1. Von Colln-Appling C, Giuliano D. A concept analysis of critical thinking: a guide for nurse educators. Nurse Educ Today 2017;49:106-109. https://doi.org/10.1016/j.nedt.2016.11.007 ArticlePubMed
• 2. Chan ZC. A systematic review on critical thinking in medical education. Int J Adolesc Med Health 2016;30:20150117. https://doi.org/10.1515/ijamh-2015-0117 Article
• 3. Ross D, Loeffler K, Schipper S, Vandermeer B, Allan GM. Do scores on three commonly used measures of critical thinking correlate with academic success of health professions trainees?: a systematic review and meta-analysis. Acad Med 2013;88:724-734. https://doi.org/10.1097/ACM.0b013e31828b0823 ArticlePubMed
• 4. Huang GC, Newman LR, Schwartzstein RM. Critical thinking in health professions education: summary and consensus statements of the Millennium Conference 2011. Teach Learn Med 2014;26:95-102. https://doi.org/10.1080/10401334.2013.857335 ArticlePubMed
• 5. Burke SM. Cultivating critical thinking using virtual interactive case studies. J Pediatr Nurs 2017;33:94-96. https://doi.org/10.1016/j.pedn.2016.12.001 ArticlePubMed
• 6. Behar-Horenstein LS, Niu L. Teaching critical thinking skills in higher education: a review of the literature. J Coll Teach Learn 2011;8:25-41. https://doi.org/10.19030/tlc.v8i2.3554 Article
• 7. Mafinejad MK, Arabshahi SK, Monajemi A, Jalili M, Soltani A, Rasouli J. Use of multi-response format test in the assessment of medical students’ critical thinking ability. J Clin Diagn Res 2017;11:LC10-LC13. https://doi.org/10.7860/JCDR/2017/24884.10607 Article
• 8. Khalili H, Hossein Zadeh M. Investigation of reliability, validity and normality Persian version of the California Critical Thinking Skills Test; form B (CCTST). J Med Educ [Internet]. J Med Educ 2003 [cited 2021 May 23];3:29-32. Available from: https://www.sid.ir/en/Journal/ViewPaper.aspx?ID=109238
• 9. Athari Z, Sharif M, Nematbakhsh M, Babamohammadi H. Evaluation of critical thinking skills in Isfahan University of Medical Sciences’ students and its relationship with their rank in university entrance exam rank. Iran J Med Educ [Internet]. 2009 [cited 2021 May 23];9:5-11. Available from: http://ijme.mui.ac.ir/browse.php?%20a_id=924&sid=1&slc_lang=en
• 10. Smith L, Gillette C, Taylor SR, Manolakis M, Dinkins M, Ramey C. A semester-long critical thinking course in the first semester of pharmacy school: impact on critical thinking skills. Curr Pharm Teach Learn 2019;11:499-504. https://doi.org/10.1016/j.cptl.2019.02.014 ArticlePubMed
• 11. Jacob SM. Analyzing critical thinking skills using online discussion forums and CCTST. Procedia Soc Behav Sci 2012;31:805-809. https://doi.org/10.1016/j.sbspro.2011.12.145 Article
• 12. Chen SL, Liang T, Lee ML, Liao IC. Effects of concept map teaching on students’ critical thinking and approach to learning and studying. J Nurs Educ 2011;50:466-469. https://doi.org/10.3928/01484834-20110415-06 ArticlePubMed
• 13. Athari ZS, Sharif SM, Nasr AR, Nematbakhsh M. Assessing critical thinking in medical sciences students in two sequential semesters: does it improve? J Educ Health Promot 2013;2:5. https://doi.org/10.4103/2277-9531.106644 ArticlePubMedPMC
• 14. Azizi-Fini I, Hajibagheri A, Adib-Hajbaghery M. Critical thinking skills in nursing students: a comparison between freshmen and senior students. Nurs Midwifery Stud 2015;4:e25721. https://doi.org/10.17795/nmsjournal25721 ArticlePubMedPMC
• 15. Liu NY, Hsu WY, Hung CA, Wu PL, Pai HC. The effect of gender role orientation on student nurses’ caring behaviour and critical thinking. Int J Nurs Stud 2019;89:18-23. https://doi.org/10.1016/j.ijnurstu.2018.09.005 ArticlePubMed

Figure & Data

References

Citations

Citations to this article as recorded by  

• Health literacy profiles of medical students in an Australian Doctor of Medicine programme: A cross‐sectional study using the Health Literacy Questionnaire
  Margo Lane, Robyn Dixon, Ken J. Donald, Robert S. Ware
  Health Promotion Journal of Australia.2023;[Epub]     CrossRef
• Evaluating and comparing critical thinking skills of residents of Tehran University of Medical Sciences
  Saeed Reza Mehrpour, Amin Hoseini Shavoun, Azita Kheiltash, Rasoul Masoomi, Roya Nasle Seraji
  BMC Medical Education.2023;[Epub]     CrossRef
• Construction of structural correlation of quantitative literacy and critical thinking, and factors affecting them in students of pre-service biology teachers
  M. Arsyad, Sri Rahayu Lestari, Murni Sapta Sari, Fatchur Rohman
  Eurasia Journal of Mathematics, Science and Technology Education.2023; 19(10): em2342.     CrossRef
• Habilidades del pensamiento crítico y liderazgo docente: propuesta con perspectiva de género para la formación inicial
  Liliana Pedraja-Rejas, Carlos Rodríguez-Cisterna
  Revista Venezolana de Gerencia.2023; 28(104): 1667.     CrossRef