Graphical abstract

Introduction

Background/rationale

In today’s rapidly evolving world, the Partnership for 21st Century Learning (P21) framework emphasizes essential skills for success in academic and professional settings [1]. Integrating these competencies with traditional subjects prepares students to apply knowledge in real-world scenarios, increasing their readiness for future challenges [2]. Supported by UNESCO (United Nations Educational, Scientific, and Cultural Organization), this educational shift calls for innovative policies, curriculum reforms, and a broader understanding of education’s role in fostering global citizenship, technological literacy, and ethical leadership [3].

Thailand’s Community Health Professional Council has outlined competencies aligned with 21st-century skills, reinforcing abilities such as critical thinking, communication, and collaboration essential for tackling public health challenges. These competencies highlight the need for research focusing on bachelor of public health programs in Thailand, supporting skills required for modern public health roles.

Measuring 21st-century skills provides insights into student capabilities, influencing course design and encouraging the teaching of these skills [4]. However, assessing these skills is challenging, particularly in tracking student progress [5]. The emphasis on embedding these skills into curricula has increased the need for reliable and valid assessment instruments. Measuring creativity, critical thinking, and collaboration requires clear frameworks that translate these abstract concepts into measurable competencies. Educational research has increasingly employed methods such as factor analysis to ensure validity and reliability [6]. Recent studies have developed diverse scales to measure 21st-century skills, emphasizing the need for context-specific tools tailored to various disciplines and student populations [7-9].

Despite this progress, there remains a gap in assessing 21st-century skills among public health undergraduates. This study addresses this gap by applying the P21 framework to a culturally specific public health context, emphasizing the unique contributions of the 21st Century Skills Assessment Scale (21CSAS). Unlike traditional health education assessments that focus on knowledge retention, 21CSAS emphasizes critical competencies such as communication and creativity, which are crucial for culturally adaptive health practices. Tailored assessments that incorporate community-specific beliefs make 21CSAS particularly valuable in diverse public health settings.

Objectives

The purpose of this study was to develop and validate a 21st-century skills assessment scale for Thai public health undergraduates using the P21 framework.

Methods

Ethics statement

The study was approved by the Human Research Ethics Committee of the Institute for Population and Social Research, Mahidol University (COA no., 2022/09-187) on December 14, 2022. Informed consent was acquired from all participants.

Study design

This study was a cross-sectional survey to validate an assessment tool.

Setting

This study was conducted in the setting of bachelor of public health programs of Thai autonomous universities accredited by the Community Public Health Council [10]. Data were collected between January and March 2023.

Participants

A 2-stage sampling method was used. In the first stage, 12 autonomous universities with accredited bachelor of public health programs were grouped into 4 regions. In the second stage, one university from each region was selected through simple random sampling. Data were collected from first to fourth-year students in good health during the 2022 academic year. Between 180–200 eligible students were recruited from each university.

Data sources/measurement

The 21CSAS was developed based on the P21 framework, focusing on 3 key domains: (1) learning and innovation skills (LI), which are critical for students to thrive in increasingly complex life and work environments. This domain includes creativity, critical thinking, communication, and collaboration; (2) information, media, and technology skills (IMT), which refer to students’ abilities to access, analyze, evaluate, and produce information and media effectively, accurately, and ethically; and (3) life and career skills (LC), which encompass work-life balance, professionalism, ethics, and the ability to work effectively across cultural differences.

The preliminary 21CSAS consisted of 21 items drafted from the definitions of the 3 domains with content validity confirmed by 3 experts, resulting in a scale content validity index (CVI) of 0.91 and item CVIs from 0.8 to 1. Face validity was validated through feedback from 30 public health undergraduates.

The questionnaire, consisting of 21 items (LI: 10, IMT: 5, LC: 6), was rated on a 5-point scale (1=strongly disagree to 5=strongly agree) and administered once to 780 participants. Of these, 727 responses (93.21%) were completed and deemed suitable for analysis. The dataset was divided into 2 subsets: one with 300 participants for exploratory factor analysis (EFA) and another with 427 participants for confirmatory factor analysis (CFA) (Dataset 1).

Variables

Demographic variables included gender and academic year. The 21CSAS consisted of 21 items, with latent variables to be identified as part of the analysis.

Bias

Self-reports can be biased, such as when participants provide socially acceptable answers (social desirability bias) or struggle to recall past behaviors (recall bias). Differences in understanding or motivation may also cause inconsistencies. To reduce biases, the following measures were taken: ensuring anonymity, using validated scales, and asking neutral questions. Nonetheless, the results should be interpreted carefully.

Study size

The sample size calculation followed the N:q rule of thumb, where N represents the number of cases, and q denotes the number of model parameters [6]. For this study, a recommended ratio of 15–20:1 for 20 parameters, a minimum of 600 participants was required across 2 datasets (300 each) for the factor analysis (EFA and CFA) utilized in this study. To account for potential data loss, 30% more participants were included, resulting in a final sample size of 780.

Statistical methods

Mplus software ver. 8.10 (Muthén & Muthén; license no., SABC80022922) with the maximum likelihood robust estimator was used to perform EFA using oblique rotation with the Geomin method. Additionally, CFA was conducted. The quality of fit of the models was assessed by meeting reference values of good quality indicators (Table 1) [11].

Results

Participants

The sample comprised 727 undergraduate students enrolled in bachelor of public health programs. The majority (87.8%, n=638) were women. The distribution by academic year was as follows: 27.1% (n=197) third-year, 26.1% (n=190) first-year, 24.8% (n=180) second-year, and 22% (n=160) fourth-year students.

Main results

Exploratory factor analysis

EFA was performed on an initial 21CSAS using a dataset of 300 participants. Initially containing 21 items, 1 item was removed, leaving a final version of 20 items. The EFA revealed a 6-factor structure with good model fit (Tables 1–3). The 6 identified dimensions are as follows: (1) Creativity and innovation skills (CI): Refers to the ability to proactively identify problems, apply knowledge to real-world situations, and develop innovative solutions through the design thinking process, which includes empathy, defining the problem, ideation, prototyping, and testing (3 items, loadings 0.32–0.77). (2) Critical thinking and problem-solving skills (CP): Refers to the capacity to think systematically and make decisions by applying logical reasoning to effectively solve problems (3 items, loadings 0.63–0.85). (3) Communication and collaboration skills (CC): Refers to the ability to work well in teams, share responsibility, and lead others ethically and collaboratively to achieve common goals (3 items, loadings 0.40–0.75). (4) IMT: Refers to the ability to analyze, create, and manage media content accurately and ethically across various platforms, using electronic devices and applications effectively while handling informational data appropriately (5 items, loadings 0.31–0.93). (5) Initiative and self-direction skills (IS): Refers to the abilities that help individuals grow and improve in both their personal lives and careers, including effective learning, curiosity, and life skills that contribute to achieving fulfillment, balance, and success (3 items, loadings 0.50–0.95). (6) Social and cross-cultural skills (SC): Refers to the ability to work effectively with people from diverse backgrounds by showing respect and understanding cultural differences (3 items, loadings 0.65–0.94).

Confirmatory factor analysis

CFA confirmed the factor structure using a dataset of 427 subjects. The 21CSAS model demonstrated a good overall fit, explaining approximately 34%–71% of the variance among items. Convergent validity was supported by item loadings ranging from 0.58 to 0.84, while discriminant validity was indicated by correlations between constructs ranging from 0.52 to 0.79 (Tables 1–3, Fig. 1).

The 21st Century Skills Assessment Scale for Thai public health students

The 21CSAS demonstrated strong psychometric properties, with notable item-discrimination and high reliability (Table 4). Both EFA and CFA validated the 6-dimensional structure, establishing the 21CSAS as a dependable and effective instrument for assessing 21st-century skills in Thai public health undergraduates.

Discussion

Key results

The 21CSAS is a validated tool developed to assess key 21st-century skills among public health undergrad students in Thailand. It evaluates 6 critical skill areas essential for modern public health practice: CI, CP, CC, IMT, IS, and SC. Notably, the 21CSAS has demonstrated both validity and reliability, effectively distinguishing skill levels across different academic years. Analysis reveals that 4th-year students achieve the highest average scores, followed by 3rd-year, 2nd-year, and 1st-year students.

Interpretation

The 21CSAS framework is vital for equipping students with the skills needed to address complex, real-world challenges. By emphasizing creativity, critical thinking, communication, and collaboration, it prepares individuals to thrive in dynamic professional environments such as public health. These competencies serve as cornerstones for tackling modern health challenges and driving meaningful solutions.

CI is the ability to generate innovative ideas and approach problems with originality. This skill empowers students in public health to think beyond conventional solutions, fostering creativity in addressing pressing issues. CI’s dimensions (ci1, ci2, ci3) reflect its role in encouraging students to develop unique strategies and collaborate effectively. By promoting creativity, students gain the tools to engage with diverse perspectives and offer groundbreaking solutions in evolving contexts.

CP centers on evaluating situations, identifying challenges, and formulating logical solutions. Public health professionals must think systematically and make decisions based on sound reasoning to address multifaceted problems. The cp1, cp2, and cp3 dimensions highlight the importance of structured, critical approaches to problem-solving, which are crucial for navigating complex health scenarios. This skill helps students break down problems and come up with practical solutions.

CC involve exchanging ideas, sharing information, and working cohesively to achieve shared goals. These dimensions (cc1, cc2, cc3) emphasize the necessity of clear communication and teamwork in resolving health-related challenges. Public health relies heavily on coordinated efforts, and the ability to work well in teams is pivotal. The interplay between creativity, problem-solving, and collaboration demonstrates the interdependence of these skills in achieving impactful results.

IMT emphasize the ability to use digital tools, critically assess media content, and ethically apply technology. Public health professionals must be adept at navigating digital platforms, interpreting data, and leveraging technology to design effective interventions. Dimensions imt1 through imt5 reflect the growing reliance on these skills in modern practice, highlighting the need to integrate technological literacy into problem-solving and communication.

IS focus on goal-setting, self-motivation, and independent learning. These skills, represented by is1, is2, and is3, enable students to take charge of their education and remain adaptable in evolving fields. Public health professionals benefit from initiative by continuously updating their knowledge and skills to address emerging challenges. Self-direction fosters resilience and equips students to adapt to various professional demands.

SC emphasize collaboration across diverse backgrounds. In today’s interconnected world, public health professionals must work effectively with individuals from various cultural and social contexts. Measured by sc1, sc2, and sc3, these skills promote inclusivity and adaptability. A connection between initiative and cross-cultural competencies, as self-directed learners are often better equipped to navigate diverse environments and build effective relationships.

By developing these essential skills, the 21CSAS framework prepares public health students to meet the field’s complex and evolving demands. Integrating creativity, problem-solving, communication, technological literacy, self-direction, and cross-cultural collaboration into education enhances job readiness and professional competence. These skills enable students to tackle pressing health challenges, advocate for health equity, and contribute meaningfully to improving global health outcomes. Through this holistic approach, the framework cultivates professionals who are not only capable of addressing contemporary health needs but are also prepared to adapt and thrive in an ever-changing landscape.

Comparison with previous studies

The 21CSAS improves previous tools by addressing weaknesses in public health education. Previous assessments, such as those validated by Cevik and Senturk [7], often lacked focus on field-specific competencies and did not fully reflect current educational demands. In contrast, the 21CSAS is tailored to these needs, with questions developed from a comprehensive literature review. Additionally, its 5-point Likert scale minimizes response bias and enhances the accuracy of results. The 21CSAS also aligns with findings from Castanheira et al. [8] regarding skill development in achieving educational goals and addresses the need for effective evaluation tools in health professions education, as highlighted by Huang et al. [9].

Public health professionals differ from students in other programs through their focus on community health and preventive measures rather than individual patient care. They are trained to understand population dynamics, social determinants of health, and systemic interventions. This necessitates skills in epidemiology, biostatistics, and health policy analysis, which are less emphasized in traditional medical training.

Additionally, the 21CSAS aligns with the competencies outlined by the Thai Community Health Professions Council, highlighting leadership, advocacy, communication, and ethical practice. By incorporating these core competencies and addressing emerging public health issues such as climate change and pandemics, the 21CSAS effectively prepares future public health leaders to confront pressing global health challenges.

Limitations

The predominantly female sample (88%) limits generalizability to gender-diverse populations. The cross-sectional design prevents causal inferences on academic year and skill development, complicating long-term impact assessment. Self-reported measures may introduce bias, as students might overestimate their skills. Additionally, the 21CSAS lacks predictive validity testing, leaving its ability to forecast real-world public health performance unexplored.

Generalizability

Focused on public health undergraduates in Thailand, this study may have broader relevance to similar educational settings in Asia and Europe. The developed assessment scale, based on the P21 framework, can extend beyond public health to fields such as health and social sciences. Adapting this scale across disciplines enables educators to assess and develop essential skills effectively, preparing students for workforce challenges and enhancing curriculum development.

Suggestions

To improve generalizability, future studies should apply the 21CSAS across diverse disciplines and cultural settings. Combining various data sources (e.g., peer assessments, direct observations, and longitudinal designs) will offer a deeper understanding of students’ 21st-century skills. Educators should analyze assessment data to pinpoint areas of difficulty, which they can address using targeted teaching methods and support. Longitudinal studies testing predictive validity would enhance the scale’s applicability and better prepare students for real-world public health challenges.

Conclusion

In conclusion, the 21CSAS demonstrates strong psychometric properties, serving as a valuable tool for assessing public health students’ 21st-century skills. The study underscores the importance of collaboration among educational institutions, public health agencies, and communities to align curricula with the evolving demands of the field, ensuring that future professionals are well-prepared to meet emerging challenges and improve the health of diverse populations.

Supplementary materials

Fig. 1.

Confirmatory factor analysis model of the 21st Century Skills Assessment Scale. CI, creativity and innovation skills; CP, critical thinking and problem-solving skills; CC, communication and collaboration skills; IMT, information, media, and technology skills; IS, initiative and self-direction skills; SC, social and cross-cultural skills.

References

• 1. Partnership for 21st Century Learning. Framework for 21st century learning definitions [Internet] Battelle for Kids; 2019 [cited 2024 Aug 3]. Available from: https://www.battelleforkids.org/wp-content/uploads/2023/11/P21_Framework_DefinitionsBFK.pdf
• 2. Trilling B, Fadel C. 21st century skills: learning for life in our times. John Wiley & Sons; 2009. 243 p.
• 3. UNESCO, Education Sector, Section of Education Policy. The right to education in the 21st century: findings from the international seminar on the evolving right to education [Internet] UNESCO; 2022 [cited 2024 Aug 20]. Available from: https://unesdoc.unesco.org/ark:/48223/pf0000381108/PDF/381108eng.pdf.multi
• 4. Koretz D. The testing charade: pretending to make schools better. University of Chicago Press; 2017. 288 p.
• 5. Holec V, Marynowski R. Does it matter where you teach?: insights from a quasi-experimental study of student engagement in an active learning classroom. Teach Learn Inq 2020;8:140-164. https://doi.org/10.20343/teachlearninqu.8.2.10 Article
• 6. Kline RB. Principles and practice of structural equation modeling. 5th ed. Guilford Press; 2023. 427 p.
• 7. Cevik M, Senturk C. Multidimensional 21st Century Skills Scale: validity and reliability study. Cypriot J Educ Sci 2019;14:11-28. https://doi.org/10.18844/cjes.v14i1.3506 Article
• 8. Castanheira CA, Jardim J, Oliveira CM. Validation of the competences for the 21st Century Skill Scale in higher education students. Cypriot J Educ Sci 2023;18:286-296. https://doi.org/10.18844/cjes.v18i1.7950 Article
• 9. Huang PH, O’Sullivan AJ, Shulruf B. Development and validation of the student ratings in clinical teaching scale in Australia: a methodological study. J Educ Eval Health Prof 2023;20:26. https://doi.org/10.3352/jeehp.2023.20.26 ArticlePubMedPMC
• 10. Courses approved by the Thai Community Public Health Council [Internet] Thai Community Public Health Council; 2022 [cited 2024 Aug 20]. Available from: https://www.ccph.or.th/diplomacy/
• 11. Hair JF Jr, Black WC, Babin BJ, Anderson RE. Multivariate data analysis. 8th ed. Cengage; 2019. 834 p.

Figure & Data

References

Citations

Citations to this article as recorded by