, Jin Seo Lee2
, Seon Young Jeong3
, Young Soon Park4*
1Department of Medical Education, Konyang University College of Medicine, Daejeon, Korea
2Division of Infectious Disease, Department of Internal Medicine, Kangdong Sacred Heart Hospital, Hallym University College of Medicine, Seoul, Korea
3Department of Nursing, Konyang University College of Nursing, Daejeon, Korea
4Department of Medical Education, Catholic Kwandong University College of Medicine, Gangneung, Korea
© 2025 Korea Health Personnel Licensing Examination Institute
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Authors’ contributions
Conceptualization: KHC. Data curation: KHC, YSP. Methodology/formal analysis/validation: KHC, JSL, SYJ, YSP. Funding acquisition: JSL. Writing–original draft: KHC, YSP. Writing–review & editing: KHC, JSL, SYJ, YSP.
Conflict of interest
No potential conflict of interest relevant to this article was reported.
Funding
This research was supported by funding for research from the Korea Disease Control and Prevention Agency (11-1790387-100532-01). This study was conducted as part of the Korea Disease Control and Prevention Agency’s private commissioned project “2022 Project for Strengthening Regional Infectious Disease Response Capacity.” The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Data availability
Data files are available from https://doi.org/10.7910/DVN/VKJTWD
Dataset 1. Pre-test and post-test raw scores.
Acknowledgments
None.
| Mean±SD | F-value | Post hoc (Bonferroni) | |
|---|---|---|---|
| Time point | 12.922*** | 2, 3> 1 | |
| 1. Pre-training | 9.53±2.03 | ||
| 2. Immediate post-training | 10.91±2.04 | ||
| 3. 1-month follow-up | 10.48±1.81 |
Values are category means calculated from individual item scores. Detailed item-level data and level definitions are provided in Supplement 2.
SD, standard deviation; COVID-19, coronavirus disease 2019; PPE, personal protective equipment.
Values are category means calculated from individual item scores. Detailed item-level data and level definitions are provided in Supplement 3.
SD, standard deviation; COVID-19, coronavirus disease 2019.
| CIPP model | Kirkpatrick | |||
|---|---|---|---|---|
| L1. Program satisfaction survey/interview | L2. Learning achievement assessment | L3. Workplace application assessment | L4. Organizational contribution assessment | |
| Context | ●○ | |||
| Input | ●○ | |||
| Process | ●○ | |||
| Product | ●○ | ● | ● | ● |
| Clinical practice type | Male | Female | Total |
|---|---|---|---|
| Endoscopic procedures | 11 (11.0) | 0 (0.0) | 11 (11.0) |
| Intravenous therapy & injection procedures | 12 (12.0) | 3 (3.0) | 15 (15.0) |
| Surgery (partial anesthesia) | 9 (9.0) | 1 (1.0) | 10 (10.0) |
| Surgery (general anesthesia) | 10 (10.0) | 2 (2.0) | 12 (12.0) |
| Consultation & prescription | 39 (39.0) | 13 (13.0) | 52 (52.0) |
| Total | 81 (81.0) | 19 (19.0) | 100 (100.0) |
| Factor (CIPP) | No. of items | Cronbach’s α | Description |
|---|---|---|---|
| Context | 3 | 0.972 | Perceptions of program necessity and learning outcomes |
| Input | 3 | 0.869 | Ease of voluntary participation, registration, and guidance |
| Process | 8 | 0.977 | Appropriateness of objectives, content, instructors, platform, Q&A |
| Product | 7 | 0.987 | Impact on personal/organizational competency, overall satisfaction |
| Total | 21 | 0.989 | Reliability for all items |
| Domain | Instrument | No. of items | Time of administration | Notes |
|---|---|---|---|---|
| Quantitative (Kirkpatrick) | ||||
| Level 1 | Training satisfaction (survey) | 21 | Immediately post-training | 5-point Likert scale |
| Level 2 | Learning achievement (test) | 15 | Pre-training, post-training, 1-month follow-up | True/false multiple-choice items |
| Level 3 | Practical application (survey) | 45 | 1-month follow-up | 5-point Likert scale |
| Level 4 | Organizational contribution (survey) | 15 | 1-month follow-up | 5-point Likert scale |
| Qualitative | Semi-structured interview guide | - | 1-month follow-up | Individual interviews |
| CIPP domain | Item | Mean±SD |
|---|---|---|
| Context | Awareness of program purpose | 3.98±0.62 |
| Perception of program necessity | 4.04±0.62 | |
| Awareness of target competency | 4.00±0.64 | |
| Input | Voluntary participation | 3.97±0.78 |
| Ease of pre-registration | 3.98±0.84 | |
| Clarity of pre-program guidance/schedule | 3.99±0.70 | |
| Process | Clarity of learning objectives | 4.00±0.68 |
| Appropriate program volume/length | 3.75±0.77 | |
| Appropriate difficulty level | 3.74±0.68 | |
| Content appropriate for achieving goals | 3.76±0.77 | |
| Instructor quality | 3.99±0.73 | |
| Quality of materials | 3.88±0.77 | |
| Online platform stability | 4.08±0.75 | |
| Availability of Q&A | 3.86±0.79 | |
| Product | Helped improve personal competency | 4.06±0.69 |
| Helped improve organizational competency | 3.91±0.75 | |
| Helpful for infectious disease response | 4.03±0.66 | |
| Motivation to apply learning in organization | 3.91±0.74 | |
| Overall satisfaction | 4.07±0.77 | |
| Willingness to recommend to colleagues | 4.04±0.72 | |
| Willingness to participate in future training | 4.09±0.71 | |
| Overall | 3.96±0.72 |
| Mean±SD | F-value | Post hoc (Bonferroni) | |
|---|---|---|---|
| Time point | 12.922 |
2, 3> 1 | |
| 1. Pre-training | 9.53±2.03 | ||
| 2. Immediate post-training | 10.91±2.04 | ||
| 3. 1-month follow-up | 10.48±1.81 |
| LOs | Specific content | Participant feedback summary |
|---|---|---|
| LO-A (COVID-19) | - Characteristics of new variants | “Helpful for actual patient care.” |
| - Vaccine goals | “The module on vaccine goals was most helpful for explaining to patients.” | |
| - Efficacy of vaccines/prior infection | “Long-COVID patterns were very relevant.” | |
| - Long COVID patterns | ||
| LO-B (respiratory viruses) | - Main transmission modes | “Helpful for actual practice.” |
| - High-risk environments for aerosol spread | “The module on prevention in the clinical environment was the most useful.” | |
| - Prevention in the clinical setting | ||
| LO-C (emerging diseases & reporting) | - Emergence of novel diseases | “Mpox was interesting, even if rare.” |
| - Mpox transmission, PPE | “I had an actual mpox case, so this was extremely helpful.” | |
| - Mpox clinical presentation, screening | “Learned the correct PPE for suspected cases.” | |
| - Reportable disease classification and reporting procedures | “The reporting procedures were very useful.” | |
| LO-D (equipment reprocessing) | - Concept and process of reprocessing | “This module felt a bit slow/dull.” |
| - Sterilization/disinfection by equipment type | “I've always been interested in sterilization.” | |
| - Reprocessing workflow in clinics | “This was new information for me; I’m glad I learned it.” | |
| “For me, LO-D was the most helpful part of the entire program.” |
| Topic category | No. of items | Mean±SD |
|---|---|---|
| COVID-19 general (1–5) | 5 | 3.33±0.90 |
| COVID-19 prevention (6–8) | 3 | 3.31±0.92 |
| Mpox & PPE (9–10) | 2 | 2.87±1.04 |
| Equipment reprocessing (11–15) | 5 | 3.11±0.95 |
| Overall | 15 | 3.19±0.86 |
| Topic category | No. of items | Mean±SD | |
|---|---|---|---|
| Development | Implementation | ||
| COVID-19-related (S1–S4/P1–P4) | 8 | 3.15±0.81 | 3.15±0.83 |
| Respiratory transmission (S5–S7/P5–P7) | 6 | 3.24±0.87 | 3.21±0.87 |
| Novel diseases (S8–S10/P8–P10) | 6 | 2.98±0.97 | 2.93±0.98 |
| Equipment/reporting (S11–S15/P11–P15) | 10 | 3.23±0.85 | 3.23±0.88 |
| Overall | 30 | 3.13±0.83 | 3.13±0.85 |
| Item (perceived awareness/practice of staff/patients regarding...) | Responses (%) | Mean±SD | ||||
|---|---|---|---|---|---|---|
| Strongly disagree | Disagree | Neutral | Agree | Strongly agree | ||
| 1. Characteristics of new COVID-19 variants | - | 11 | 41 | 40 | 8 | 3.45±0.80 |
| 2. Main goals of COVID-19 vaccination | - | 12 | 32 | 47 | 9 | 3.53±0.82 |
| 3. Effect of vaccines/prior infection on prevention | - | 11 | 34 | 45 | 10 | 3.54±0.82 |
| 4. Patterns of long COVID | - | 14 | 37 | 42 | 7 | 3.42±0.82 |
| 5. Main transmission modes of respiratory viruses | 1 | 11 | 30 | 46 | 12 | 3.57±0.88 |
| 6. High-risk environments for aerosol spread | - | 8 | 35 | 46 | 11 | 3.60±0.79 |
| 7. Prevention of respiratory virus transmission in clinic | - | 7 | 38 | 45 | 10 | 3.58±0.77 |
| 8. Emergence and impact of novel diseases | - | 8 | 43 | 39 | 10 | 3.51±0.78 |
| 9. Staff use appropriate PPE for suspected mpox | 4 | 17 | 38 | 34 | 7 | 3.23±0.95 |
| 10. Staff screen/diagnose suspected mpox | 4 | 22 | 40 | 26 | 8 | 3.12±0.98 |
| 11. Staff awareness of reporting classification/methods | 1 | 13 | 43 | 35 | 8 | 3.36±0.85 |
| 12. Staff awareness of reprocessing concept/process | - | 6 | 42 | 44 | 8 | 3.54±0.73 |
| 13. Staff select appropriate disinfection/sterilization | - | 3 | 43 | 44 | 10 | 3.61±0.71 |
| 14. Staff verify sterilization status of reprocessed equipment | - | 5 | 48 | 37 | 10 | 3.52±0.75 |
| 15. Staff apply reprocessing procedures in the institution | - | 4 | 48 | 39 | 9 | 3.53±0.72 |
| Overall | 3.47±0.70 | |||||
●: quantitative method, ○: qualitative method.
Values are presented as number (%).
SD, standard deviation.
SD, standard deviation. P<0.001.
LO, learning outcomes; COVID-19, coronavirus disease 2019; PPE, personal protective equipment.
Values are category means calculated from individual item scores. Detailed item-level data and level definitions are provided in SD, standard deviation; COVID-19, coronavirus disease 2019; PPE, personal protective equipment.
Values are category means calculated from individual item scores. Detailed item-level data and level definitions are provided in SD, standard deviation; COVID-19, coronavirus disease 2019.
SD, standard deviation; COVID-19, coronavirus disease 2019; PPE, personal protective equipment.