KHPLEI
Search
- Page Path
- HOME > Search
Research articles
-
Validity evaluation of a mock examination including multimedia-assisted items for the Korean Nursing Licensing Examination: a mixed-methods study with a counterbalanced design
-
Sujin Shin
, Inyoung Lee, Heejeong Kim, Rhayun Song, Jun-Ah Song, Hyun Sook Yi, Janghee Park, Mi Kyoung Yim, Minjae Lee, Subin Yu - J Educ Eval Health Prof. 2026;23:21. Published online July 21, 2026
- DOI: https://doi.org/10.3352/jeehp.2026.23.21 [Epub ahead of print]
- 631 View
- 106 Download
-
Abstract
PDF - Purpose
This study aimed to evaluate the validity of multimedia-assisted items (MAIs) in a mock examination by examining their psychometric properties and nursing students’ perceptions.
Methods
This methodological study used a within-group counterbalanced design in which fourth-year nursing students in South Korea completed an online mock examination consisting of paired text-based items and MAIs. Psychometric properties were examined using classical test theory and item response theory, and learner perceptions and response processes were explored through post-examination surveys and focus group interviews.
Results
Among 515 participants, MAIs demonstrated overall difficulty and discrimination comparable to those of text-based items in both classical test theory and item response theory analyses, with similar test characteristic curves across counterbalanced groups, supporting measurement equivalence across item formats. Post-test surveys indicated that participants perceived MAIs as realistic reflections of clinical practice and as more helpful for problem solving and ability assessment than text-based items. Qualitative findings from focus group interviews (n=15) further supported the educational value of MAIs for assessing clinical judgment while emphasizing the need for appropriate multimedia use. An order effect was also observed: presenting MAIs first led to higher item response theory discrimination parameters in subsequent text-based items (P=0.020).
Conclusion
MAIs demonstrated psychometric properties comparable to those of text-based items and were positively perceived as valid tools for assessing clinical judgment and practical competence. These findings support the feasibility of incorporating multimedia-based assessment into the Korean Nursing Licensing Examination as part of a rigorous computer-based testing framework.
- Proposal for setting a passing score for the Korean Nursing Licensing Examination
-
Janghee Park, Mi Kyoung Yim, Sujin Shin, Rhayun Song, Jun-Ah Song, Inyoung Lee, Heejeong Kim, Minjae Lee
- J Educ Eval Health Prof. 2025;22:25. Published online September 8, 2025
- DOI: https://doi.org/10.3352/jeehp.2025.22.25
- 2,830 View
- 246 Download
-
Abstract
PDF
Supplementary Material - Purpose
The Korean Nursing Licensing Examination (KNLE) is planning to transition to a computer-based test (CBT). This study aims to propose a reasonable and efficient method for setting passing scores.
Methods
A standard-setting (passing score setting) analysis was conducted using an expert panel over the past 3 years of the national nursing examination. The standard-setting method was modified from Angoff, and the validity of the passing score was verified through the Hofstee method. The standard-setting workshop was conducted in 2 stages: first, a pilot workshop for 2 subjects, followed by a second workshop where 6 additional subjects were selected based on the pilot results. For items with an actual correct answer rate of 90% or higher, the estimated correct answer rate for minimum competency was calculated using the observed correct answer rate. A survey and discussion with the expert panel were also conducted regarding the standard-setting procedures and results.
Results
The passing score for the national nursing examination was calculated using the new method, and the score was slightly higher than the existing score. The nursing subject had similar results; however, the legal subjects varied.
Conclusion
The modified Angoff and Hofstee methods were successfully applied to the KNLE. Using the actual correct answer rate as an indicator to derive expected minimum competency was shown to be effective. This approach could streamline future standard-setting processes, particularly when converting to CBT.
First
Prev
