Suggestion for item allocation to 8 nursing activity categories of the Korean Nursing Licensing Examination: a survey-based descriptive study

Background/rationale

The Korean Nursing Licensing Examination (KNLE) is a representative criterion-referenced assessment, based on the latest guidelines for nursing practice. It evaluates the minimum knowledge and skills that nursing graduates must possess to efficiently perform their duties [1]. However, the subjects that this licensing examination should cover are under constant debate. It has been suggested that the learning objectives of the 8 major nursing subjects overlap and that the KNLE may not be adequately equipped to appropriately evaluate the job competency of new nursing graduates [2,3]. Hence, continuing efforts are being made to verify the job-based knowledge of nursing graduates and to resolve the issue of overlap among the objectives of nursing subjects [4-7]. As part of this effort, Kim et al. [1] reviewed the feasibility of linking nurse jobs with the licensing examination. Song et al. [3] proposed subjects for an integrated nursing examination by extracting 1,303 statements from 481 knowledge statements. Consequently, 134 activity statements based on graduate nurse job activities in the field of nursing practice and 481 knowledge statements have been developed for the KNLE. These knowledge statements can be grouped into the following 8 activity categories: management of care and improvement of professionalism, safety and infection control, management of potential risk, basic care, physiological integrity and maintenance, pharmacological and parenteral therapies, psychosocial integrity and maintenance, and health promotion and maintenance. However, despite ongoing efforts to design a more appropriate licensing examination, the process of sharing information, collecting opinions, and securing agreement among academic societies and related organizations, such as those involved in nursing education, and nursing practice experts is still insufficient.

Objectives

This study aims to suggest the number of test items in each of 8 nursing activity categories of the KNLE. The specific goals of the study were as follows: first, to elicit opinions on the distribution of the number of test items from 7 major academic societies affiliated with the Korean Society of Nursing Science; and second, to obtain perspectives from 4 professional associations on those opinions from 7 academic societies. It is hoped that the results will help resolve redundancy among the objectives of the 8 nursing subjects by proposing job-based item development for the KNLE.

Ethics statement

Institutional Review Board approval was not required for this process, because the survey questionnaire items did not contain information about the individual characteristics of the participants. Instead, it had information about the policies of societies and associations. No sensitive or individually identifiable information was obtained from the participants.

Study design

This was a survey-based descriptive study and content analysis, described according to the STROBE (Strengthening the Reporting of Observational studies in Epidemiology) statement and 6 steps for conducting thematic analysis [8].

Setting

A job-based item model for the KNLE to resolve redundancy among the objectives of the 8 nursing subjects was provided by the authors (Supplement 1). Two opinion surveys were conducted from members of 7 academic societies to assess the suitability of this model. The first opinion survey was conducted from March 19 to 31, 2021. The second opinion survey was conducted from April 30 to May 14, 2021, after sharing the results of the first survey. Approximately 12–27 members from all academic societies and associations participated in the opinion-collection process. Next, 4 experts from 4 nursing associations reviewed the opinion-survey results from May 21 to June 4, 2021. The review results were summarized by content analysis. Official e-mails were sent to each academic society and association, and the executives of each academic society and association gave their opinions. After summarizing the survey and analysis results, these results were compared with those reported by Tak et al. [9] and the National Council License Examination for Registered Nurses (NCLEX-RN) of the United States [10]. The overall workflow of the study is depicted in Fig. 1. Although a public hearing was held, the content was not reflected in this article. For more information on the public hearing, refer to Supplement 2.

Fig. 1.

Flow of the study. KNLE, Korean Nursing Licensing Examination; NCLEX-RN, National Council License Examination for Registered Nurses.

Participants

Seven academic societies participated in this survey, including the following: the Korean Academic Society of Community Health Nursing, the Korean Academic Society of Nursing Administration, the Korean Academic Society of Fundamentals of Nursing, the Korean Society of Adult Nursing, the Korean Academic Society of Child Health Nursing, the Korean Academic Society of Psychiatric and Mental Health Nursing, and the Korean Society of Women Health Nursing. The 4 expert associations participating in the review were the Korean Society of Nursing Science, the Korean Association of College of Nursing, the Korean Deans Association of Nursing College, and the Korean Hospital Nurses Association. The executives from the 7 academic societies included nursing education experts, while the reviewers from the associations included both nursing educators and practice experts.

Variables

The primary outcome was the number of items for each category.

Data sources/measurement

The results of the opinion survey of the KNLE among the 7 academic societies and 4 associations were used as data sources. The survey questionnaire for 7 academic societies consisted of the items and the responses by the participants (Supplement 3). A more precise source report for the article is available in Supplement 2.

Bias

There was no bias in selecting participants because this was a descriptive study in which no comparison was conducted.

Study size

We did not perform any sample size estimation because this was a descriptive study without any further analysis.

Statistical methods

The collected quantitative data were organized through content analysis by focusing on key concepts and analyzed with descriptive statistics such as frequencies and percentages.

Participants

Table 1 presents the participating societies and associations.

Table 1.

Participating societies and associations

Number of items suggested for the 8 categories by 7 academic societies

The KNLE items comprising 8 activity categories based on the results of the second opinion survey of the 7 member societies are listed in Table 2 (Dataset 1). The exam items included the following: management of care and improvement of professionalism, 41 items; health promotion and maintenance, 33 items; basic care, 37 items; physiological integrity and maintenance, 63 items; psychosocial integrity and maintenance, 41 items; safety and infection control, 16 items; pharmacological and parenteral therapies, 11 items; and management of potential risk, 31 items. In total, 273 items were confirmed. Two items on nursing history were exempted. Another 20 test items related to medical laws were also exempted.

Table 2.

Survey results from 7 academic societies on the number of items in each nursing activity category

Opinions of experts from 4 professional associations

The 4 professional associations emphasized that the safety and infection control activity category and pharmacological and parenteral therapies activity category play major roles in clinical and community nursing practice in real-life situations. Therefore, in the guideline on item-making, the proportion of items in the pharmacological and parenteral therapy activity category increased from 10% (in the guideline on item-making presented by Tak et al. [9] in 2019) to 12%, and that of the safety and infection control activity category also increased from 10% to 12%. The results of the opinion survey of the 4 expert associations on the KNLE standards for each activity category presented by the 7 academic societies mentioned above are summarized in Table 3.

Table 3.

Opinions of experts from the 4 expert associations on the item model for the KNLE (units: number of comments)

Comparison with research on other examination items

We compared the results obtained with those of the NCLEX-RN evaluation conducted in 2019 [9] and those reported by Tak et al. [9] in a study that was also conducted in 2019 (Table 4). Among the items presented by Tak et al. [9], the percentage of items related to the management of care and improvement of professionalism, basic care, psychosocial integrity and maintenance, and health promotion and maintenance activity categories were the same as that determined by the opinion surveys conducted for this study. In the examination subjects developed by Tak et al. [9], the percentage of items in the management of potential risk activity category was 16% and that of items in the physiological integrity and maintenance activity category was 20%, which were higher than those in the NCLEX-RN Test Plan (2019) [10]. Referring to the opinion-survey results, we noted that the maximum percentage of the management of risk potential activity category in the distribution of the NCLEX-RN Test Plan (2019) was 15% (9%–15%) [10]. The percentage of the physiological integrity and maintenance activity category in the NCLEX-RN Test Plan (2019) was adjusted to 17%, corresponding to the maximum range of the distribution (11%–17%) [10]. In the opinion survey of the academies, the percentages of items in the safety and infection control and pharmacological and parenteral therapies activity categories were 4%–6%p lower than those in the item model suggested by Tak et al. [9] and 6%–10%p lower than those in the NCLEX-RN Test Plan (2019) [10].

Table 4.

A revised guideline on item-making based on the model for the Korean Nursing Licensing Examination to expresses the item distribution (item model) based on each activity category

Interpretation

This study proposes a guideline for item-making based on the item model of Tak et al. [9] in 2019 and the NCLEX-RN Test Plan (2019 NCLEX-RN), opinion surveys of 7 academic societies and 4 associations, interviews with experts on clinical and community nursing practice, and the results of a content validity survey. The distribution of items in 4 activity categories—management of care and improvement of professionalism, basic care, physiological integrity and maintenance, and health promotion and maintenance—were maintained. However, the number of items in the management of potential risk and psychosocial integrity and maintenance areas were adjusted downward. The number of items in the areas of pharmacological and parenteral therapies and safety and infection control was increased.

In the expert opinion survey of the associations, some differences were noted in the understanding of the following 4 activity categories: management of potential risk, basic care, physiological integrity and maintenance, and pharmacological and parenteral therapies. To gain a clear understanding of the 8 activity categories, the NCLEX-RN Test Plan should be referred to [10] and the definition and goals for each activity category should be presented more clearly. An opinion was expressed that the 8 activity categories should be arranged systematically in a sequence. Furthermore, a specialist society needs to be established to collect opinions and reach a consensus regarding clinical and community nursing practice. Additionally, the 481 knowledge statements have different levels of specificity for each group. Some of the knowledge statements are not linked to any major subject, while others are linked to 2 or more major subjects. Therefore, the process of collecting opinions and reaching a consensus regarding knowledge statements should be modified and take precedence over other procedures. Because the competency of new nursing graduates is directly related to the quality of nursing and patient safety [11,12], it is necessary to prepare a guideline on item-making that can redirect the aim of the examination to a knowledge-verification model to evaluate examinees’ competency.

Limitations

Given the basic research nature of this study, which investigated the opinions of experts on the job-based integrated national examination model and a guideline on item-making for new nursing graduates, there is a possibility that it lacks objective justification.

Generalizability

The problems and directions for improvement proposed in this study relate to the KNLE. The study results are based on the cultural and situational background of the Republic of Korea. Hence, the results of this study have limited applicability to nursing licensing examinations of other countries or licensing examinations for other job groups.

Suggestions for further studies

Future studies should increase the level of objective legitimacy achieved in this study. For this, a task force team composed of nursing education and practice experts should be formed, and a consensus should be reached through sufficient discussion and careful collection of experts’ opinions.

Conclusion

The study results suggest that the distribution of items in the following 4 activity categories should be maintained: management of care and improvement of professionalism, basic care, physiological integrity and maintenance, and health promotion and maintenance. However, the number of items of the management of potential risk and the psychosocial integrity and maintenance area should be reduced. The number of items in pharmacological and parenteral therapies, and safety and infection control should be increased.