Revised evaluation objectives of the Korean Dentist Clinical Skill Test: a survey study and focus group interviews

Background/rationale

In 2009, the Korean Dental Association decided to introduce a clinical skill test as part of the Korean Dental Licensing Examination (KDLE). The objectives of the clinical skill test were developed to assess the minimal competencies that new dentists should possess, based on dental practitioners’ competencies in conjunction with the dental practitioner job analysis. In 2011, national-level dental practitioner competencies were developed to establish evaluation objectives for the Korean Dentist Clinical Skill Test [1]. The Korean Health Personnel Licensing Examination Institute (KHPLEI) conducted the first dental practitioner job analysis in 2000 and the second dental practitioner job analysis in 2012 [2]. In 2013, a study was conducted on the categories and item types of the clinical skill test, revising the selection criteria of the test’s items within the national-level dental practitioner competencies, which were developed in 2011 (Supplement 1) [3]. The researchers extracted “task elements” that could be tested in the clinical skill test from the second dental practitioner job analysis and correlated 36 extracted task elements with the national-level dental practitioner competencies. Subsequently, the affairs committee determined the clinical skill test’s assessment categories and types of test items.

In 2017, KHPLEI published evaluation objectives for the Korean Dentist Clinical Skill Test, which currently consists of process and outcome evaluations (Table 1). The “process evaluation” is administered in the form of an objective structured clinical skill examination, which includes a combination of one or more of the 9 domains of assessment [4]. The “outcome evaluation” is a form of bench test using a dental mannequin simulator. The evaluation objectives list 22 items to be evaluated (Table 2).

Table 1.

Item types and evaluation areas of the Korean Dentist Clinical Skill Test

Table 2.

The evaluation objectives of the Korean Dentist Clinical Skill Test

The first clinical skill test took place in 2021, and 3 examinations were conducted by the end of 2023. In 2023, the third dental practitioner job analysis was published [5]. Given that the competencies necessary for dentists, dental education, and the practice environment are continually evolving, it is imperative to periodically review the evaluation objectives of the Korean Dentist Clinical Skill Test.

Objectives

The study aimed to propose a plan for revising the Korean Dentist Clinical Skill Test based on the opinions of those involved in the examination after reviewing the evaluation objectives based on the national-level dental practitioner competencies, dental school educational competencies, and the third dental practitioner job analysis.

Ethics statement

This study was approved by the Institutional Review Board (IRB) of Kyung Hee University Dental Hospital (IRB no., KH-DT23032).

Study design

This was a survey-based study with focus group interviews. The literature review on the clinical skill test objectives was based on the national-level dental practitioner competencies, dental school educational competencies, and the third dental practitioner job analysis.

Setting

Reviewing the validity of the clinical skill test’s evaluation objectives

The design for examining the validity of the evaluation objectives of the Korean Dentist Clinical Skill Test is presented in Fig. 1. The evaluation objectives were reviewed based on the national-level dental practitioner competencies and the educational competencies of dental schools across the country. Next, the examination objectives were reviewed based on the results of the third dental practitioner job analysis.

Fig. 1.

Study design for validating the evaluation objectives of the Korean Dental Clinical Skill Test.

Educational competency data from 11 dental schools nationwide were collected. From the educational competency data and the national-level dental practitioner competencies, “competency elements” relevant to the 22 clinical skill test objectives were extracted. The authors discussed the validity of the competency elements as clinical skill test objectives and the feasibility of test item development for those competency elements.

The third dental practitioner job analysis published in 2023 was based on the second dental practitioner job analysis [5]. In the third dental practitioner job analysis, dental practitioners working in private dental clinics with at least 5 years of practical experience were asked to rate their perceptions of task importance using a 5-point Likert scale (1, not important: 2, somewhat important; 3, moderately important: 4 points, important: 5, extremely important). In this study, task elements related to dentist competency with an average importance rating of 4 or higher were selected. The authors assessed the feasibility and ease of implementing the task elements as examination objectives.

Participants

The examinees were students who attempted the 76th KDLE in 2023, and the former examinees were licensed dentists who had taken the KDLE in 2021 and 2022. The participating professors were those who had participated in the examination as either test item developers or test examiners. The dental school faculty members were affiliated with the 11 dental schools in Korea.

Data sources/measurement

Three surveys were conducted to gather data for this study. The first survey targeted current and former examinees, assessing their perceptions of the evaluation objectives and the alignment of these objectives with their school curricula (Supplement 2). The second and third surveys were administered to the participating professors and dental school faculty members, respectively, focusing on their perceptions of the validity of the 22 evaluation objectives (Supplements 3, 4). For the participating professors (test item developers and test examiners), an additional item on the overlap between the 22 evaluation objectives and the dental specialty areas was included (Supplement 3). All surveys employed a 5-point Likert scale.

Bias

All participated in the survey voluntarily. There may have been selection bias because individuals with a positive attitude toward the clinical skill test may have participated in the survey more actively.

Study size

The sample size was not predetermined, as all participants took part voluntarily.

Statistical methods

The survey responses were subjected to descriptive analysis. The chi-square test was used to compare responses between the groups. All statistical analyses were performed using IBM SPSS ver. 25.0 (IBM Corp.), and the statistical significance level was set at α=0.05.

Focus group interviews with clinical skill test experts

Two focus group interviews were conducted with experts with extensive experience participating in the clinical skill test for the process and outcome evaluations. During the focus group interviews, participants discussed which evaluation objectives should be retained, deleted, revised, or added. Each focus group interview included 5 clinical skill test experts.

Reviewing the validity of the clinical skill test’s evaluation objectives

When the extracted competency elements were compared to the evaluation objectives, it was found that some competency elements were not included in the clinical skill test. These competency elements were “periodontal curettage,” “supportive periodontal therapy,” “diagnosis and treatment planning–simple soft tissue diseases,” “incision and drainage,” “biopsy,” “treatment of oromaxillofacial pain,” and “emergency rescue skills.” The results of our review on the feasibility and ease of implementing these competency elements as clinical skill tests are presented in Table 3. Among the task elements identified in the third dental practitioner job analysis, those related to dentist competency that had an average importance rating of 4 or higher were selected and assessed for alignment with the evaluation objectives. The results of the validity assessment and the feasibility of incorporating these task elements as evaluation objectives are presented in Table 4.

Table 3.

Competency elements not included in the clinical skill test objectives

Table 4.

Task elements not included in the clinical skill test objectives

Survey perceptions of the clinical skill test’s evaluation objectives

In total, 116 current and 35 former examinees completed the survey, and the results are available in ^{Dataset 1,} and summarized in Table 5. No statistically significant differences were found between the perceptions of the current and former examinees.

Table 5.

Survey results of current and former examinees regarding the evaluation objectives of the clinical skill test

The survey on the perceptions of the validity of clinical skill test objectives included responses from 16 test item developers, 32 test examiners, and 24 dental school faculty members. The results of the survey are available in ^{Dataset 2,} and summarized in Table 6. For most items, the percentages of “agree” and “strongly agree” were high, with no significant differences between the groups. There were differences between groups on some items, such as “4. Pulp test,” where the percentage of “strongly agree” was relatively high among dental school professors. For “15. Space management for primary and mixed dentition,” the percentage of test item developers who strongly disagreed was relatively high.

Table 6.

Survey results on the participating professors (test item developers and test examiners) and dental school faculty members’ perceptions of the validity of the clinical skill test’s evaluation objectives

The responses of the participating professors to the survey item “Do the evaluation objectives overlap with dental specialty areas?” are available in ^{Dataset 3,} and summarized in Table 7. There were no significant differences between the groups on any of the items. Most of the items had a high percentage of “disagree” or “strongly disagree” responses, implying that the evaluation objectives do not overlap with the dental specialty areas. However, “2. Oromaxillofacial cranial nerve examination,” “3. Temporomandibular disorder palpation test,” and “15. Space management for primary and mixed dentition” were more likely to be perceived as overlapping with dental specialty areas than other items.

Table 7.

Survey results on the participating professors’ (test item developers and test examiners) perceptions of the overlap between the clinical skill test's evaluation objectives and dental specialty areas

Focus group interviews with clinical skill test experts

Clinical skill test experts agreed that most of the evaluation objectives should be retained considering the competencies required for new dentists. However, in the focus group interview for the “process evaluation,” some participants suggested that “2. Oromaxillofacial cranial nerve examination” and “15. Space management for primary and mixed dentition” were inappropriate for evaluating the competencies of new dentists. Regarding the objectives that require revision, “6. Intraoral X-ray taking” was discussed due to practical difficulties in exam administration. In the focus group interview for the “outcomes evaluation,” the experts suggested that the material-specific evaluation objectives should be revised to include various restorative and prosthetic materials. Additionally, several experts proposed adding items related to dental laboratory procedures, preparation of abutment teeth for removable partial dentures, and pediatric dentistry to the evaluation objectives.

Key results

In this study, the evaluation objectives of the Korean Dentist Clinical Skill Test were reviewed based on the national-level dental practitioner competencies, dental school curricula, and the third dental practitioner job analysis. The evaluation objectives for the Korean Dentist Clinical Skill Test were developed in conjunction with the development of dental practitioners’ competencies. Dental schools in Korea are working toward implementing performance-based education based on the national-level dental practitioner competencies. This study confirmed that the clinical skill test’s evaluation objectives encompass most of the basic competencies expected of dentists graduating from dental schools. However, it was found that some competencies required of new dentists, such as “emergency rescue skills,” “planning and performing simple fixed partial denture treatment,” and “planning and performing simple removable partial denture treatment,” were not included in the evaluation objectives.

Interpretation

The survey respondents recognized that a majority of the clinical skill test’s objectives were appropriate to assess the competence of new dentists. However, some evaluation objectives needed reviewing given the environment in which the clinical skill test is administered and considering some objective overlap with dental specialty areas.

Regarding “2. Oromaxillofacial cranial nerve examination,” numerous opinions from professors participating in the examination and from dental schools indicated that it was inadequate as an evaluation objective. This is likely because they believed that the topic falls under a specialty area. Moreover, in the job analysis data, the cranial nerve exam was rated low in importance and frequency.

For professors participating in the clinical skill test, many respondents stated that “3. Temporomandibular disorder palpation test” falls under a specialty area. The focus group interviews also indicated that this item was difficult to adequately assess via the clinical skill test.

Many dental school faculty members considered “6. Intraoral X-ray taking” to be an appropriate evaluation objective, whereas professors participating in the examination felt that it was inappropriate. These differing views may stem from the recognition that competency in intraoral radiography is essential for new dentists, but it can be challenging to access radiographic equipment within the constraints of the current clinical skill test environment. The focus group interviews suggested that competency should be assessed by other means, such as multimedia questions, which are being considered for inclusion in the KDLE.

Many professors involved in the clinical skill test agreed that “15. Space management for primary and mixed dentition” was unsuitable as an evaluation objective. This may be due to the perception that this area is specialized and the challenges of creating assessment questions.

The percentage of participants who agreed that “18. Amalgam restoration in posterior teeth” was valid as a clinical skill test objective was relatively lower than that for other evaluation objectives. In the focus group interview, participants recommended either removing or revising the objective related to amalgam restorations, noting that amalgam is now seldom used. It was suggested that the evaluation objective should be revised to “intracanal restorations” to include a range of restorative materials including composite resins rather than limiting the evaluation objective to amalgam.

Limitations

The limitations of this study include its focus on reviewing the clinical skill test’s evaluation objectives based primarily on the national-level dental practitioner competencies, dental school curricula, and dentist job analysis—all of which are specific to Korea. Future studies should also consider the implementation and evolution of clinical skill tests in other countries to provide a more comprehensive perspective on potential improvements for the KDLE.

Suggestions

Emergency rescue skills, which constitute a competency directly related to patient life, are included as an evaluation objective in national examinations for other healthcare professions, such as physicians and emergency medical technicians. This item is related to “7.3. Perform emergency medical procedures that may occur in a dental practice” in the national-level dental practitioner competencies. The dental practitioner job analysis includes “B3-6 Perform cardiopulmonary resuscitation,” which was identified as a low-frequency but high-importance item. Given that the primary purpose of the KDLE is to assess minimal competency as a healthcare provider, it is reasonable to consider adding test items related to emergency rescue skills, which can be evaluated using a first aid simulator.

“Planning and performing simple fixed partial denture treatment” and “planning and performing simple removable partial denture treatment” are among the important competencies for dentists, relating to “6.19. Can effectively design and perform simple fixed and removable partial denture, and full denture treatments” in the national-level dental practitioner competencies and “B4-6 Perform a fixed partial denture treatment” and “B4-8 Perform a removable partial denture treatment” in the dental practitioner job analysis. These competencies cover a wide range of clinical skill-related situations and competency levels. Therefore, a comprehensive discussion should be held on the scope and level of these competencies. Regarding the evaluation method, considering that the competencies related to prosthetic treatments are currently evaluated by the “outcome evaluation” using a dental mannequin simulator, it could be possible to evaluate “planning and performing simple fixed partial denture treatment” and “planning and performing simple removable partial denture treatment” via the “outcome evaluation.”

Conclusion

Based on an analysis of the national-level dental practitioner competencies, dental school educational competencies, dental practitioner job analysis, and the perspectives of those involved in the clinical skill test, this study found that most evaluation objectives were valid. However, future revisions should incorporate assessments related to “emergency rescue skills” and “planning and performing prosthetic treatment.” Modifications to the evaluation objectives of the Korean Dentist Clinical Skill Test should involve input from both dental education institutions and specialty societies to achieve a collective consensus among dental professionals.