Skip Navigation
Skip to contents

JEEHP : Journal of Educational Evaluation for Health Professions

OPEN ACCESS
SEARCH
Search

Search

Page Path
HOME > Search
32 "Licensure"
Filter
Filter
Article category
Keywords
Publication year
Authors
Funded articles
Research article
ChatGPT (GPT-4) passed the Japanese National License Examination for Pharmacists in 2022, answering all items including those with diagrams: a descriptive study  
Hiroyasu Sato, Katsuhiko Ogasawara
J Educ Eval Health Prof. 2024;21:4.   Published online February 28, 2024
DOI: https://doi.org/10.3352/jeehp.2024.21.4
  • 1,427 View
  • 227 Download
AbstractAbstract PDFSupplementary Material
Purpose
The objective of this study was to assess the performance of ChatGPT (GPT-4) on all items, including those with diagrams, in the Japanese National License Examination for Pharmacists (JNLEP) and compare it with the previous GPT-3.5 model’s performance.
Methods
The 107th JNLEP, conducted in 2022, with 344 items input into the GPT-4 model, was targeted for this study. Separately, 284 items, excluding those with diagrams, were entered into the GPT-3.5 model. The answers were categorized and analyzed to determine accuracy rates based on categories, subjects, and presence or absence of diagrams. The accuracy rates were compared to the main passing criteria (overall accuracy rate ≥62.9%).
Results
The overall accuracy rate for all items in the 107th JNLEP in GPT-4 was 72.5%, successfully meeting all the passing criteria. For the set of items without diagrams, the accuracy rate was 80.0%, which was significantly higher than that of the GPT-3.5 model (43.5%). The GPT-4 model demonstrated an accuracy rate of 36.1% for items that included diagrams.
Conclusion
Advancements that allow GPT-4 to process images have made it possible for LLMs to answer all items in medical-related license examinations. This study’s findings confirm that ChatGPT (GPT-4) possesses sufficient knowledge to meet the passing criteria.
Technical report
Item difficulty index, discrimination index, and reliability of the 26 health professions licensing examinations in 2022, Korea: a psychometric study
Yoon Hee Kim, Bo Hyun Kim, Joonki Kim, Bokyoung Jung, Sangyoung Bae
J Educ Eval Health Prof. 2023;20:31.   Published online November 22, 2023
DOI: https://doi.org/10.3352/jeehp.2023.20.31
  • 943 View
  • 81 Download
AbstractAbstract PDFSupplementary Material
Purpose
This study presents item analysis results of the 26 health personnel licensing examinations managed by the Korea Health Personnel Licensing Examination Institute (KHPLEI) in 2022.
Methods
The item difficulty index, item discrimination index, and reliability were calculated. The item discrimination index was calculated using a discrimination index based on the upper and lower 27% rule and the item-total correlation.
Results
Out of 468,352 total examinees, 418,887 (89.4%) passed. The pass rates ranged from 27.3% for health educators level 1 to 97.1% for oriental medical doctors. Most examinations had a high average difficulty index, albeit to varying degrees, ranging from 61.3% for prosthetists and orthotists to 83.9% for care workers. The average discrimination index based on the upper and lower 27% rule ranged from 0.17 for oriental medical doctors to 0.38 for radiological technologists. The average item-total correlation ranged from 0.20 for oriental medical doctors to 0.38 for radiological technologists. The Cronbach α, as a measure of reliability, ranged from 0.872 for health educators-level 3 to 0.978 for medical technologists. The correlation coefficient between the average difficulty index and average discrimination index was -0.2452 (P=0.1557), that between the average difficulty index and the average item-total correlation was 0.3502 (P=0.0392), and that between the average discrimination index and the average item-total correlation was 0.7944 (P<0.0001).
Conclusion
This technical report presents the item analysis results and reliability of the recent examinations by the KHPLEI, demonstrating an acceptable range of difficulty index and discrimination index values, as well as good reliability.
Research articles
Implementation strategy for introducing a clinical skills examination to the Korean Oriental Medicine Licensing Examination: a mixed-method modified Delphi study  
Chan-Young Kwon, Sanghoon Lee, Min Hwangbo, Chungsik Cho, Sangwoo Shin, Dong-Hyeon Kim, Aram Jeong, Hye-Yoon Lee
J Educ Eval Health Prof. 2023;20:23.   Published online July 17, 2023
DOI: https://doi.org/10.3352/jeehp.2023.20.23
  • 1,928 View
  • 145 Download
AbstractAbstract PDFSupplementary Material
Purpose
This study investigated the validity of introducing a clinical skills examination (CSE) to the Korean Oriental Medicine Licensing Examination through a mixed-method modified Delphi study.
Methods
A 3-round Delphi study was conducted between September and November 2022. The expert panel comprised 21 oriental medicine education experts who were officially recommended by relevant institutions and organizations. The questionnaires included potential content for the CSE and a detailed implementation strategy. Subcommittees were formed to discuss concerns around the introduction of the CSE, which were collected as open-ended questions. In this study, a 66.7% or greater agreement rate was defined as achieving a consensus.
Results
The expert panel’s evaluation of the proposed clinical presentations and basic clinical skills suggested their priorities. Of the 10 items investigated for building a detailed implementation strategy for the introduction of the CSE to the Korean Oriental Medicine Licensing Examination, a consensus was achieved on 9. However, the agreement rate on the timing of the introduction of the CSE was low. Concerns around 4 clinical topics were discussed in the subcommittees, and potential solutions were proposed.
Conclusion
This study offers preliminary data and raises some concerns that can be used as a reference while discussing the introduction of the CSE to the Korean Oriental Medicine Licensing Examination.
Suggestion for item allocation to 8 nursing activity categories of the Korean Nursing Licensing Examination: a survey-based descriptive study  
Kyunghee Kim, So Young Kang, Younhee Kang, Youngran Kweon, Hyunjung Kim, Youngshin Song, Juyeon Cho, Mi-Young Choi, Hyun Su Lee
J Educ Eval Health Prof. 2023;20:18.   Published online June 12, 2023
DOI: https://doi.org/10.3352/jeehp.2023.20.18
  • 1,785 View
  • 114 Download
AbstractAbstract PDFSupplementary Material
Purpose
This study aims to suggest the number of test items in each of 8 nursing activity categories of the Korean Nursing Licensing Examination, which comprises 134 activity statements including 275 items. The examination will be able to evaluate the minimum ability that nursing graduates must have to perform their duties. Methods: Two opinion surveys involving the members of 7 academic societies were conducted from March 19 to May 14, 2021. The survey results were reviewed by members of 4 expert associations from May 21 to June 4, 2021. The results for revised numbers of items in each category were compared with those reported by Tak and his colleagues and the National Council License Examination for Registered Nurses of the United States. Results: Based on 2 opinion surveys and previous studies, the suggestions for item allocation to 8 nursing activity categories of the Korean Nursing Licensing Examination in this study are as follows: 50 items for management of care and improvement of professionalism, 33 items for safety and infection control, 40 items for management of potential risk, 28 items for basic care, 47 items for physiological integrity and maintenance, 33 items for pharmacological and parenteral therapies, 24 items for psychosocial integrity and maintenance, and 20 items for health promotion and maintenance. Twenty other items related to health and medical laws were not included due to their mandatory status. Conclusion: These suggestions for the number of test items for each activity category will be helpful in developing new items for the Korean Nursing Licensing Examination.
Adequacy of the examination-based licensing system and a training-based licensing system for midwifery license according to changes in childbirth medical infrastructure in Korea: a survey-based descriptive study  
Yun Mi Kim, Sun Hee Lee, Sun Ok Lee, Mi Young An, Bu Youn Kim, Jum Mi Park
J Educ Eval Health Prof. 2023;20:15.   Published online May 22, 2023
DOI: https://doi.org/10.3352/jeehp.2023.20.15
  • 1,246 View
  • 68 Download
AbstractAbstract PDFSupplementary Material
Purpose
The number of Korean midwifery licensing examination applicants has steadily decreased due to the low birth rate and lack of training institutions for midwives. This study aimed to evaluate the adequacy of the examination-based licensing system and the possibility of a training-based licensing system.
Methods
A survey questionnaire was developed and dispatched to 230 professionals from December 28, 2022 to January 13, 2023, through an online form using Google Surveys. Descriptive statistics were used to analyze the results.
Results
Responses from 217 persons (94.3%) were analyzed after excluding incomplete responses. Out of the 217 participants, 198 (91.2%) agreed with maintaining the current examination-based licensing system; 94 (43.3%) agreed with implementing a training-based licensing system to cover the examination costs due to the decreasing number of applicants; 132 (60.8%) agreed with establishing a midwifery education evaluation center for a training-based licensing system; 163 (75.1%) said that the quality of midwifery might be lowered if midwives were produced only by a training-based licensing system, and 197 (90.8%) said that the training of midwives as birth support personnel should be promoted in Korea.
Conclusion
Favorable results were reported for the examination-based licensing system; however, if a training-based licensing system is implemented, it will be necessary to establish a midwifery education evaluation center to manage the quality of midwives. As the annual number of candidates for the Korean midwifery licensing examination has been approximately 10 in recent years, it is necessary to consider more actively granting midwifery licenses through a training-based licensing system.
Technical report
Development of examination objectives based on nursing competency for the Korean Nursing Licensing Examination: a validity study  
Sujin Shin, Gwang Suk Kim, Jun-Ah Song, Inyoung Lee
J Educ Eval Health Prof. 2022;19:19.   Published online August 22, 2022
DOI: https://doi.org/10.3352/jeehp.2022.19.19
  • 2,620 View
  • 209 Download
  • 3 Web of Science
  • 3 Crossref
AbstractAbstract PDFSupplementary Material
Purpose
This study aimed to develop the examination objectives based on nursing competency of the Korean Nursing Licensing Examination.
Methods
This is a validity study to develop the examination objectives based on nursing competency. Data were collected in December 2021. We reviewed the literature related to changing nurse roles and on the learning objectives for the Korea Medical Licensing Examination and other health personnel licensing examinations. Thereafter, we created a draft of the nursing problems list for examination objectives based on the literature review, and the content validity was evaluated by experts. A final draft of the examination objectives is presented and discussed.
Results
A total of 4 domains, 12 classes, and 85 nursing problems for the Korean Nursing Liscensing Examination were developed. They included the essentials of objectives, related factors, evaluation goals, related activity statements, related clients, related settings, and specific outcomes.
Conclusion
This study developed a draft of the examination objectives based on clinical competency that were related to the clinical situations of nurses and comprised appropriate test items for the licensing examination. Above results may be able to provide fundamental data for item development that reflects future nursing practices.

Citations

Citations to this article as recorded by  
  • A validity study of COMLEX-USA Level 3 with the new test design
    Xia Mao, John R. Boulet, Jeanne M. Sandella, Michael F. Oliverio, Larissa Smith
    Journal of Osteopathic Medicine.2024; 124(6): 257.     CrossRef
  • A Survey on Perceptions of the Direction of Korean Medicine Education and National Licensing Examination
    Han-Byul Cho, Won-Suk Sung, Jiseong Hong, Yeonseok Kang, Eun-Jung Kim
    Healthcare.2023; 11(12): 1685.     CrossRef
  • Suggestion for item allocation to 8 nursing activity categories of the Korean Nursing Licensing Examination: a survey-based descriptive study
    Kyunghee Kim, So Young Kang, Younhee Kang, Youngran Kweon, Hyunjung Kim, Youngshin Song, Juyeon Cho, Mi-Young Choi, Hyun Su Lee
    Journal of Educational Evaluation for Health Professions.2023; 20: 18.     CrossRef
Research articles
Is it possible to introduce an interview to the Korean Medical Licensing Examination to assess professional attributes?: a survey-based observational study  
Seung-Joo Na, HyeRin Roh, Kyung Hee Chun, Kyung Hye Park, Do-Hwan Kim
J Educ Eval Health Prof. 2022;19:10.   Published online May 10, 2022
DOI: https://doi.org/10.3352/jeehp.2022.19.10
  • 3,251 View
  • 288 Download
AbstractAbstract PDFSupplementary Material
Purpose
This study aimsed to gather opinions from medical educators on the possibility of introducing an interview to the Korean Medical Licensing Examination (KMLE) to assess professional attributes. Specifically following topics were dealt with: the appropriate timing and tool to assess unprofessional conduct; ; the possiblity of prevention of unprofessional conduct by introducing an interview to the KMLE; and the possibility of implementation of an interview to the KMLE.
Methods
A cross-sectional study approach based on a survey questionnaire was adopted. We analyzed 104 pieces of news about doctors’ unprofessional conduct to determine the deficient professional attributes. We derived 24 items of unprofessional conduct and developed the questionnaire and surveyed 250 members of the Korean Society of Medical Education 2 times. Descriptive statistics, cross-tabulation analysis, and Fisher’s exact test were applied to the responses. The answers to the open-ended questions were analyzed using conventional content analysis.
Results
In the survey, 49 members (19.6%) responded. Out of 49, 24 (49.5%) responded in the 2nd survey. To assess unprofessional conduct, there was no dominant timing among basic medical education (BME), KMLE, and continuing professional development (CPD). There was no overwhelming assessment tool among written examination, objective structured clinical examination, practice observation, and interview. Response rates of “impossible” (49.0%) and “possible” (42.9%) suggested an interview of the KMLE prevented unprofessional conduct. In terms of implementation, “impossible” (50.0%) was selected more often than “possible” (33.3%).
Conclusion
Professional attributes should be assessed by various tools over the period from BME to CPD. Hence, it may be impossible to introduce an interview to assess professional attributes to the KMLE, and a system is needed such as self-regulation by the professional body rather than licensing examination.
Level of professional ethics awareness and medical ethics competency of dental hygienists and dental hygiene students: the need to add ethics items to the Korean Dental Hygienist Licensing Examination  
Yoon-Sook Hwang, Jong-Hwa Jang, Kyung-Hee Kang, Minji Kim, Jeong-Ran Park, Sohyun Son, Sun-Mi Lee, Da-Yee Jeung, Jung-Eun Ha, Su-Min Hong, Young-Eun Jang
J Educ Eval Health Prof. 2020;17:34.   Published online November 17, 2020
DOI: https://doi.org/10.3352/jeehp.2020.17.34
  • 5,825 View
  • 165 Download
AbstractAbstract PDFSupplementary Material
Purpose
This study aimed to evaluate the level of professional ethics awareness and medical ethics competency in order to assess the potential need for ethics items to be included on the Korean Dental Hygienist Licensing Examination.
Methods
In total, 358 clinical dental hygienists and dental hygiene students completed a structured questionnaire to evaluate their level of ethical awareness and medical ethics competency. The sub-factors of medical ethics were classified into relationships with patients, medical and social relations, and individual specialized fields.
Results
Only 32.1% of participants indicated that they had taken a course on professional ethics in the university curriculum, but 95.2% of respondents considered professional ethics to be important. The overall score for medical ethics competency was average (3.37 out of 5). The score for relationships with patients was 3.75 points, followed by medical and social relations (3.19 points) and individual specialized fields (3.16 points). The level of professional ethics awareness was higher among participants who had taken a course on professional ethics than among those who had not done so or who did not remember whether they had done so.
Conclusion
Dental hygienists were aware of the importance of professional ethics, but their medical ethics competency was moderate. Therefore, medical ethics should be treated as a required subject in the university curriculum, and medical ethics competency evaluations should be strengthened by adding ethics items to the Korean Dental Hygienist Licensing Examination.
Using the Angoff method to set a standard on mock exams for the Korean Nursing Licensing Examination  
Mi Kyoung Yim, Sujin Shin
J Educ Eval Health Prof. 2020;17:14.   Published online April 22, 2020
DOI: https://doi.org/10.3352/jeehp.2020.17.14
  • 8,256 View
  • 201 Download
  • 8 Web of Science
  • 6 Crossref
AbstractAbstract PDFSupplementary Material
Purpose
This study explored the possibility of using the Angoff method, in which panel experts determine the cut score of an exam, for the Korean Nursing Licensing Examination (KNLE). Two mock exams for the KNLE were analyzed. The Angoff standard setting procedure was conducted and the results were analyzed. We also aimed to examine the procedural validity of applying the Angoff method in this context.
Methods
For both mock exams, we set a pass-fail cut score using the Angoff method. The standard setting panel consisted of 16 nursing professors. After the Angoff procedure, the procedural validity of establishing the standard was evaluated by investigating the responses of the standard setters.
Results
The descriptions of the minimally competent person for the KNLE were presented at the levels of general and subject performance. The cut scores of first and second mock exams were 74.4 and 76.8, respectively. These were higher than the traditional cut score (60% of the total score of the KNLE). The panel survey showed very positive responses, with scores higher than 4 out of 5 points on a Likert scale.
Conclusion
The scores calculated for both mock tests were similar, and were much higher than the existing cut scores. In the second simulation, the standard deviation of the Angoff rating was lower than in the first simulation. According to the survey results, procedural validity was acceptable, as shown by a high level of confidence. The results show that determining cut scores by an expert panel is an applicable method.

Citations

Citations to this article as recorded by  
  • Comparing Estimated and Real Item Difficulty Using Multi-Facet Rasch Analysis
    Ayfer SAYIN, Sebahat GÖREN
    Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi.2023; 14(4): 440.     CrossRef
  • Application of computer-based testing in the Korean Medical Licensing Examination, the emergence of the metaverse in medical education, journal metrics and statistics, and appreciation to reviewers and volunteers
    Sun Huh
    Journal of Educational Evaluation for Health Professions.2022; 19: 2.     CrossRef
  • Possibility of using the yes/no Angoff method as a substitute for the percent Angoff method for estimating the cutoff score of the Korean Medical Licensing Examination: a simulation study
    Janghee Park
    Journal of Educational Evaluation for Health Professions.2022; 19: 23.     CrossRef
  • Development of examination objectives based on nursing competency for the Korean Nursing Licensing Examination: a validity study
    Sujin Shin, Gwang Suk Kim, Jun-Ah Song, Inyoung Lee
    Journal of Educational Evaluation for Health Professions.2022; 19: 19.     CrossRef
  • Possibility of independent use of the yes/no Angoff and Hofstee methods for the standard setting of the Korean Medical Licensing Examination written test: a descriptive study
    Do-Hwan Kim, Ye Ji Kang, Hoon-Ki Park
    Journal of Educational Evaluation for Health Professions.2022; 19: 33.     CrossRef
  • Comparing the cut score for the borderline group method and borderline regression method with norm-referenced standard setting in an objective structured clinical examination in medical school in Korea
    Song Yi Park, Sang-Hwa Lee, Min-Jeong Kim, Ki-Hwan Ji, Ji Ho Ryu
    Journal of Educational Evaluation for Health Professions.2021; 18: 25.     CrossRef
Technical report
Improvement of the qualification system in Korea for foreign nurses eligible to take Korean nursing licensing examination  
Young Whee Lee, Yeon Ok Suh, Kyoung Sook Chae
J Educ Eval Health Prof. 2019;16:26.   Published online September 4, 2019
DOI: https://doi.org/10.3352/jeehp.2019.16.26
  • 10,925 View
  • 144 Download
  • 1 Web of Science
  • 1 Crossref
AbstractAbstract PDFSupplementary Material
Purpose
We aimed to review and provide a quality improvement for the document utilized by the relevant Korean government body to verify and evaluate foreign university/college graduates’ eligibility for nursing and qualification to take the Korean nursing licensing examination.
Methods
This was a descriptive study. We analyzed the current Korean qualification system for foreign graduates to Korean nursing licensing examination and the same system utilized in some other countries. Then, we created a draft of the reviewed qualification standards document based on the 2 prior analyses and their comparisons, and applied a questionnaire in an open hearing with 5 experts to enhance the draft’s quality. Finally, we presented and discussed the final draft.
Results
The reviewed criteria of the qualification standards included confirming whether the foreign graduate’s university has an accreditation provided by its relevant government body, the exclusion of foreign graduates’ provision of several documents previously required, a minimum number of credits (1,000 hours) for their original course, a 3-year minimum enrollment period for their original course, and a mandatory reassessment of the foreign graduates’ university recognition in a 5-year cycle.
Conclusion
We believe that by creating a review draft that addresses the flaws of the current document utilized to determine the qualification for foreign graduates to take the Korean nursing licensing examination, we have simplified it for a better understanding of the application process. We hope that this draft will contribute to a more objective and equitable qualification process for foreign university nurse graduates in Korea.

Citations

Citations to this article as recorded by  
  • Recognition of nursing qualification and credentialing pathway of Filipino nurses in Finland: A qualitative study
    Floro Cubelo, Maliheh Nekouei Marvi Langari, Krista Jokiniemi, Hannele Turunen
    International Nursing Review.2023;[Epub]     CrossRef
Research articles
The relationship of examinees’ individual characteristics and perceived acceptability of smart device-based testing to test scores on the practice test of the Korea Emergency Medicine Technician Licensing Examination  
Eun Young Lim, Mi Kyoung Yim, Sun Huh
J Educ Eval Health Prof. 2018;15:33.   Published online December 27, 2018
DOI: https://doi.org/10.3352/jeehp.2018.15.33
  • 19,243 View
  • 233 Download
  • 2 Web of Science
  • 2 Crossref
AbstractAbstract PDFSupplementary Material
Purpose
Smart device-based testing (SBT) is being introduced into the Republic of Korea’s high-stakes examination system, starting with the Korean Emergency Medicine Technician Licensing Examination (KEMTLE) in December 2017. In order to minimize the effects of variation in examinees’ environment on test scores, this study aimed to identify any associations of variables related to examinees’ individual characteristics and their perceived acceptability of SBT with their SBT practice test scores.
Methods
Of the 569 candidate students who took the KEMTLE on September 12, 2015, 560 responded to a survey questionnaire on the acceptability of SBT after the examination. The questionnaire addressed 8 individual characteristics and contained 2 satisfaction, 9 convenience, and 9 preference items. A comparative analysis according to individual variables was performed. Furthermore, a generalized linear model (GLM) analysis was conducted to identify the effects of individual characteristics and perceived acceptability of SBT on test scores.
Results
Among those who preferred SBT over paper-and-pencil testing, test scores were higher for male participants (mean± standard deviation [SD], 4.36± 0.72) than for female participants (mean± SD, 4.21± 0.73). According to the GLM, no variables evaluated— including gender and experience with computer-based testing, SBT, or using a tablet PC—showed a statistically significant relationship with the total score, scores on multimedia items, or scores on text items.
Conclusion
Individual characteristics and perceived acceptability of SBT did not affect the SBT practice test scores of emergency medicine technician students in Korea. It should be possible to adopt SBT for the KEMTLE without interference from the variables examined in this study.

Citations

Citations to this article as recorded by  
  • Application of computer-based testing in the Korean Medical Licensing Examination, the emergence of the metaverse in medical education, journal metrics and statistics, and appreciation to reviewers and volunteers
    Sun Huh
    Journal of Educational Evaluation for Health Professions.2022; 19: 2.     CrossRef
  • Evaluation of Student Satisfaction with Ubiquitous-Based Tests in Women’s Health Nursing Course
    Mi-Young An, Yun-Mi Kim
    Healthcare.2021; 9(12): 1664.     CrossRef
Comparison of standard-setting methods for the Korean Radiological Technologist Licensing Examination: Angoff, Ebel, bookmark, and Hofstee  
Janghee Park, Duck-Sun Ahn, Mi Kyoung Yim, Jaehyoung Lee
J Educ Eval Health Prof. 2018;15:32.   Published online December 26, 2018
DOI: https://doi.org/10.3352/jeehp.2018.15.32
  • 19,309 View
  • 253 Download
  • 11 Web of Science
  • 9 Crossref
AbstractAbstract PDFSupplementary Material
Purpose
This study aimed to compare the possible standard-setting methods for the Korean Radiological Technologist Licensing Examination, which has a fixed cut score, and to suggest the most appropriate method.
Methods
Six radiological technology professors set standards for 250 items on the Korean Radiological Technologist Licensing Examination administered in December 2016 using the Angoff, Ebel, bookmark, and Hofstee methods.
Results
With a maximum percentile score of 100, the cut score for the examination was 71.27 using the Angoff method, 62.2 using the Ebel method, 64.49 using the bookmark method, and 62 using the Hofstee method. Based on the Hofstee method, an acceptable cut score for the examination would be between 52.83 and 70, but the cut score was 71.27 using the Angoff method.
Conclusion
The above results suggest that the best standard-setting method to determine the cut score would be a panel discussion with the modified Angoff or Ebel method, with verification of the rated results by the Hofstee method. Since no standard-setting method has yet been adopted for the Korean Radiological Technologist Licensing Examination, this study will be able to provide practical guidance for introducing a standard-setting process.

Citations

Citations to this article as recorded by  
  • Setting standards for a diagnostic test of aviation English for student pilots
    Maria Treadaway, John Read
    Language Testing.2024;[Epub]     CrossRef
  • Third Year Veterinary Student Academic Encumbrances and Tenacity: Navigating Clinical Skills Curricula and Assessment
    Saundra H. Sample, Elpida Artemiou, Darlene J. Donszelmann, Cindy Adams
    Journal of Veterinary Medical Education.2024;[Epub]     CrossRef
  • The challenges inherent with anchor-based approaches to the interpretation of important change in clinical outcome assessments
    Kathleen W. Wyrwich, Geoffrey R. Norman
    Quality of Life Research.2023; 32(5): 1239.     CrossRef
  • Possibility of independent use of the yes/no Angoff and Hofstee methods for the standard setting of the Korean Medical Licensing Examination written test: a descriptive study
    Do-Hwan Kim, Ye Ji Kang, Hoon-Ki Park
    Journal of Educational Evaluation for Health Professions.2022; 19: 33.     CrossRef
  • Comparison of the validity of bookmark and Angoff standard setting methods in medical performance tests
    Majid Yousefi Afrashteh
    BMC Medical Education.2021;[Epub]     CrossRef
  • Comparing the cut score for the borderline group method and borderline regression method with norm-referenced standard setting in an objective structured clinical examination in medical school in Korea
    Song Yi Park, Sang-Hwa Lee, Min-Jeong Kim, Ki-Hwan Ji, Ji Ho Ryu
    Journal of Educational Evaluation for Health Professions.2021; 18: 25.     CrossRef
  • Using the Angoff method to set a standard on mock exams for the Korean Nursing Licensing Examination
    Mi Kyoung Yim, Sujin Shin
    Journal of Educational Evaluation for Health Professions.2020; 17: 14.     CrossRef
  • Performance of the Ebel standard-setting method for the spring 2019 Royal College of Physicians and Surgeons of Canada internal medicine certification examination consisting of multiple-choice questions
    Jimmy Bourque, Haley Skinner, Jonathan Dupré, Maria Bacchus, Martha Ainslie, Irene W. Y. Ma, Gary Cole
    Journal of Educational Evaluation for Health Professions.2020; 17: 12.     CrossRef
  • Similarity of the cut score in test sets with different item amounts using the modified Angoff, modified Ebel, and Hofstee standard-setting methods for the Korean Medical Licensing Examination
    Janghee Park, Mi Kyoung Yim, Na Jin Kim, Duck Sun Ahn, Young-Min Kim
    Journal of Educational Evaluation for Health Professions.2020; 17: 28.     CrossRef
Usefulness of the DETECT program for assessing the internal structure of dimensionality in simulated data and results of the Korean nursing licensing examination  
Dong Gi Seo, Younyoung Choi, Sun Huh
J Educ Eval Health Prof. 2017;14:32.   Published online December 27, 2017
DOI: https://doi.org/10.3352/jeehp.2017.14.32
  • 25,548 View
  • 262 Download
  • 3 Web of Science
  • 4 Crossref
AbstractAbstract PDFSupplementary Material
Purpose
The dimensionality of examinations provides empirical evidence of the internal test structure underlying the responses to a set of items. In turn, the internal structure is an important piece of evidence of the validity of an examination. Thus, the aim of this study was to investigate the performance of the DETECT program and to use it to examine the internal structure of the Korean nursing licensing examination.
Methods
Non-parametric methods of dimensional testing, such as the DETECT program, have been proposed as ways of overcoming the limitations of traditional parametric methods. A non-parametric method (the DETECT program) was investigated using simulation data under several conditions and applied to the Korean nursing licensing examination.
Results
The DETECT program performed well in terms of determining the number of underlying dimensions under several different conditions in the simulated data. Further, the DETECT program correctly revealed the internal structure of the Korean nursing licensing examination, meaning that it detected the proper number of dimensions and appropriately clustered the items within each dimension.
Conclusion
The DETECT program performed well in detecting the number of dimensions and in assigning items for each dimension. This result implies that the DETECT method can be useful for examining the internal structure of assessments, such as licensing examinations, that possess relatively many domains and content areas.

Citations

Citations to this article as recorded by  
  • Meanings of Rough Sex across Gender, Sexual Identity, and Political Ideology: A Conditional Covariance Approach
    Dubravka Svetina Valdivia, Debby Herbenick, Tsung-chieh Fu, Heather Eastman-Mueller, Lucia Guerra-Reyes, Molly Rosenberg
    Journal of Sex & Marital Therapy.2022; 48(6): 579.     CrossRef
  • The accuracy and consistency of mastery for each content domain using the Rasch and deterministic inputs, noisy “and” gate diagnostic classification models: a simulation study and a real-world analysis using data from the Korean Medical Licensing Examinat
    Dong Gi Seo, Jae Kum Kim
    Journal of Educational Evaluation for Health Professions.2021; 18: 15.     CrossRef
  • Estimation of item parameters and examinees’ mastery probability in each domain of the Korean Medical Licensing Examination using a deterministic inputs, noisy “and” gate (DINA) model
    Younyoung Choi, Dong Gi Seo
    Journal of Educational Evaluation for Health Professions.2020; 17: 35.     CrossRef
  • Linear programming method to construct equated item sets for the implementation of periodical computer-based testing for the Korean Medical Licensing Examination
    Dong Gi Seo, Myeong Gi Kim, Na Hui Kim, Hye Sook Shin, Hyun Jung Kim
    Journal of Educational Evaluation for Health Professions.2018; 15: 26.     CrossRef
Perception survey on the introduction of clinical performance examination as part of the national nursing licensing examination in Korea  
Su Jin Shin, Yeong Kyeong Kim, Soon-Rim Suh, Duk Yoo Jung, Yunju Kim, Mi Kyoung Yim
J Educ Eval Health Prof. 2017;14:26.   Published online October 25, 2017
DOI: https://doi.org/10.3352/jeehp.2017.14.26
  • 32,103 View
  • 297 Download
  • 2 Web of Science
  • 5 Crossref
AbstractAbstract PDF
Purpose
The purpose of this study was to analyze opinions about the action plan for implementation of clinical performance exam as part of the national nursing licensing examination and presents the expected effects of the performance exam and aspects to consider regarding its implementation.
Methods
This study used a mixed-methods design. Quantitative data were collected by a questionnaire survey, while qualitative data were collected by focus group interviews with experts. The survey targeted 200 nursing professors and clinical nurses with more than 5 years of work experience, and the focus group interviews were conducted with 28 of professors, clinical instructors, and nurses at hospitals.
Results
First, nursing professors and clinical specialists agreed that the current written tests have limitations in evaluating examinees’ ability, and that the introduction of a clinical performance exam will yield positive results. Clinical performance exam is necessary to evaluate and improve nurses’ work ability, which means that the implementation of a performance exam is advisable if its credibility and validity can be verified. Second, most respondents chose direct performance exams using simulators or standardized patients as the most suitable format of the test.
Conclusion
In conclusion, the current national nursing licensing exam is somewhat limited in its ability to identify competent nurses. Thus, the time has come for us to seriously consider the introduction of a performance exam. The prerequisites for successfully implementing clinical performance exam as part of the national nursing licensing exam are a professional training process and forming a consortium to standardize practical training.

Citations

Citations to this article as recorded by  
  • The Clinical Nursing Competency Assessment System of Ghana: Perspectives of Key Informants
    Oboshie Anim-Boamah, Christmal Dela Christmals, Susan Jennifer Armstrong
    SAGE Open.2022; 12(2): 215824402210899.     CrossRef
  • Adaptation of Extended Reality Smart Glasses for Core Nursing Skill Training Among Undergraduate Nursing Students: Usability and Feasibility Study
    Sun Kyung Kim, Youngho Lee, Hyoseok Yoon, Jongmyung Choi
    Journal of Medical Internet Research.2021; 23(3): e24313.     CrossRef
  • Nursing Students’ Experiences on Clinical Competency Assessment in Ghana
    Oboshie Anim-Boamah, Christmal Dela Christmals, Susan Jennifer Armstrong
    Nurse Media Journal of Nursing.2021; 11(3): 278.     CrossRef
  • Clinical nursing competency assessment: a scoping review
    Oboshie Anim-Boamah, Christmal Dela Christmals, Susan Jennifer Armstrong
    Frontiers of Nursing.2021; 8(4): 341.     CrossRef
  • Factors Influencing the Success of the National Nursing Competency Examination taken by the Nursing Diploma Students in Yogyakarta
    Yulia Wardani
    Jurnal Ners.2020; 14(2): 172.     CrossRef
Review article
Overview and current management of computerized adaptive testing in licensing/certification examinations  
Dong Gi Seo
J Educ Eval Health Prof. 2017;14:17.   Published online July 26, 2017
DOI: https://doi.org/10.3352/jeehp.2017.14.17
  • 39,125 View
  • 372 Download
  • 12 Web of Science
  • 10 Crossref
AbstractAbstract PDF
Computerized adaptive testing (CAT) has been implemented in high-stakes examinations such as the National Council Licensure Examination-Registered Nurses in the United States since 1994. Subsequently, the National Registry of Emergency Medical Technicians in the United States adopted CAT for certifying emergency medical technicians in 2007. This was done with the goal of introducing the implementation of CAT for medical health licensing examinations. Most implementations of CAT are based on item response theory, which hypothesizes that both the examinee and items have their own characteristics that do not change. There are 5 steps for implementing CAT: first, determining whether the CAT approach is feasible for a given testing program; second, establishing an item bank; third, pretesting, calibrating, and linking item parameters via statistical analysis; fourth, determining the specification for the final CAT related to the 5 components of the CAT algorithm; and finally, deploying the final CAT after specifying all the necessary components. The 5 components of the CAT algorithm are as follows: item bank, starting item, item selection rule, scoring procedure, and termination criterion. CAT management includes content balancing, item analysis, item scoring, standard setting, practice analysis, and item bank updates. Remaining issues include the cost of constructing CAT platforms and deploying the computer technology required to build an item bank. In conclusion, in order to ensure more accurate estimations of examinees’ ability, CAT may be a good option for national licensing examinations. Measurement theory can support its implementation for high-stakes examinations.

Citations

Citations to this article as recorded by  
  • Validation of the cognitive section of the Penn computerized adaptive test for neurocognitive and clinical psychopathology assessment (CAT-CCNB)
    Akira Di Sandro, Tyler M. Moore, Eirini Zoupou, Kelly P. Kennedy, Katherine C. Lopez, Kosha Ruparel, Lucky J. Njokweni, Sage Rush, Tarlan Daryoush, Olivia Franco, Alesandra Gorgone, Andrew Savino, Paige Didier, Daniel H. Wolf, Monica E. Calkins, J. Cobb S
    Brain and Cognition.2024; 174: 106117.     CrossRef
  • The current utilization of the patient-reported outcome measurement information system (PROMIS) in isolated or combined total knee arthroplasty populations
    Puneet Gupta, Natalia Czerwonka, Sohil S. Desai, Alirio J. deMeireles, David P. Trofa, Alexander L. Neuwirth
    Knee Surgery & Related Research.2023;[Epub]     CrossRef
  • Evaluating a Computerized Adaptive Testing Version of a Cognitive Ability Test Using a Simulation Study
    Ioannis Tsaousis, Georgios D. Sideridis, Hannan M. AlGhamdi
    Journal of Psychoeducational Assessment.2021; 39(8): 954.     CrossRef
  • Accuracy and Efficiency of Web-based Assessment Platform (LIVECAT) for Computerized Adaptive Testing
    Do-Gyeong Kim, Dong-Gi Seo
    The Journal of Korean Institute of Information Technology.2020; 18(4): 77.     CrossRef
  • Transformaciones en educación médica: innovaciones en la evaluación de los aprendizajes y avances tecnológicos (parte 2)
    Veronica Luna de la Luz, Patricia González-Flores
    Investigación en Educación Médica.2020; 9(34): 87.     CrossRef
  • Introduction to the LIVECAT web-based computerized adaptive testing platform
    Dong Gi Seo, Jeongwook Choi
    Journal of Educational Evaluation for Health Professions.2020; 17: 27.     CrossRef
  • Computerised adaptive testing accurately predicts CLEFT-Q scores by selecting fewer, more patient-focused questions
    Conrad J. Harrison, Daan Geerards, Maarten J. Ottenhof, Anne F. Klassen, Karen W.Y. Wong Riff, Marc C. Swan, Andrea L. Pusic, Chris J. Sidey-Gibbons
    Journal of Plastic, Reconstructive & Aesthetic Surgery.2019; 72(11): 1819.     CrossRef
  • Presidential address: Preparing for permanent test centers and computerized adaptive testing
    Chang Hwi Kim
    Journal of Educational Evaluation for Health Professions.2018; 15: 1.     CrossRef
  • Updates from 2018: Being indexed in Embase, becoming an affiliated journal of the World Federation for Medical Education, implementing an optional open data policy, adopting principles of transparency and best practice in scholarly publishing, and appreci
    Sun Huh
    Journal of Educational Evaluation for Health Professions.2018; 15: 36.     CrossRef
  • Linear programming method to construct equated item sets for the implementation of periodical computer-based testing for the Korean Medical Licensing Examination
    Dong Gi Seo, Myeong Gi Kim, Na Hui Kim, Hye Sook Shin, Hyun Jung Kim
    Journal of Educational Evaluation for Health Professions.2018; 15: 26.     CrossRef

JEEHP : Journal of Educational Evaluation for Health Professions