Journal of Educational Evaluation for Health Professions

Volume 2 (1); January 2005

Original Articles

Correlations between the scores of computerized adaptive testing, paper and pencil tests, and the Korean Medical Licensing Examination: Mee Young Kim, Yoon Hwan Lee, Sun Huh; J Educ Eval Health Prof. 2005;2(1):113-118. Published online June 30, 2005; DOI: https://doi.org/10.3352/jeehp.2005.2.1.113

42,488 View
161 Download
3 Crossref

To evaluate the usefulness of computerized adaptive testing (CAT) in medical school, the General Examination for senior medical students was administered as a paper and pencil test (P&P) and using CAT. The General Examination is a graduate examination, which is also a preliminary examination for the Korean Medical Licensing Examination (KMLE). The correlations between the results of the CAT and P&P and KMLE were analyzed. The correlation between the CAT and P&P was 0.8013 (p=0.000); that between the CAT and P&P was 0.7861 (p=0.000); and that between the CAT and KMLE was 0.6436 (p=0.000). Six out of 12 students with an ability estimate below 0.52 failed the KMLE. The results showed that CAT could replace P&P in medical school. The ability of CAT to predict whether students would pass the KMLE was 0.5 when the criterion of the theta value was set at -0.52 that was chosen arbitrarily for the prediction of pass or failure.

Citations

Citations to this article as recorded by

Analysis on Validity and Academic Competency of Mock Test for Korean Medicine National Licensing Examination Using Item Response Theory
Han Chae, Eunbyul Cho, SeonKyoung Kim, DaHye Choi, Seul Lee
Keimyung Medical Journal.2023; 42(1): 7. CrossRef
Application of Computerized Adaptive Testing in Medical Education
Sun Huh
Korean Journal of Medical Education.2009; 21(2): 97. CrossRef
Estimation of an Examinee's Ability in the Web-Based Computerized Adaptive Testing Program IRT-CAT
Yoon-Hwan Lee, Jung-Ho Park, In-Yong Park
Journal of Educational Evaluation for Health Professions.2006; 3: 4. CrossRef

Students' Attitude toward and Acceptability of Computerized Adaptive Testing in Medical School and their Effect on the Examinees' Ability: Mee Young Kim, Sun Huh; J Educ Eval Health Prof. 2005;2(1):105-111. Published online June 30, 2005; DOI: https://doi.org/10.3352/jeehp.2005.2.1.105

31,504 View
170 Download
3 Crossref

Abstract PDF

An examinee's ability can be evaluated precisely using computerized adaptive testing (CAT), which is shorter than written tests and more efficient in terms of the duration of the examination. We used CAT for the second General Examination of 98 senior students in medical college on November 27, 2004. We prepared 1,050 pre-calibrated test items according to item response theory, which had been used for the General Examination administered to senior students in 2003. The computer was programmed to pose questions until the standard error of the ability estimate was smaller than 0.01. To determine the students' attitude toward and evaluation of CAT, we conducted surveys before and after the examination, via the Web. The mean of the students' ability estimates was 0.3513 and its standard deviation was 0.9097 (range -2.4680 to +2.5310). There was no significant difference in the ability estimates according to the responses of students to items concerning their experience with CAT, their ability to use a computer, or their anxiety before and after the examination (p>0.05). Many students were unhappy that they could not recheck their responses (49%), and some stated that there were too few examination items (24%). Of the students, 79 % had no complaints concerning using a computer and 63% wanted to expand the use of CAT. These results indicate that CAT can be implemented in medical schools without causing difficulties for users.

Citations

Citations to this article as recorded by

Computer‐Based Testing and Construction of an Item Bank Database for Medical Education in Korea
Sun Huh
Korean Medical Education Review.2014; 16(1): 11. CrossRef
Can computerized tests be introduced to the Korean Medical Licensing Examination?
Sun Huh
Journal of the Korean Medical Association.2012; 55(2): 124. CrossRef
Application of Computerized Adaptive Testing in Medical Education
Sun Huh
Korean Journal of Medical Education.2009; 21(2): 97. CrossRef

Development of Standards and Criteria for Accreditation of Baccalaureate Nursing Education Program: Yang Heui Ahn, Kyung Sook Park, Soon Ok Yang, Kyung Rim Shin, Mi Ja Kim; J Educ Eval Health Prof. 2005;2(1):87-103. Published online June 30, 2005; DOI: https://doi.org/10.3352/jeehp.2005.2.1.87

26,587 View
159 Download
7 Crossref

Abstract PDF

The goal of accreditation is to ensure that the education provided by an institution of higher education meets an acceptable level of quality. This study developed standards and criteria for accreditation of baccalaureate nursing education programs, by comparing accreditation in South Korea and in the United States, and validating standards and criteria. A main comparative analysis was made between Nursing League for Nursing Accrediting Commission (NLNAC) standards, Commission on Collegiate Nursing Education (CCNE) standards for accreditation of baccalaureate nursing education programs, and Korean Accreditation Board of Nursing standards for accreditation of nursing education programs. The research team developed and validated standards and criteria for South Korean baccalaureate nursing education programs. Using the results of the analysis, revisions are proposed to standards for accreditation of baccalaureate nursing education programs, and categorized into 24 criteria and six domains: mission and governance, curriculum and teaching-learning practices, students, faculty, resources, effectiveness. Further studies are required to refine the standards and criteria and make them sophisticated enough to be applied globally.

Citations

Citations to this article as recorded by

Development and Psychometric Testing of a Self-Rated Scale Based on National Nursing Core Competency Standards
Al D. Biag, Vanessa L. Belen
Journal of Nursing Measurement.2022; 30(1): 75. CrossRef
The Positive Effects of Accreditation on Graduate Medical Education Programs in Singapore
Kathleen D. Holt, Rebecca S. Miller, Lauren M. Byrne, Susan H. Day
Journal of Graduate Medical Education.2019; 11(4s): 213. CrossRef
Analysis of Survey of Interdisciplinary Unification of Biomedical Laboratory Science
Hong Sung Kim, Ji-Hyuk Kang, Man-Gil Yang, Chang-Eun Park, Kyung-A Shin, Pil Seung Kwon
The Korean Journal of Clinical Laboratory Science.2018; 50(3): 289. CrossRef
Development and Evaluation of Korean Nurses’ Core Competency Scale (KNCCS)
Won-Hee Lee, Sanghee Kim, Jihyoun An
Open Journal of Nursing.2017; 07(05): 599. CrossRef
Direction of the Community Health Nursing Practice Education Focusing on the Nursing Education Accreditation Criteria
Kyung Ja June, Young Ran Chin, Hee Girl Kim, Chun Mi Kim, Yeon Yi Song, Souk Young Kim, Hanju Lee, Ihn Sook Jeong, Ki Soon Seo, Kyung Won Choi
Journal of Korean Academy of Community Health Nursing.2016; 27(2): 183. CrossRef
A study on Korean nursing students' educational outcomes
Kasil Oh, Yang Heui Ahn, Hyang-Yeon Lee, Sook-Ja Lee, In-Ja Kim, Kyung-Sook Choi, Myung-Sook Ko
Journal of Educational Evaluation for Health Professions.2011; 8: 3. CrossRef
Faculty Perceptions of the Improvement and Effort for the Achievement of Quality in Nursing Education through Accreditation in Baccalaureate Nursing Programs
Kon Hee Kim
Korean Medical Education Review.2011; 13(2): 59. CrossRef

Construct Validity of the Korean Dental Licensing Examination using Confirmatory Factor Analysis: Mi Kyoung Yim, Yoon Hee Kim; J Educ Eval Health Prof. 2005;2(1):75-86. Published online June 30, 2005; DOI: https://doi.org/10.3352/jeehp.2005.2.1.75

34,481 View
161 Download

Abstract PDF: Confirmatory factor analysis based on a measurement model of a structural equation model was used to test the construct validity of 13 subjects in the Korean Dental Licensing Examination (KDLE). The results of 1,086 examinees who wrote the KDLE in 2004 were analyzed. The thirteen subjects were classified into 62 major categories and 122 intermediate categories. There were 364 items. A hierarchical model was constructed, including major and intermediate categories. The impact of the variables was determined by the standardized regression coefficient that related latent and measured variables in the measurement model. The KDLE showed a high goodness-of-fit with a root mean square error of approximation of 0.030 and a non-normed fit index of 0.998. When the latent variables for the major and intermediate categories were analyzed, the standardized regression coefficients of all of the subjects, with the exception of Health and Medical Legislation, were significant. From the result, we concluded that the 13 subjects showed constructive validity. In addition, the study model and data were very compatible. The subject Health and Medical Legislation had a low explanatory impact with respect to testing the ability of dentists to perform their jobs. This study suggests that similar psychometric studies are needed before integrating or deleting subjects on the KDLE, and to improve item development.

Dental Hygienist Job Analysis for Item Development for the Korean Dental Hygienists' Licensing Examination: Sook Hyang Kim, Kyung Hee Song, Hyeon Sook Kwun, Seol Aak Kim, Jong Hwa Jang, Chang Hee Kim, Min Jung Kim, Jin Soo Kim, Sang Hwan Oh; J Educ Eval Health Prof. 2005;2(1):59-74. Published online June 30, 2005; DOI: https://doi.org/10.3352/jeehp.2005.2.1.59

65,535 View
150 Download
2 Crossref

Abstract PDF

This study aimed to develop standard items for the Korean Dental Hygienists' Licensing Examination; these items were also earmarked for use in developing the curriculum for dental hygienists, and in writing a job description, based on a job analysis using the Developing A Curriculum Method (DACUM). It also aimed to understand the significance and frequency of task elements that dental hygienists perform. Data were collected by means of a mail survey, in the form of self-entry, from a sample of dental hygienists registered with the Korean Dental Hygienists' Association. In all, 260 responses were analyzed. The tasks of dental hygienists were divided into four categories, 93 tasks, and 494 task elements. There were 281 elements (61%) that scored higher than 3.5 in significance, in the 4-scale items, and 480 elements (98%) that scored higher than 3.0. There were 30 elements (6%) that scored higher than 3.5 in frequency, and 140 elements (29%) that scored higher than 3.0 in frequency. Overall, 130 out of 494 elements (27%) scored higher than 3.0 for both significance and frequency. Therefore, those 130 elements should be included as items in the Korean Dental Hygienists' Licensing Examination. The results can also be used for curriculum development and as the basis of a job description for dental hygienists.

Citations

Citations to this article as recorded by

Performance of ChatGPT on the Korean National Examination for Dental Hygienists
Soo-Myoung Bae, Hye-Rim Jeon, Gyoung-Nam Kim, Seon-Hui Kwak, Hyo-Jin Lee
Journal of Dental Hygiene Science.2024; 24(1): 62. CrossRef
The Dental Hygienists’ Perception of the National Practical Examination
Da-Kyung Ko, Sung-Suk Bae
Journal of Dental Hygiene Science.2016; 16(6): 488. CrossRef

The Validity of Subjects in Korean Dental Technicians' Licensing Examination: Woong-chul Kim, Won-chul Lee, Young-seok Sohn, Sae-yoon Oh, Bu-sob Kim, Chin-ho Yu, Ji-hwan Kim; J Educ Eval Health Prof. 2005;2(1):43-57. Published online June 30, 2005; DOI: https://doi.org/10.3352/jeehp.2005.2.1.43

26,258 View
150 Download
4 Crossref

Abstract PDF

This study prepared a basic framework for the development and improvement of Korean Dental Technicians' Licensing Examination, based on actual test questions. A peer review was conducted to ensure relevance to current practices in dental technology. For the statistical analysis, 1000 dental laboratory technicians were selected; specialists in dental laboratory technology (laboratory owners, educators, etc.) were involved in creating valid and reliable questions. Results indicated that examination subjects should be divided into three categories: basic dental laboratory theory, dental laboratory specialties, and a practical examination. To ensure relevance to current practice, there should be less emphasis on basic dental laboratory theory, including health-related laws, and more emphasis on dental laboratory specialties. Introduction to dental anatomy should be separated from oral anatomy and tooth morphology; and fixed prosthodontics should be separated from crown and bridge technology and dental ceramics technology. Removable orthodontic appliance technology should be renamed 'orthodontic laboratory technology'. There should be less questions related to health related law, oral anatomy, dental hygiene, dental materials science and inlay, while the distribution ratio of questions related to tooth morphology should be maintained. There should be a decrease in the distribution ratio of questions related to crown and bridge technology, dental ceramics technology, complete dentures and removable partial dentures technology, and orthodontic laboratory technology. In the practical examination, the current multiple choice test should be replaced with tooth carving using wax or plaster. In dental laboratory specialties, subjects related to contemporary dental laboratory technology should be included in the test items.

Citations

Citations to this article as recorded by

A study of the opening status of the compulsory courses for the dental technologist license
Soon-Suk Kwon
Journal of Korean Acedemy of Dental Technology.2021; 43(3): 106. CrossRef
A study on the reorganization of the curriculum for produce of field-oriented dental technicians
Kim Jae-Hong, Kim Ki-Baek, Kim Won-Soo
Journal of Korean Acedemy of Dental Technology.2019; 41(4): 359. CrossRef
The Study on the improvement of dental technician practical examination II
Eun-Jeong Bae, Ki-Baek Kim, Woong-Chul Kim, Ji-Hwan Kim, Chin-Ho Yu, Gyu-Sun Lee
Journal of Korean Acedemy of Dental Technology.2014; 36(1): 39. CrossRef
The Study on the improvement of dental technician practical examination I
Eun-Jeong Bae, Woong-Chul Kim, In-Sung Chung, Sang-Yong Nam, Ji-Hwan Kim
Journal of Korean Acedemy of Dental Technology.2012; 34(4): 413. CrossRef

A study of the validity of the Korean Nurses' Licensing Examination: Hyang Yeon Lee, Cho Ja Kim, Sook Ja Lee, Ho Ran Park, In Sook Lee, Hoo Ja Kim, Young Mi Park; J Educ Eval Health Prof. 2005;2(1):31-42. Published online June 30, 2005; DOI: https://doi.org/10.3352/jeehp.2005.2.1.31

32,075 View
183 Download
5 Crossref

Abstract PDF

This study tested the validity of subjects in the Korean Nurses' Licensing Examination (KNLE). To determine the validity of test items in the KNLE, the items testing each subject in the examination and all of the test items were compared. The homogeneity and proper degree of conceptual diversity of the items in the examination were tested by comparing the frameworks of the test items in the National Council Licensure Examination for Registered Nurses (NCLEX-RN) and the KNLE. The validity of the KNLE was determined by examining the correlations between the scores that the same individuals obtained on the KNLE in 2002 and their college marks. This study showed that basic core items appeared repeatedly in the KNLE, and items in the examination were selected according to the importance of each subject examined. Therefore, a new test system based on united subjects should be considered to solve these problems. Further studies are needed to develop a framework for classifying the test items that should be included as united subjects through considering the job description and goals of study of nurses.

Citations

Citations to this article as recorded by

Item development process and analysis of 50 case-based items for implementation on the Korean Nursing Licensing Examination
In Sook Park, Yeon Ok Suh, Hae Sook Park, So Young Kang, Kwang Sung Kim, Gyung Hee Kim, Yeon-Hee Choi, Hyun-Ju Kim
Journal of Educational Evaluation for Health Professions.2017; 14: 20. CrossRef
Relevance of the test content of the Korean Nursing Licensing Examination to nursing job
In Sook Park, Yeon Ok Suh, Hae Sook Park, Soo Yeon Ahn, So Young Kang, Kwang Sung Kim
Journal of Educational Evaluation for Health Professions.2016; 13: 23. CrossRef
The Relevance between Pathophysiological Subject and Examination Workbook Items for National Nurse Licensure Examination in South Korea and the United States
Myung Sook Park, Hee Jung Choi, Youn Jung Kim, Hee Kyung Chang, Sun Ju Chang, Haeyoung Lee
Journal of Korean Biological Nursing Science.2016; 18(4): 264. CrossRef
A Six-Year Study of Relationship between Academic Performance in Dental Hygiene School and Performance on the Korean Dental Hygiene Licensing Examination at Yonsei University
So-Jung Mun, Hie-Jin Noh, Hyun-Sun Jeon, Ji-Eun Heo, Won-Gyun Chung
Journal of dental hygiene science.2014; 14(3): 332. CrossRef
A Comparative Study of the Nurse Licensure Exam Systems between the United States and South Korea
Chanyeong Kwak, Soon-Nyoung Yun, Shin-Jeong Kim
Korean Journal of Adult Nursing.2013; 25(6): 622. CrossRef

Applicability of Item Response Theory to the Korean Nurses' Licensing Examination: Geum-Hee Jeong, Mi Kyoung Yim; J Educ Eval Health Prof. 2005;2(1):23-29. Published online June 30, 2005; DOI: https://doi.org/10.3352/jeehp.2005.2.1.23

35,031 View
161 Download
3 Crossref

Abstract PDF

To test the applicability of item response theory (IRT) to the Korean Nurses' Licensing Examination (KNLE), item analysis was performed after testing the unidimensionality and goodness-of-fit. The results were compared with those based on classical test theory. The results of the 330-item KNLE administered to 12,024 examinees in January 2004 were analyzed. Unidimensionality was tested using DETECT and the goodness-of-fit was tested using WINSTEPS for the Rasch model and Bilog-MG for the two-parameter logistic model. Item analysis and ability estimation were done using WINSTEPS. Using DETECT, Dmax ranged from 0.1 to 0.23 for each subject. The mean square value of the infit and outfit values of all items using WINSTEPS ranged from 0.1 to 1.5, except for one item in pediatric nursing, which scored 1.53. Of the 330 items, 218 (42.7%) were misfit using the two-parameter logistic model of Bilog-MG. The correlation coefficients between the difficulty parameter using the Rasch model and the difficulty index from classical test theory ranged from 0.9039 to 0.9699. The correlation between the ability parameter using the Rasch model and the total score from classical test theory ranged from 0.9776 to 0.9984. Therefore, the results of the KNLE fit unidimensionality and goodness-of-fit for the Rasch model. The KNLE should be a good sample for analysis according to the IRT Rasch model, so further research using IRT is possible.

Citations

Citations to this article as recorded by

Item difficulty index, discrimination index, and reliability of the 26 health professions licensing examinations in 2022, Korea: a psychometric study
Yoon Hee Kim, Bo Hyun Kim, Joonki Kim, Bokyoung Jung, Sangyoung Bae
Journal of Educational Evaluation for Health Professions.2023; 20: 31. CrossRef
Study on the Academic Competency Assessment of Herbology Test using Rasch Model
Han Chae, Soo Jin Lee, Chang-ho Han, Young Il Cho, Hyungwoo Kim
Journal of Korean Medicine.2022; 43(2): 27. CrossRef
Can computerized tests be introduced to the Korean Medical Licensing Examination?
Sun Huh
Journal of the Korean Medical Association.2012; 55(2): 124. CrossRef

Review Article

The New Horizon for Evaluations in Medical Education in Korea: Sang-Ho Baik; J Educ Eval Health Prof. 2005;2(1):7-22. Published online June 30, 2005; DOI: https://doi.org/10.3352/jeehp.2005.2.1.7

33,831 View
215 Download
7 Crossref

Abstract PDF

Over the last two decades, there have been a number of significant changes in the evaluation system in medical education in Korea. One major improvement in this respect has been the listing of learning objectives at medical schools and the construction of a content outline for the Korean Medical Licensing Examination that can be used as a basis of evaluation. Item analysis has become a routine method for obtaining information that often provides valuable feedback concerning test items after the completion of a written test. The use of item response theory in analyzing test items has been spreading in medical schools as a way to evaluate performance tests and computerized adaptive testing. A series of recent studies have documented an upward trend in the adoption of the objective structured clinical examination (OSCE) and clinical practice examination (CPX) for measuring skill and attitude domains, in addition to tests of the knowledge domain. There has been an obvious increase in regional consortiums involving neighboring medical schools that share the planning and administration of the OSCE and CPX; this includes recruiting and training standardized patients. Such consortiums share common activities, such as case development and program evaluation. A short history and the pivotal roles of four organizations that have brought about significant changes in the examination system are discussed briefly.

Citations

Citations to this article as recorded by

Presidential address: Adoption of a clinical skills examination for dental licensing, implementation of computer-based testing for the medical licensing examination, and the 30th anniversary of the Korea Health Personnel Licensing Examination Institute
Yoon-Seong Lee
Journal of Educational Evaluation for Health Professions.2022; 19: 1. CrossRef
Effectiveness of Medical Education Assessment Consortium Clinical Knowledge Mock Examination (2011‐2016)
Sang Yeoup Lee, Yeli Lee, Mi Kyung Kim
Korean Medical Education Review.2018; 20(1): 20. CrossRef
Long for wonderful leadership in a new era of the Korean Association of Medical Colleges
Young Hwan Lee
Korean Journal of Medical Education.2014; 26(3): 163. CrossRef
Major Reforms and Issues of the Medical Licensing Examination Systems in Korea
Sang-Ho Baik
Korean Medical Education Review.2013; 15(3): 125. CrossRef
A Study on the Feasibility of a National Practical Examination in the Radiologic Technologist
Soon-Yong Son, Tae-Hyung Kim, Jung-Whan Min, Dong-Kyoon Han, Sung-Min Ahn
Journal of the Korea Academia-Industrial cooperation Society.2011; 12(5): 2149. CrossRef
The Relationship between Senior Year Examinations at a Medical School and the Korean Medical Licensing Examination
Ki Hoon Jung, Ho Keun Jung, Kwan Lee
Korean Journal of Medical Education.2009; 21(1): 17. CrossRef
What Qualities Do Medical School Applicants Need to Have? - Secondary Publication
Yera Hur, Sun Kim
Yonsei Medical Journal.2009; 50(3): 427. CrossRef

First
Prev
Page of 1
Next
Last

Previous issues