Journal of Educational Evaluation for Health Professions

Technical report

Feasibility of applying computerized adaptive testing to the Clinical Medical Science Comprehensive Examination in Korea: a psychometric study: Jeongwook Choi, Sung-Soo Jung, Eun Kwang Choi, Kyung Sik Kim, Dong Gi Seo; J Educ Eval Health Prof. 2025;22:29. Published online October 1, 2025; DOI: https://doi.org/10.3352/jeehp.2025.22.29

1,149 View
182 Download

Abstract PDF Supplementary Material: Purpose
This study aimed to investigate the feasibility of transitioning the Clinical Medical Science Comprehensive Examination (CMSCE) to computerized adaptive testing (CAT) in Korea, thereby providing greater opportunities for medical students to accurately compare their clinical competencies with peers nationwide and to monitor their own progress.
Methods
A medical self-assessment using CAT was conducted from March to June 2023, involving 1,541 medical students who volunteered from 40 medical colleges in Korea. An item bank consisting of 1,145 items from previously administered CMSCE examinations (2019–2021) hosted by the Medical Education Assessment Corporation was established. Items were selected through 2-stage filtering, based on classical test theory (discrimination index above 0.15) and item response theory (discrimination parameter estimates above 0.6 and difficulty parameter estimates between –5 and +5). Maximum Fisher information was employed as the item selection method, and maximum likelihood estimation was used for ability estimation.
Results
The CAT was successfully administered without significant issues. The stopping rule was set at a standard error of measurement of 0.25, with a maximum of 50 items for ability estimation. The mean ability score was 0.55, with an average of 28 items administered per student. Students at extreme ability levels reached the maximum of 50 items due to the limited availability of items at appropriate difficulty levels.
Conclusion
The medical self-assessment CAT, the first of its kind in Korea, was successfully implemented nationwide without significant problems. These results indicate strong potential for expanding the use of CAT in medical education assessments.

Research article

Validation of the 21st Century Skills Assessment Scale for public health students in Thailand: a methodological study: Suphawadee Panthumas, Kaung Zaw, Wirin Kittipichai; J Educ Eval Health Prof. 2024;21:37. Published online December 10, 2024; DOI: https://doi.org/10.3352/jeehp.2024.21.37

5,155 View
403 Download

Abstract PDF Supplementary Material: Purpose
This study aimed to develop and validate the 21st Century Skills Assessment Scale (21CSAS) for Thai public health (PH) undergraduate students using the Partnership for 21st Century Skills framework.
Methods
A cross-sectional survey was conducted among 727 first- to fourth-year PH undergraduate students from 4 autonomous universities in Thailand. Data were collected using self-administered questionnaires between January and March 2023. Exploratory factor analysis (EFA) was used to explore the underlying dimensions of 21CSAS, while confirmatory factor analysis (CFA) was conducted to test the hypothesized factor structure using Mplus software (Muthén & Muthén). Reliability and item discrimination were assessed using Cronbach’s α and the corrected item-total correlation, respectively.
Results
EFA performed on a dataset of 300 students revealed a 20-item scale with a 6-factor structure: (1) creativity and innovation; (2) critical thinking and problem-solving; (3) information, media, and technology; (4) communication and collaboration; (5) initiative and self-direction; and (6) social and cross-cultural skills. The rotated eigenvalues ranged from 2.12 to 1.73. CFA performed on another dataset of 427 students confirmed a good model fit (χ2/degrees of freedom=2.67, comparative fit index=0.93, Tucker-Lewis index=0.91, root mean square error of approximation=0.06, standardized root mean square residual=0.06), explaining 34%–71% of variance in the items. Item loadings ranged from 0.58 to 0.84. The 21CSAS had a Cronbach’s α of 0.92.
Conclusion
The 21CSAS proved be a valid and reliable tool for assessing 21st century skills among Thai PH undergraduate students. These findings provide insights for educational system to inform policy, practice, and research regarding 21st-century skills among undergraduate students.

Reviews

A proposal for the future of medical education accreditation in Korea: Ki-Young Lim; J Educ Eval Health Prof. 2020;17:32. Published online October 21, 2020; DOI: https://doi.org/10.3352/jeehp.2020.17.32

7,505 View
146 Download
4 Web of Science
8 Crossref

Abstract PDF Supplementary Material

For the past 20 years, the medical education accreditation program of the Korean Institute of Medical Education and Evaluation (KIMEE) has contributed significantly to the standardization and improvement of the quality of basic medical education in Korea. It should now contribute to establishing and promoting the future of medical education. The Accreditation Standards of KIMEE 2019 (ASK2019) have been adopted since 2019, with the goal of achieving world-class medical education by applying a learner-centered curriculum using a continuum framework for the 3 phases of formal medical education: basic medical education, postgraduate medical education, and continuing professional development. ASK2019 will also be able to promote medical education that meets community needs and employs systematic assessments throughout the education process. These are important changes that can be used to gauge the future of the medical education accreditation system. Furthermore, globalization, inter-professional education, health systems science, and regular self-assessment systems are emerging as essential topics for the future of medical education. It is time for the medical education accreditation system in Korea to observe and adopt new trends in global medical education.

Citations

Citations to this article as recorded by

Development of Clinical Reasoning and History-Taking Remediation Training
Giray Kolcu, Sebahat Ulusan, Mukadder İnci Başer Kolcu
Journal of Medical Education and Family Medicine.2025; 2(1): 6. CrossRef
Perceived effects of accreditation on education quality and health-related job outcomes: scales validation and correlates in Lebanon
Deema Rahme, Daniele Saade, Hala Sacre, Chadia Haddad, Samah Tawil, Samar Merhi, Randa Aoun, Aline Hajj, Fouad Sakr, Jihan Safwan, Marwan Akel, Rony M. Zeenny, Pascale Salameh
BMC Medical Education.2025;[Epub] CrossRef
Analyzing the characteristics of mission statements in Korean medical schools based on the Korean Doctor’s Role framework
Ye Ji Kang, Soomin Lee, Hyo Jeong Lee, Do-Hwan Kim
Korean Journal of Medical Education.2024; 36(1): 99. CrossRef
Challenges and potential improvements in the Accreditation Standards of the Korean Institute of Medical Education and Evaluation 2019 (ASK2019) derived through meta-evaluation: a cross-sectional study
Yoonjung Lee, Min-jung Lee, Junmoo Ahn, Chungwon Ha, Ye Ji Kang, Cheol Woong Jung, Dong-Mi Yoo, Jihye Yu, Seung-Hee Lee
Journal of Educational Evaluation for Health Professions.2024; 21: 8. CrossRef
Accreditation standards items of post-2nd cycle related to the decision of accreditation of medical schools by the Korean Institute of Medical Education and Evaluation
Kwi Hwa Park, Geon Ho Lee, Su Jin Chae, Seong Yong Kim
Korean Journal of Medical Education.2023; 35(1): 1. CrossRef
Continuing Professional Development of Pharmacists and The Roles of Pharmacy Schools
Hyemin Park, Jeong-Hyun Yoon
Korean Journal of Clinical Pharmacy.2022; 32(4): 281. CrossRef
Definition of character for medical education based on expert opinions in Korea
Yera Hur
Journal of Educational Evaluation for Health Professions.2021; 18: 26. CrossRef
Special reviews on the history and future of the Korean Institute of Medical Education and Evaluation to memorialize its collaboration with the Korea Health Personnel Licensing Examination Institute to designate JEEHP as a co-official journal
Sun Huh
Journal of Educational Evaluation for Health Professions.2020; 17: 33. CrossRef

Is accreditation in medical education in Korea an opportunity or a burden?: Hanna Jung, Woo Taek Jeon, Shinki An; J Educ Eval Health Prof. 2020;17:31. Published online October 21, 2020; DOI: https://doi.org/10.3352/jeehp.2020.17.31

9,487 View
176 Download
13 Web of Science
17 Crossref

Abstract PDF Supplementary Material

The accreditation process is both an opportunity and a burden for medical schools in Korea. The line that separates the two is based on how medical schools recognize and utilize the accreditation process. In other words, accreditation is a burden for medical schools if they view the accreditation process as merely a formal procedure or a means to maintain accreditation status for medical education. However, if medical schools acknowledge the positive value of the accreditation process, accreditation can be both an opportunity and a tool for developing medical education. The accreditation process has educational value by catalyzing improvements in the quality, equity, and efficiency of medical education and by increasing the available options. For the accreditation process to contribute to medical education development, accrediting agencies and medical schools must first be recognized as partners of an educational alliance working together towards common goals. Secondly, clear guidelines on accreditation standards should be periodically reviewed and shared. Finally, a formative self-evaluation process must be introduced for institutions to utilize the accreditation process as an opportunity to develop medical education. This evaluation system could be developed through collaboration among medical schools, academic societies for medical education, and the accrediting authority.

Citations

Citations to this article as recorded by

Curriculum development as a governance process: insights from a new PharmD program
Geneviève Gauthier, Françoise Moreau-Johnson, Christine Landry, Tim Dubé, Claire Touchie
Advances in Health Sciences Education.2026;[Epub] CrossRef
Equity in Basic Medical Education accreditation standards: a scoping review protocol
Neelofar Shaheen, Usman Mahboob, Ahsan Sethi, Muhammad Irfan
BMJ Open.2025; 15(1): e086661. CrossRef
Making Medical Education Socially Accountable in Australia and Southeast Asia: A Systematic Review
Jyotsna Rimal, Ashish Shrestha, Elizabeth Cardell, Stephen Billett, Alfred King-yin Lam
Medical Science Educator.2025; 35(3): 1767. CrossRef
Impacts of the Accreditation Process for Undergraduate Medical Schools: A Scoping Review
Leticia C. Girotto, Karynne B. Machado, Roberta F. C. Moreira, Milton A. Martins, Patrícia Z. Tempski
The Clinical Teacher.2025;[Epub] CrossRef
Benefits, Challenges, and Opportunities of LCME Accreditation: A National Survey
Colleen Hayden, David Muller, Allan R. Tunkel, Reena Karani, Jennifer Christner, Abbas Hyderi, Robert Fallar
Medical Science Educator.2025; 35(5): 2533. CrossRef
Impact of accreditation on medical education quality improvement in 82 medical schools in Japan: a descriptive study
Nobuo Nara
Journal of Educational Evaluation for Health Professions.2025; 22: 22. CrossRef
Two decades of accreditation in Chilean medical education: outcomes and lessons learned
Oscar Jerez Yañez, Carlos Schade Carter, Miguel Altamirano Rivas, Bárbara Carrasco García
BMC Medical Education.2025;[Epub] CrossRef
Global research agenda for medical education regulation: findings from a nominal group consensus exercise
Valdes Roberto Bollela, Vanessa Burch, Kadambari Dharanipragada, Janneke Frambach, Janet Grant, Lois Haruna-Cooper, Homa Kabiri, James Kelly, Maria-Athina Martimianakis, Fernando Menezes da Silva, Lamiaa Mohsen, John-george Nicholson, Mohammed Ahmed Rashi
BMJ Global Health.2025; 10(12): e016014. CrossRef
To prove or improve? Examining how paradoxical tensions shape evaluation practices in accreditation contexts
Betty Onyura, Abigail J. Fisher, Qian Wu, Shrutikaa Rajkumar, Sarick Chapagain, Judith Nassuna, David Rojas, Latika Nirula
Medical Education.2024; 58(3): 354. CrossRef
ASPIRE for excellence in curriculum development
John Jenkins, Sharon Peters, Peter McCrorie
Medical Teacher.2024; 46(5): 633. CrossRef
Challenges and potential improvements in the Accreditation Standards of the Korean Institute of Medical Education and Evaluation 2019 (ASK2019) derived through meta-evaluation: a cross-sectional study
Yoonjung Lee, Min-jung Lee, Junmoo Ahn, Chungwon Ha, Ye Ji Kang, Cheol Woong Jung, Dong-Mi Yoo, Jihye Yu, Seung-Hee Lee
Journal of Educational Evaluation for Health Professions.2024; 21: 8. CrossRef
Accreditation standards items of post-2nd cycle related to the decision of accreditation of medical schools by the Korean Institute of Medical Education and Evaluation
Kwi Hwa Park, Geon Ho Lee, Su Jin Chae, Seong Yong Kim
Korean Journal of Medical Education.2023; 35(1): 1. CrossRef
The Need for the Standards for Anatomy Labs in Medical School Evaluation and Accreditation
Yu-Ran Heo, Jae-Ho Lee
Anatomy & Biological Anthropology.2023; 36(3): 81. CrossRef
Seal of Approval or Ticket to Triumph? The Impact of Accreditation on Medical Student Performance in Foreign Medical Council Examinations
Saurabh RamBihariLal Shrivastava, Titi Savitri Prihatiningsih, Kresna Lintang Pratidina
Indian Journal of Medical Specialities.2023; 14(4): 249. CrossRef
Internal evaluation in the faculties affiliated to zanjan university of medical sciences: Quality assurance of medical science education based on institutional accreditation
Alireza Abdanipour, Farhad Ramezani‐Badr, Ali Norouzi, Mehdi Ghaemi
Journal of Medical Education Development.2022; 15(46): 61. CrossRef
Development of Mission and Vision of College of Korean Medicine Using the Delphi Techniques and Big-Data Analysis
Sanghee Yeo, Seong Hun Choi, Su Jin Chae
Journal of Korean Medicine.2021; 42(4): 176. CrossRef
Special reviews on the history and future of the Korean Institute of Medical Education and Evaluation to memorialize its collaboration with the Korea Health Personnel Licensing Examination Institute to designate JEEHP as a co-official journal
Sun Huh
Journal of Educational Evaluation for Health Professions.2020; 17: 33. CrossRef

Research articles

Development of a self-assessment tool for resident doctors’ communication skills in India: Upendra Baitha, Piyush Ranjan, Siddharth Sarkar, Charu Arora, Archana Kumari, Sada Nand Dwivedi, Asmita Patil, Nayer Jamshed; J Educ Eval Health Prof. 2019;16:17. Published online June 24, 2019; DOI: https://doi.org/10.3352/jeehp.2019.16.17

17,229 View
303 Download
13 Web of Science
13 Crossref

Abstract PDF Supplementary Material

Purpose
Effective communication skills are essential for resident doctors to provide optimum patient care. This study was conducted to develop and validate a questionnaire for the self-assessment of resident doctors’ communication skills in India.
Methods
This was a mixed-methods study conducted in 2 phases. The first phase consisted of questionnaire development, including the identification of relevant literature, focus group discussions with residents and experts from clinical specialties, and pre-testing of the questionnaire. The second phase involved administering the questionnaire survey to 95 residents from the Departments of Medicine, Emergency Medicine, Pediatrics, and Surgery at the All India Institute of Medical Sciences, New Delhi, India in April 2019. Internal consistency was tested and the factor structure was analyzed to test construct validity.
Results
The questionnaire consisted of 3 sections: (A) 4 items on doctor-patient conflicts and the role of communication skills in avoiding these conflicts, (B) 29 items on self-assessment of communication skills in different settings, and (C) 8 items on barriers to practicing good communication skills. Sections B and C had good internal consistency (Cronbach α: 0.885 and 0.771, respectively). Section C had a 2-factor solution, and the barriers were classified as ‘training’ and ‘infrastructure’ factors.
Conclusion
This appears to be a valid assessment tool of resident doctors’ communication skills, with potential utility for identifying gaps in communication skills and developing communication skills modules.

Citations

Citations to this article as recorded by

Evaluating Communication Skills in Health Professions Education: A Systematic Review of Assessment Tools
Kelly Macauley, Jacque Bradford, Cathy Stucker, Carol Likens, Lin Wu
The Clinical Teacher.2026;[Epub] CrossRef
Are we ready for the feedback? A qualitative exploratory study on residents’ awareness and readiness for multisource (360°) feedback in medical residency
Sumera Nisar, Ayesha Jamal, Husna Irfan Thalib, Sariya Khan, Sayeeda Mehveen, Aseef Rehman, Ghala AlSughayyir, Ammar Yassir Bafarat, Hiba Hisham Alshikh Ali
BMJ Open.2025; 15(12): e101924. CrossRef
Development and Validation of a Comprehensive Nursing Competence Assessment Questionnaire (CNCAQ) to Determine the Perceived Clinical Competence of Nursing Graduates
Sunita Srivastava, Hariprasath Pandurangan, Anil Kumar
Nursing & Midwifery Research Journal.2024; 20(2): 96. CrossRef
Socio‐behaviour change intervention in health care professionals: Impact and effectiveness
Chinmay Shah, Fouzia Shersad
Medical Education.2024; 58(11): 1422. CrossRef
Communication skills of residents: are they as good as they think?
Namra Qadeer Shaikh, Ali Aahil Noorali, Asma Altaf Hussain Merchant, Noreen Afzal, Maryam Pyar Ali Lakhdir, Komal Abdul Rahim, Syeda Fatima Shariq, Rida Ahmad, Saqib Kamran Bakhshi, Saad Bin Zafar Mahmood, Shayan Shah, Muhammad Rizwan Khan, Muhammad Tariq
Medical Education Online.2024;[Epub] CrossRef
Leveraging the vantage point – exploring nurses’ perception of residents’ communication skills: a mixed-methods study
Komal Abdul Rahim, Maryam Pyar Ali Lakhdir, Noreen Afzal, Asma Altaf Hussain Merchant, Namra Qadeer Shaikh, Ali Aahil Noorali, Umar Tariq, Rida Ahmad, Saqib Kamran Bakhshi, Saad bin Zafar Mahmood, Muhammad Rizwan Khan, Muhammed Tariq, Adil H. Haider
BMC Medical Education.2023;[Epub] CrossRef
Developing a communication-skills training curriculum for resident-physicians to enhance patient outcomes at an academic medical centre: an ongoing mixed-methods study protocol
Hamna Shahbaz, Ali Aahil Noorali, Maha Inam, Namra Qadeer, Asma Altaf Hussain Merchant, Adnan Ali Khan, Noreen Afzal, Komal Abdul Rahim, Ibrahim Munaf, Rida Ahmad, Muhammad Tariq, Adil H Haider
BMJ Open.2022; 12(8): e056840. CrossRef
A cross-sectional evaluation of communication skills and perceived barriers among the resident doctors at a tertiary care center in India
Amandeep Singh, Piyush Ranjan, Archana Kumari, Siddharth Sarkar, Tanveer Kaur, Ramesh Aggarwal, Ashish Datt Upadhyay, Biswaroop Chakrawarty, Jamshed Nayer, Mohit Joshi, Avinash Chakrawarty
Journal of Education and Health Promotion.2022; 11(1): 425. CrossRef
What do clinical resident doctors think about workplace violence? A qualitative study comprising focus group discussions and thematic analysis from a tertiary care center of India
Amandeep Singh, Piyush Ranjan, Siddharth Sarkar, Tarang Preet Kaur, Roshan Mathew, Dinesh Gora, Ajay Mohan, Jaswant Jangra
Journal of Family Medicine and Primary Care.2022; 11(6): 2678. CrossRef
Development and validation of a questionnaire to assess preventive practices against COVID-19 pandemic in the general population
Ayush Agarwal, Piyush Ranjan, Priyanka Rohilla, Yellamraju Saikaustubh, Anamika Sahu, Sada Nand Dwivedi, Aakansha, Upendra Baitha, Arvind Kumar
Preventive Medicine Reports.2021; 22: 101339. CrossRef
Development and Validation of a Comprehensive Questionnaire to Assess Interpersonal Discord (Bullying, Harassment, and Discrimination) at the Workplace in a Healthcare Setting
Amandeep Singh, Piyush Ranjan, Tanveer Kaur, Siddharth Sarkar, Ashish D Upadhyay, Upendra Baitha, Prayas Sethi, Ranveer S Jadon, Pankaj Jorwal
Cureus.2021;[Epub] CrossRef
Development and Validation of a Questionnaire to Evaluate Workplace Violence in Healthcare Settings
Archana Kumari, Amandeep Singh, Piyush Ranjan, Siddharth Sarkar, Tanveer Kaur, Ashish D Upadhyay, Kirti Verma, Vignan Kappagantu, Ajay Mohan, Upendra Baitha
Cureus.2021;[Epub] CrossRef
The value of communicating with patients in their first language
Piyush Ranjan, Archana Kumari, Charu Arora
Expert Review of Pharmacoeconomics & Outcomes Research.2020; 20(6): 559. CrossRef

Learning through multiple lenses: analysis of self, peer, nearpeer, and faculty assessments of a clinical history-taking task in Australia: Kylie Fitzgerald, Brett Vaughan; J Educ Eval Health Prof. 2018;15:22. Published online September 18, 2018; DOI: https://doi.org/10.3352/jeehp.2018.15.22

26,207 View
306 Download
5 Web of Science
7 Crossref

Abstract PDF Supplementary Material

Purpose
Peer assessment provides a framework for developing expected skills and receiving feedback appropriate to the learner’s level. Near-peer (NP) assessment may elevate expectations and motivate learning. Feedback from peers and NPs may be a sustainable way to enhance student assessment feedback. This study analysed relationships among self, peer, NP, and faculty marking of an assessment and students’ attitudes towards marking by those various groups.
Methods
A cross-sectional study design was used. Year 2 osteopathy students (n= 86) were invited to perform self and peer assessments of a clinical history-taking and communication skills assessment. NPs and faculty also marked the assessment. Year 2 students also completed a questionnaire on their attitudes to peer/NP marking. Descriptive statistics and the Spearman rho coefficient were used to evaluate relationships across marker groups.
Results
Year 2 students (n= 9), NPs (n= 3), and faculty (n= 5) were recruited. Correlations between self and peer (r= 0.38) and self and faculty (r= 0.43) marks were moderate. A weak correlation was observed between self and NP marks (r= 0.25). Perceptions of peer and NP marking varied, with over half of the cohort suggesting that peer or NP assessments should not contribute to their grade.
Conclusion
Framing peer and NP assessment as another feedback source may offer a sustainable method for enhancing feedback without overloading faculty resources. Multiple sources of feedback may assist in developing assessment literacy and calibrating students’ self-assessment capability. The small number of students recruited suggests some acceptability of peer and NP assessment; however, further work is required to increase its acceptability.

Citations

Citations to this article as recorded by

Learning how to suture: Should learners observe a demonstration of someone who is experienced or inexperienced to improve their own performance?
Portia Kalun, Ranil Sonnadara
The American Journal of Surgery.2025; 243: 116276. CrossRef
Transitioning to competency-based education in nursing: a scoping review of curriculum review and revision strategies
Zakaria Ahmed Mani
BMC Nursing.2025;[Epub] CrossRef
The extent and quality of evidence for osteopathic education: A scoping review
Andrew MacMillan, Patrick Gauthier, Luciane Alberto, Arabella Gaunt, Rachel Ives, Chris Williams, Dr Jerry Draper-Rodi
International Journal of Osteopathic Medicine.2023; 49: 100663. CrossRef
History and physical exam: a retrospective analysis of a clinical opportunity
David McLinden, Krista Hailstone, Sue Featherston
BMC Medical Education.2023;[Epub] CrossRef
How Accurate Are Our Students? A Meta-analytic Systematic Review on Self-assessment Scoring Accuracy
Samuel P. León, Ernesto Panadero, Inmaculada García-Martínez
Educational Psychology Review.2023;[Epub] CrossRef
Evaluating the Academic Performance of Mustansiriyah Medical College Teaching Staff vs. Final-Year Students Failure Rates
Wassan Nori, Wisam Akram , Saad Mubarak Rasheed, Nabeeha Najatee Akram, Taqi Mohammed Jwad Taher, Mustafa Ali Kassim Kassim, Alexandru Cosmin Pantazi
Al-Rafidain Journal of Medical Sciences ( ISSN 2789-3219 ).2023; 5(1S): S151. CrossRef
History-taking level and its influencing factors among nursing undergraduates based on the virtual standardized patient testing results: Cross sectional study
Jingrong Du, Xiaowen Zhu, Juan Wang, Jing Zheng, Xiaomin Zhang, Ziwen Wang, Kun Li
Nurse Education Today.2022; 111: 105312. CrossRef

First
Prev
Page of 1
Next
Last

Search