Purpose This study assessed the performance of 6 generative artificial intelligence (AI) platforms on the learning objectives of medical arthropodology in a parasitology class in Korea. We examined the AI platforms’ performance by querying in Korean and English to determine their information amount, accuracy, and relevance in prompts in both languages.
Methods From December 15 to 17, 2023, 6 generative AI platforms—Bard, Bing, Claude, Clova X, GPT-4, and Wrtn—were tested on 7 medical arthropodology learning objectives in English and Korean. Clova X and Wrtn are platforms from Korean companies. Responses were evaluated using specific criteria for the English and Korean queries.
Results Bard had abundant information but was fourth in accuracy and relevance. GPT-4, with high information content, ranked first in accuracy and relevance. Clova X was 4th in amount but 2nd in accuracy and relevance. Bing provided less information, with moderate accuracy and relevance. Wrtn’s answers were short, with average accuracy and relevance. Claude AI had reasonable information, but lower accuracy and relevance. The responses in English were superior in all aspects. Clova X was notably optimized for Korean, leading in relevance.
Conclusion In a study of 6 generative AI platforms applied to medical arthropodology, GPT-4 excelled overall, while Clova X, a Korea-based AI product, achieved 100% relevance in Korean queries, the highest among its peers. Utilizing these AI platforms in classrooms improved the authors’ self-efficacy and interest in the subject, offering a positive experience of interacting with generative AI platforms to question and receive information.
Citations
Citations to this article as recorded by
Opportunities, challenges, and future directions of large language models, including ChatGPT in medical education: a systematic scoping review Xiaojun Xu, Yixiao Chen, Jing Miao Journal of Educational Evaluation for Health Professions.2024; 21: 6. CrossRef
Artificial intelligence (AI) is expected to affect various fields of medicine substantially and has the potential to improve many aspects of healthcare. However, AI has been creating much hype, too. In applying AI technology to patients, medical professionals should be able to resolve any anxiety, confusion, and questions that patients and the public may have. Also, they are responsible for ensuring that AI becomes a technology beneficial for patient care. These make the acquisition of sound knowledge and experience about AI a task of high importance for medical students. Preparing for AI does not merely mean learning information technology such as computer programming. One should acquire sufficient knowledge of basic and clinical medicines, data science, biostatistics, and evidence-based medicine. As a medical student, one should not passively accept stories related to AI in medicine in the media and on the Internet. Medical students should try to develop abilities to distinguish correct information from hype and spin and even capabilities to create thoroughly validated, trustworthy information for patients and the public.
Citations
Citations to this article as recorded by
Radiology as a Specialty in the Era of Artificial Intelligence: A Systematic Review and Meta-analysis on Medical Students, Radiology Trainees, and Radiologists Amir Hassankhani, Melika Amoukhteh, Parya Valizadeh, Payam Jannatdoust, Paniz Sabeghi, Ali Gholamrezanezhad Academic Radiology.2024; 31(1): 306. CrossRef
Strategies for Implementing Machine Learning Algorithms in the Clinical Practice of Radiology Allison Chae, Michael S. Yao, Hersh Sagreiya, Ari D. Goldberg, Neil Chatterjee, Matthew T. MacLean, Jeffrey Duda, Ameena Elahi, Arijitt Borthakur, Marylyn D. Ritchie, Daniel Rader, Charles E. Kahn, Walter R. Witschey, James C. Gee Radiology.2024;[Epub] CrossRef
Towards integration of artificial intelligence into medical devices as a real-time recommender system for personalised healthcare: State-of-the-art and future prospects Talha Iqbal, Mehedi Masud, Bilal Amin, Conor Feely, Mary Faherty, Tim Jones, Michelle Tierney, Atif Shahzad, Patricia Vazquez Health Sciences Review.2024; 10: 100150. CrossRef
The Knowledge of Students at Bursa Faculty of Medicine towards Artificial Intelligence: A Survey Study Deniz GÜVEN, Elif Güler KAZANCI, Ayşe ÖREN, Livanur SEVER, Pelin ÜNLÜ Journal of Bursa Faculty of Medicine.2024; 2(1): 20. CrossRef
Preparing healthcare leaders of the digital age with an integrative artificial intelligence curriculum: a pilot study Soo Hwan Park, Roshini Pinto-Powell, Thomas Thesen, Alexander Lindqwister, Joshua Levy, Rachael Chacko, Devina Gonzalez, Connor Bridges, Adam Schwendt, Travis Byrum, Justin Fong, Shahin Shasavari, Saeed Hassanpour Medical Education Online.2024;[Epub] CrossRef
A scoping review of artificial intelligence in medical education: BEME Guide No. 84 Morris Gordon, Michelle Daniel, Aderonke Ajiboye, Hussein Uraiby, Nicole Y. Xu, Rangana Bartlett, Janice Hanson, Mary Haas, Maxwell Spadafore, Ciaran Grafton-Clarke, Rayhan Yousef Gasiea, Colin Michie, Janet Corral, Brian Kwan, Diana Dolmans, Satid Thamma Medical Teacher.2024; : 1. CrossRef
Artificial Intelligence Readiness Status of Medical Faculty Students Büşra EMİR, Tulin YURDEM, Tulin OZEL, Toygar SAYAR, Teoman Atalay UZUN, Umit AKAR, Unal Arda COLAK Konuralp Tıp Dergisi.2024; 16(1): 88. CrossRef
Potential applications of ChatGPT in obstetrics and gynecology in Korea: a review article YooKyung Lee, So Yun Kim Obstetrics & Gynecology Science.2024; 67(2): 153. CrossRef
ChatGPT in dentomaxillofacial radiology education Hilal Peker Öztürk, Hakan Avsever, Buğra Şenel, Şükran Ayran, Mustafa Çağrı Peker, Hatice Seda Özgedik, Nurten Baysal Journal of Health Sciences and Medicine.2024; 7(2): 224. CrossRef
Twelve tips for addressing ethical concerns in the implementation of artificial intelligence in medical education Russell Franco D’Souza, Mary Mathew, Vedprakash Mishra, Krishna Mohan Surapaneni Medical Education Online.2024;[Epub] CrossRef
A novel adaptive cubic quasi‐Newton optimizer for deep learning based medical image analysis tasks, validated on detection of COVID‐19 and segmentation for COVID‐19 lung infection, liver tumor, and optic disc/cup Yan Liu, Maojun Zhang, Zhiwei Zhong, Xiangrong Zeng Medical Physics.2023; 50(3): 1528. CrossRef
Clinical informatics training in medical school education curricula: a scoping review Humairah Zainal, Joshua Kuan Tan, Xin Xiaohui, Julian Thumboo, Fong Kok Yong Journal of the American Medical Informatics Association.2023; 30(3): 604. CrossRef
Are ChatGPT’s knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination?: a descriptive study Sun Huh Journal of Educational Evaluation for Health Professions.2023; 20: 1. CrossRef
Exploring the views of Singapore junior doctors on medical curricula for the digital age: A case study Humairah Zainal, Xin Xiaohui, Julian Thumboo, Fong Kok Yong, Conor Gilligan PLOS ONE.2023; 18(3): e0281108. CrossRef
Artificial Intelligence Teaching as Part of Medical Education: Qualitative Analysis of Expert Interviews Lukas Weidener, Michael Fischer JMIR Medical Education.2023; 9: e46428. CrossRef
Investigating Students’ Perceptions towards Artificial Intelligence in Medical Education Ali Jasem Buabbas, Brouj Miskin, Amar Ali Alnaqi, Adel K. Ayed, Abrar Abdulmohsen Shehab, Shabbir Syed-Abdul, Mohy Uddin Healthcare.2023; 11(9): 1298. CrossRef
Performance and risks of ChatGPT used in drug information: an exploratory real-world analysis Benedict Morath, Ute Chiriac, Elena Jaszkowski, Carolin Deiß, Hannah Nürnberg, Katrin Hörth, Torsten Hoppe-Tichy, Kim Green European Journal of Hospital Pharmacy.2023; : ejhpharm-2023-003750. CrossRef
A closer look at the current knowledge and prospects of artificial intelligence integration in dentistry practice: A cross-sectional study Zuhal Y. Hamd, Wiam Elshami, Sausan Al Kawas, Hanan Aljuaid, Mohamed M. Abuzaid Heliyon.2023; 9(6): e17089. CrossRef
ChatGPT and the Future of Digital Health: A Study on Healthcare Workers’ Perceptions and Expectations Mohamad-Hani Temsah, Fadi Aljamaan, Khalid H. Malki, Khalid Alhasan, Ibraheem Altamimi, Razan Aljarbou, Faisal Bazuhair, Abdulmajeed Alsubaihin, Naif Abdulmajeed, Fatimah S. Alshahrani, Reem Temsah, Turki Alshahrani, Lama Al-Eyadhy, Serin Mohammed Alkhate Healthcare.2023; 11(13): 1812. CrossRef
The Impact of Artificial Intelligence on the Preference of Radiology as a Future Specialty Among Medical Students at Jazan University, Saudi Arabia: A Cross-Sectional Study Khalid M Hakami, Mohammed Alameer, Essa Jaawna, Abdulrahman Sudi, Bahiyyah Bahkali, Amnah Mohammed, Abdulaziz Hakami, Mohamed Salih Mahfouz, Abdulaziz H Alhazmi, Turki M Dhayihi Cureus.2023;[Epub] CrossRef
Application of artificial intelligence in medical education: focus on the application of ChatGPT for clinical medical education Hyeonmi Hong, Youngjoon Kang, Youngjon Kim, Bomsol Kim Journal of Medicine and Life Science.2023; 20(2): 53. CrossRef
Medical Students’ Perspectives on Artificial Intelligence in Radiology:
The Current Understanding and Impact on Radiology as a Future
Specialty Choice Ali Alamer Current Medical Imaging Formerly Current Medical Imaging Reviews.2023;[Epub] CrossRef
Psychometric properties of the persian version of the Medical Artificial Intelligence Readiness Scale for Medical Students (MAIRS-MS) AmirAli Moodi Ghalibaf, Maryam Moghadasin, Ali Emadzadeh, Haniye Mastour BMC Medical Education.2023;[Epub] CrossRef
Views of Veterinary Faculty Students on the Concept of Artificial Intelligence and Its Use in Veterinary Medicine Practices: An Example of XXXX University Faculty of Veterinary Medicine Nigar YERLİKAYA, Özgül KÜÇÜKASLAN Ankara Üniversitesi Veteriner Fakültesi Dergisi.2023;[Epub] CrossRef
A Pilot Remote Curriculum to Enhance Resident and Medical Student Understanding of Machine Learning in Healthcare Seth M. Meade, Sebastian Salas-Vega, Matthew R. Nagy, Swetha J. Sundar, Michael P. Steinmetz, Edward C. Benzel, Ghaith Habboub World Neurosurgery.2023; 180: e142. CrossRef
Medical Students’ Knowledge and Attitudes about Artificial Intelligence: A Cross-Sectional Survey Amber EKER, Ahmet Asım ÇALIŞKAN, Aysel ZORALİ, Bensu KAYNAK, Mehmet Erhan DERİN Tıp Eğitimi Dünyası.2023; 22(68): 41. CrossRef
El camino a futuro de la pediatría: Nuevas oportunidades con la inteligencia artificial en la atención infantil Wagner Rios-Garcia, Mayli M. Condori-Orosco, Cyntia J. Huasasquiche Investigación e Innovación Clínica y Quirúrgica Pediátrica.2023; 1(2): 71. CrossRef
Percepciones de estudiantes de Medicina sobre el impacto de la inteligencia artificial en radiología G. Caparrós Galán, F. Sendra Portero Radiología.2022; 64(6): 516. CrossRef
Finding the needle by modeling the haystack: Pulmonary embolism in an emergency patient with cardiorespiratory manifestations Davide Luciani, Alessandro Magrini, Carlo Berzuini, Antonello Gavazzi, Paolo Canova, Tiziano Barbui, Guido Bertolini Expert Systems with Applications.2022; 189: 116066. CrossRef
SHIFTing artificial intelligence to be responsible in healthcare: A systematic review Haytham Siala, Yichuan Wang Social Science & Medicine.2022; 296: 114782. CrossRef
AUGMENTING CBME CURRICULUM WITH ARTIFICIAL INTELLIGENCE COURSES – A FUTURISTIC APPROACH. Yogesh Bahurupi, Ashwini A Mahadule, Prashant M Patil, Vartika Saxena INDIAN JOURNAL OF APPLIED RESEARCH.2022; : 46. CrossRef
Artificial Intelligence in Pediatric Pathology: The Extinction of a Medical Profession or the Key to a Bright Future? Ananda van der Kamp, Tomas J. Waterlander, Thomas de Bel, Jeroen van der Laak, Marry M. van den Heuvel-Eibrink, Annelies M. C. Mavinkurve-Groothuis, Ronald R. de Krijger Pediatric and Developmental Pathology.2022; 25(4): 380. CrossRef
Artificial Intelligence Education for the Health Workforce: Expert Survey of Approaches and Needs Kathleen Gray, John Slavotinek, Gerardo Luis Dimaguila, Dawn Choo JMIR Medical Education.2022; 8(2): e35223. CrossRef
Advancements in Oncology with Artificial Intelligence—A Review Article Nikitha Vobugari, Vikranth Raja, Udhav Sethi, Kejal Gandhi, Kishore Raja, Salim R. Surani Cancers.2022; 14(5): 1349. CrossRef
Needs, Challenges, and Applications of Artificial Intelligence in Medical Education Curriculum Joel Grunhut, Oge Marques, Adam T M Wyatt JMIR Medical Education.2022; 8(2): e35587. CrossRef
Promoting Research, Awareness, and Discussion on AI in Medicine Using #MedTwitterAI: A Longitudinal Twitter Hashtag Analysis Faisal A. Nawaz, Austin A. Barr, Monali Y. Desai, Christos Tsagkaris, Romil Singh, Elisabeth Klager, Fabian Eibensteiner, Emil D. Parvanov, Mojca Hribersek, Maria Kletecka-Pulker, Harald Willschke, Atanas G. Atanasov Frontiers in Public Health.2022;[Epub] CrossRef
Communication training for pharmacy students with standard patients using artificial intelligence Naoto Nakagawa, Keita Odanaka, Hiroshi Ohara, Shigeki Kisara Currents in Pharmacy Teaching and Learning.2022; 14(7): 854. CrossRef
Artificial intelligence in healthcare: Should it be included in the medical curriculum? A students’ perspective MANISHI BANSAL, ANKUSH JINDAL The National Medical Journal of India.2022; 35: 56. CrossRef
Undergraduate Medical Students’ and Interns’ Knowledge and Perception of Artificial Intelligence in Medicine Nisha Jha, Pathiyil Ravi Shankar, Mohammed Azmi Al-Betar, Rupesh Mukhia, Kabita Hada, Subish Palaian Advances in Medical Education and Practice.2022; Volume 13: 927. CrossRef
Perceptions of US Medical Students on Artificial Intelligence in Medicine: Mixed Methods Survey Study David Shalom Liu, Jake Sawyer, Alexander Luna, Jihad Aoun, Janet Wang, Lord Boachie, Safwan Halabi, Bina Joe JMIR Medical Education.2022; 8(4): e38325. CrossRef
Artificial intelligence in medical education: a cross-sectional needs assessment M. Murat Civaner, Yeşim Uncu, Filiz Bulut, Esra Giounous Chalil, Abdülhamit Tatli BMC Medical Education.2022;[Epub] CrossRef
Medical students’ perceptions of the impact of artificial intelligence in radiology G. Caparrós Galán, F. Sendra Portero Radiología (English Edition).2022; 64(6): 516. CrossRef
Medical Education 4.0: A Neurology Perspective Zaitoon Zafar, Muhammad Umair, Filzah Faheem, Danish Bhatti , Junaid S Kalia Cureus.2022;[Epub] CrossRef
AI in the hands of imperfect users Kristin M. Kostick-Quenet, Sara Gerke npj Digital Medicine.2022;[Epub] CrossRef
Trust and medical AI: the challenges we face and the expertise needed to overcome them Thomas P Quinn, Manisha Senadeera, Stephan Jacobs, Simon Coghlan, Vuong Le Journal of the American Medical Informatics Association.2021; 28(4): 890. CrossRef
Attitude of Brazilian dentists and dental students regarding the future role of artificial intelligence in oral radiology: a multicenter survey Ruben Pauwels, Yumi Chokyu Del Rey Dentomaxillofacial Radiology.2021; 50(5): 20200461. CrossRef
Key Principles of Clinical Validation, Device Approval, and Insurance Coverage Decisions of Artificial Intelligence Seong Ho Park, Jaesoon Choi, Jeong-Sik Byeon Korean Journal of Radiology.2021; 22(3): 442. CrossRef
Basic of machine learning and deep learning in imaging for medical physicists Luigi Manco, Nicola Maffei, Silvia Strolin, Sara Vichi, Luca Bottazzi, Lidia Strigari Physica Medica.2021; 83: 194. CrossRef
Inteligencia artificial y simulación en urología J. Gómez Rivas, C. Toribio Vázquez, C. Ballesteros Ruiz, M. Taratkin, J.L. Marenco, G.E. Cacciamani, E. Checcucci, Z. Okhunov, D. Enikeev, F. Esperto, R. Grossmann, B. Somani, D. Veneziano Actas Urológicas Españolas.2021; 45(8): 524. CrossRef
Regulating AI in Health Care: The Challenges of Informed User Engagement Olya Kudina Hastings Center Report.2021; 51(5): 6. CrossRef
Are We Ready to Integrate Artificial Intelligence Literacy into Medical School Curriculum: Students and Faculty Survey Elena A Wood, Brittany L Ange, D Douglas Miller Journal of Medical Education and Curricular Development.2021; 8: 238212052110240. CrossRef
A Conference-Friendly, Hands-on Introduction to Deep Learning for Radiology Trainees Walter F. Wiggins, M. Travis Caton, Kirti Magudia, Michael H. Rosenthal, Katherine P. Andriole Journal of Digital Imaging.2021; 34(4): 1026. CrossRef
Artificial intelligence and simulation in urology J. Gómez Rivas, C. Toribio Vázquez, C. Ballesteros Ruiz, M. Taratkin, J.L. Marenco, G.E. Cacciamani, E. Checcucci, Z. Okhunov, D. Enikeev, F. Esperto, R. Grossmann, B. Somani, D. Veneziano Actas Urológicas Españolas (English Edition).2021; 45(8): 524. CrossRef
Accelerating the Appropriate Adoption of Artificial Intelligence in Health Care: Protocol for a Multistepped Approach David Wiljer, Mohammad Salhia, Elham Dolatabadi, Azra Dhalla, Caitlin Gillan, Dalia Al-Mouaswas, Ethan Jackson, Jacqueline Waldorf, Jane Mattson, Megan Clare, Nadim Lalani, Rebecca Charow, Sarmini Balakumar, Sarah Younus, Tharshini Jeyakumar, Wanda Petean JMIR Research Protocols.2021; 10(10): e30940. CrossRef
Artificial Intelligence in Undergraduate Medical Education: A Scoping Review Juehea Lee, Annie Siyu Wu, David Li, Kulamakan (Mahan) Kulasegaram Academic Medicine.2021; 96(11S): S62. CrossRef
Artificial Intelligence Evidence-Based Current Status and Potential for Lower Limb Vascular Management Xenia Butova, Sergey Shayakhmetov, Maxim Fedin, Igor Zolotukhin, Sergio Gianesini Journal of Personalized Medicine.2021; 11(12): 1280. CrossRef
Artificial Intelligence Education Programs for Health Care Professionals: Scoping Review Rebecca Charow, Tharshini Jeyakumar, Sarah Younus, Elham Dolatabadi, Mohammad Salhia, Dalia Al-Mouaswas, Melanie Anderson, Sarmini Balakumar, Megan Clare, Azra Dhalla, Caitlin Gillan, Shabnam Haghzare, Ethan Jackson, Nadim Lalani, Jane Mattson, Wanda Pete JMIR Medical Education.2021; 7(4): e31043. CrossRef
The Journal Citation Indicator has arrived for Emerging Sources Citation Index journals, including the Journal of Educational Evaluation for Health Professions, in June 2021 Sun Huh Journal of Educational Evaluation for Health Professions.2021; 18: 20. CrossRef
Ethical Challenges of Artificial Intelligence in Health Care: A Narrative Review Aaron T. Hui, Shawn S. Ahn, Carolyn T. Lye, Jun Deng Ethics in Biology, Engineering and Medicine: An International Journal.2021; 12(1): 55. CrossRef
Bayesian networks: Making the most of a history Rami Abbass, Usmaan Bhatti, Shad Asinger The Clinical Teacher.2021; 18(2): 140. CrossRef
Fundamentals in Artificial Intelligence for Vascular Surgeons Juliette Raffort, Cédric Adam, Marion Carrier, Fabien Lareyre Annals of Vascular Surgery.2020; 65: 254. CrossRef
Extending capabilities of artificial intelligence for decision-making and healthcare education Mohd Javaid, Abid Haleem, IbrahimHaleem Khan, Raju Vaishya, Abhishek Vaish Apollo Medicine.2020; 17(1): 53. CrossRef
Artificial intelligence with multi-functional machine learning platform development for better healthcare and precision medicine Zeeshan Ahmed, Khalid Mohamed, Saman Zeeshan, XinQi Dong Database.2020;[Epub] CrossRef
Artificial Intelligence Education and Tools for Medical and Health Informatics Students: Systematic Review A Hasan Sapci, H Aylin Sapci JMIR Medical Education.2020; 6(1): e19285. CrossRef
Evaluation of epidemiological lectures using peer instruction: focusing on the importance of ConcepTests Toshiharu Mitsuhashi PeerJ.2020; 8: e9640. CrossRef
Artificial Intelligence in Small Bowel Endoscopy: Current Perspectives and Future Directions Dinesh Meher, Mrinal Gogoi, Pankaj Bharali, Prajna Anirvan, Shivaram Prasad Singh Journal of Digestive Endoscopy.2020; 11(04): 245. CrossRef
Key principles of clinical validation, device approval, and insurance coverage decisions of artificial intelligence Seong Ho Park, Jaesoon Choi, Jeong-Sik Byeon Journal of the Korean Medical Association.2020; 63(11): 696. CrossRef
Artificial intelligence-based education assists medical students’ interpretation of hip fracture Chi-Tung Cheng, Chih-Chi Chen, Chih-Yuan Fu, Chung-Hsien Chaou, Yu-Tung Wu, Chih-Po Hsu, Chih-Chen Chang, I-Fang Chung, Chi-Hsun Hsieh, Ming-Ju Hsieh, Chien-Hung Liao Insights into Imaging.2020;[Epub] CrossRef
Current Status and Future Direction of Artificial Intelligence in Healthcare and Medical Education Jin Sup Jung Korean Medical Education Review.2020; 22(2): 99. CrossRef
Introducing Artificial Intelligence Training in Medical Education Ketan Paranjape, Michiel Schinkel, Rishi Nannan Panday, Josip Car, Prabath Nanayakkara JMIR Medical Education.2019; 5(2): e16048. CrossRef