Graphical abstract

Background

O’Connor and ChatGPT [1] wrote an editorial, the opening paragraphs of which were written by ChatGPT, an artificial intelligence (AI) chatbot. ChatGTP was trained by a model using reinforcement learning from human feedback, using the same methods as InstructGPT (GPT: generative pre-trained transformer) [2]. AI chatbots such as ChatGPT could provide tutoring and homework help by answering questions and providing explanations to help students understand complex concepts. However, there are concerns that the use of AI software by students to write university assessments could diminish the value of the assessments and the overall quality of the university program [1]. After the release of ChatGPT to the public on November 30, 2022, it became a hot topic, particularly in education. Stokel-Walker [3] also noted that ChatGPT, an AI-powered chatbot that generates intelligent-sounding text in response to user prompts, including homework assignments and exam-style questions, has caused concern. Medical students must be able to evaluate the accuracy of medical information generated by AI and have the abilities to create reliable, validated information for patients and the public [4]. Therefore, it is necessary to determine how accurately ChatGPT, a recently developed AI chatbot, can solve questions on medical examinations. This comparison of ChatGPT’s abilities may provide insights into whether—and if so, how—medical students could use ChatGPT for their learning.

Objectives

This study aimed to compare the knowledge and interpretation ability of ChatGPT with those of medical students in Korea by administering a parasitology examination. This subject is required in medical schools in Korea. Specifically, the following were investigated: (1) the scores of ChatGPT compared to those of the medical students; (2) the correct answer rate of ChatGPT according to items’ knowledge level; and (3) the acceptability of ChatGPT’s explanations as reflecting current parasitology knowledge, as evaluated by the author.

Ethics statement

This was not a study of human subjects, but an analysis of the results of an educational examination routinely conducted at medical colleges. Therefore, neither receiving approval from the institutional review board nor obtaining informed consent was required.

Study design

This is a descriptive study to compare the ability of ChatGPT to answer questions with that of medical students.

Setting

On January 1, 2023 (Seoul time), a parasitology examination with identical items to those administered to first-year medical students at Hallym University on December 12, 2022, using computer-based testing (Supplement 1), was administered to ChatGPT (version December 15, 2022). The answers given by ChatGPT were compared to those of the medical students. Parasitology classes for medical students began on October 31, 2022, and ended on December 8, 2022. There were 16 hours of lectures and 32 hours of laboratory practice.

Participants

Seventy-seven medical students took the parasitology on December 12, 2022. ChatGPT was counted as one examinee. There were no exclusion criteria.

Variables

The items’ knowledge level and the examinees’ scores were the variables.

Data sources and measurement

The response data of 77 medical students on the parasitology examination and ChatGPT were compared. The correct answer rate according to items’ level of knowledge was analyzed. The author also evaluated the acceptability of the explanations provided by ChatGPT (Supplement 2, Fig. 1), and classified the acceptability as good, needing revision, and unacceptable.

Bias

There was no bias in the selection of examinees. All students who attended the parasitology lecture course were included.

Study size

Sample size estimation was not required because all target students were included, and one AI platform was added.

Statistical methods

Descriptive statistics were used to analyze the chatbot’s score. A comparative analysis was conducted using DBSTAT version 5.0 (DBSTAT).

Score of ChatGPT and comparison with the medical students’ performance

According to data from Dataset 1, ChatGPT correctly answered 48 out of 79 items (60.8%). This score was lower than the average score of 77 medical students, which was 71.8 out of 79 (90.8%), with a minimum score of 65 (89.0%) and a maximum score of 74 (93.7%).

Comparison of ChatGPT’s correct answer rate according to items’ knowledge level

Table 1 shows ChatGPT’s responses according to items’ knowledge level. The chi-square test yielded results of χ²=3.02, degrees of freedom (df)=2, with a significance level of 0.05 (χ²=5.99). This result indicates that the relationship between the 2 variables was not significant (P=0.2206).

Acceptability of ChatGPT’s explanations

Table 2 shows the acceptability of ChatGPT’s explanations according to the correctness of the answer. The chi-square test showed results of χ²=51.62, df=2, with a significance level of 0.05 (χ²=5.99). This result indicates that the relationship between the 2 variables was significant (P=0.0000).

Key results

ChatGPT’s performance was lower than that of medical students. The correct answer rate shown by ChatGPT was not related to the items’ knowledge level. However, there was an association between acceptable explanations and correct answers.

Interpretation

ChatGPT’s correct answer rate of 60.8% was not necessarily an indicator of poor performance, as the questions were not easy for medical students to answer correctly. The considerably higher average score (89.6%) of the medical students may have been due to their prior learning of parasitology and the fact that the examination was administered 4 days after the class. If the examination had been taken 1 or 2 months after the class, the students’ performance scores might have been lower. Some incorrect answers may have been due to the following factors: first, ChatGPT is currently unable to interpret figures, graphs, and tables as a student can, so the author had to describe these materials in text form. Second, some epidemiological data unique to Korea were outside ChatGPT’s knowledge. Some of those data are only available in Korean or are not searchable online. Third, ChatGPT sometimes did not understand multiple-choice questions where the examinee must select the best answer out of multiple options. ChatGPT sometimes selected 2 or more options, as it has not yet been trained to do otherwise.

There was no significant difference in the correct answer rate according to the knowledge level of the items. However, this may vary in other examinations and may have been a unique phenomenon for this parasitology exam. ChatGPT’s explanations of the question items were generally acceptable if it made a correct selection. However, the explanations for 7 items needed to be updated or revised because they contained incorrect information. This finding suggests that ChatGPT’s knowledge in specific fields (e.g., parasitology) remains insufficient. If the incorrect option was selected, the explanation was unacceptable or needed revision in 90.0% of items. This result was anticipated, as students’ explanations for incorrect selections are also usually unacceptable. Sometimes, GPT could not select best answer but, the explanation is acceptable. Example is the item number 39.

Comparison with previous studies

There have been no reported studies in the literature databases, including PubMed, Scopus, and Web of Science, on the comparability of ChatGPT’s performance to that of students on medical examinations.

Limitations

The input for the question items for ChatGPT was not precisely the same as for the medical students. The chatbot cannot receive information in graphs, figures, and tables, so this information was re-described by the author. Additionally, the interpretation of the explanations and correct answers may vary according to the perspectives of different parasitologists, although the author has worked in the field of parasitology for 40 years (1982–2022) in Korea. Best practices for patient care may also vary according to the region and medical environment.

Generalizability

The above results cannot be generalized directly to other subjects or medical schools, as chatbots will likely continue to evolve rapidly through user feedback. A future trial with the same items may yield different results. The present results reflect the abilities of ChatGPT on January 1, 2023.

Implications for medical/health students and professors to use ChatGPT

Currently, ChatGPT’s level of knowledge and interpretation is not sufficient to be used by medical students, especially in medical school exams. This may also be the case for high-stakes exams, including health licensing exams. However, I believe that ChatGPT’s knowledge and interpretation abilities will improve rapidly through deep learning, similar to AlphaGo’s ability [5]. Therefore, medical/health professors and students should be mindful of how to incorporate this AI platform into medical/health education soon. Furthermore, AI should be integrated into the medical school curriculum, and some schools have already adopted it [6].

Conclusion

ChatGPT’s knowledge and interpretation ability in answering this parasitology examination are not yet comparable to those of medical students in Korea. However, these abilities will likely improve through deep learning. Medical/health professors and students should be aware of the progress of this AI chatbot and consider its potential adoption in learning and education.

Acknowledgments

Supplementary materials

Fig. 1.

Screenshot of ChatGPT’s answer to a question item from a parasitology examination for medical students at Hallym University.

References

• 1. O’Connor S. Open artificial intelligence platforms in nursing education: tools for academic progress or abuse? Nurse Educ Pract 2022;66:103537. https://doi.org/10.1016/j.nepr.2022.103537 ArticlePubMed
• 2. OpenAI. ChatGPT Dec 15 version [Internet]. 2022 [cited 2023 Jan 1]. Available from: https://chat.openai.com/chat
• 3. Stokel-Walker C. AI bot ChatGPT writes smart essays: should professors worry? Nature 2022 Dec 9 [Epub]. https://doi.org/10.1038/d41586-022-04397-7 Article
• 4. Park SH, Do KH, Kim S, Park JH, Lim YS. What should medical students know about artificial intelligence in medicine? J Educ Eval Health Prof 2019;16:18. https://doi.org/10.3352/jeehp.2019.16.18 ArticlePubMedPMC
• 5. Silver D, Huang A, Maddison CJ, Guez A, Sifre L, van den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M, Dieleman S, Grewe D, Nham J, Kalchbrenner N, Sutskever I, Lillicrap T, Leach M, Kavukcuoglu K, Graepel T, Hassabis D. Mastering the game of Go with deep neural networks and tree search. Nature 2016;529:484-489. https://doi.org/10.1038/nature16961 ArticlePubMed
• 6. Hu R, Fan KY, Pandey P, Hu Z, Yau O, Teng M, Wang P, Li A, Ashraf M, Singla R. Insights from teaching artificial intelligence to medical students in Canada. Commun Med (Lond) 2022;2:63. https://doi.org/10.1038/s43856-022-00125-4 ArticlePubMedPMC

Figure & Data

References

Citations

Citations to this article as recorded by  

• Performance of ChatGPT on the India Undergraduate Community Medicine Examination: Cross-Sectional Study
  Aravind P Gandhi, Felista Karen Joesph, Vineeth Rajagopal, P Aparnavi, Sushma Katkuri, Sonal Dayama, Prakasini Satapathy, Mahalaqua Nazli Khatib, Shilpa Gaidhane, Quazi Syed Zahiruddin, Ashish Behera
  JMIR Formative Research.2024; 8: e49964.     CrossRef
• Large Language Models and Artificial Intelligence: A Primer for Plastic Surgeons on the Demonstrated and Potential Applications, Promises, and Limitations of ChatGPT
  Jad Abi-Rafeh, Hong Hao Xu, Roy Kazan, Ruth Tevlin, Heather Furnas
  Aesthetic Surgery Journal.2024; 44(3): 329.     CrossRef
• Redesigning Tertiary Educational Evaluation with AI: A Task-Based Analysis of LIS Students’ Assessment on Written Tests and Utilizing ChatGPT at NSTU
  Shamima Yesmin
  Science & Technology Libraries.2024; 43(4): 355.     CrossRef
• Unveiling the ChatGPT phenomenon: Evaluating the consistency and accuracy of endodontic question answers
  Ana Suárez, Víctor Díaz‐Flores García, Juan Algar, Margarita Gómez Sánchez, María Llorente de Pedro, Yolanda Freire
  International Endodontic Journal.2024; 57(1): 108.     CrossRef
• Bob or Bot: Exploring ChatGPT's Answers to University Computer Science Assessment
  Mike Richards, Kevin Waugh, Mark Slaymaker, Marian Petre, John Woodthorpe, Daniel Gooch
  ACM Transactions on Computing Education.2024; 24(1): 1.     CrossRef
• A systematic review of ChatGPT use in K‐12 education
  Peng Zhang, Gemma Tur
  European Journal of Education.2024;[Epub]     CrossRef
• Evaluating ChatGPT as a self‐learning tool in medical biochemistry: A performance assessment in undergraduate medical university examination
  Krishna Mohan Surapaneni, Anusha Rajajagadeesan, Lakshmi Goudhaman, Shalini Lakshmanan, Saranya Sundaramoorthi, Dineshkumar Ravi, Kalaiselvi Rajendiran, Porchelvan Swaminathan
  Biochemistry and Molecular Biology Education.2024; 52(2): 237.     CrossRef
• Examining the use of ChatGPT in public universities in Hong Kong: a case study of restricted access areas
  Michelle W. T. Cheng, Iris H. Y. YIM
  Discover Education.2024;[Epub]     CrossRef
• Performance of ChatGPT on Ophthalmology-Related Questions Across Various Examination Levels: Observational Study
  Firas Haddad, Joanna S Saade
  JMIR Medical Education.2024; 10: e50842.     CrossRef
• Assessment of Artificial Intelligence Platforms With Regard to Medical Microbiology Knowledge: An Analysis of ChatGPT and Gemini
  Jai Ranjan, Absar Ahmad, Monalisa Subudhi, Ajay Kumar
  Cureus.2024;[Epub]     CrossRef
• A comparative vignette study: Evaluating the potential role of a generative AI model in enhancing clinical decision‐making in nursing
  Mor Saban, Ilana Dubovi
  Journal of Advanced Nursing.2024;[Epub]     CrossRef
• Comparison of the Performance of GPT-3.5 and GPT-4 With That of Medical Students on the Written German Medical Licensing Examination: Observational Study
  Annika Meyer, Janik Riese, Thomas Streichert
  JMIR Medical Education.2024; 10: e50965.     CrossRef
• From hype to insight: Exploring ChatGPT's early footprint in education via altmetrics and bibliometrics
  Lung‐Hsiang Wong, Hyejin Park, Chee‐Kit Looi
  Journal of Computer Assisted Learning.2024; 40(4): 1428.     CrossRef
• A scoping review of artificial intelligence in medical education: BEME Guide No. 84
  Morris Gordon, Michelle Daniel, Aderonke Ajiboye, Hussein Uraiby, Nicole Y. Xu, Rangana Bartlett, Janice Hanson, Mary Haas, Maxwell Spadafore, Ciaran Grafton-Clarke, Rayhan Yousef Gasiea, Colin Michie, Janet Corral, Brian Kwan, Diana Dolmans, Satid Thamma
  Medical Teacher.2024; 46(4): 446.     CrossRef
• Üniversite Öğrencilerinin ChatGPT 3,5 Deneyimleri: Yapay Zekâyla Yazılmış Masal Varyantları
  Bilge GÖK, Fahri TEMİZYÜREK, Özlem BAŞ
  Korkut Ata Türkiyat Araştırmaları Dergisi.2024; (14): 1040.     CrossRef
• Tracking ChatGPT Research: Insights From the Literature and the Web
  Omar Mubin, Fady Alnajjar, Zouheir Trabelsi, Luqman Ali, Medha Mohan Ambali Parambil, Zhao Zou
  IEEE Access.2024; 12: 30518.     CrossRef
• Potential applications of ChatGPT in obstetrics and gynecology in Korea: a review article
  YooKyung Lee, So Yun Kim
  Obstetrics & Gynecology Science.2024; 67(2): 153.     CrossRef
• Application of generative language models to orthopaedic practice
  Jessica Caterson, Olivia Ambler, Nicholas Cereceda-Monteoliva, Matthew Horner, Andrew Jones, Arwel Tomos Poacher
  BMJ Open.2024; 14(3): e076484.     CrossRef
• Opportunities, challenges, and future directions of large language models, including ChatGPT in medical education: a systematic scoping review
  Xiaojun Xu, Yixiao Chen, Jing Miao
  Journal of Educational Evaluation for Health Professions.2024; 21: 6.     CrossRef
• The advent of ChatGPT: Job Made Easy or Job Loss to Data Analysts
  Abiola Timothy Owolabi, Oluwaseyi Oluwadamilare Okunlola, Emmanuel Taiwo Adewuyi, Janet Iyabo Idowu, Olasunkanmi James Oladapo
  WSEAS TRANSACTIONS ON COMPUTERS.2024; 23: 24.     CrossRef
• ChatGPT in dentomaxillofacial radiology education
  Hilal Peker Öztürk, Hakan Avsever, Buğra Şenel, Şükran Ayran, Mustafa Çağrı Peker, Hatice Seda Özgedik, Nurten Baysal
  Journal of Health Sciences and Medicine.2024; 7(2): 224.     CrossRef
• Performance of ChatGPT on the Korean National Examination for Dental Hygienists
  Soo-Myoung Bae, Hye-Rim Jeon, Gyoung-Nam Kim, Seon-Hui Kwak, Hyo-Jin Lee
  Journal of Dental Hygiene Science.2024; 24(1): 62.     CrossRef
• Medical knowledge of ChatGPT in public health, infectious diseases, COVID-19 pandemic, and vaccines: multiple choice questions examination based performance
  Sultan Ayoub Meo, Metib Alotaibi, Muhammad Zain Sultan Meo, Muhammad Omair Sultan Meo, Mashhood Hamid
  Frontiers in Public Health.2024;[Epub]     CrossRef
• Unlock the potential for Saudi Arabian higher education: a systematic review of the benefits of ChatGPT
  Eman Faisal
  Frontiers in Education.2024;[Epub]     CrossRef
• Does the Information Quality of ChatGPT Meet the Requirements of Orthopedics and Trauma Surgery?
  Adnan Kasapovic, Thaer Ali, Mari Babasiz, Jessica Bojko, Martin Gathen, Robert Kaczmarczyk, Jonas Roos
  Cureus.2024;[Epub]     CrossRef
• Exploring the Profile of University Assessments Flagged as Containing AI-Generated Material
  Daniel Gooch, Kevin Waugh, Mike Richards, Mark Slaymaker, John Woodthorpe
  ACM Inroads.2024; 15(2): 39.     CrossRef
• Comparing the Performance of ChatGPT-4 and Medical Students on MCQs at Varied Levels of Bloom’s Taxonomy
  Ambadasu Bharatha, Nkemcho Ojeh, Ahbab Mohammad Fazle Rabbi, Michael Campbell, Kandamaran Krishnamurthy, Rhaheem Layne-Yarde, Alok Kumar, Dale Springer, Kenneth Connell, Md Anwarul Majumder
  Advances in Medical Education and Practice.2024; Volume 15: 393.     CrossRef
• The emergence of generative artificial intelligence platforms in 2023, journal metrics, appreciation to reviewers and volunteers, and obituary
  Sun Huh
  Journal of Educational Evaluation for Health Professions.2024; 21: 9.     CrossRef
• ChatGPT, a Friend or a Foe in Medical Education: A Review of Strengths, Challenges, and Opportunities
  Mahdi Zarei, Maryam Zarei, Sina Hamzehzadeh, Sepehr Shakeri Bavil Oliyaei, Mohammad-Salar Hosseini
  Shiraz E-Medical Journal.2024;[Epub]     CrossRef
• Augmenting intensive care unit nursing practice with generative AI: A formative study of diagnostic synergies using simulation‐based clinical cases
  Chedva Levin, Moriya Suliman, Etti Naimi, Mor Saban
  Journal of Clinical Nursing.2024;[Epub]     CrossRef
• Artificial intelligence chatbots for the nutrition management of diabetes and the metabolic syndrome
  Farah Naja, Mandy Taktouk, Dana Matbouli, Sharfa Khaleel, Ayah Maher, Berna Uzun, Maryam Alameddine, Lara Nasreddine
  European Journal of Clinical Nutrition.2024; 78(10): 887.     CrossRef
• Large language models in healthcare: from a systematic review on medical examinations to a comparative analysis on fundamentals of robotic surgery online test
  Andrea Moglia, Konstantinos Georgiou, Pietro Cerveri, Luca Mainardi, Richard M. Satava, Alfred Cuschieri
  Artificial Intelligence Review.2024;[Epub]     CrossRef
• Is ChatGPT Enhancing Youth’s Learning, Engagement and Satisfaction?
  Christina Sanchita Shah, Smriti Mathur, Sushant Kr. Vishnoi
  Journal of Computer Information Systems.2024; : 1.     CrossRef
• Comparison of ChatGPT, Gemini, and Le Chat with physician interpretations of medical laboratory questions from an online health forum
  Annika Meyer, Ari Soleman, Janik Riese, Thomas Streichert
  Clinical Chemistry and Laboratory Medicine (CCLM).2024;[Epub]     CrossRef
• Performance of ChatGPT-3.5 and GPT-4 in national licensing examinations for medicine, pharmacy, dentistry, and nursing: a systematic review and meta-analysis
  Hye Kyung Jin, Ha Eun Lee, EunYoung Kim
  BMC Medical Education.2024;[Epub]     CrossRef
• Role of ChatGPT in Dentistry: A Review
  Pratik Surana, Priyanka P. Ostwal, Shruti Vishal Dev, Jayesh Tiwari, Kadire Shiva Charan Yadav, Gajji Renuka
  Research Journal of Pharmacy and Technology.2024; : 3489.     CrossRef
• Exploring the Current Applications and Effectiveness of ChatGPT in Nursing: An Integrative Review
  Yuan Luo, Yiqun Miao, Yuhan Zhao, Jiawei Li, Ying Wu
  Journal of Advanced Nursing.2024;[Epub]     CrossRef
• A Scoping Review on the Educational Applications of Generative AI in Primary and Secondary Education
  Solmoe Ahn, Jeongyoon Lee, Jungmin Park, Soyoung Jung, Jihoon Song
  The Journal of Korean Association of Computer Education.2024; 27(6): 11.     CrossRef
• Applicability of ChatGPT in Assisting to Solve Higher Order Problems in Pathology
  Ranwir K Sinha, Asitava Deb Roy, Nikhil Kumar, Himel Mondal
  Cureus.2023;[Epub]     CrossRef
• Issues in the 3rd year of the COVID-19 pandemic, including computer-based testing, study design, ChatGPT, journal metrics, and appreciation to reviewers
  Sun Huh
  Journal of Educational Evaluation for Health Professions.2023; 20: 5.     CrossRef
• Emergence of the metaverse and ChatGPT in journal publishing after the COVID-19 pandemic
  Sun Huh
  Science Editing.2023; 10(1): 1.     CrossRef
• Assessing the Capability of ChatGPT in Answering First- and Second-Order Knowledge Questions on Microbiology as per Competency-Based Medical Education Curriculum
  Dipmala Das, Nikhil Kumar, Langamba Angom Longjam, Ranwir Sinha, Asitava Deb Roy, Himel Mondal, Pratima Gupta
  Cureus.2023;[Epub]     CrossRef
• Evaluating ChatGPT's Ability to Solve Higher-Order Questions on the Competency-Based Medical Education Curriculum in Medical Biochemistry
  Arindam Ghosh, Aritri Bir
  Cureus.2023;[Epub]     CrossRef
• Overview of Early ChatGPT’s Presence in Medical Literature: Insights From a Hybrid Literature Review by ChatGPT and Human Experts
  Omar Temsah, Samina A Khan, Yazan Chaiah, Abdulrahman Senjab, Khalid Alhasan, Amr Jamal, Fadi Aljamaan, Khalid H Malki, Rabih Halwani, Jaffar A Al-Tawfiq, Mohamad-Hani Temsah, Ayman Al-Eyadhy
  Cureus.2023;[Epub]     CrossRef
• ChatGPT for Future Medical and Dental Research
  Bader Fatani
  Cureus.2023;[Epub]     CrossRef
• ChatGPT in Dentistry: A Comprehensive Review
  Hind M Alhaidry, Bader Fatani, Jenan O Alrayes, Aljowhara M Almana, Nawaf K Alfhaed
  Cureus.2023;[Epub]     CrossRef
• Can we trust AI chatbots’ answers about disease diagnosis and patient care?
  Sun Huh
  Journal of the Korean Medical Association.2023; 66(4): 218.     CrossRef
• Large Language Models in Medical Education: Opportunities, Challenges, and Future Directions
  Alaa Abd-alrazaq, Rawan AlSaad, Dari Alhuwail, Arfan Ahmed, Padraig Mark Healy, Syed Latifi, Sarah Aziz, Rafat Damseh, Sadam Alabed Alrazak, Javaid Sheikh
  JMIR Medical Education.2023; 9: e48291.     CrossRef
• Early applications of ChatGPT in medical practice, education and research
  Sam Sedaghat
  Clinical Medicine.2023; 23(3): 278.     CrossRef
• A Review of Research on Teaching and Learning Transformation under the Influence of ChatGPT Technology
  璇 师
  Advances in Education.2023; 13(05): 2617.     CrossRef
• Performance of GPT-3.5 and GPT-4 on the Japanese Medical Licensing Examination: Comparison Study
  Soshi Takagi, Takashi Watari, Ayano Erabi, Kota Sakaguchi
  JMIR Medical Education.2023; 9: e48002.     CrossRef
• ChatGPT’s quiz skills in different otolaryngology subspecialties: an analysis of 2576 single-choice and multiple-choice board certification preparation questions
  Cosima C. Hoch, Barbara Wollenberg, Jan-Christoffer Lüers, Samuel Knoedler, Leonard Knoedler, Konstantin Frank, Sebastian Cotofana, Michael Alfertshofer
  European Archives of Oto-Rhino-Laryngology.2023; 280(9): 4271.     CrossRef
• Analysing the Applicability of ChatGPT, Bard, and Bing to Generate Reasoning-Based Multiple-Choice Questions in Medical Physiology
  Mayank Agarwal, Priyanka Sharma, Ayan Goswami
  Cureus.2023;[Epub]     CrossRef
• The Intersection of ChatGPT, Clinical Medicine, and Medical Education
  Rebecca Shin-Yee Wong, Long Chiau Ming, Raja Affendi Raja Ali
  JMIR Medical Education.2023; 9: e47274.     CrossRef
• The Role of Artificial Intelligence in Higher Education: ChatGPT Assessment for Anatomy Course
  Tarık TALAN, Yusuf KALINKARA
  Uluslararası Yönetim Bilişim Sistemleri ve Bilgisayar Bilimleri Dergisi.2023; 7(1): 33.     CrossRef
• Comparing ChatGPT’s ability to rate the degree of stereotypes and the consistency of stereotype attribution with those of medical students in New Zealand in developing a similarity rating test: a methodological study
  Chao-Cheng Lin, Zaine Akuhata-Huntington, Che-Wei Hsu
  Journal of Educational Evaluation for Health Professions.2023; 20: 17.     CrossRef
• Examining Real-World Medication Consultations and Drug-Herb Interactions: ChatGPT Performance Evaluation
  Hsing-Yu Hsu, Kai-Cheng Hsu, Shih-Yen Hou, Ching-Lung Wu, Yow-Wen Hsieh, Yih-Dih Cheng
  JMIR Medical Education.2023; 9: e48433.     CrossRef
• Assessing the Efficacy of ChatGPT in Solving Questions Based on the Core Concepts in Physiology
  Arijita Banerjee, Aquil Ahmad, Payal Bhalla, Kavita Goyal
  Cureus.2023;[Epub]     CrossRef
• ChatGPT Performs on the Chinese National Medical Licensing Examination
  Xinyi Wang, Zhenye Gong, Guoxin Wang, Jingdan Jia, Ying Xu, Jialu Zhao, Qingye Fan, Shaun Wu, Weiguo Hu, Xiaoyang Li
  Journal of Medical Systems.2023;[Epub]     CrossRef
• Artificial intelligence and its impact on job opportunities among university students in North Lima, 2023
  Doris Ruiz-Talavera, Jaime Enrique De la Cruz-Aguero, Nereo García-Palomino, Renzo Calderón-Espinoza, William Joel Marín-Rodriguez
  ICST Transactions on Scalable Information Systems.2023;[Epub]     CrossRef
• Revolutionizing Dental Care: A Comprehensive Review of Artificial Intelligence Applications Among Various Dental Specialties
  Najd Alzaid, Omar Ghulam, Modhi Albani, Rafa Alharbi, Mayan Othman, Hasan Taher, Saleem Albaradie, Suhael Ahmed
  Cureus.2023;[Epub]     CrossRef
• Opportunities, Challenges, and Future Directions of Generative Artificial Intelligence in Medical Education: Scoping Review
  Carl Preiksaitis, Christian Rose
  JMIR Medical Education.2023; 9: e48785.     CrossRef
• Exploring the impact of language models, such as ChatGPT, on student learning and assessment
  Araz Zirar
  Review of Education.2023;[Epub]     CrossRef
• Evaluating the reliability of ChatGPT as a tool for imaging test referral: a comparative study with a clinical decision support system
  Shani Rosen, Mor Saban
  European Radiology.2023; 34(5): 2826.     CrossRef
• ChatGPT and the AI revolution: a comprehensive investigation of its multidimensional impact and potential
  Mohd Afjal
  Library Hi Tech.2023;[Epub]     CrossRef
• The Significance of Artificial Intelligence Platforms in Anatomy Education: An Experience With ChatGPT and Google Bard
  Hasan B Ilgaz, Zehra Çelik
  Cureus.2023;[Epub]     CrossRef
• Is ChatGPT’s Knowledge and Interpretative Ability Comparable to First Professional MBBS (Bachelor of Medicine, Bachelor of Surgery) Students of India in Taking a Medical Biochemistry Examination?
  Abhra Ghosh, Nandita Maini Jindal, Vikram K Gupta, Ekta Bansal, Navjot Kaur Bajwa, Abhishek Sett
  Cureus.2023;[Epub]     CrossRef
• Ethical consideration of the use of generative artificial intelligence, including ChatGPT in writing a nursing article
  Sun Huh
  Child Health Nursing Research.2023; 29(4): 249.     CrossRef
• Potential Use of ChatGPT for Patient Information in Periodontology: A Descriptive Pilot Study
  Osman Babayiğit, Zeynep Tastan Eroglu, Dilek Ozkan Sen, Fatma Ucan Yarkac
  Cureus.2023;[Epub]     CrossRef
• Efficacy and limitations of ChatGPT as a biostatistical problem-solving tool in medical education in Serbia: a descriptive study
  Aleksandra Ignjatović, Lazar Stevanović
  Journal of Educational Evaluation for Health Professions.2023; 20: 28.     CrossRef
• Assessing the Performance of ChatGPT in Medical Biochemistry Using Clinical Case Vignettes: Observational Study
  Krishna Mohan Surapaneni
  JMIR Medical Education.2023; 9: e47191.     CrossRef
• Performance of ChatGPT, Bard, Claude, and Bing on the Peruvian National Licensing Medical Examination: a cross-sectional study
  Betzy Clariza Torres-Zegarra, Wagner Rios-Garcia, Alvaro Micael Ñaña-Cordova, Karen Fatima Arteaga-Cisneros, Xiomara Cristina Benavente Chalco, Marina Atena Bustamante Ordoñez, Carlos Jesus Gutierrez Rios, Carlos Alberto Ramos Godoy, Kristell Luisa Teresa
  Journal of Educational Evaluation for Health Professions.2023; 20: 30.     CrossRef
• ChatGPT’s performance in German OB/GYN exams – paving the way for AI-enhanced medical education and clinical practice
  Maximilian Riedel, Katharina Kaefinger, Antonia Stuehrenberg, Viktoria Ritter, Niklas Amann, Anna Graf, Florian Recker, Evelyn Klein, Marion Kiechle, Fabian Riedel, Bastian Meyer
  Frontiers in Medicine.2023;[Epub]     CrossRef
• Medical students’ patterns of using ChatGPT as a feedback tool and perceptions of ChatGPT in a Leadership and Communication course in Korea: a cross-sectional study
  Janghee Park
  Journal of Educational Evaluation for Health Professions.2023; 20: 29.     CrossRef
• FROM TEXT TO DIAGNOSE: CHATGPT’S EFFICACY IN MEDICAL DECISION-MAKING
  Yaroslav Mykhalko, Pavlo Kish, Yelyzaveta Rubtsova, Oleksandr Kutsyn, Valentyna Koval
  Wiadomości Lekarskie.2023; 76(11): 2345.     CrossRef
• Using ChatGPT for Clinical Practice and Medical Education: Cross-Sectional Survey of Medical Students’ and Physicians’ Perceptions
  Pasin Tangadulrat, Supinya Sono, Boonsin Tangtrakulwanich
  JMIR Medical Education.2023; 9: e50658.     CrossRef
• Below average ChatGPT performance in medical microbiology exam compared to university students
  Malik Sallam, Khaled Al-Salahat
  Frontiers in Education.2023;[Epub]     CrossRef
• ChatGPT: "To be or not to be" ... in academic research. The human mind's analytical rigor and capacity to discriminate between AI bots' truths and hallucinations
  Aurelian Anghelescu, Ilinca Ciobanu, Constantin Munteanu, Lucia Ana Maria Anghelescu, Gelu Onose
  Balneo and PRM Research Journal.2023; 14(Vol.14, no): 614.     CrossRef
• ChatGPT Review: A Sophisticated Chatbot Models in Medical & Health-related Teaching and Learning
  Nur Izah Ab Razak, Muhammad Fawwaz Muhammad Yusoff, Rahmita Wirza O.K. Rahmat
  Malaysian Journal of Medicine and Health Sciences.2023; 19(s12): 98.     CrossRef
• Application of artificial intelligence chatbots, including ChatGPT, in education, scholarly work, programming, and content generation and its prospects: a narrative review
  Tae Won Kim
  Journal of Educational Evaluation for Health Professions.2023; 20: 38.     CrossRef
• Trends in research on ChatGPT and adoption-related issues discussed in articles: a narrative review
  Sang-Jun Kim
  Science Editing.2023; 11(1): 3.     CrossRef
• Information amount, accuracy, and relevance of generative artificial intelligence platforms’ answers regarding learning objectives of medical arthropodology evaluated in English and Korean queries in December 2023: a descriptive study
  Hyunju Lee, Soobin Park
  Journal of Educational Evaluation for Health Professions.2023; 20: 39.     CrossRef
• What will ChatGPT revolutionize in the financial industry?
  Hassnian Ali, Ahmet Faruk Aysan
  Modern Finance.2023; 1(1): 116.     CrossRef