-
ChatGPT (GPT-4) passed the Japanese National License Examination for Pharmacists in 2022, answering all items including those with diagrams: a descriptive study
-
Hiroyasu Sato
, Katsuhiko Ogasawara
-
J Educ Eval Health Prof. 2024;21:4. Published online February 28, 2024
-
DOI: https://doi.org/10.3352/jeehp.2024.21.4
-
-
2,595
View
-
284
Download
-
4
Web of Science
-
7
Crossref
-
Abstract
PDF Supplementary Material
- Purpose
The objective of this study was to assess the performance of ChatGPT (GPT-4) on all items, including those with diagrams, in the Japanese National License Examination for Pharmacists (JNLEP) and compare it with the previous GPT-3.5 model’s performance.
Methods The 107th JNLEP, conducted in 2022, with 344 items input into the GPT-4 model, was targeted for this study. Separately, 284 items, excluding those with diagrams, were entered into the GPT-3.5 model. The answers were categorized and analyzed to determine accuracy rates based on categories, subjects, and presence or absence of diagrams. The accuracy rates were compared to the main passing criteria (overall accuracy rate ≥62.9%).
Results The overall accuracy rate for all items in the 107th JNLEP in GPT-4 was 72.5%, successfully meeting all the passing criteria. For the set of items without diagrams, the accuracy rate was 80.0%, which was significantly higher than that of the GPT-3.5 model (43.5%). The GPT-4 model demonstrated an accuracy rate of 36.1% for items that included diagrams.
Conclusion Advancements that allow GPT-4 to process images have made it possible for LLMs to answer all items in medical-related license examinations. This study’s findings confirm that ChatGPT (GPT-4) possesses sufficient knowledge to meet the passing criteria.
-
Citations
Citations to this article as recorded by 
- Qwen-2.5 Outperforms Other Large Language Models in the Chinese National Nursing Licensing Examination: Retrospective Cross-Sectional Comparative Study
Shiben Zhu, Wanqin Hu, Zhi Yang, Jiani Yan, Fang Zhang JMIR Medical Informatics.2025; 13: e63731. CrossRef - ChatGPT (GPT-4V) Performance on the Healthcare Information Technologist Examination in Japan
Kai Ishida, Eisuke Hanada Cureus.2025;[Epub] CrossRef - Potential of ChatGPT to Pass the Japanese Medical and Healthcare Professional National Licenses: A Literature Review
Kai Ishida, Eisuke Hanada Cureus.2024;[Epub] CrossRef - Performance of Generative Pre-trained Transformer (GPT)-4 and Gemini Advanced on the First-Class Radiation Protection Supervisor Examination in Japan
Hiroki Goto, Yoshioki Shiraishi, Seiji Okada Cureus.2024;[Epub] CrossRef - Performance of ChatGPT‐3.5 and ChatGPT‐4o in the Japanese National Dental Examination
Osamu Uehara, Tetsuro Morikawa, Fumiya Harada, Nodoka Sugiyama, Yuko Matsuki, Daichi Hiraki, Hinako Sakurai, Takashi Kado, Koki Yoshida, Yukie Murata, Hirofumi Matsuoka, Toshiyuki Nagasawa, Yasushi Furuichi, Yoshihiro Abiko, Hiroko Miura Journal of Dental Education.2024;[Epub] CrossRef - An exploratory assessment of GPT-4o and GPT-4 performance on the Japanese National Dental Examination
Masaki Morishita, Hikaru Fukuda, Shino Yamaguchi, Kosuke Muraoka, Taiji Nakamura, Masanari Hayashi, Izumi Yoshioka, Kentaro Ono, Shuji Awano The Saudi Dental Journal.2024; 36(12): 1577. CrossRef - Evaluating the Accuracy of ChatGPT in the Japanese Board-Certified Physiatrist Examination
Yuki Kato, Kenta Ushida, Ryo Momosaki Cureus.2024;[Epub] CrossRef
|