Skip Navigation
Skip to contents

JEEHP : Journal of Educational Evaluation for Health Professions

OPEN ACCESS
SEARCH
Search

Author index

Page Path
HOME > Browse articles > Author index
Search
Hiroyasu Sato 1 Article
ChatGPT (GPT-4) passed the Japanese National License Examination for Pharmacists in 2022, answering all items including those with diagrams: a descriptive study  
Hiroyasu Sato, Katsuhiko Ogasawara
J Educ Eval Health Prof. 2024;21:4.   Published online February 28, 2024
DOI: https://doi.org/10.3352/jeehp.2024.21.4
  • 2,595 View
  • 284 Download
  • 4 Web of Science
  • 7 Crossref
AbstractAbstract PDFSupplementary Material
Purpose
The objective of this study was to assess the performance of ChatGPT (GPT-4) on all items, including those with diagrams, in the Japanese National License Examination for Pharmacists (JNLEP) and compare it with the previous GPT-3.5 model’s performance.
Methods
The 107th JNLEP, conducted in 2022, with 344 items input into the GPT-4 model, was targeted for this study. Separately, 284 items, excluding those with diagrams, were entered into the GPT-3.5 model. The answers were categorized and analyzed to determine accuracy rates based on categories, subjects, and presence or absence of diagrams. The accuracy rates were compared to the main passing criteria (overall accuracy rate ≥62.9%).
Results
The overall accuracy rate for all items in the 107th JNLEP in GPT-4 was 72.5%, successfully meeting all the passing criteria. For the set of items without diagrams, the accuracy rate was 80.0%, which was significantly higher than that of the GPT-3.5 model (43.5%). The GPT-4 model demonstrated an accuracy rate of 36.1% for items that included diagrams.
Conclusion
Advancements that allow GPT-4 to process images have made it possible for LLMs to answer all items in medical-related license examinations. This study’s findings confirm that ChatGPT (GPT-4) possesses sufficient knowledge to meet the passing criteria.

Citations

Citations to this article as recorded by  
  • Qwen-2.5 Outperforms Other Large Language Models in the Chinese National Nursing Licensing Examination: Retrospective Cross-Sectional Comparative Study
    Shiben Zhu, Wanqin Hu, Zhi Yang, Jiani Yan, Fang Zhang
    JMIR Medical Informatics.2025; 13: e63731.     CrossRef
  • ChatGPT (GPT-4V) Performance on the Healthcare Information Technologist Examination in Japan
    Kai Ishida, Eisuke Hanada
    Cureus.2025;[Epub]     CrossRef
  • Potential of ChatGPT to Pass the Japanese Medical and Healthcare Professional National Licenses: A Literature Review
    Kai Ishida, Eisuke Hanada
    Cureus.2024;[Epub]     CrossRef
  • Performance of Generative Pre-trained Transformer (GPT)-4 and Gemini Advanced on the First-Class Radiation Protection Supervisor Examination in Japan
    Hiroki Goto, Yoshioki Shiraishi, Seiji Okada
    Cureus.2024;[Epub]     CrossRef
  • Performance of ChatGPT‐3.5 and ChatGPT‐4o in the Japanese National Dental Examination
    Osamu Uehara, Tetsuro Morikawa, Fumiya Harada, Nodoka Sugiyama, Yuko Matsuki, Daichi Hiraki, Hinako Sakurai, Takashi Kado, Koki Yoshida, Yukie Murata, Hirofumi Matsuoka, Toshiyuki Nagasawa, Yasushi Furuichi, Yoshihiro Abiko, Hiroko Miura
    Journal of Dental Education.2024;[Epub]     CrossRef
  • An exploratory assessment of GPT-4o and GPT-4 performance on the Japanese National Dental Examination
    Masaki Morishita, Hikaru Fukuda, Shino Yamaguchi, Kosuke Muraoka, Taiji Nakamura, Masanari Hayashi, Izumi Yoshioka, Kentaro Ono, Shuji Awano
    The Saudi Dental Journal.2024; 36(12): 1577.     CrossRef
  • Evaluating the Accuracy of ChatGPT in the Japanese Board-Certified Physiatrist Examination
    Yuki Kato, Kenta Ushida, Ryo Momosaki
    Cureus.2024;[Epub]     CrossRef

JEEHP : Journal of Educational Evaluation for Health Professions
TOP