International Journal of Osteopathic Medicine, 2026/06, 60(online 2026/03/04):100816. doi: 10.1016/j.ijosm.2026.100816, type of study: observational study

Full text    (https://www.journalofosteopathicmedicine.com/article/S1746-0689(26)00008-8/abstract)

Keywords:

AI [1994]
artificial intelligence [15]
ChatGPT [6]
exams [36]
learning [109]
observational study [246]
osteopathic medicine [2109]
osteopathic principles [98]
USA [1752]

Abstract:

Artificial intelligence (AI) is increasingly being integrated into medical education. While ChatGPT (Chat Generative Pre-Trained Transformer) has demonstrated the ability to pass standardized exams such as the United States Medical Licensing Examinations (USMLE) Steps 1-3 and outperform some medical students on clinical reasoning questions, its utility for osteopathic medical licensing remains unexplored. This study evaluated ChatGPT-3.5?s performance on 113 multiple-choice questions sourced from Comprehensive Osteopathic Medical Licensing Examination COMLEX-USA Level 1 (COMLEX-L1) practice materials and two physiology textbooks: Ganong's Review of Medical Physiology, 26e, and Case Files: Physiology, 2e. Questions involving images were excluded. ChatGPT was prompted to select and justify answers, which were then evaluated for accuracy. Results showed that ChatGPT correctly answered 56% of COMLEX-L1 questions, including only 33.33% in the Osteopathic Principles and Practice (OPP) category. In contrast, it achieved 82.35% accuracy on questions from Ganong?s and 58.82% on Case Files, with a combined textbook accuracy of 77.65%. These findings indicate that while ChatGPT can effectively support physiology learning, it currently lacks the precision required for COMLEX-L1 preparation. Future studies should explore model refinement, command optimization, and enhanced integration with osteopathic-specific resources to better support osteopathic medical students preparing for board examinations.

Abstract German language:
(translated by Ostlib): Artificial intelligence (AI) is increasingly being integrated into medical education. While ChatGPT (Chat Generative Pre-Trained Transformer) has demonstrated the ability to pass standardized exams such as the United States Medical Licensing Examinations (USMLE) Steps 1-3 and outperform some medical students on clinical reasoning questions, its utility for osteopathic medical licensing remains unexplored. This study evaluated ChatGPT-3.5?s performance on 113 multiple-choice questions sourced from Comprehensive Osteopathic Medical Licensing Examination COMLEX-USA Level 1 (COMLEX-L1) practice materials and two physiology textbooks: Ganong's Review of Medical Physiology, 26e, and Case Files: Physiology, 2e. Questions involving images were excluded. ChatGPT was prompted to select and justify answers, which were then evaluated for accuracy. Results showed that ChatGPT correctly answered 56% of COMLEX-L1 questions, including only 33.33% in the Osteopathic Principles and Practice (OPP) category. In contrast, it achieved 82.35% accuracy on questions from Ganong?s and 58.82% on Case Files, with a combined textbook accuracy of 77.65%. These findings indicate that while ChatGPT can effectively support physiology learning, it currently lacks the precision required for COMLEX-L1 preparation. Future studies should explore model refinement, command optimization, and enhanced integration with osteopathic-specific resources to better support osteopathic medical students preparing for board examinations.