Annals of Coloproctology

Display

Sort by :

Original Article

Colorectal cancer

How appropriately can generative artificial intelligence platforms, including GPT-4, Gemini, Bing, and Wrtn, answer questions about colon cancer in the Korean language?: Sun Huh; Ann Coloproctol. 2025;41(3):190-197. Published online June 25, 2025; DOI: https://doi.org/10.3393/ac.2024.00122.0017

6,354 View
64 Download
3 Web of Science
4 Citations

Purpose
This study aims to assess the performance of 4 generative artificial intelligence (AI) platforms—Gemini (formerly Bard), Bing, GPT-4, and Wrtn—in answering questions about colon cancer in the Korean language. Two main research questions guided this study. First, which AI platform provides the most accurate answers? Second, can these AI-generated answers be reliably used to educate patients and their families about colon cancer?
Methods
Ten questions selected by the author were posed to the 4 generative AI platforms on February 22, 2024. Two colorectal surgeons in Korea, each with over 20 years of clinical experience, independently evaluated the answers provided by these generative AI platforms.
Results
The generative AI platforms scored an average of 5.5 out of 10 points. Wrtn achieved the highest score at 6 points, followed by GPT-4 and Gemini, each with 5.5, and Bing, scoring 5 points. The weighted κ for inter-rater reliability was 0.597 (P<0.001). The generative AI platforms performed well in explaining the occult blood test for cancer screening, keyhole surgery, and dietary recommendations for cancer prevention. However, they demonstrated significant limitations in answering more complex topics, such as estimating survival rates following surgery, choosing targeted therapy after surgery, and accurately reporting the mortality rate due to colon cancer in Korea.
Conclusion
The findings suggest that using these generative AI platforms as educational resources for patients and their families regarding colon cancer is premature. Further training on colorectal diseases is required before these AI platforms can be considered reliable information sources for the general public in Korea.

Citations

Citations to this article as recorded by

Expert Review on the Quality of Responses to the Questions of Multiple Myeloma Patients: A Validation Study of the Medical Artificial Intelligence System “Myelobot”
Aleksander Sergeevich Luchinin, O. E. Ochirova, V. G. Potapenko, V. V. Ryabchikova
Clinical Oncohematology.2026; 19(1): 81. CrossRef
Agentic artificial intelligence is the future of cancer detection and diagnosis
Sayedur Rahman, Md. Tanzib Hosain, Nafiz Fahad, Md. Kishor Morol, Md. Jakir Hossen
Array.2026; 29: 100676. CrossRef
Role of Medical Editors in the Age of Generative Artificial Intelligence
Sun Huh
Healthcare Informatics Research.2025; 31(4): 317. CrossRef
Temporal evolution of large language models (LLMs) in oncology
Zilin Qiu, Aimin Jiang, Chang Qi, Wenyi Gan, Lingxuan Zhu, Weiming Mou, Dongqiang Zeng, Mingjia Xiao, Guangdi Chu, Shengkun Peng, Hank Z. H. Wong, Lin Zhang, Hengguo Zhang, Xinpei Deng, Quan Cheng, Bufu Tang, Yaxuan Wang, Jian Zhang, Anqi Lin, Peng Luo
Journal of Translational Medicine.2025;[Epub] CrossRef

First
Prev
Page of 1
Next
Last

Search