TY - GEN
T1 - GPT-4 in Clinical Practice
T2 - 2024 International Conference on Informatics, Management, and Technology in Healthcare, ICIMTH 2024
AU - Shah-Mohammadi, Fatemeh
AU - Finkelstein, Joseph
N1 - Publisher Copyright:
© 2025 The Authors.
PY - 2025/4/8
Y1 - 2025/4/8
N2 - Accurate extraction of patient symptoms and signs from clinical notes is essential for effective diagnosis, treatment planning, and research. In this study, we evaluate the capability of GPT-4, specifically GPT-4o, in extracting symptoms and signs from nursing notes within the MIMIC-III dataset. We experimented with two temperature settings (1 and 0.3) to explore the impact of model diversity and consistency on extraction accuracy. Performance metrics include precision, specificity, recall, and F1-score. The results show that a higher temperature (1) led to more creative and varied outputs, with a mean precision of 79% and specificity of 96%, but also exhibited variability, with a minimum precision of 24%. Conversely, at a lower temperature (0.3), precision was more conservative but dropped significantly, with a mean precision of 45% and minimum of 0%. High recall and specificity at optimal temperature setting indicates that GPT-4 holds promise as an assistive tool in clinical practice for symptom and sign extraction tasks.
AB - Accurate extraction of patient symptoms and signs from clinical notes is essential for effective diagnosis, treatment planning, and research. In this study, we evaluate the capability of GPT-4, specifically GPT-4o, in extracting symptoms and signs from nursing notes within the MIMIC-III dataset. We experimented with two temperature settings (1 and 0.3) to explore the impact of model diversity and consistency on extraction accuracy. Performance metrics include precision, specificity, recall, and F1-score. The results show that a higher temperature (1) led to more creative and varied outputs, with a mean precision of 79% and specificity of 96%, but also exhibited variability, with a minimum precision of 24%. Conversely, at a lower temperature (0.3), precision was more conservative but dropped significantly, with a mean precision of 45% and minimum of 0%. High recall and specificity at optimal temperature setting indicates that GPT-4 holds promise as an assistive tool in clinical practice for symptom and sign extraction tasks.
KW - GPT
KW - Symptom extraction
KW - Zero-shot learning
UR - https://www.scopus.com/pages/publications/105003240788
U2 - 10.3233/SHTI250054
DO - 10.3233/SHTI250054
M3 - Conference contribution
C2 - 40200451
AN - SCOPUS:105003240788
T3 - Studies in Health Technology and Informatics
SP - 86
EP - 90
BT - Envisioning the Future of Health Informatics and Digital Health
A2 - Mantas, John
A2 - Hasman, Arie
A2 - Zoulias, Emmanouil
A2 - Karitis, Konstantinos
A2 - Gallos, Parisis
A2 - Diomidous, Marianna
A2 - Zogas, Spyridon
A2 - Charalampidou, Martha
PB - IOS Press BV
Y2 - 13 December 2024 through 15 December 2024
ER -