Leveraging Large Language Models for Colorectal Cancer Symptom Extraction from MIMIC-IV Clinical Notes

The study compared various methods for extracting 46 colorectal cancer symptoms from 2,704 patient discharge notes in the MIMIC-IV database. Four approaches were tested: dictionary-based rules, named entity recognition (NER), and two large language models (Claude Haiku and Gemini 3.5 Flash) without prior training. Results showed that Gemini 3.5 Flash achieved the best performance with an F1 score of 0.70, followed by Claude Haiku with an F1 score of 0.63. Both models substantially outperformed traditional methods (dictionary rules with F1 0.44 and NER with F1 0.38). Interestingly, additional negation rules degraded model performance. The study concluded that large language models offer a scalable and accurate alternative to manual document review without requiring specific training or institution-specific rule development.