LLM-enabled Natural History Study Analysis to Support Rare Disease Research

Rare diseases affect approximately 300 million people worldwide, yet research needed for diagnosis and treatment is often scattered across unstructured sources. Natural history studies are a key source of this evidence, but manual extraction of information is time-consuming and not scalable. A research team developed an automated information extraction system using three open-source language models (Athena-v3-AWQ, Google Gemma3-27B, and Meta Llama-3.1-70B-Instruct) to extract data from PubMed abstracts. All three models processed abstracts with success rates exceeding 99 percent. The Gemma model achieved the best performance with 68 percent accuracy according to expert evaluation and the fastest runtime (approximately 16 minutes for 3,547 abstracts). The results demonstrate that locally deployed open-source language models can efficiently extract structured information from studies at scale, thereby accelerating evidence synthesis in rare disease research.