Evaluating agentic simulation for local public health estimation

Researchers developed LJMPopSim, a population simulation framework using large language models that combines U.S. Census and Centers for Disease Control and Prevention data to create synthetic individuals and simulate their health behaviors. The system was developed using historical Hawaii data and tested on 2022 data from Hawaii and New York State, focusing on colorectal cancer screening and mammography. Mean absolute error ranged from 3.5 to 15.0 percentage points, and correlation between simulated and observed prevalence ranged from 0.26 to 0.69. Colorectal cancer screening predictions better preserved geographic differences but systematically overestimated prevalence, while mammography achieved lower absolute error but weaker geographic correlation. Prediction errors were greatest in communities with lower observed screening prevalence. The research demonstrates that synthetic agents based on language models can create population-level patterns similar to real-world health behaviors, though calibration and subgroup performance remain key challenges.