Automating clinical trial outcome identification and misreporting detection using RegCheck

RegCheck is an automated system based on a large language model (LLM) developed to identify clinical trial outcomes and detect outcome misreporting. The research involved validation on 62 clinical trials from five major medical journals and comparison with manual assessment from the COMPare Trials project. RegCheck achieved 91.2% success rate in outcome extraction, 83.6% accuracy in classifying outcomes as primary, secondary, or non-prespecified, and 85.6% accuracy in detecting outcome misreporting. After resolving discrepancies with original human judgements, the accuracy of outcome misreporting detection increased to 94.8%. The average cost of running the system was 5.94 USD per paper. Automated outcome checking may provide a scalable way to support editors, peer reviewers, and authors in detecting outcome switching and improving the quality of clinical trial reporting.