Continuous Value Tokenization Improves Medical Event Foundation Models

A research team trained transformer models with 47 million parameters on MIMIC-IV database data from 364,627 patients and 375 million observations. They compared three tokenization methods for numeric values such as lab results, vital signs, and time intervals: Discrete (binned values), Continuous Factored, and Continuous Fused. Continuous Fused tokenization reduced sequence length by 34 percent and improved numeric value prediction accuracy by 30.25 percent. For ICD code prediction, Continuous Fused tokenization performed better (AU-PRC 0.457 versus 0.446), while DRG prediction and ED disposition determination showed comparable performance across all methods. Continuous value tokenization provides substantial efficiency and precision gains without requiring modifications to standard transformer architecture.