Head of Clinical Data Science InsightRX, Ontario, Canada
Disclosure(s):
Jasmine Hughes, PhD: No relevant disclosure to display
Objectives: Model-informed precision dosing (MIPD) relies on selecting a pharmacokinetic model that accurately reflects individual patient characteristics. However, model performance varies across patient subpopulations, posing challenges for clinical implementation[1]. We compared two vancomycin model selection strategies: a machine learning model[2] and a structured algorithm guided by expert-defined principles and real-world data, which we term an Evidence-plus-Expertise (E+E) algorithm.
Methods: De-identified retrospectively analysed data extracted from the InsightRX Nova MIPD platform supported algorithm development (N = 240,000+ patients). These data included patient weight, height, sex, serum creatinine, and age. Additional metrics were also derived from these data, including body mass index (BMI) and estimated glomerular filtration rate. Both algorithms were developed to select one model from a pre-defined curated list of candidate models, used only data available at the start of therapy (i.e., before the availability of therapeutic drug monitoring samples), and were evaluated on a hold-out data set (N = 100,000+ patients). The ML model used a multi-label classification strategy to rank candidate models by the probability that predictions would fall within 80-125% of observed concentrations. The E+E algorithm was based on seven principles for evaluating PK models, including development cohort similarity, physiological suitability such as model structure and covariate structure, and criteria for evaluating model performance. Candidate model predictive performance was evaluated across 129 predefined subpopulations based on age, serum creatinine, BMI and sex. These statistical results were integrated with the expert-defined principles through a structured rule-based algorithm to assign a preferred model to each subpopulation.
Results: Relative to single-model approaches, the ML model reduced root mean squared error (RMSE) by 2-35% and the E+E algorithm reduced RMSE by 5-33%. Relative to a simple two-model BMI-based algorithm, the ML model reduced RMSE by 6.3% compared to a reduction of 7.1% by the E+E algorithm. ML model feature importance analysis confirmed that age, BMI, renal function, and sex were the most influential predictors, consistent with the E+E subpopulation framework. The E+E algorithm was implemented in Nova as Gemini.
Conclusions: Both approaches provided similar improvements in model predictions compared to the standard of care. The E+E algorithm, which is essentially a decision tree, offers enhanced explainability and operational transparency, facilitating implementation in clinical workflows where interpretability is critical. ML model performance may be limited by the data available as features; with increasingly rich data sets, this approach would be expected to scale better.
Citations: 1. Hughes MA, Lee T, Faldasz JD, Hughes JH. Impacts of age and BMI on vancomycin model choice in a Bayesian software: Lessons from a very large multi-site retrospective study. Pharmacotherapy. 2024 Oct;44(10):794-802. doi: 10.1002/phar.4613. Epub 2024 Oct 9. PMID: 39382218. 2. van Os W, O’Jeanson A, Troisi C, Liu C, Hughes J, Tong D, Brooks J, Keizer R. Data-driven model selection for model-informed precision dosing: a case study with vancomycin. PAGE 32 (2024) Abstract 11024. www.page-meeting.org/?abstract=11024
