(T-021) Utilizing Generative Deep Learning and Bayesian Neural Network for Pharmacometrics Covariate Analysis

Huy Vo – Quantitative Pharmacology and Data Analytics – BMS; Urvi Aras – Clinical Pharmacology – BMS; Huynh Yen Thanh Bach – Clinical Pharmacology – BMS; Anna Kondic – Quantitative Pharmacology and Data Analytics – BMS; Loan Pham, Director – Clinical Pharmacology – BMS

Objectives: The utility and feasibility of Neural network (NN) and Bayesian in PK-PD analysis were investigated. Recently, Hierarchical Deep Compartment Modeling (HDCM) workflow was shown to be able to quantify uncertainty of NN model predicted parameters and was successfully tested on synthetic data from 10 subjects [1]. While conceptually appealing, this approach incurs prohibitive run time for typical clinical datasets due to the complexity of the high-dimensional Bayesian posterior distribution.

To enable the full covariate modeling in PK-PD analyses with practical run time, this research utilized flow-based generative deep learning [2] in addition to NN and Bayesian inference. A novel approach was developed, which trains normalizing flows (NFs) [2] to generate both samples and their associated probabilities from the posterior distribution. This approach allows breaking down Bayesian inference from a large dataset into manageable subtasks that can be completed sequentially while accumulating all information from the whole dataset.

Methods: The workflow was implemented in Julia to perform PopPK/PD analysis on simulated dataset for a hypothetical small molecule drug with pharmacodynamic activity following stimulatory response function. The data included 302 subjects from phase 1 and phase 2 studies, with formulation, subject type, weight, gender, age as PK covariates and 3 PD covariates (PDcov1-3).
The covariate-PK/PD parameter relationships were modeled by NN. Loose priors and standard normal distributions were set for fixed, random effect and NN parameters. The initial analysis step processed one sub-dataset (first subtask). The No-U-Turn Sampler (NUTS) [3] was used to draw samples from the intermediate posterior distribution. A NF [2] was trained to establish a probability distribution from these samples. The NF-induced distribution became the prior distribution input for the second analysis step, with the likelihood evaluated only for the next sub-dataset. This iterative procedure was repeated until the whole dataset was analyzed.

Results: A six-step RealNVP[4] architecture using NN conditioners of 2 hidden layers, width 100, was adequate for carrying posterior information forward across 30 sub-datasets. NUTS produced well-mixing chains (R ̂ < 1.01). Full covariate analysis testing was completed within 3 days on a 64-core compute node. Posterior predictive checks confirmed that predicted concentrations/PD responses based on the posterior distributions captured observed drug concentrations/responses. Forest plots generated from the NN-predicted covariate effects correctly identified covariate-parameter relationships consistent with NONMEM-based Stepwise Covariate Modeling approach [5].

Conclusions: Bayesian updates enabled by normalizing flows was shown to be a promising direction for improving scalability of NN-based PopPK/PD analyses. The newly developed method adds flexibility to analysis planning, allowing the posterior distribution to be provisionally generated from existing data while waiting for additional clinical studies, from which new information can be readily incorporated.

Citations: [1] A. Elmokadem, M. Wiens, T. Knab, K. Utsey, S. P. Callisto, and D. Kirouac, “Hierarchical deep compartment modeling: A workflow to leverage machine learning and Bayesian inference for hierarchical pharmacometric modeling,” Clinical and Translational Science, vol. 17, no. 10, p. e70045, 2024, doi: 10.1111/cts.70045.
[2] G. Papamakarios, E. Nalisnick, D. J. Rezende, S. Mohamed, and B. Lakshminarayanan, “Normalizing Flows for Probabilistic Modeling and Inference,” Journal of Machine Learning Research, Vol. 22, 2021, p. 1-64.
[3] M. D. Hoffman and A. Gelman, “The no-U-turn sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo,” Journal of Machine Learning Research, Vol. 15, 2014, p. 1351-1381
[4] L. Dinh, J. Sohl-Dickstein, and S. Bengio, “Density estimation using Real NVP,” Feb. 27, 2017, arXiv: arXiv:1605.08803. doi: 10.48550/arXiv.1605.08803.
[5] K. Sanghavi et al., “Covariate modeling in pharmacometrics: General points for consideration,” CPT: Pharmacometrics & Systems Pharmacology, vol. 13, no. 5, pp. 710–728, 2024, doi: 10.1002/psp4.13115.

Keywords: Pharmacometrics covariate analysis, Neural Network, Flow-based generative deep learning, Bayesian inference