Associate Director, Clinical Pharmacology Arcus Biosciences, Inc., United States
Disclosure(s):
Ian Linsmeier: No relevant disclosure to display
Objectives: A crucial aspect of every Pharmacometrics (PMX) project is the quality control (QC) process used to verify the accuracy, reproducibility, and sensibility of the results. However, there is minimal industry guidance on the topic to date, and inconsistent QC review execution may adversely affect project quality1–3. Thus, there is a growing need for more standardized and reproducible QC review workflows across the industry. Version control systems—such as git and GitHub—are increasingly being used by PMX teams to track file changes over time and collaborate on software-based projects4.
Our objective was to develop an effective QC process using GitHub’s code review functionality with the following design features: (1) fit-for-purpose reviews with variable scope and scale, (2) QC initiation at any time during the project lifecycle, (3) fully reproducible analyses, and (4) seamless integration into existing PMX workflows without sacrificing efficiency.
Methods: PMX projects are version controlled in a local git repository and synced to GitHub, a web-hosting platform for remote storage and collaboration. Before initiating a QC, the analyst updates the repository’s “main” branch by merging in recent changes. The analyst also includes a reproducible analysis script, which automatically reproduces the primary analysis results when executed.
The analyst then creates a new branch off the “main” branch and opens a pull request (PR) for conducting the QC on GitHub. Next they curate the key source files (e.g., code, models, etc.) for review by appending QC request comments to make each file visible within the GitHub PR. In the PR description, they document the relevant file paths, QC review scope, purpose, project location, and software versions. Finally, the analyst assigns a reviewer and requests a PR review via GitHub.
The QC reviewer clones the project repository and runs the reproducible analysis script to generate the results. Then they check each file specified in the PR description and document any findings by adding comments to the file on GitHub. The QC reviewer selects “Request changes” if there are any findings after completing the PR review.
The analyst addresses the findings identified by the reviewer and responds to each comment on GitHub by describing any changes they made. Once all comments are addressed, another review is requested on the same PR. The process is repeated until there are no additional findings, at which point the reviewer “Approves” the PR review and merges the changes into the main branch, marking the end of the QC.
Results: By leveraging GitHub code review functionality, this workflow results in a fully traceable QC review process that incorporates all documentation into the repository’s version control history. Custom R code was developed to automate project reproducibility, with further automation possible via GitHub Actions.
Conclusions: This GitHub-based QC workflow provides a standardized framework that can be easily adopted by PMX teams of any size and is compatible with other repository hosting sites, such as GitLab and Bitbucket. This process has been used to efficiently QC PMX models in Phase 3 enabling regulatory submissions for domvanalimab/zimberelimab, casdatifan, quemliclustat, and etrumadenant.
Citations: [1] Owen, J. S. & Fiedler-Kelly, J. Introduction to Population Pharmacokinetic / Pharmacodynamic Analysis with Nonlinear Mixed Effects Models. (Wiley, 2014). [2] Workgroup, E. M. et al. Good Practices in Model-Informed Drug Discovery and Development: Practice, Application, and Documentation. CPT: Pharmacometrics & Systems Pharmacology 5, 93–122 (2016). [3] Bonate, P. L. et al. Guidelines for the Quality Control of Population Pharmacokinetic–Pharmacodynamic Analyses: an Industry Perspective. The AAPS Journal 14, 749 (2012). [4] About GitHub and Git. GitHub Docs at
