Scientific Associate Director Amgen, Inc., United States
Disclosure(s):
Ari Pritchard-Bell: No relevant disclosure to display
Objectives: Manual implementation of pharmacometric analyses requires significant expertise, is time-intensive, and prone to inconsistencies across analysts. While large language models (LLMs) show promise for automation, their direct application lacks the stability and validation required for regulatory submissions. We developed a novel multi-agent framework that combines structured constraints with iterative refinement to generate high quality pharmacometric analyses while maintaining transparency and reproducibility.
Methods: Our tool employs an agent-based LLM architecture: 1) An agent that reads and processes analysis plans using AWS bedrock hosted LLMs to identify required analyses and success criteria with structured output constraints; 2) An agent that iteratively generates and validates R code with built-in quality checkpoints, applying human-like reasoning for error resolution and code refinement; and 3) A systematic validation framework that executes generated code, provides feedback, and documents the complete analytical thought process. The framework includes automated handling of data validation, population filtering, missing data patterns, and model diagnostics. The system was evaluated using example pharmacometric analyses from multiple therapeutic areas.
Results: The framework successfully automated complex analyses while maintaining rigorous submission quality standards. The first agent achieved >95% accuracy in identifying required analyses from analysis plans. Generated code included comprehensive data validation, proper population filtering, and standardized output formats. The iterative refinement process enabled stable convergence to valid solutions within 8-10 cycles on average. Quality control checkpoints effectively identified potential issues in data handling, model specification, and results interpretation. The system maintained complete reproducibility through detailed documentation of analysis decisions and systematic version control. Example applications demonstrated successful automation of analyses like pharmacokinetic-pharmacodynamic characterization and exposure-response relationships.
Conclusions: This work represents a significant advance in automating pharmacometric analyses by combining the flexibility of LLMs with rigorous quality requirements. The multi-agent architecture with iterative refinement enables reliable automation while maintaining transparency and reproducibility. This approach has broad applications in clinical pharmacology and could significantly accelerate drug development while improving analysis consistency. The framework's ability to generate high quality, well-documented analyses with built-in QC steps provides a scalable solution for standardizing pharmacometric workflows.
Citations: [1] Chase, H. (2022, October 17). LangChain. https://github.com/langchain-ai/langchain
