Simulation in Complex and Innovative Trials

Organizers: Fang Chen (SAS), Freda Conner (Amgen)
Chair: Fang Chen (SAS)
Vice Chair: Clay Thompson (SAS)

Fei Chen (J&J)
Matt Psioda (UNC)
Melanie Quintana (Berry Consultant)
Ruixue (Ree) Wang (Merck)


Title: Simulation is the only route to planning a basket trial
Speaker: Fei Chen (J&J)

Tumor-agnostic basket trials assume genomic alterations as the main driver for tumor response. But usually there are many tumor histologies included, and a naïve pooled analysis across tumor types may introduce bias and result in inflation of false positive error rate.  Separate analysis for each tumor type, however, is less efficient and lacks power for those tumors with limited sample sizes. Bayesian Hierarchical Model (BHM) is usually used in this type of trial design, allowing more powerful evaluation of treatment effect within each histology through borrowing across histologies. It can also incorporate multiple futility analyses at the individual histology level to exclude tumor histologies of inadequate treatment effect, reducing heterogeneity of treatment effects in the remaining histologies, and allow a pooled final analysis for a histology-agnostic claim. Through extensive simulations, we show that the BHM leads to improved statistical power and more precise futility stopping. It limits enrollment of patients with histologies that do not respond to and increases the probability of success of the trial. Most interestingly, we will explore how type-I error should be defined for basket studies, and to what extent false positive rate should be controlled without overly compromising statistical power.


Title: Performing comprehensive simulations to evaluate adaptive Bayesian trials that incorporate information borrowing
Speaker: Matt Psioda (UNC)

According to U.S. Food and Drug Administration guidance on adaptive designs, “for simulations intended to estimate Type I error probability, hypothetical clinical trials would be simulated under a series of assumptions compatible with the null hypothesis.” In the context of adaptive designs that borrow information across arms in a trial or from a source outside the trial (e.g., from external controls) evaluating type I error probabilities (and other important characteristics such as power, accuracy of point estimators, and expected sample size) is difficult. This is because the timing and number of analyses, rate of patient accrual relative to outcome ascertainment, agreement between model parameters and prior information, and the method chosen for information borrowing all interact to determine the operating characteristics for the design. Weighing the pros and cons of an adaptive, information borrowing design requires exploring the design’s operating characteristics over an array of plausible assumptions for the set of design features that are not completely controllable (e.g., enrollment rate) or are unknown (e.g., true parameter values). In this talk we illustrate how one can use simulation to comprehensively evaluate the properties of sequential monitoring trial designs that incorporates information borrowing from an external data source.

Title: Evidence-Based Clinical Trial Simulation and Design Using Historical Data
Speaker: Melanie Quintana (Berry Consultant)

There is a growing need to learn from our past in clinical trial design.  In light of this, numerous efforts are being made to promote the creation of shared disease-specific databases with patient-level historical control and observational data.  With these efforts in mind, we discuss how we can synthesize and make better use of this historical information to design more informed and powerful clinical trials.  One such use of historical disease-specific databases is to create realistic virtual patient simulation models to feed clinical trial simulations and better understand operating characteristics of potential future clinical designs.  Throughout this presentation, we highlight the use of the shared Pooled Resource Open-Access ALS Clinical Trials Database (PRO-ACT) in designing a platform trial for amyotrophic lateral sclerosis (ALS) using.  The PRO-ACT database provides patient-level longitudinal data from placebo and treatment arms from 23 Phase II/III clinical trials and is an exemplary effort to share data and learn from past studies.  Access to rich patient-level data within the PRO-ACT database provides many advantages in designing the ALS platform trial, including informing the creation of realistic clinical trial simulations to optimize key design elements such as the primary endpoint and analysis method, key inclusion / exclusion criteria, sample size and length of follow-up.

Title: Practical thinking of group sequential design modification after interim analysis
Speaker: Ruixue (Ree) Wang (Merck)

With the fast development of immune-oncology, It is not unusual that we have to re-design clinical trials due to emerging data, such as a new stand of care is approved, new findings about biomarker effective size, later line of treatment effect and so on. Re-designing includes adding a subpopulation, adding or removing primary objectives or interim analyses, changing statistical assumption or multiplicity strategy. Those changes are major and challenging especially if we want to change it after interim analysis. One of the concerns would be how to control the overall type I error at pre-specified level. In this presentation, we consider a group sequential design with pre-planned design revision evaluation via simulation for type I error control while study team keeps blinded.