Utilizing Real-World Data in Late-Stage Clinical Trials – Methodologies and Example Sharing
Chair: Yeh-Fong Chen (FDA)
Vice Chair: MiaoMiao Ge (Boehringer-ingelheim)
Speaker: Huan Wang (FDA)
Title: Challenges and Strategies to Properly Utilize Historical or Real-World Data in Clinical Trials
Abstract: Single-arm clinical trials utilizing external control in some cases can have advantages over randomized clinical trials. The causal treatment effect of interest for single arm studies is usually the average treatment effect on the treated (ATT) rather than the average treatment effect (ATE). Although methods have already been developed to estimate the ATT, the selection and use of these methods require a thorough comparison and in-depth understanding of the advantages and disadvantages of these methods.
In this study, we conducted simulations under different identifiability assumptions to compare the performance metrics, including the bias, standard deviation (SD), mean squared error (MSE), power as well as type I error, for a variety of methods, including regression model, propensity score matching, Mahalanobis distance matching, coarsened exact matching, inverse propensity weighting, augmented inverse propensity weighting (AIPW), AIPW with Super Learner, and targeted maximum likelihood estimator (TMLE) with Super Learner.
According to our simulations under most scenarios we examined, matching-based methods preserve the type I error better than non-matching-based method. However, they in general have worse performance in the estimation accuracy compared to doubly robust methods given that the identifiability assumptions are not severely violated. We also proposed a procedure to help trialists conduct a single arm or hybrid studies utilizing external information.
Speaker: Ruthie Davi (3DS)
Title: Illustration of Propensity Score Weighted External Control with Single Arm Phase 2 Trial and Hybrid Randomized and External Control in Phase 3 Design in Recurrent Glioblastoma
Abstract: Interest in utilizing external controls for purposes of medical product development is growing. Such external controls may be created from historical clinical trials data, registry data, or real-world data and generally use statistical methods, such as propensity score methods, to provide baseline balance between the external control and the investigational group. External controls can be advantageous to enhance the interpretation of single arm trials or augment randomized controlled trials in some indications.
This talk will present how external controls are being used in the development of a product to treat recurrent glioblastoma. This includes creation of a propensity score weighted external control aligned to a single arm phase 2 study and a planned design for the phase 3 confirmatory study utilizing a hybrid control that includes both randomized and external control patients. Inverse probability treatment weighting for estimation of the average treatment effect on the treated (also called weighting by odds) is utilized and will be described.
Speaker: Jia Guan (Boehringer Ingelheim)
Title: Bias minimization in data integration of clinical trial data and real-world data
Abstract: Real-world data (RWD) has been increasingly discussed and used in the context of external control arm, hybrid control arm, and Bayesian framework in clinical trials, in particular in rare disease or unethical situation. However, bias arise from multiple sources would over- or under-estimate the treatment effect as the characteristics of the patient population and the approach of data collection varies from clinical trial and real-world data. Bias minimization plays an essential role to generate robust results and reliable causal inference. The emerging epidemiological and statistical methods have been proven to mitigate the selection bias and confounding bias, well in data integration of clinical trial data and RWD, measurement bias remains a challenge when RWD has been routinely collected in certain measurement. We would propose and discuss the application of quantitative bias analysis approach to reduce the systematic bias caused by measurement difference. Flatiron Health database is widely known and used for evidence generation in oncology, well Response Evaluation Criteria in Solid Tumours (RECIST) -based response is regarded as one of gold standard endpoints. However, the measurement bias between RECIST-based response and real-world response impedes utility of the RWD. We would take the response data as an example to illustrate how to correct the measurement bias and enhance the treatment effect estimation.
Speaker: Pallavi Mishra-Kalyani (FDA)
Title: Externally Controlled Clinical Trials: Statistical Considerations and Regulatory Experience
Abstract: In general, randomized controlled trials are preferred for providing evidence of drug efficacy, as they allow for a comparison of treatment arms with minimal concern for confounding by known and unknown factors. However, in certain disease settings, randomization is not feasible. In these cases, incorporating external control data into the study design can be an effective way to expand the interpretability of the results of an experimental arm by introducing the ability to perform a formal or informal comparative analysis.
This presentation will provide statistical considerations for the use of external control data intended to provide evidence of efficacy for regulatory purposes. In particular, current FDA guidance on this topic will be discussed, followed by a discussion of study design and analysis considerations when incorporating external control data in a clinical trial. Finally, some examples of the use of external controls in oncology marketing applications will be discussed.
Discussant: Helen Li (QRMedSci LLC)
