ICH E9 (R1)-Statistical Principles for Clinical Trials: Addendum: Estimands and Sensitivity Analysis in Clinical Trials

Yeh-Fong Chen (FDA), Rakhi Kilaru (PPD), Miaomiao Ge (Boehringer-Ingelheim), Roann Seay (Prahs)
Chair: Roann Seay (Prahs)
Vice Chair: Yeh-Fong Chen (FDA)

Miguel Garcia (Boehringer-Ingelheim)
Joe Hirman (Pacific Northwest Statistical Consulting)
Natalia Kan-Dobrosky (PPD)
Munish Mehra (Tigermed)


Title: Study Populations, Convenience Samples and Estimands
Speaker: Joe Hirman (Pacific Northwest Statistical Consulting)

ICH E9 (R1) introduces the concept of estimands which it explains ‘defines in detail what needs to be estimated to address a specific scientific question of interest.’ This attempt to add increased formality and specificity to statistics, as it is applied to clinical trials, is welcome. However, ICH E9, including R1, does not provide a coherent framework of starting assumptions that would allow the performance of various estimators or tests to be evaluated under the restrictions imposed by a randomized, clinical trial. With the generality that ICH E9 is written one must look elsewhere to determine a metric to evaluate the performance of the tools of statistics (estimators and hypothesis tests). Standard metrics, such as unbiasedness, type I error rates and power, can be readily used for this purpose. However, these metrics require a probability space. The question posed is what probability spaces are reasonable in the analysis of data from randomized clinical trials. Specifically, given that the subjects, sites and medical practice employed in a clinical trial are not a random sample from any population, what probability spaces can reasonably be employed to evaluate estimates of the estimands? How then is this probability space communicated to the medical community and patients? ICH E9 hints at the answer when it states “preservation of the initial randomisation in analysis is important … in providing a secure foundation for statistical tests” and “factors on which randomisation has been stratified should be accounted for later in the analysis.”

Title: Implementing Estimands in Trials: Detailed Clinical Objectives
Speaker: Miguel Garcia (Boehringer-Ingelheim)

In this talk we present the concept of Detailed Clinical Objectives (DCOs) as a means to engage clinicians and prospectively implement estimands from the very earliest stages of trial design. With DCOs, we reframe estimands as an extension to the widely-understood, pre-existing concept of objectives: Since estimands must align with objectives, precise objective definitions not only lead to suitable estimands, but also make clear the explicit intent behind trial design. DCOs are based on required components derived from those in ICH E9(R1) and the PICO model for evidence-based medicine. A simple workflow and template are used to facilitate discussions and CTP inclusion. An example from a clinical trial will be presented. In addition to the concept, we also present some early experiences of implementing this technique.

Title: Estimands in clinical trials with treatment switching
Speaker: Natalia Kan-Dobrosky (PPD)

An addendum of the ICH E9 guideline on Statistical Principles for Clinical Trials was released in November 2019 introducing the estimand framework. This new framework aims to align trial objectives and statistical analyses by requiring a precise definition of the population quantity of interest, i.e. the estimand. Estimands should explicitly account for intercurrent events, i.e. events, which occur after treatment initiation but before observing the study endpoint, such as the start of new therapy when the endpoint is overall survival (OS). A working group was initiated to foster understanding and consistent implementation of the estimand framework in oncology clinical trials. This work summarizes the group’s recommendations for appropriate estimands in the presence of treatment switching, one of the key intercurrent events in oncology clinical trials.
Traditionally, analysis of OS in the confirmatory study is performed ignoring treatment switching (treatment-policy estimand). If patients from the control group switch more frequently to the treatment which prolongs OS, than patients in the investigational group, the true survival benefit of the investigational treatment itself is likely to be underestimated. Causal inference methodologies accounting for treatment switching (hypothetical estimand) such as rank-preserving structural failure time models and inverse probability weighting have been proposed and applied in oncology trials for the analysis of OS, to mitigate this bias, providing further perspectives on the added value of novel therapies, e.g. to payers and patients. We present different choices of estimands, illustrate those estimands using case studies, and discuss how those choices may impact study design, data collection, trial conduct, analysis, and interpretation.

Title: From Efficacy Endpoints to Estimands, communicating effectively with non-statisticians what has changed, why and the benefits of following ICH E9R1.
Speaker: Munish Mehra (Tigermed)

People familiar with the design, conduct and analysis of clinical trials understand efficacy endpoints and their analyses, however the term estimand is not familiar to most non-statisticians. While many groups and large companies are moving forward implementing ICH E9 R1 and the incorporation of Estimands into their protocols, most smaller companies have yet to understand let alone incorporate the components of an estimand in their trial design. The final definition of an Estimand in ECH E9 R1 “A precise description of the treatment effect reflecting the clinical question posed by the trial objective. It summarizes at a population-level what the outcomes would be in the same patients under different treatment conditions being compared.“ and the Wikipedia entry [1/2/2010] “An estimand is a parameter which is to be estimated in a statistical analysis. The term is used to more clearly distinguish the target of inference from the function to obtain this parameter (i.e., the estimator) and the specific value obtained from a given data set (i.e., the estimate)” require a Statistician to translate the terminology! Regulatory agencies have not consistently asked for it in late stage trials making it harder for industry to follow. What is needed is a way to simplify and effectively communicate to the key concepts around use of estimands and sensitivity analyses to non-statisticians. This presentation will share current status of industry understanding and adoption and suggest ways to improve understanding as statisticians communicate the concepts of ICH E9-R1 to non-statisticians.