By: Damian L Keter, PT, DPT, PhD
Background:
Comparative effectiveness studies are the cornerstone of medicine and health sciences research. They have a goal of finding ‘the best’ treatment for each associated condition. In comparative effectiveness studies, statistical models are able to provide ‘average’ treatment effects, which are often used to establish standardized mean difference between the interventions; however, it is clear across interventional studies that the ‘average’ effect is not to be consistently expected. Whereas interventional design focuses on central tendency (mean or median of the population), one may more importantly consider the dispersion of data around that point.
Heterogeneity of treatment effects (HTE) refers to the variation in how different individuals respond to the same treatment or intervention. HTE are often represented by standard deviations which are impacted by outliers, demonstrating individuals who respond ‘differently’ to the intervention than the ‘average’ results. There are a number of ways in which HTE can be analyzed or managed in secondary analyses including subgroup analysis of covariates and specific statistical methods to identify heterogeneity [1]. HTE is critically important in interpretation of results in interventional studies; however, they are often poorly reported [1]. Two factors must be considered when understanding HTE in interventional effectiveness trials: 1) what factors contribute to HTE, 2) the limitations and challenges in attempting to control HTE.
Factors contributing to heterogeneity of treatment effects:
A significant proportion of treatment effect may be attributed to non-specific effects- meaning those effects considered outside of the specific treatment mechanisms of the intervention itself [2]. Furthermore, individuals presented with the same stimuli display different responses, which has been proposed to be related to their personal physiological adaptability to pain [3-4]. This has been projected to be one of the reasons orthopaedic manual therapy elicits variable treatment responses (often displayed as ‘responders’ and ‘non-responders’ to treatments) although often reported as efficacious to the ‘average’ individual [5]. Treatment effect should therefore be viewed as the sum of a multitude of contributors that interact with one another and are known to influence outcomes. (Fig1)
Fig1: Factors contributing to treatment effectiveness in interventional trials.
Limitations and challenges in attempting to control heterogeneity of treatment effects:
At first glance, the easy answer is to control HTE through attempting standardization of: 1) patients by narrowing inclusion criteria; 2) environment by using a lab-based design with high levels of control; and 3) intervention through a prescriptive design. This concept is challenged by current limited understanding of the multitude of factors (patient, provider, environmental, systemic), which influence the outcome and therefore must be considered [6]. Increased standardization also results in a reduction in external validity to the clinical setting as patients, environment, and interventions are highly dynamic and pragmatic. While highly controlled designs may be theorized to reduce HTE, it would be impractical to think they would markedly narrow to a homogenous treatment effect. Even so, if the study design is controlled to capture more of the ‘average’ responders and omit the outliers, does it improve meaningfulness of the results any more than what is reported with HTE? In essence, omitting the outliers from research does not reduce the chance that those patients similar to the outliers will walk into the clinic. (Fig 2).
Fig 2: The clinical setting is unable to narrow inclusion criteria as is possible in clinical trial design therefore limiting the external validity of highly controlled clinical trials.
Summary:
An expectation of HTE should be the norm within interventional studies. Comparative effectiveness studies should look to identify which intervention works better for the ‘average’ individual, but should use secondary analysis measures to establish if certain subgroups of individuals (clinical phenotypes) respond differently to the intervention based on a cluster of patient, environmental, or interventional factors [7]. Effect modeling procedures have been proposed to appreciate interaction between intervention and baseline covariates with the goal of developing absolute treatment effects (those anticipated within a certain population/subgroup) with the goal of improving translational clinical value [8]. During study design, investigators should appreciate the benefits and risks of increasing/decreasing control as well as different prospective versus post hoc analysis techniques and should develop studies appropriately based on their purpose.
Patient management, including the use of manual therapies [9], has shifted towards person-centered models as research has established the individuality of experiences in healthcare, therefore research should look to work within these constructs as well appreciating that the days of searching for the ‘holy grail’ of interventions is behind us, but rather a focus should be on identifying which intervention is best suited for which patient, at what time.
References:
- Gabler NB, Duan N, Liao D, Elmore JG, Ganiats TG, Kravitz RL. Dealing with heterogeneity of treatment effects: is the literature up to the challenge? Trials. 2009;10(1):43. doi:10.1186/1745-6215-10-43
- Ezzatvar Y, Dueñas L, Balasch-Bernat M, Lluch-Girbés E, Rossettini G. Which Portion of Physiotherapy Treatments’ Effect Is Not Attributable to the Specific Effects in People with Musculoskeletal Pain? A Meta-Analysis of Randomized Placebo-Controlled Trials. Journal of Orthopaedic & Sports Physical Therapy. 2024;54(6):391-399. doi:10.2519/jospt.2024.12126
- Wan DWL, Arendt-Nielsen L, Wang K, Xue CC, Wang Y, Zheng Z. Pain Adaptability in Individuals With Chronic Musculoskeletal Pain Is Not Associated With Conditioned Pain Modulation. The Journal of Pain. 2018;19(8):897-909. doi:10.1016/j.jpain.2018.03.002
- Zheng Z, Wang K, Yao D, Xue CCL, Arendt-Nielsen L. Adaptability to pain is associated with potency of local pain inhibition, but not conditioned pain modulation: A healthy human study. Pain. 2014;155(5):968-976. doi:10.1016/j.pain.2014.01.024
- Keter D, Cook C, Learman K, Griswold D. Time to evolve: the applicability of pain phenotyping in manual therapy. J Man Manip Ther. 2022;30(2):61-67. doi:10.1080/10669817.2022.2052560”
- Keter D, Loghmani MT, Rossettini G, Esteves JE, Cook CE. Context is Complex: Challenges and opportunities dealing with contextual factors in manual therapy mechanisms research. International Journal of Osteopathic Medicine. Published online January 2, 2025. doi:10.1016/j.ijosm.2025.100750
- Edwards RR, Dworkin RH, Turk DC, et al. Patient phenotyping in clinical trials of chronic pain treatments: IMMPACT recommendations. Pain. 2016;157(9):1851-1871. doi:10.1097/j.pain.0000000000000602
- Kent DM, Paulus JK, Van Klaveren D, et al. The Predictive Approaches to Treatment effect Heterogeneity (PATH) Statement. Ann Intern Med. 2020;172(1):35. doi:10.7326/M18-3667
- Keter D, Hutting N, Vogsland R, Cook CE. Integrating Person-Centered Concepts and Modern Manual Therapy. JOSPT Open. 2023;2(1):60-70. doi:10.2519/josptopen.2023.0812
