Thorough Research Questions should have Layers

Order Questions

Most of our musculoskeletal treatments fall within a ‘gray zone’ of effectiveness¹. Nearly none of our treatments are strongly effective all of the time and few have no benefit all of the time. This is one of the reasons we see outcomes that are consistently similar across well-designed musculoskeletal studies. To enhance treatment benefits and advance precision medicine, pain researchers have begun to move beyond the first-order question of “Does the treatment work?² and have begun studying second and third order questions such as “how does this treatment work” (second order) and “who does this treatment benefit” (third order).

First Order Questions

Presently, first order questions in musculoskeletal management are fairly well studied. To investigate a first-order question, one requires a randomized trial that compares two or more interventions. In these studies, the comparator matters since the effect of the intervention is being compared against the effect of another intervention. When two reasonably useful interventions are compared, we mostly find similar effects in musculoskeletal research, suggesting no superiority among most interventions^3-7. And it’s worth reminding oneself that no significant difference in studies of two reasonable interventions does not mean that the interventions have no effect, it just means that the interventions have a similar influence on clinical outcomes. We can’t really assume an intervention has no effect unless it is compared well against a true placebo. Designing a good placebo-controlled (or sham) trial is not easy.

Second Order Questions

Second-order questions such as “how does the treatment work” have emerged as a critical question for both developing theory and enhancing treatment efficacy. In theory, each treatment is hypothesized to elicit a “mechanism” and this mechanism may be specific or shared. Specific treatment mechanisms are unique to an intervention and are considered the primary reason the treatment works. For example, resisted exercise is theorized to work because in mechanisms studies it increases in muscle fiber size and neural adaptations, which leads to improved strength and endurance⁸. Manual therapy is theorized to work because in mechanisms studies it exhibits both peripheral and central influences, and reduces muscle spasm, which leads to pain modulation and improved mobility⁹. Shared treatment mechanisms occur when two seemingly different treatments (e.g. manual therapy and resistance exercise) are found to exert their effects on clinical outcomes via similar mechanisms (e.g. therapeutic alliance).

Although second order questions help us understand “how does the treatment work”, the capture of mechanisms in isolation (especially in a laboratory study or preclinical work) only tells part of the story. The truth is, mechanisms are present in almost all forms of treatment, even the dubious ones. For example, craniosacral mobilization of the head may actually elicit neurophysiological changes in a patient during a preclinical study. However, whether these neurophysiological changes are related to changes seen in clinical outcomes is presently unknown.

Third Order Questions

Third-order questions ask “who does this treatment benefit?” At present, this has remained elusive in all forms of conservative and surgical approaches. These studies require responder analyses, which are evaluating using many different designs. By definition, a responder is an individual who improves because of the treatment they received, not just “after” a treatment they received.

Studies such as these are actually much more difficult to perform than people think. Although promising in context, responder analyses run the risk of placing too much emphasis on algorithms and not taking the patient’s complex background and needs, such as culture, values, preferences, and beliefs, into consideration. Further, these analyses often dichotomize clinical outcomes to determine a “success” creating thresholds. Thresholds that are used to determine a responder (e.g., minimally clinically important difference) are likely unalike across social and psychological risk groups and need to be considered when interpreting clinical results.

What areas do we need to study the most?

I would argue that second order questions have the most potential yield in research. But to answer “how” and “why”, we need a unique study design that concurrently captures specific and shared mechanisms prior to and during the capture of clinical outcomes. The design would require the repeated sampling of subjects’ specific and potential shared treatment mechanisms in a longitudinal fashion, allowing exploration the association of early changes in mechanisms to long-term clinical outcomes. If a mechanistic factor plays a causal role in outcome, change in that factor (either specific or shared) must precede change in the clinical outcome.

Presently, this is well studied in the psychological literature¹⁰, but remains poorly studied in musculoskeletal care. Completion of these studies is challenging since it is time consuming, expensive, a burden to the patients, and requires a fairly large sample size (mediation analyses are required). Nonetheless, we’ve sunk millions of dollars into answering the question of “Does the treatment work?” and the answer is nearly always “yes” for nearly every treatment! It’s time we take things to the next level and understand why we are seeing the changes in the clinical outcomes that we study.

Written by: Chad E Cook PT, PhD, FAPTA

Twitter @chadcookpt

Professor, Department of Orthopaedics, Duke University, Durham, NC. 27516

Competing interests: A portion of Dr Cook’s salary is funded by the NIH/VA/DoD and the Center of Excellence in Manual and Manipulative Therapy at Duke University.

References

1. Cook CE, Bonnet F, Maragano N, Garcia AN, Vielitz A, Riley SP. What is the believability of evidence that is read or heard by physical therapists? Braz J Phys Ther. 2022 Jul 8;26(4):100428. doi: 10.1016/j.bjpt.2022.100428. Epub ahead of print. PMID: 35849892.
2. Day MA, Jensen MP. Understanding pain treatment mechanisms: a new direction in outcomes research. Pain. 2022 Mar 1;163(3):406-407.
3. Chmielewski TL, George SZ, Tillman SM, et al. Low- versus high-intensity plyometric exercise during rehabilitation after anterior cruciate ligament reconstruction. Am J Sports Med. 2016;44:609–617.
4. Cook CE, George SZ, Keefe F. Different interventions, same outcomes? Here are four good reasons. Br J Sports Med. 2018;52(15):951-952.
5. Day MA, Ehde DM, Burns J, et al. A randomized trial to examine the mechanisms of cognitive, behavioral and mindfulness-based psychosocial treatments for chronic pain: Study protocol. Contemp Clin Trials. 2020;93:106000.
6. Di Blasi Z, Harkness E, Ernst E, Georgiou A, Kleijnen J. Influence of context effects on health outcomes: a systematic review. Lancet. 2001;357(9258):757-762.
7. O’Keeffe M, Purtill H, Kennedy N, et al. Comparative Effectiveness of Conservative Interventions for Nonspecific Chronic Spinal Pain: Physical, Behavioral/Psychologically Informed, or Combined? A Systematic Review and Meta-Analysis. J Pain. 2016 Jul;17(7):755-774.
8. Gabriel DA, Kamen G, Frost G. Neural adaptations to resistive exercise: mechanisms and recommendations for training practices. Sports Med. 2006;36(2):133-149
9. Bialosky JE, Beneciuk JM, Bishop MD, et al. Unraveling the Mechanisms of Manual Therapy: Modeling an Approach. J Orthop Sports Phys Ther. 2018;48(1):8-18.
10. Day MA, Ehde DM, Burns J, et al. A randomized trial to examine the mechanisms of cognitive, behavioral and mindfulness-based psychosocial treatments for chronic pain: Study protocol. Contemp Clin Trials. 2020;93:106000.