The American Journal of Bioethics has recently published three articles authored by members of the Regulatory/Ethics core group describing various questions related to research on medical practices:
Patients’ views concerning research on medical practices: implications for consent(Weinfurt et al. 2015) describes the results of focus group sessions that elicited a range of patients’ views and opinions about different types of research on usual medical practices. The authors state that “our data suggest that effective policy and guidance will involve balancing different patients’ interests and potentially different sets of interests for different types of research studies on usual medical practices.”
Ethics of research in usual care settings: data on point(Sugarman 2016) introduces a special five-article supplement in the American Journal of Bioethics, stating that the “growing empirical ethics literature regarding research in usual care settings provides data to inform conceptual and policy debates regarding this research and suggests areas that require further study.”
These publications were supported by a bioethics supplement awarded to the Regulatory/Ethics Core group by the NIH’s Office of the Director.
Investigators from the STOP CRC pragmatic trial, an NIH Collaboratory Demonstration Project, have recently published an article in the journal eGEMs describing solutions to issues that arose in the trial’s implementation phase. STOP CRC tests a program to improve colorectal cancer screening rates in a collaborative network of Federally Qualified Health Centers by mailing fecal immunochemical testing (FIT) kits to screen-eligible patients at clinics in the intervention arm. Clinics in the control arm provided opportunistic colorectal-cancer screening to patients at clinic visits in Year 1 and implemented the intervention in Year 2. In this cluster-randomized trial, clinics are the unit of analysis, rather than individual patients, with the primary outcome being the proportion of screen-eligible patients at each clinic who complete a FIT.
The team dealt with various challenges that threatened the validity of their primary analysis, one of which related to potential contamination of the primary outcome due to the timing of the intervention rollout: for control participants, the Year 2 intervention actively overlapped with the Year 1 control measurements. The other challenge was due to a lack of synchronization between the measurement and accrual windows. To deal with these issues, the team had to slightly modify the study design in addition to developing a few sensitivity analyses to better estimate the true impact of the intervention.
“While the nature of the challenges we encountered are not unique to pragmatic trials, we believe they are likely to be more common in such trials due to both the types of designs commonly used in such studies and the challenges of implementing system-based interventions within freestanding health clinics.” (Vollmer et al. eGEMs 2015)
The Publish EDM Forum Community publishes eGEMs (generating evidence & methods to improve patient outcomes) and provides free and open access to this methods case study. Readers can access the article here.
A new article published in the journal Trials provides a look at how the Pragmatic–Explanatory Continuum Indicator Summary, or PRECIS, rating system can be applied to clinical trials designs in order to examine where a given study sits on the spectrum of explanatory versus pragmatic clinical trials.
The PRECIS-2 criteria are used to rate study designs as more or less “pragmatic” according to multiple domains that include participant eligibility, recruitment methods, setting, organization, analysis methods, primary outcomes, and more. In this context, “pragmatic” refers to trials that are designed to study a therapy or intervention in a “real world” setting similar or identical to the one in which the therapy will actually be used. Pragmatic trials stand in contrast to explanatory trials, which are typically designed to demonstrate the safety and efficacy of an intervention under highly controlled conditions and in carefully selected groups of participants, but which may also be difficult to generalize to larger or more varied populations.
PRECIS-2 Wheel. Kirsty Loudon et al. BMJ 2015;350:bmj.h2147. Copyright 2015 by British Medical Journal Publishing Group. Used by permission.
Clinical trials are almost never wholly “explanatory” or wholly “pragmatic.” Instead, many studies exist somewhere on a spectrum between these two categories. However, understanding how these different attributes apply to trials can help researchers design studies that are optimally fit for purpose, whether that purpose is to describe a biological mechanism (as in an explanatory trial) or to show how effective an intervention is when used across a broad population of patients (as in a pragmatic trial).
In their article in Trials, authors Karin Johnson, Gila Neta, and colleagues applied PRECIS-2 criteria to 5 pragmatic clinical trials (PCTs) being conducted through the NIH Collaboratory. Each trial was found to rate as “highly pragmatic” across the multiple PRECIS-2 domains, highlighting the tool’s potential usefulness in guiding decisions about study design, but also revealing a number of challenges in applying it and interpreting the results.
Study authors Johnson and Neta will be discussing their findings during the NIH Collaboratory’s Grand Rounds on Friday, January 22, 2016 (an archived version of the presentation will be available the following week).
Johnson KE, Neta G, Dember LM, Coronado GD, Suls J, Chambers DA, Rundell S, Smith DH, Liu B, Taplin S, Stoney CM, Farrell MM, Glasgow RE. Use of PRECIS ratings in the National Institutes of Health (NIH) Health Care Systems Research Collaboratory. Trials. 2016;17(1):32. doi: 10.1186/s13063-016-1158-y. PMID: 26772801. PMCID: PMC4715340.
You can read more about the NIH Collaboratory PCTs featured as part of this project at the following links:ABATE (Active Bathing to Eliminate Infection)
LIRE (A pragmatic trial of Lumbar Image Reporting with Epidemiology)
PPACT (Collaborative Care for Chronic Pain in Primary Care)
STOP-CRC (Strategies & Opportunities to Stop Colon Cancer in Priority Populations)
TIME (Time to Reduce Mortality in End-Stage Renal Disease)
Additional Resources
An introductory slide set on PCTs (by study author Karin Johnson) is available from the Living Textbook:
Introduction to Pragmatic Clinical Trials
The University of Colorado Denver - Anschutz Medical Campus publishes an electronic textbook on pragmatic trials:
Pragmatic Trials: A workshop Handbook
The NIH Collaboratory’s Health Care Systems Interactions Core has published a document entitled Lessons Learned from the NIH Health Care Systems Research Collaboratory Demonstration Projects. The Principal Investigators of each of the Demonstration Projects shared their trial-specific experience with the Core to develop the document, which presents problems and solutions for initiation and implementation of pragmatic clinical trials (PCTs). Lessons learned are divided into the following categories: build partnerships, define clinically important questions, assess feasibility, involve stakeholders in study design, consider institutional review board and regulatory issues, and assess potential issues with biostatistics and the analytic plan.
In contrast to traditional randomized controlled clinical trials where data are prospectively collected, many pragmatic clinical trials use data that were primarily collected for clinical purposes and are secondarily used for research. The chapter describes the steps a prospective researcher will take to acquire and use EHR data:
Gain permission to use the data. When a prospective researcher wishes to use data, a data use agreement (DUA) is usually required that describes the purpose of the research and the proposed use of the data. This section also describes use of de-identified data and limited data sets.
Understand fundamental differences in context. Data collected in routine care settings reflect standard procedures at an individual’s healthcare facility, and are not collected in a standard, structured manner.
Assess the availability of health record data. Few assumptions can be made about what is available from an organization’s healthcare records; up-front, detailed discussions about data element collection over time at each facility is required.
Understand the available data. A secondary data user must understand both the data meaning and the data quality; both can vary greatly across organizations and affect a study’s ability to support research conclusions.
Identify populations and outcomes of interest. Because healthcare facilities are obligated to provide only the minimum necessary data to answer a research question, investigators must identify the needed patients and data elements with specificity and sensitivity to answer the research question given the available data.
Consider record linkage. Studies using data from multiple records and sources will require matching data to ensure they refer to the correct patient.
Manage the data. The investigator is responsible for receiving, managing, and processing data and must demonstrate that the data are reproducible and support research conclusions.
Archive and share the data after the study. Data may be archived and shared to ensure reproducibility, enable auditing for quality assurance and regulatory compliance, or to answer other questions about the research.
Gloria Coronado, PhD, and Beverly Green, MD, MPH, Principal Investigators, STOP CRC Trial
Drs. Beverly Green and Gloria Coronado and colleagues have published an article in Clinical Trialsdescribing the challenges of recruiting participants into large, multisite pragmatic clinical trials—particularly at the health system level. STOP CRC is one of the NIH Collaboratory’s pragmatic clinical trial Demonstration Projects, which are intended to provide a framework of implementation methods and best practices to enable participation of varied health care systems in clinical research.
STOP CRC is testing a culturally tailored, health care system–based program to improve colorectal cancer screening rates in a community-based collaborative network of federally qualified health centers. The authors observed that recruiting sites to participate in pragmatic trials is time-intensive and involves both preparing materials and organizing face-to-face meetings with staff and clinic leaders. Yet little is known about the characteristics of nonparticipating sites and clinic-level factors that may influence willingness to participate in a pragmatic trial.
“Our findings underscore the importance of assessing and reporting recruitment success at the organizational and/or clinic level in order to know the external validity of the findings and may inform future efforts to select and recruit health systems to participate in pragmatic research.” (Coronado, et al. Clin Trials 2015)
The NIH Collaboratory’s Health Care Systems Interactions Core has published a guidance document entitled Considerations for Training Front-Line Staff and Clinicians on Pragmatic Clinical Trial Procedures. The purpose of this guidance is to help pragmatic clinical trial (PCT) teams plan training for study procedures that involve front-line clinicians and staff. The content was developed by drawing on trial-specific experience from the NIH Collaboratory Demonstration Projects. The document describes how training for PCTs will differ from training conducted for typical research studies, and includes a list of specific considerations, real-world examples, a checklist for PCT training design, and links to additional resources.
Other tools available from the Health Care Systems Interactions Core include an introduction to PCTs slide set.
LIRE is studying the effect of inserting epidemiologic benchmarks for common imaging findings into lumbar spine imaging reports being delivered to primary care physicians. The primary goal is to measure whether the intervention reduces subsequent spine-related tests and treatments. All outcomes are captured passively through the electronic health record. The authors state that if successful, such a low-cost intervention could potentially be applied to diagnostic tests for other conditions. LIRE has a projected sample size of more than 160,000 patients across an estimated >2000 primary care physicians at 4 health systems. Enrollment will continue through 2016.
“LIRE is a pragmatic cluster randomized trial of a minimal-risk intervention that we believe can serve as a model for future pragmatic trials.”
(Jarvik JG, et al. Contemp Clin Trials 2015)
Earlier this year, the NIH Collaboratory conducted a series of interviews with the principal investigators of its first round of pragmatic clinical trial Demonstration Projects. These projects completed a pilot phase before scaling up to full implementation in 2014-2015. The purpose of the interviews was to share challenges and lessons learned that may help future pragmatic trials. The interviews have now been archived on the Living Textbook:
The NIH Collaboratory will continue to share information learned as these trials progress. Regular updates are also provided during weekly Grand Rounds webinars. For more information on these trials, visit the Demonstration Project websites.
Few clinical trials are entirely explanatory (done in an idealized setting) or entirely pragmatic (done in a usual-care setting); rather, trials are situated somewhere along a continuum of applicability. Pragmatic clinical trials are trials designed with pragmatic qualities and are intended to inform decision makers, including patients, clinicians, administrators, and policymakers, about the relative benefits, burdens, and risks of a health intervention.
To help trialists assess how closely their trial’s design matches its intended purpose, a group of trialists and methodologists developed a design tool, the Pragmatic–Explanatory Continuum Indicator Summary, or PRECIS. Originally implemented in 2008, the wheel-shaped indicator tool recently underwent a revision, leading to PRECIS-2. The revised, validated tool guides trialists to prospectively consider the design of their trial along 9 domains: eligibility criteria, recruitment, setting, organization, flexibility (delivery), flexibility (adherence), follow-up, primary outcome, and primary analysis.
PRECIS-2 Wheel*
*Kirsty Loudon et al. BMJ 2015;350:bmj.h2147. Copyright 2015 by British Medical Journal Publishing Group. Used by permission.
