Tag Archives: NIH Collaboratory

STOP CRC Trial: Analytic Challenges and Pragmatic Solutions


Investigators from the STOP CRC pragmatic trial, an NIH Collaboratory Demonstration Project, have recently published an article in the journal eGEMs describing solutions to issues that arose in the trial’s implementation phase. STOP CRC tests a program to improve colorectal cancer screening rates in a collaborative network of Federally Qualified Health Centers by mailing fecal immunochemical testing (FIT) kits to screen-eligible patients at clinics in the intervention arm. Clinics in the control arm provided opportunistic colorectal-cancer screening to patients at clinic visits in Year 1 and implemented the intervention in Year 2. In this cluster-randomized trial, clinics are the unit of analysis, rather than individual patients, with the primary outcome being the proportion of screen-eligible patients at each clinic who complete a FIT.

The team dealt with various challenges that threatened the validity of their primary analysis, one of which related to potential contamination of the primary outcome due to the timing of the intervention rollout: for control participants, the Year 2 intervention actively overlapped with the Year 1 control measurements. The other challenge was due to a lack of synchronization between the measurement and accrual windows. To deal with these issues, the team had to slightly modify the study design in addition to developing a few sensitivity analyses to better estimate the true impact of the intervention.

“While the nature of the challenges we encountered are not unique to pragmatic trials, we believe they are likely to be more common in such trials due to both the types of designs commonly used in such studies and the challenges of implementing system-based interventions within freestanding health clinics.” (Vollmer et al. eGEMs 2015)

The Publish EDM Forum Community publishes eGEMs (generating evidence & methods to improve patient outcomes) and provides free and open access to this methods case study. Readers can access the article here.

Applying PRECIS Ratings to Collaboratory Pragmatic Trials

A new article published in the journal Trials provides a look at how the  Pragmatic–Explanatory Continuum Indicator Summary, or PRECIS, rating system can be applied to clinical trials designs in order to examine where a given study sits on the spectrum of explanatory versus pragmatic clinical trials.

The PRECIS-2 criteria are used to rate study designs as more or less “pragmatic” according to multiple domains that include participant eligibility, recruitment methods, setting, organization, analysis methods, primary outcomes, and more. In this context, “pragmatic” refers to trials that are designed to study a therapy or intervention in a “real world” setting similar or identical to the one in which the therapy will actually be used. Pragmatic trials stand in contrast to explanatory trials, which are typically designed to demonstrate the safety and efficacy of an intervention under highly controlled conditions and in carefully selected groups of participants, but which may also be difficult to generalize to larger or more varied populations.

Schematic of PRECIS-2 Wheel used to evaluate where a given trial design resides upon the explanatory-pragmatic spectrum.
PRECIS-2 Wheel.  Kirsty Loudon et al. BMJ 2015;350:bmj.h2147. Copyright 2015 by British Medical Journal Publishing Group. Used by permission.

Clinical trials are almost never wholly “explanatory” or wholly “pragmatic.” Instead, many studies exist somewhere on a spectrum between these two categories. However, understanding how these different attributes apply to trials can help researchers design studies that are optimally fit for purpose, whether that purpose is to describe a biological mechanism (as in an explanatory trial) or to show how effective an intervention is when used across a broad population of patients (as in a pragmatic trial).

In their article in Trials, authors Karin Johnson, Gila Neta, and colleagues  applied PRECIS-2 criteria to 5 pragmatic clinical trials (PCTs) being conducted through the NIH Collaboratory. Each trial was found to rate as “highly pragmatic” across the multiple PRECIS-2 domains, highlighting the tool’s potential usefulness in guiding decisions about study design, but also revealing a number of challenges in applying it and interpreting the results.

Study authors Johnson and Neta will be discussing their findings during the NIH Collaboratory’s Grand Rounds on Friday, January 22, 2016 (an archived version of the presentation will be available the following week).


Johnson KE, Neta G, Dember LM, Coronado GD, Suls J, Chambers DA, Rundell S, Smith DH, Liu B, Taplin S, Stoney CM, Farrell MM, Glasgow RE. Use of PRECIS ratings in the National Institutes of Health (NIH) Health Care Systems Research Collaboratory. Trials. 2016;17(1):32. doi: 10.1186/s13063-016-1158-y. PMID: 26772801. PMCID: PMC4715340.
You can read more about the NIH Collaboratory PCTs featured as part of this project at the following links:

ABATE (Active Bathing to Eliminate Infection)

LIRE (A pragmatic trial of Lumbar Image Reporting with Epidemiology)

PPACT (Collaborative Care for Chronic Pain in Primary Care)

STOP-CRC (Strategies & Opportunities to Stop Colon Cancer in Priority Populations)

TIME (Time to Reduce Mortality in End-Stage Renal Disease)
Additional Resources

An introductory slide set on PCTs (by study author Karin Johnson) is available from the Living Textbook:

Introduction to Pragmatic Clinical Trials

The University of Colorado Denver - Anschutz Medical Campus publishes an electronic textbook on pragmatic trials:

Pragmatic Trials: A workshop Handbook

 

 

 

New Lessons Learned Document Draws on Experiences of Demonstration Projects

The NIH Collaboratory’s Health Care Systems Interactions Core has published a document entitled Lessons Learned from the NIH Health Care Systems Research Collaboratory Demonstration Projects. The Principal Investigators of each of the Demonstration Projects shared their trial-specific experience with the Core to develop the document, which presents problems and solutions for initiation and implementation of pragmatic clinical trials (PCTs). Lessons learned are divided into the following categories: build partnerships, define clinically important questions, assess feasibility, involve stakeholders in study design, consider institutional review board and regulatory issues, and assess potential issues with biostatistics and the analytic plan.

Other tools available from the Health Care Systems Interactions Core include a guidance document entitled Considerations for Training Front-Line Staff and Clinicians on Pragmatic Clinical Trial Procedures and an introduction to PCTs slide set.

New Living Textbook Chapter on Acquiring and Using Electronic Health Record Data for Research

Topic ChaptersMeredith Nahm Zozus and colleagues from the NIH Collaboratory’s Phenotypes, Data Standards, and Data Quality Core have published a new Living Textbook chapter about key considerations for secondary use of electronic health record (EHR) data for clinical research.

In contrast to traditional randomized controlled clinical trials where data are prospectively collected, many pragmatic clinical trials use data that were primarily collected for clinical purposes and are secondarily used for research. The chapter describes the steps a prospective researcher will take to acquire and use EHR data:

  • Gain permission to use the data. When a prospective researcher wishes to use data, a data use agreement (DUA) is usually required that describes the purpose of the research and the proposed use of the data. This section also describes use of de-identified data and limited data sets.
  • Understand fundamental differences in context. Data collected in routine care settings reflect standard procedures at an individual’s healthcare facility, and are not collected in a standard, structured manner.
  • Assess the availability of health record data. Few assumptions can be made about what is available from an organization’s healthcare records; up-front, detailed discussions about data element collection over time at each facility is required.
  • Understand the available data. A secondary data user must understand both the data meaning and the data quality; both can vary greatly across organizations and affect a study’s ability to support research conclusions.
  • Identify populations and outcomes of interest. Because healthcare facilities are obligated to provide only the minimum necessary data to answer a research question, investigators must identify the needed patients and data elements with specificity and sensitivity to answer the research question given the available data.
  • Consider record linkage. Studies using data from multiple records and sources will require matching data to ensure they refer to the correct patient.
  • Manage the data. The investigator is responsible for receiving, managing, and processing data and must demonstrate that the data are reproducible and support research conclusions.
  • Archive and share the data after the study. Data may be archived and shared to ensure reproducibility, enable auditing for quality assurance and regulatory compliance, or to answer other questions about the research.

Lessons Learned from NIH Collaboratory Pragmatic Trials


Earlier this year, the NIH Collaboratory conducted a series of interviews with the principal investigators of its first round of pragmatic clinical trial Demonstration Projects. These projects completed a pilot phase before scaling up to full implementation in 2014-2015. The purpose of the interviews was to share challenges and lessons learned that may help future pragmatic trials. The interviews have now been archived on the Living Textbook:

Demonstration Project Interviews

The NIH Collaboratory will continue to share information learned as these trials progress. Regular updates are also provided during weekly Grand Rounds webinars. For more information on these trials, visit the Demonstration Project websites.


Special Issue Published on Ethical & Regulatory Complexities of Pragmatic Clinical Trials


Tools for ResearchA new series of 12 articles published in a special issue of the journal Clinical Trials addresses ethical and regulatory challenges particular to pragmatic clinical research. Pragmatic clinical trials are designed to efficiently provide answers to important clinical questions, yet they present special challenges in conforming to the ethical and regulatory guidelines that were developed for more traditional clinical research. The special issue describes these challenges and begins to outline possible solutions that will protect the rights and welfare of research participants while allowing pragmatic clinical trials to gather much-needed evidence for informing healthcare decisions. An introductory article is followed by 11 articles addressing individual topics, such as alteration of informed consent, privacy, gatekeepers, and defining minimal risk research. The effort was funded by the NIH Health Care Systems Research Collaboratory, with additional support from the Patient-Centered Outcomes Research Institute (PCORI), and involved diverse groups of stakeholders, including researchers, patient advocates, bioethicists, and regulatory experts. Robert M. Califf, MD, and Jeremy Sugarman, MD, MPH, were editors of the special issue.

For more information:


Minimizing the Burden of Practical Research: Case Studies from the NIH Collaboratory


In a forthcoming article in Healthcare, Dr. Eric Larson and colleagues present practical advice based on case studies from the National Institutes of Health (NIH) Health Care Systems Research Collaboratory. Physician–scientists, health services researchers, and delivery system leaders provide insight from their experience launching a pragmatic clinical trial (PCT) as part of the Collaboratory.

The authors make 5 recommendations:

  • Establish a partnership from the get-go
  • Do a pilot project
  • Take advantage of existing hospital and health system infrastructure
  • Minimize the impact on clinical workflow
  • Remember that even high-priority research questions must be balanced with the systems’ greatest priority: providing good healthcare to patients.

The authors note that researchers need to be flexible and prepared to adjust the study design to the workflow and culture of the system.

Reference: Larson E, Tachibana C, Thompson E, et al. Trials without tribulations: Minimizing the burden of pragmatic research on healthcare systems. Healthcare. 2015; in press. doi:10.1016/j.hjdsi.2015.07.005

Watch Dr. Larson’s Grand Rounds Presentation from June 2013: Trials, Not Tribulations: Minimizing the Burden of Research on Health Care Systems


Closer Partnerships Needed Between Researchers and Healthcare Executives


Larson-JohnsonIn a commentary published this week in Modern Healthcare, Eric Larson, MD, MPH, and Karin Johnson, PhD, of Group Health Research Institute, argue that greater collaboration is needed between clinical researchers and healthcare system executives to address a “a gap between research approaches and delivery system needs.” Perspectives gathered through a survey and Institute of Medicine workshop with healthcare executives indicated that research is not conducted fast enough or designed in a way that facilitates translation of evidence into clinical practice. The NIH Collaboratory and National Patient-Centered Clinical Research Network (PCORnet) are cited as examples of effective partnerships between researchers and healthcare leaders; these research programs are addressing high-priority clinical questions and generating actionable knowledge. According to Drs. Larson and Johnson, pragmatic clinical trials and big data offer opportunities to create a learning health system, but this will require combining the perspectives and expertise of researchers and stakeholders from healthcare delivery systems. Drs. Larson and Johnson are part of the NIH Collaboratory’s Health Care Systems Interactions Core, a working group that “aims to support and facilitate productive collaboration between researchers, clinicians, and health system leaders.”

Related links:


Groundbreaking Suicide Prevention Trial has Enrolled Initial Patients

March 5, 2015

Dr. Greg Simon and the Suicide Prevention Team have enrolled the first participants in the Pragmatic Trial of Population-Based Programs to Prevent Suicide Attempt. This groundbreaking study was developed by researchers at Group Health Cooperative in Seattle, Washington, Health Partners Medical Group in Minnesota, and Kaiser Permanente of Colorado, in collaboration with patients who have experienced suicidal thoughts or survived suicide attempts themselves.

Over 9 million adults in the United States experience suicidal thoughts, and more than 1 million adults attempt suicide each year. However, patients at risk for suicidal behavior are not routinely identified, and successful interventions for depression and suicide are not routinely implemented. New evidence suggests that patients who report frequent thoughts of death or self-harm on a commonly-used depression questionnaire are at higher risk for suicide attempt and death over the following year.

This study aims to address the significant problem of suicide by identifying patients who are at risk for suicidal behavior and testing two suicide prevention strategies. Patients at participating institutions will complete a standard depression severity questionnaire during routine clinical care, and the results will be stored in their electronic health records (EHR). Investigators will use the responses in the EHR to identify at-risk individuals, and once identified, the patients will be randomly assigned to either usual care or to two treatment programs. The first is a collaborative care-management approach; the second is an online skills training program called “Now Matters Now,” which is designed to help people manage painful emotions and stressful situations.

Over the next 5 years, the team plans to enroll 19,500 adult patients. The study is an NIH Collaboratory Demonstration Project being overseen by the National Institute of Mental Health (NIMH).

Congratulations to Dr. Simon and his team for their achievement!

 

PCORnet “Writing for Research” Webinars Available via Living Textbook


“Writing for the Clinical Research Setting, “A 4-part webinar series sponsored by the National Patient-Centered Clinical Research Network (PCORnet) is now available on Rethinking Clinical Trials. PCORnet Originally given in the fall of 2014 as part of the PCORnet “Office Hours” series, the recorded sessions are presented by Living Textbook managing editor Jonathan McCall, MS, and can be accessed as streaming video under the Tools for Research tab, or directly here.

The series provides a basic introduction to various facets of writing for the clinical research environment. Individual sessions, each roughly 1 hour in length, include the following topics:

  • Writing Peer-Reviewed Research Articles
  • Organizing and Writing Your White Paper
  • Writing Guidance Documents
  • Managing the Process of Writing and Publication