Preclinical Reproducibility

The reproducibility rate of preclinical is studies are continuing to garner increased attention as they are unacceptably low. By some sources 51%-89% of preclinical studies are not reproducible.

Considering how many resources are pouring into preclinical research these rates are unacceptable. Preclinical studies currently cost $56.4 billion dollars annually accounting for 49% of all life science research in the united states annually.

When combined these statistics reveal that over $28 billion dollars annually are spent on irreproducible preclinical studies.

Clinical trial difficulties

  • Only 11% of phase one trials move forward, 5% for cancer and 8% for neurological disease
  • 45% of phase II/III fail
  • Most promising preclinical trials show resistance to translation
  • Main cause is insufficient activity

Effect of clinical trial failure on patients

  • Creates false hope for patients
  • Exposes them to physical risk

Institutional effects

  • Undermines integrity of institutions of investigators
  • Undermines scientific peer review process

Drug development effects

  • Increases cost
  • Lengthens development timeline

Causes

  • Negative/controversial results less likely to be reported
  • Authors are the largest contributors of publication bias, because of ‘lack of interest’ especially when results are negative or
  • Journal editors have difficulties getting negative or nonsignificant papers reviewed by external reviewers and tend to select papers for publication that are most exciting or papers that make the journal “look good.”
  • Lack of GCP
  • Inaccuracies imported from preclinical studies

Solutions

  • Clinical registrations
  • The declaration of Helsinki – Ethical Principles for Medical Research Involving Human Subjects, requires trial registration for all studies concerning human subjects, ‘Every clinical trial must be registered in a publicly accessible database before recruitment of the first subject’.
  • GLP
  • Audit preclinical irreproducibilityeconomics-of-irreproducibility

    Preclinical irreproducibility

    • 49% of all life science research spending
    • $28B spent annually on irreproducible results

    Sources of irreproducibility

    • 36% Biological reagents
    • 28% Study design
    • 25% Data analysis and reporting
    • 11% Lab protocols

    Effects of data analysis and reporting

    • In animal studies of stroke, publication bias relatively overestimates the efficacy of interventions by 31.1%
    • Unpublished studies mask distort effect size
    • Unpublished studies lead to unnecessary replication of experiments

    Justification for preclinical registration

    • Bring standards more inline with clinical trial registration
    • Responsibility to protect the animal subjects from undue harm
    • Protects both institutions and individuals downstream who rely on preclinical data

    Three main objections

    • It jeopardizes the strategic advantage of drug developers by enabling free-riding on investments in clinical development.
    • If the logic supporting good disclosure practice extends to preclinical studies, does it extend to basic science as well?
    • Cost to investigators

    Answers to objections

    • Moratorium on the release of registered data for a predetermined amount of time
    • Standards set for registration to only include preclinical In Vivo models
    • Cost could be shifted to government regulatory agencies and funding agencies and away from investigators.