RCT Trial Banks: Supporting Evidence-based Practice Through Informatics

Origin: www.medrecinst.com
Date: October 2003

Ida Sim, MD, PhD¹
Kathleen M. Hunter, RN, PhD²

Randomized controlled trials (RCTs) are one of the best sources of evidence for the scientific practice of medicine, but substantial gaps exist between everyday practice and "best practice" as defined by RCTs.

In recent years, computer-based approaches to managing the clinical literature have become widespread, and hopes are high that information technology can help close the proof-to-practice gap and improve healthcare quality. However, because clinical research results usually are published only in text, and computers cannot read text articles, computers are effectively illiterate of the evidence they are supposed to help clinicians apply. Without knowing the clinical details of RCTs, today's decision support systems cannot deliver RCT evidence in ways that meet the information needs of practicing clinicians.

Individual clinicians struggle to keep up with clinical trial findings and new medical knowledge, but the lack of a coordinated plan for bringing that evidence from the literature to practice and the often inaccurate search systems available to clinicians make this an arduous task. It is no wonder that clinical practice often lags months and even years behind the best practice shown by science. Today's clinicians may best be served by information from systematic reviews that critically appraise and synthesize evidence from all relevant RCTs. However, systematic reviews also are difficult and time-consuming to perform because (1) study methods and results are incompletely reported and (2) information must be manually abstracted from text articles. Although reporting problems have decreased with the widely adopted CONSORT reporting recommendations, systematic reviewers, such as those from the Cochrane Collaboration, could benefit gready from corpplete databases of RCT information.

In summary, while billions of dollars are spent every year conducting RCTs, results of these RCTs are difficult for both clinicians and systematic reviewers to find, interpret, or apply to clinical practice. The result is an inefficient transfer of evidence from research to practice and missed opportunities for improving health outcomes.

The Trial Bank Project seeks to change the way clinical trials are published in the academic medical literature. In collaboration with the Annals of Internal Medicine and the Journal of the American Medical Association, authors of accepted trial manuscripts are asked to co-publish their trials in a machine-understandable electronic "trial bank" called RCT Bank. These co-published trials can be browsed at RCT Presenter (http://rctbank.ucsf.edu/Presenter/).

TRIAL BANKS

Definition

Trial banks are electronic databases that contain descriptions of the design, execution, and summary-level results of individual RCTs. Trial descriptions in a trial bank must be sufficiendy detailed and complete to support full evaluation of the trials' internal and external validity.

Therefore, trial bank reports have to be more comprehensive than the report written for journal publication. Extended comprehensiveness includes information on a trial's administration, design, subjects and recruitment, treatment assignment, interventions, follow-up, outcomes, measurements, and results. Trial banks differ from other trial databases such as the Cochrane Controlled Trials Register and clinicaltrials.gov (www.clinicaltrials.gov) in a crucial way: RCT Bank can capture all the information items necessary for systematic reviewing, while these other trial databases contain only sufficient information for recruiting subjects or for determining a trial's existence.

Fig. 1 Direct Links: RCT Presenter offers help for critiquing trials. Here links from the User's Guide to the Literature link directly to the relevant information for this trial.

Benefits

The sharing of a common data representation and a common clinical vocabulary is a critical aspect of trial banks. With these in place, an interoperating system will automatically route and map a user's query to all trial banks in the system-all interoperating trial banks will appear as one to users. The Trial Bank Project offers a first attempt at defIning a common trial- bank data representation.

Trial banks-either singly or when integrated with other information systems-can improve search precision to support users' interest in specific relevant studies, critical appraisal of retrieved studies, and application of RCT fIndings to patient care.

Integration of electronic health records with trial banks will benefit clinical practice through automatic notification to clinicians of new trial results pertinent to individual patients.

Electronic health record integration with trial banks also will provide the evidence upon which decision-support systems will base their recommendations. Trial banks will be part of a critical informatics infrastructure for computer-assisted evidence-based practice.

Researchers will benefit by being able to pose questions about RCTs that are of much larger scope than is possible today: What are the sample sizes and powers of National Institutes of Health (NIH)-funded trials? What is the most common definition of sudden cardiac death among cardiology trials?

Systematic reviewers will become some of the most important users of RCT reports, while clinicians will turn increasingly to the results of systematic reviews for their primary source of research evidence.

SYSTEMATIC REVIEWS

Systematic reviews employ a methodology for minimizing bias in the retrieval and interpretation of scientific literature. If the reviews include a quantitative synthesis of trial results, they are called quantitative overviews or meta-analyses. The methodology in systematic reviews encompasses five core tasks: trial retrieval, judgment of internal validity, judgment of generalizability, quantitative synthesis, and contextual interpretation. In the Trial Bank Project, these tasks are decomposed into subtasks, methods, and information needs, which determine the concepts in the RCT Bank data model.

Trial Retrieval

This task is concerned with the retrieval of all trials relevant to the overall study question. It can be divided into (1) query capture and (2) query matching to the trials in the trial bank. Information needed to develop relevant queries and achieve comprehensive retrieval includes criteria for trial inclusion and exclusion, enrollee clinical characteristics, the interventions, and the outcome definitions.

Judgment of Internal Validity

This is an assessment of the extent to which a trial's design and execution will yield unbiased estimates of the true underlying effect. Specific information elements include:

Appropriateness of the trial's statistical design

Degree of intervention assignment bias

Extent to which the intervention groups were comparable

Degree of any intervention-related bias

Presence of co-interventions that may have confounded the results

Meaningful outcome variables

Extent of any outcome assessment or measurement bias

Presence of follow-up bias

Appropriate analysis of results

Presence of biases introduced by trial personnel

Fig. 2: The main RCT Presenter screen

Judgment of Generalizability

Research synthesizers determine if a trial's results can be generalized to one or more settings different from that of the original trial.

Quantitative Synthesis If a set of clinical trials address a similar research question and are of sufficient statistical and clinical similarity, statistical techniques such as meta-analysis can be used to synthesize the quantitative results to increase the statistical power and precision for detecting therapeutic effects.

 Contextual Interpretation

During research synthesis, the description and outcomes of a clinical trial are interpreted within the full scientific, ethical, and socioeconomic context of the condition under investigation.

CONSORT PLUS

Randomized trials are the gold standard experiment for clinical knowledge, yet journal reports of trials are often incomplete and ambiguous. The CONSORT statement is an important research tool that takes an evidence-based approach to improve the quality of reports of randomized trials. The statement is available in six languages and has been endorsed by prominent medical journals such as The Lancet, Annals of Internal Medicine, and the Journal of the American MedicalAssociation. Its critical value to researchers, healthcare providers, peer reviewers, journal editors, and health policy makers is the guarantee of integrity in the reported results of research.

CONSORT comprises a checklist and flow diagram to help improve the quality of reports of randomized controlled trials. It offers a standard way for researchers to report trials. The checklist includes items, based on evidence, that need to be addressed in the report; the flow diagram provides readers with a clear picture of the progress of all participants in the trial, from the time they are randomized until the end of their involvement. The intent is to make the experimental process clearer, flawed or not, so that users of the data can more appropriately evaluate its validity for their purposes.

CONSORT Plus is an extension of the CONSORT requirements to trial-bank publishing. CONSORT Plus is more extensive than CONSORT, and it imposes data integrity constraints that are not possible in text-based reporting. Below are the concept categories of the items in the CONSORT Plus guidelines. To see the details on individual items, go to the web site: http://rctbank.ucsf.edu/consort/cplus.html.

Administration

Study Design

Subjects and Recruitment

Randomization and Treatment Assignment

Interventions

Outcomes and Analysis

Outcomes Assessment

Follow up

Results

Publications

Fig. 3 Summary Report: Clicking on the Summary Report link in the main page bring ups a summary of the trial

RCT SCHEMA

The data model for RCT Bank (called the RCT Schema) is guaranteed to contain all the trial information needed for rigorously applying RCT evidence to practice. As noted earlier, this model is derived from a task analysis of systematic reviewing.

§          Root

o       Concept

§         Intervention-concept

§         Intervention-logic

§         Conditional-logic

§         Simple-logic

§         Blocking-logic

§         Option-logic

§         Chain-logic

§         Compliance-result

§         Intervention

§         Placebo-or-sham

§         No-treatment

§         Usual-care

§         Drug

§         Device

§         Procedure

§         Other-intervention

§         Cointervention

§         Intervention-arm

§         Experimental-arm

§         Comparison-arm

§         Intervention-step

§         Non-drug-intervention-step

§         Drug-step

§         Compliance-details

§         Blinding-concept

§         Blinding-efficacy

§         Blinding-methods

§         Blinding

§         Administrative-concept

§         Registry-id

§         Institution

§         Funder

§         Study-committee

§         Ethics

§         Person

§         Investigator

§         Trial-subject

§         Study-site

§         Publication-concept

§         Publication

§         Journal-publication

§         Abstract

§         Text-subsection

§         Pub-uid

§         Authorship

§         Outcome-concept

§         Outcome

§         Study-outcome

§         Side-effect

§         Ancillary-outcome

§         Secondary-outcome

§         Primary-outcome

§         Baseline

§         Outcome-value-entity

§         Regression-variable-value

§         Cox-variable

§         Other-value

§         Proportion-value

§         Continuous-value

§         Categorical-value

§         Count-value

§         Result-entity

§         Subgroup-regression

§         Regression-analysis-and-results

§         Datapoint-entity

§         Summary-datapoint

§         Baseline-datapoint

§         Blinding-efficacy-datapoint

§         Kaplan-meier-datapoint

§         Generic-datapoint

§         Datapoint-of-pop

§         Datapoint-of-crossover-population

§         Cointervention-datapoint

§         Single-time-x-comparison

§         All-comparisons-at-time-x

§         Survival-analysis-and-results

§         Kaplan-meier-timepoint

§         Miscellaneous-outcome-entity

§         Range

§         Category

§         Outcome-assessment

§         Special-variable-information

§         Cost

§         Categorical

§         Scored-instrument

§         Life-year

§         Rate

§         Design-concept

§         Stopping-rule

§         Regression-analysis-and-results

§         Study-monitoring

§         Study-objective

§         Hypothesis-concept

§         Secondary-hypothesis

§         Primary-hypothesis

§         Trial-design

§         Sample-size-calculation

§         Statistical-analysis-and-results

§         Survival-analysis-and-results

§         Protocol-concept

§         Secondary-study-protocol

§         Executed-secondary-study-protocol

§         Intended-secondary-study-protocol

§         Outcomes-followup

§         Reason

§         Withdrawal-reason

§         Reason-off-assigned-intervention

§         Reason-not-eligible

§         Reason-not-enrolled

§         Reason-not-randomized

§         Reason-excluded-postrand

§         Reason-outcome-not-assessed

§         Protocol

§         Intended-protocol

§         Executed-protocol

§         Protocol-change

§         Follow-up

§         Follow-up-activity

§         Follow-up-compliance

§         Treatment -assignment-concept

§         Treatment-assignment

§         Randomization

§         Fixed-randomization

§         Adaptive-randomization

§         Population-concept

§         Site-enrollment

§         Recruitment

§         Population

§         Analyzed-population

§         Subgroup-population

§         Study-arm-population

§         Crossover-population

§         All-subjects

§         Excluded-population

§         Not-eligible-population

§         Not-enrolled-population

§         Not-randomized-population

§         Excluded-postrand-population

§         Recruited-population

§         Screened-population

§         Eligible-population

§         Enrolled-population

§         Randomized-population

§         Recruitment-flowchart

§         Primary-recruitment-flowchart

§         Subgroup

§         Generic-concept

§         Situation

§         State

§         Non-concurrent-situations

§         Concurrent-situations

§         Event-entity

§         Regularly-recurring-event

§         Single-or-irregular-event

§         Rule-concept

§         Age-gender-rule

§         Ethnicity-language-rule

§         Rule-entity

§         Base-rule

§         Recursive-rule

§         Clinical-rule

§         Inclusion-rule

§         Exclusion-rule

§         Time-entity

§         Date

§         Time-range

§         Duration

§         Fuzzy-duration

§         Interval

§         Single-anchored-interval

§         Double-anchored-interval

§         Anchored-time

§         Timepoint

§         Interval

§         Single-anchored-interval

§         Double-anchored-interval

§         Term-information

o       Trial

o       Trial-details

§         Fraud-details

§         Correction-details

§         Retraction-details

§         Stopping-details

§         Background-details

§         Conclusion-details

§         Administrative-details

§         Primary-administrative-details

§         Secondary-administrative-details

§         Trial-entry-details

§         Publication-details

§         Erratum

o       Secondary-study

RCT Schema is an Ocelot frame-based ontology consisting of 188 frames and 601 unique slots. The schema is capable of capturing multi-armed trials and a wide range of interventions (e.g., drugs, procedures, devices, behavioral interventions) and outcomes (e.g., dichotomous, continuous, categorical, survival, cost) in any clinical discipline. RCT Schema cannot yet capture crossover, follow up, and clustered randomization trials, but RCT Schema is continuously being enhanced.

SUMMARY

RCT Bank captures essential RCT information into an open-access, electronic knowledge base that is specifically designed to support rigorous, computer-assisted, evidence-based practice. Just as we need to capture clinical data into computer-understandable electronic medical records to facilitate clinical decision support and improve healthcare quality, so too do we need to capture RCT evidence into computer-understandable trial banks. The new publishing model of trial bank publishing is founded on the principle that scientific knowledge should be disseminated in the form that best facilitates its use; simply publishing science in electronic text is not good enough anymore. The Trial Bank Project seeks to fundamentally change the way RCT evidence is disseminated to the scientific and clinical community to provide a foundation for advanced decision support that can help us translate evidence to practice more efficiently and effectively.

ENDNOTES

¹ Ida Sim, MD, PhD is Assistant Professor of Medicine, and Associate Director for Medical Infonnatics, Program in Biological and Medical Informatics, University of California, San Francisco (sim@medicine.ucsf.edu)

 ² Kathleen Hunter, RN, PhD is Managing Editor, Health IT Advisory Report and President, K&D Hunter Associates, Inc. in Lithia, FL (dmillho1@tampabay.rr.com)

 ³ Trial Banks: An Informatics Infrastructure for Evidence-Based Medicine, 2003, http://rctbank.ucsf.edu/home/introduction.html