Evidence-based practice for researchers concentrates on the types, quality, and conduct of research studies that gather evidence that may be informative for clinical practice or clinical decision making. It also involves the process of gathering and synthesizing scientific evidence from various sources and translating it to be applied to clinical practice.
Study Designs That Gather Evidence
The design of a study has great impact on the trustworthiness of its results and hence, the level of scientific evidence it can provide. Various entities have come up with a hierarchical system for grading levels of scientific evidence based on the strengths and weaknesses of various study designs. In general, the highest level of evidence that can inform clinical practice comes from randomized controlled trials and systematic reviews or meta-analyses of randomized controlled trials. However, in some cases other studies designs (cohort, case-control, cross-sectional, case studies, etc.) can also provide information useful for clinical decision-making. Below is a list of common study designs and the situations in which they may best be used to inform clinical practice.
Summary of different study designs
Study Design |
Description |
Answers Questions About |
|---|---|---|
Meta-analysis |
A special case of a systematic review in which data is statistically combined from multiple studies to calculate a summary measure of effect. (What is a Meta-analysis)? http://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/Meta-An.pdf) |
Treatment Prevention Harm |
Systematic Reviews |
A systematic, objective, transparent approach to summarizing the scientific literature on a single topic. (What is a systematic review? http://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/Syst-review.pdf) |
Treatment Prevention Harm |
Randomized Controlled Trial |
A group of subjects are randomly assigned to treatment/intervention and control group and outcomes are measured prospectively. |
Treatment Prevention Harm |
Cohort |
A group of subjects, usually without disease/outcome at baseline, are observed over time for the development or progression of disease. Exposures or risk factors are hypothesized or known and assessed prior to development of disease. |
Incidence Risk or Preventive Factors Prognosis |
Case-control |
Patients are grouped based on disease/no disease status and risk factors for disease are retrospectively recalled or collected from medical records. |
Risk or Preventive Factors |
Cross-sectional |
Risk factor and disease status are collected at a single point in time. |
Prevalence |
Case Report / Case Series |
Detailed report of a single patient or groups of patients. |
Unique or rare conditions New treatment regimes |
Finding and Interpreting Systematic Reviews and Meta-analyses
Finding, interpreting, and applying information from individual randomized controlled trials or systematic reviews and meta-analyses to your clinical practice is a crucial part of evidence-based practice. Researchers also need ready access to the most up-to-date research in clinical practice to assess gaps and areas with high need for further investigation.
General Evidence-Based Practice databases
Cochrane Collaboration –is considered one of the definitive sources for systematic reviews and meta-analyses informing healthcare decision-making. Multiple resources are available including the Cochrane Library (available at www.cochrane.org) a database of systematic reviews, resources and training on how to do systematic reviews, free software for data analysis, and reporting guidelines.
Cochrane databases (available at http://www.cochrane.org/) include:
- The Cochrane Database of Systemic Reviews (Cochrane Reviews)
- The Database of Abstracts of Reviews of Effects (DARE)
- The Cochrane Central Register of Controlled Trials (CENTRAL)
- The Cochrane Database of Methodology Reviews (Methodology Reviews)
- The Cochrane Methodology Register (Methodology Register)
- Health Technology Assessment Database (HTA)
- NHS Economic Evaluation Database (NHS EED)
Turning Research into Practice (TRIP) database – systematic reviews organized by specialty including geriatric medicine, rheumatology, and others. The search engine will search the TRIP database as well as selected specialty journals. http://www.tripdatabase.com/index.html
Agency for Healthcare Research and Quality – provides a wide variety of information on evidence-based healthcare, funding opportunities, data from surveys, healthcare quality and patient safety. Includes a special section on Nursing Research: http://www.ahrq.gov
U.S. Preventive Services Task Force (USPSTF) Guide to Clinical Preventive Services – recommendations on screening, counseling, and preventive medication topics and includes clinical considerations. http://www.ahrq.gov/clinic/pocketgd.htm
Centers for Disease Control and Prevention Guide to Community Preventive Services - Evidence-based recommendations for programs and policies to promote population health. http://www.thecommunityguide.org/
Health Services/Technology Assessment Text (HSTAT) - HSTAT is part of the expanded Health Services Research Information Program coordinated by the National Library of Medicine. HSTAT works to improve the organization and dissemination of the results of health services research, including practice guidelines and technology assessments. One-stop shopping for evidence reviews from multiple sources including the Agency for Healthcare Research and Quality, U.S. Preventive Services Task Force evidence reviews, Reports of the Surgeon General, and other governmental and non-governmental evidence reports. http://www.ncbi.nlm.nih.gov/books/bv.fcgi?rid=hstat
Netting the Evidence - University of Sheffield. – a search engine devoted to finding resources on the methods of evidence-based practice. http://www.google.com/cse/home?cx=004326897958477606950%3Adjcbsrxkatm
- Rheumatology Relevant Databases
- Primary Care/Internal Medicine/Rheumatology
Best Practice (Evidence-based Medicine) – a digest for primary care and internal medicine. http://ebm.bmj.com/
BMJ Clinical Evidence – reliable and relevant reviews of healthcare interventions. http://clinicalevidence.bmj.com/ceweb/index.jsp
Essential Evidence Plus – clinical questions and reviews geared toward family practice in the form of Info Poems (Patient Oriented Evidence that Matters). http://www.essentialevidenceplus.com/
Evidence-based Rheumatology text – text with searchable CD and online access to updated reviews. http://www.blackwellpublishing.com/medicine/bmj/rheumatology/default.asp
Cochrane Library Review Groups – Several Cochrane Review groups publish reviews relevant to rheumatology and musculoskeletal health including the Musculoskeletal Group, the Bone, Joint, and Muscle Trauma Group, the Pain, Palliative and Supportive Care Group, etc. http://www.mrw.interscience.wiley.com/cochrane/cochrane_clsysrev_crglist_fs.html
Bandolier – reviews on arthritis, bone, and joint problems. http://www.medicine.ox.ac.uk/bandolier/booth/booths/bones.html
- Physical Therapy
Physiotherapy Evidence Database (PEDro) - details and abstracts of randomized controlled trials, systematic reviews and evidence-based clinical practice guidelines in physiotherapy which are rated based on quality. http://www.pedro.org.au/
Hooked on Evidence – sponsored by the American Physical Therapy Association (APTA) as a resource on the effectiveness of physical therapy interventions. http://www.hookedonevidence.org/
Kessler Medical Rehabilitation Research and Education Center - Rehab Trials – a clinical trial registry for rehabilitation studies. http://www.kesslerfoundation.org/research/trials
- Occupational Therapy
OT Search – search engine managed by the American Occupational Therapy Association. http://www1.aota.org/otsearch/
OT Seeker – critically appraised and rated reviews of OT interventions. http://www.otseeker.com/
OT CATs – critically appraised topics on occupational therapy from the University of Western Sydney. http://www.otcats.com/index.html
- Nursing
BMJ Evidence-based Nursing - a journal dedicated to evidence-based nursing practice. http://ebn.bmj.com/
Academic Center for Evidence-based Nursing (ACE) – University of Texas Health Science Center in San Antonio. Offers summer institutes on evidence-based nursing. http://www.acestar.uthscsa.edu/
Evidence-based Practice Tutorial -from the University of Minnesota. http://www.biomed.lib.umn.edu/learn/ebp/
Search filters for CINAHL – information on how to narrow literature searches to find relevant information. http://www.urmc.rochester.edu/hslt/miner/digital_library/tip_sheets/Cinahl_eb_filters.pdf
- Guides for interpreting systematic reviews and meta-analyses
Navigating the scientific literature can be a daunting task for busy clinicians and researchers. Systematic reviews and meta-analyses may provide a condensed synthesis on a given topic, but as with any scientific study, have inherent strengths and weaknesses. Several published guides are available to help readers digest and apply the information gleaned from these types of Evidence-Based Practice studies.
University of Berkeley Guide for Reading and Interpreting Systematic Reviews - http://www.medepi.net/meta/lectures/Lecture_6A_Reading_Interpreting_Reviews.ppt
User’s Guide to the Medical Literature: How to Use a Treatment Recommendation – one of a series of articles published in JAMA. http://jama.ama-assn.org/cgi/content/full/281/19/1836
Reviewing a Review (Bandolier) – an index for assessing the quality of a review. http://www.medicine.ox.ac.uk/bandolier/band42/b42-3.html
Duke University Evaluating a Systematic Review - http://www.mclibrary.duke.edu/training/pdaformat/overviewpda.html
- Primary Care/Internal Medicine/Rheumatology
Tools for Evaluating Study Quality and Levels of Evidence
The type of study design and how a study is executed, data analyzed and reported all play an integral role in the overall study quality and its ability to inform scientific evidence. A variety of different study quality rating tools and systems to rate the level of evidence exist. A sample of tools and level of evidence scales are provided below:
Tools to rate study quality
Agency for Healthcare Research and Quality (AHRQ) – a review of multiple tools used to rate study quality.
http://www.ncbi.nlm.nih.gov/books/bv.fcgi?rid=hstat1.section.86876
PEDro Scale – developed for use in studies of physical therapy interventions.
http://www.pedro.org.au/
Jadad Scale – a widely used 5-item tool used to rate the quality of randomized controlled trials.
See: Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clinical Trials. 1996; 17:1-12.
Downs and Black Checklist – can be used to rate the quality of both randomized and non-randomized studies.
See: Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomized and non-randomized studies of health care interventions. J Epidemiology Community Health. 1998; 52: 377-384.
Newcastle-Ottawa Scale (NOS) – for non-randomized studies.
http://www.ohri.ca/programs/clinical_epidemiology/oxford.htm
Scottish Intercollegiate Guidelines Network (SIGN) – has checklists for a wide variety of study designs.
http://www.sign.ac.uk/methodology/checklists.html
Levels of evidence
Sackett’s Level of Evidence
In the late 1990’s, David Sackett was one of the first to describe a hierarchy for rating levels of evidence of studies in clinical care. His original scheme has been expanded to include a Grade (A-D) and levels (1-5 and a-c) designation. Scientific journals and other entities have adopted similar systems, but this one seems to be one of the most commonly used.
http://www.eboncall.org/content/levels.html
Agency for Healthcare Research and Quality (AHRQ)
Summary of the Evidence Report entitled Systems to Rate the Strength of Scientific Evidence.
http://www.ncbi.nlm.nih.gov/books/bv.fcgi?rid=hstat1.chapter.70996
Centre for Evidence-Based Medicine
http://www.cebm.net/index.aspx?o=1025
Standardized Reporting Guidelines
Reporting guidelines are helpful resources for both the planning and publication of research studies. First developed to assist authors with the critical pieces of information to include in published manuscripts, are now also important for doing systematic reviews and meta-analyses and during the planning phase of many clinical and observational studies.
Systematic Reviews/Meta-analysis Reporting Guides
MOOSE – Meta-analysis of Observational Studies in Epidemiology: A Proposal for Reporting.
http://www.consort-statement.org/mod_product/uploads/MOOSE%20Statement%202000.pdf
QUORUM – Improving the Quality of Meta-Analyses.
http://www.consort-statement.org/mod_product/uploads/QUOROM%20Statement%201999.pdf
Controlled Trials and Observational Study Reporting Guides
CONSORT – Consolidated Standards of Reporting Trials.
http://www.consort-statement.org/
STROBE – Strengthening the Reporting of Observational Studies in Epidemiology.
http://www.strobe-statement.org/
TREND - Transparent Reporting of Evaluations with Nonrandomized Designs.
http://www.cdc.gov/trendstatement/docs/AJPH_Mar2004_Trendstatement.pdf
STARD - Standards for Reporting of Diagnostic Accuracy.
http://www.clinchem.org/cgi/reprint/49/1/7
