Skip to Main Content

NURP 410/640: Advanced Research Methods: Evidence-Based Practice

Nursing 410/640 Research Methods

"Where does clinical evidence come from?"

Evidence-Based Medicine and Practice Defined

Evidence-Based Medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research . . . External clinical evidence both invalidates previously accepted diagnostic tests and treatments and replaces them with new ones that are more powerful, more accurate, more efficacious, and safer (Sackett et al. 1996).

Evidence-based practice is a broader term used for many health fields and described as "a problem-solving approach to the delivery of health care that integrates the best evidence from studies and patient care data with clinician expertise and patient preferences and values. When delivered in a context of caring and in a supportive organizational culture, the highest quality of care and best patient outcomes can be achieved" (Melnyk et al., 2009).


Evidence-Based Practice Triad Diagram

This diagram shows the three components of evidence-based practice as it relates to making clinical decisions for individuals. The library can help you with finding resources containing the most current or relevant evidence while you learn how to apply it.

EvidenceEBPPatient PreferencesClinical ExperienceAgeSexCultureBest most relevant evidenceDynaMed, Cochrane LibraryPeer Reviewed studiesClinical practice experienceClinical care teamsFamilySocioeconomicWork
Evidence-Based Practice Triad
Read More

Sources

Ackley, B. (2015). Evidence-based practice. In Understanding Medical Surgical Nursing, 5th ed (pp. 12–18). F.A. Davis Company.

Melnyk, B. M., Fineout-Overholt, E., Stillwell, S. B., & Williamson, K. M. (2009). Evidence-based practice: Step by step: Igniting a spirit of inquiry: An essential foundation for evidence-based practice. AJN, The American Journal of Nursing, 109(11), 49.

Melnyk, B. M., Fineout-Overholt, E., Stillwell, S. B., & Williamson, K. M. (2010). Evidence-based practice: Step by step: The seven steps of evidence-based practice. AJN, American Journal of Nursing, 110, 51-53. https://doi.org/10.1097/01.NAJ.0000366056.06605.d2

Sackett, D. L., Rosenberg, W. M., Gray, J. M., Haynes, R. B., & Richardson, W. S. (1996). Evidence based medicine: What it is and what it isn't. BMJ 312. pp. 71-72

Schub, E., Walsh, K. & Pravikoff D. (Ed.) (2017). Evidence-based nursing practice: Implementing [Skill Set]. Nursing Reference Center Plus


Polit–Beck Evidence Hierarchy/Levels of Evidence Scale for Therapy Questions

VIIILevel VIII: Nonresearch source (e.g., internal evidence,Level VII: Qualitative study/descriptive studyLevel VI: Systematic review/metasynthesis of qualitative studiesLevel V: Nonexperimental/observational studyLevel IV: Systematic review of nonexperimental (observational) studies Level III: Nonrandomized trial (quasi-experiment) Level II: Randomized controlled trial (RCT)Level I: Systematic review/meta-analysis of RCTsVIIVIVIVIIIIII Level VIII: Nonresearch source (e.g., internal evidence, expert opinion)Level VII: Qualitative study/descriptive studyLevel VI: Systematic review/metasynthesis of qualitative studiesLevel V: Nonexperimental/observational studyLevel IV: Systematic review of nonexperimental (observational) studies Level III: Nonrandomized trial (quasi-experiment) Level II: Randomized controlled trial (RCT)Level I: Systematic review/meta-analysis of RCTs
From Polit and Beck (2021, Figure 2.2, p. 29)
The Point

"Figure 2.2 [in context of book] shows our eight-level evidence hierarchy for Therapy/intervention questions. This hierarchy ranks sources of evidence with respect the readiness of an intervention to be put to use in practice" (Polit & Beck, 2021, p. 28). Levels are ranked on risk of bias - level one being the least bias, level eight being the most biased. There are several types of levels of evidence scales designed for answering different questions. "An evidence hierarchy for Prognosis questions, for example, is different from the hierarchy for Therapy questions" (p. 29).

Advantages of Levels of Evidence Scales

"Through controls imposed by manipulation, comparison, and randomization, alternative explanations can be discredited. It is because of this strength that meta-analyses of RCTs, which integrate evidence from multiple experiments, are at the pinnacle of the evidence hierarchies for Therapy questions" (p. 188).


"Tip: Traditional evidence hierarchies or level of evidence scales (e.g., Figure 2.2), rank evidence sources almost exclusively based on the risk of internal validity threats" (p. 217).


Systematic reviews can provide researchers with knowledge that prior evidence shows. This can help clarify established efficacy of a treatment without unnecessary and thus unethical research. Greenhalgh (2019) illustrates this citing Dean Fergusson and colleagues (2005) systematic review on a clinical surgical topic (p. 128).


Limits of Levels of Evidence Scales

Regarding the importance of real-world clinical practice settings, and the conflicting tradeoffs between internal and external validity, Polit and Beck (2021) write, "the first (and most prevalent) approach is to emphasize one and sacrifice another. Most often, it is external validity that is sacrificed. For example, external validity is not even considered in ranking evidence in level of evidence scales" (p. 221). ... From an EBP perspective, it is important to remember that drawing inferences about causal relationships relies not only on how high up on the evidence hierarchy a study is (Figure 2.2), but also, for any given level of the hierarchy, how successful the researcher was in managing study validity and balancing competing validity demands" (p. 222).


Polit and Beck note Levin (2014) that an evidence hierarchy "is not meant to provide a quality rating for evidence retrieved in the search for an answer" (p. 6), and as the Oxford Center for Evidence-Based Medicine concurs that evidence scales are, 'NOT intended to provide you with a definitive judgment about the quality of the evidence. There will inevitably be cases where "lower-level" evidence...will provide stronger than a "higher level" study (Howick et al., 2011, p.2)'" (p. 30).


Polit and Beck (2021) further explain the difference between levels of evidence and quality of evidence related to appraisal: "The first appraisal issue is the extent to which the findings in a research report are valid. That is, were the study methods sufficiently rigorous that the evidence has a low risk of bias? Melnyk and Fineout-Overholt (2019) propose the following formula:

Level of evidence (e.g., Figure 2.2) + Quality of evidence = Strength of evidence.



Thus, in coming to a conclusion about the quality of the evidence, it is insufficient to simply 'level' the evidence using an LOE scale–it must also be appraised" (p. 36). Greenhalgh (2019) likewise acknowledges, "on the negative side, systematic reviews can replicate and magnify flaws in the original studies (e.g. if all primary studies considered a drug at sub-therapeutic dose, the overall - misleading - conclusion may be that the drug has 'no effect')" (p. 19).

The 6S Model of Levels of Evidence

Level 6Level 5Level 4Level 3Level 2Level 1Original articles published in journalsBrief overviews of single studies, with commentaryBrief overviews of systematic reviews, with commentaryEvidence-based clinicalpractice guidelines orclinical pathwaysComputerized clinical information systems with automatic decision supports Systematic reviews or meta-analysesSynopsis of StudiesSynthesesStudiesSynopsis of SynthesesSummariesSystems SystemsLevel 1Computerized clinical information systems with automatic decision supports SummariesLevel 2Evidence-based clinical practice guidelines or clinical pathways Synopsis of SynthesesLevel 3Level 4Systematic reviews ormeta-analyses SythesesLevel 5Brief overviews of singlestudies, with commentaryLevel 6Original articles published in journals Synopsis of StudiesStudies
Modified from DiCenso, Bayley, & Haynes (2009)
The Point

"The 6S hierarchy does not imply a gradient of evidence in terms of quality, but rather in terms of ease in retrieving relevant evidence to address a clinical question. At all levels, the evidence should be assessed for quality and relevance" (Polit & Beck, 2021, p. 24, Tip box).


The 6S Pyramid proposes a structure of quantitative evidence where articles that include pre-appraised and pre-synthesized studies are located at the top of the hierarchy (McMaster U., n.d.).


It can help to consider the level of evidence that a document represents, for example, a scientific article that summarizes and analyses many similar articles may provide more insight than the conclusion of a single research article. This is not to say that summaries can not be flawed, nor does it suggest that rare case studies should be ignored. The aim of health research is the well-being of all people, therefore it is important to use current evidence in light of patient preferences negotiated with clinical expertise.

Other Gradings in Levels of Evidence

Schub & Walsh (2017) explain that:

While it is accepted that the strongest evidence is derived from meta-analyses, various evidence grading systems exist. for example:

The Johns Hopkins Nursing Evidence-Based Practice model ranks evidence from level I to level V, as follows (Seben et al., 2010):
  • Level I: Meta-analysis of randomized clinical trials (RCTs); experimental studies; RCTs
  • Level II: Quasi-experimental studies
  • Level III: Non-experimental or qualitative studies
  • Level IV: Opinions of nationally recognized experts based on research evidence or an expert consensus panel
  • Level V: Opinions of individual experts based on non-research evidence (e.g., case studies, literature reviews, organizational experience, and personal experience)
The American Association of Critical-Care Nurses (AACN) evidence level system, updated in 2009, ranks evidence as follows (Armola et al., 2009):
  • Level A: Meta-analysis of multiple controlled studies or meta-synthesis of qualitative studies with results that consistently support a specific action, intervention, or treatment
  • Level B: Well-designed, controlled randomized or non-randomized studies with results that consistently support a specific action, intervention, or treatment
  • Level C: Qualitative, descriptive, or correlational studies, integrative or systematic reviews, or RCTs with inconsistent results
  • Level D: Peer-reviewed professional organizational standards, with clinical studies to support recommendations
  • Level E: Theory-based evidence from expert opinion or multiple case reports
  • Level M: Manufacturers’ recommendations (2017)

EBM Pyramid and EBM Page Generator


Click on the pyramid text or scroll to view resources described by each level of evidence. Page source: EBM Pyramid and EBM Page Generator for Evidence-Based Mental Health Resources by Jan Glover, David Izzo, Karen Odato and Lei Wang.

Unfiltered are resources that are primary sources describing original research. Randomized controlled trials, cohort studies, case-controlled studies, and case series/reports are considered unfiltered information.

Filtered are resources that are secondary sources which summarize and analyze the available evidence. They evaluate the quality of individual studies and often provide recommendations for practice. Systematic reviews, critically-appraised topics, and critically-appraised individual articles are considered filtered information.


Sources

Armola, R. R., Bourgault, A. M., Halm, M. A., Board, R. M., Bucher, L., Harrington, L., ... Medina, J. (2009). AACN levels of evidence. What's new? Critical Care Nurse, 29(4), 70-73. doi:10.4037/ccn2009969

DiCenso, A., Bayley, L., & Haynes, R. B. (2009). Accessing pre-appraised evidence: Fine-tuning the 5S model into a 6S model. BMJ Evidence-Based Nursing, 12(4) https://ebn.bmj.com/content/12/4/99.2.short

Fergusson, D., Glass, K. C., Hutton, B., & Shapiro, S. (2005). Randomized controlled trials of Aprotinin in cardiac surgery: Could clinical equipoise have stopped the bleeding?. Clinical Trials, 2(3), 218-232.

Glover, J., Izzo, D., Odato, K. & Wang, L. (2008). Evidence-based mental health resources. EBM Pyramid and EBM Page Generator. Copyright 2008. All Rights Reserved. Retrieved April 28, 2020 from https://web.archive.org/web/20200219181415/http://www.dartmouth.edu/~biomed/resources.htmld/guides/ebm_psych_resources.html Note. Document removed from host. Old link used with the WayBack Machine of the Internet Archive to retrieve the original webpage on 2/10/21 http://www.dartmouth.edu/~biomed/resources.htmld/guides/ebm_psych_resources.html

Greenhalgh, T. (2019). How to read a paper: The basics of evidence-based medicine and healthcare. (Sixth ed.). Wiley Blackwell.

Haynes, R. B. (2001). Of studies, syntheses, synopses, and systems: The “4S” evolution of services for finding current best evidence. BMJ Evidence-Based Medicine, 6(2), 36-38.

Haynes, R. B. (2006). Of studies, syntheses, synopses, summaries, and systems: the “5S” evolution of information services for evidence-based healthcare decisions. BMJ Evidence-Based Medicine, 11(6), 162-164.

McMaster University (n.d.). 6S Search Pyramid Tool https://www.nccmt.ca/capacity-development/6s-search-pyramid

Polit, D., & Beck, C. (2019). Nursing research: Generating and assessing evidence for nursing practice. Wolters Kluwer Health.

Schub, E., Walsh, K. & Pravikoff D. (Ed.) (2017). Evidence-based nursing practice: Implementing [Skill Set]. Nursing Reference Center Plus

Seben, S., March, K. S., & Pugh, L. C. (2010). Evidence-based practice: The forum approach. American Nurse Today, 5(11), 32-34.


Evidence-Based Databases

Databases @ Simmons (See also Nursing - Point-of-Care Databases)
Open Access (See also Nursing - Open Resources)

Using the PICO or PICOT Model

Implementing evidence-based practice involves forming clinical questions about the people who are under your care in a given condition. Forming a clinical question is an early step in the process of evidence-based practice. PICO is a useful way of formulating clinical research questions and a well-build question or problem should include the four components of the model:

  • P = Patient population
  • I = Intervention
  • C = Comparison or comparison group
  • O = Outcome(s)
  • T = Time period (if applicable)
Patient populationInterventionComparisonOutcomeIn newborn infants in the neonatal intensive care unit (NICU), does oral sucrose administration provide better nonpharmacologic pain management than non-nutritive sucking during heel sticks?Newborn infants in the neonatal intensive care unit (NICU)Oral sucrose administrationnon-nutritive suckingBetter nonpharmacologic pain management during heel sticksExampleSearch languagePICOnewborn infant OR newborn“neonatal intensive care unit”“oral sucrose”“non-nutritive sucking”heel stick OR heel lance OR heel nonpharmacologic pain management
Example from Evidence-based nursing practice: Implementing by Schub & Walsh. Note. PICOT includes Time periods if applicable

PICO Search Tools

The PICO model can be used in any database, but there are also other search systems designed to work with PICO phrases


Sources

Schub, E., Walsh, K. & Pravikoff D. (Ed.) (2017). Evidence-based nursing practice: Implementing [Skill Set]. Nursing Reference Center Plus


Other Evidence-Based Resources

Related Websites

Health Statistics

Data and statistics can be difficult to search for. It's helpful to think about the following when you're searching for this kind of information: It takes time to collect, analyze, and publish data. Sometimes the most recent available data is a few years old. National and state data are more prevalent than city/town data. Sometimes you'll find published data sets, which contain raw, unanalyzed data. See the Tips for Searching Google box on this page for help finding analyzed, ready-to-use statistics. Data can be taken out of context, cherry-picked, or manipulated to support a particular point of view. If you're not sure you can trust a source, try to verify the information in a second source

You'll find some "best bets" for finding demographic and health statistics below. You'll probably save yourself some time by searching these first. But if you can't still find the information you're looking for, see what you can find by searching Google. These tips will help you do a more effective Google search: Look for sites that end in .gov. City and town websites have lots of local demographic and health information. If you find a .org site, look into the organization. Would they have any reason to present biased information? Look for reports, overviews, and snapshots when you're looking for quick facts. You'll need a lot of time (and probably some advanced tools and knowledge) to find the information you want in a data set. If you find a source that cites data from somewhere else, try to find the original source. Pay attention to when data was collected. A report published this year could cite data that's much older.