WHAT IS EVIDENCE-BASED MEDICINE?
Remember when we used to treat every otitis media with antibiotics? These recommendations came about because we applied logical reasoning to observational studies. If bacteria cause an acute otitis media, then antibiotics should help it resolve sooner, with less morbidity. Yet, when rigorously studied (via a systematic review), we found little benefit to this intervention.
The underlying premise of evidence-based medicine (EBM) is the evaluation of medical interventions and the literature that supports those interventions, in a systematic fashion. EBM hopes to encourage treatments proven to be effective and safe. And when insufficient data exists, it hopes to inform you on how to safely proceed.
EBM uses end points of real patient outcomes, morbidity, mortality, and risk. It focuses less on intermediate outcomes (bone density) and more on patient conditions (hip fractures).
Implementing EBM requires three components: the best medical evidence, the skill and experience of the provider, and the values of the patients. Should this patient be screened for prostate cancer? It depends on what is known about the test, on what you know of its benefits and harms, your ability to communicate that information, and that patient's informed choice.
This book hopes to address the first EBM component, providing you access to the best information in a quick format. Although not every test or treatment has this level of detail, many of the included interventions here use systematic review literature support.
The language of medical statistics is useful in interpreting the concepts of EBM. Below is a list of these terms, with examples to help take the confusion and mystery out of their use.
Prevalence: proportion of people in a population who have a disease (in the United States, 0.3% [3 in 1,000] people >50 years have colon cancer)
Incidence: How many new cases of a disease occur in a population during an interval of time; for example, “the estimated incidence of colon cancer in the United States is 104,000 in 2005.”
Sensitivity: Percentage of people with disease who test positive; for mammography, the sensitivity is 71-96%.
Specificity: Percentage of people without disease who test negative; for mammography, the specificity is 94-97%.
Suppose you saw ML, a 53-year-old woman, for a health maintenance visit, ordered a screening mammogram, and the report demonstrates an irregular area of microcalcifications. She is waiting in your office to receive her test results, what can you tell her?
Sensitivity and specificity refer to characteristics of people who are known to have disease (sensitivity) or those who are known not to have disease (specificity). But, what you have is an abnormal test result. To better explain this result to ML, you need the positive predictive value.
Positive predictive value (PPV): Percentage of positive test results that are truly positive; the PPV for a woman aged 50 to 59 years is approximately 22%. That is to say that only 22% of abnormal screening mammograms in this group truly identified cancer. The other 78% are false positives.
You can tell ML only one out of five abnormal mammograms correctly identify cancer; the four are false positives, but the only way to know which mammogram is correct is to do further testing.
The corollary of the PPV is the negative predictive value (NPV), which is the percentage of negative test results that are truly negative.
The PPV and NPV tests are population-dependent, whereas the sensitivity and specificity are characteristics of the test, and have little to do with the patient in front of you. So when you receive an abnormal lab result, especially a screening test such as mammography, understand their limits based on their PPV and NPV.
Treatment information is a little different. In discerning the statistics of randomized controlled trials of interventions, first consider an example. The Scandinavian Simvastatin Survival Study (4S) (Lancet. 1994;344:1383-1389) found using simvastatin in patients at high risk for heart disease for 5 years resulted in death for 8% of simvastatin patients versus 12% of those on placebo; this results in a relative risk of 0.70, a relative risk reduction of 33%, and a number needed to treat of 25.
There are two ways of considering the benefits of an intervention with respect to a given outcome. The absolute risk reduction is the difference in the percentage of people with the condition before and after the intervention. Thus, if the incidence of myocardial infarction (MI) was 12% for the placebo group and 8% for the simvastatin group, the absolute risk reduction is 4% (12% - 8% = 4%).
The relative risk reduction reflects the improvement in the outcome as a percentage of the original rate and is commonly used to exaggerate the benefit of an intervention. Thus, if the risk of MI were reduced by simvastatin from 12% to 8%, then the relative risk reduction would be 33% (4%/12% = 33%); 33% sounds better than 4%, but the 4% is the absolute risk reduction and reflects the true outcome.
Absolute risk reduction is usually a better measure of clinical significance of an intervention. For instance, in one study, the treatment of mild hypertension has been shown to have relative risk reduction of 40% over 5 years (40% fewer strokes in the treated group). However, the absolute risk reduction was only 1.3%. Because mild hypertension is not strongly associated with strokes, aggressive treatment of mild hypertension yields only a small clinical benefit. Don't confuse relative risk reduction with relative risk.
Absolute (or attributable) risk (AR): the percentage of people in the placebo or intervention group who reach an end point; in the simvastatin study, the absolute risk of death was 8%.
Relative risk (RR): the risk of disease of those treated or exposed to some intervention (i.e., simvastatin) divided by those in the placebo group or who were untreated
— If RR is <1.0, it reduces risk—the smaller the number, the greater the risk reduction.
— If RR is >1.0, it increases risk—the greater the number, the greater the risk increase.
Relative risk reduction (RRR): the relative decrease in risk of an end point compared to the percentage of that end point in the placebo group
If you are still confused, just remember that the RRR is an over-estimation of the actual effect.
Number needed to treat (NNT): This is the number of people who need to be treated by an intervention to prevent one adverse outcome. A “good” NNT can be a large number (>100) if risk of serious outcome is great. If the risk of an outcome is not that dangerous, then lower (<25) NNTs are preferred.
The NNT should be compared to a similar statistic, the number needed to harm (NNH). This is the number of people who have to be given treatment before one excess side effect or harm occurs. When the NNT is compared to the NNH, you and the patient can judge whether the benefit of the intervention is great enough to outweigh the risk of harm.
To help you interpret diagnostic and treatment recommendations within The 5-Minute Clinical Consult, we have graded the best information within the text and highlighted this content.
An “A” grade means the reference is from the highest quality resource, such as a systematic review. A systematic review is a summary of the medical literature on a given topic that uses strict, explicit methods to perform a thorough search of the literature and then provides a critical appraisal of individual studies, concluding in a recommendation. The most prestigious collection of systematic reviews is from the Cochrane Collaboration (www.cochrane.org).
A “B” grade means the data referenced comes from high-quality randomized controlled trials performed to minimize bias in their outcome. Bias is anything that interferes with the truth; in the medical literature, it is often unintentional, but it is much more common than we appreciate. In short, always assume some degree of bias exists in any research endeavor.
A “C” grade implies the reference used does not meet the A or B requirements; they are often treatments recommended by consensus groups (such as the American Cancer Society). In some cases, they may be the standards of care. But implicit in a group's recommendation is the bias of the author or the group that supports the reference. For example, the American Urological Society's recommendation around screening for prostate cancer may be motivated by their narrow scope and financial benefit. Compare this to the recommendations of the U.S. Preventive Services Task Force (www.ahrq.gov), which recommends against screening for prostate cancer.
Bias is anything that interferes with the truth. There are many types of bias that should be considered by the publishers of medical information. Below describes a number of bias types that often affect our care without us knowing it is present.
Publication bias occurs when research is not published; this is often when a study finds data that does not support an intervention. The motivation to publish information that “didn't work” is low. It is estimated that up to 40% of all medical research never gets published. When you read of an effective intervention, wonder if other studies did not show benefit and went unpublished.
Comparator bias occurs when research compares an intervention to placebo, when placebo is not the standard of care. Knowing a new antibiotic is more effective than placebo for treating a condition is not helpful if you typically use a drug or procedure. Why not release research comparing the new drug to the standard of care? Sometimes the new treatment is no better than the current standard. And if a study was done to see if the new is better than the old and not published, you have an example of publication bias.
Selection bias involves choosing study populations that might be different than the average patient or just reporting a just subset of study participants from a study. Either will result in the data being skewed because it can only be applied to small subset of people.
Attrition bias and the concept of intention to treat. Attrition bias is when researchers do not fully acknowledge and address how a study deals with participants who do not adhere to the research protocol or drop out completely. Intention to treat analysis hopes to diminish attrition bias by statistically considering the nonadhering or dropped out patients as unsuccessfully benefiting from the intervention.
Commercial (funder) bias involves who paid for the research being done, and do they have a vested interest in the outcome. If the developer of a new drug does a large study, or a researcher has a personal financial interest in seeing a study succeed, they may consciously or unconsciously alter what is reported in a study. The data may be accurate, but until this is studied by less vested interests, some feel its outcome cannot be clinically applied.
Have you been annoyed how one week you learn of a randomized controlled trial that supports a treatment, to be followed the next week with a contradictory article? Statisticians have figured out how to resolve this using something called a systematic review.
A systematic review gathers all the literature on a topic, say using antibiotics to treat otitis media, and combines the data to determine if the sum of all the trials tells a different story than any single trial. The large number of participants in this type of research results in a much more statistically (and clinically) significant conclusion than any single paper. Want more? Check this out: http://community.cochrane.org/about-us/evidence-based-health-care.
A meta-analysis is a quantitative systematic review and demonstrates its outcomes in the form of a forest plot. The bottom line with interpretation of a forest plot is to look for the diamond on the bottom. If it is to LEFT of the vertical line, it means risk of an outcome was reduced by the intervention. If it is fully to the RIGHT, then risk of that outcome was increased. And if the diamond touches the vertical line, it means there was no statistical influence of the intervention on the outcome.
We hope this brief introduction to EBM has been informative, clear, and helpful. If any of the information above seems unclear, or if you have a question, please contact us via www.5MinuteConsult.com.