Methodology of evidence-based medicine (EBM) О.G.Puzanova National O.O. Bohomolets medical university Chair of computer sciences
“Medical practitioners are overloaded with information… Every day they encounter questions that need to be answered in order to make the best decisions about patient care. This is where EBM comes in”. P.Glasziou, C.Del Mar, 2003 “By the year 2020, 90% of clinical decisions will be supported by accurate, timely, and up-to-date clinical information and will reflect the best available evidence”. IOM Roundtable on Evidence-Based Medicine
What is evidence-based medicine (EBM)? The term EBM was first used by David Sackett and his colleagues at McMaster University in Ontario, Canada, in the early 1990s. Sackett D, Rosenberg W, Gray J, Haynes R, Richardson W. Evidence based medicine: what it is and what it isn’t. BMJ 1996;312:71-2 D.Sackett: “Evidence-based medicine is the integration of best research evidence with clinical expertise and patient values”
Which doctor do you want? William Osler, 1900 Wise & experienced Smart young doctor
In its original formulation, EBM reduced an emphasis on unsystematic clinical experience and pathophysiological rationale, and promoted the examination of evidence from clinical research: “EBM is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients”. The latter term is “evidence-based practice” (EBP).
The purpose of EBM is to alert clinicians to important advances in different fields of medicine by selecting from the biomedical literature those original and review articles whose results are most likely to be both true and useful.
By individual clinical expertise we mean the proficiency and judgment that individual clinicians acquire through clinical experience and clinical practice. By best available external clinical evidence we mean clinically relevant research, often from the basic sciences of medicine, but especially from patient centred clinical research into - the accuracy and precision of diagnostic tests (including the clinical examination), - the power of prognostic markers, - and the efficacy and safety of therapeutic, rehabilitative, and preventive regimens.
Without current best evidence, practice risks becoming rapidly out of date, to the detriment of patients. Without clinical expertise, practice risks becoming tyrannised by evidence, for even excellent external evidence may be inapplicable to or inappropriate for an individual patient.
External clinical evidence both invalidates previously accepted diagnostic tests and treatments and replaces them with new ones that are more powerful, more accurate, more efficacious, and safer. External clinical evidence can inform, but can never replace individual clinical expertise, and it is this expertise that decides whether the external evidence applies to the individual patient at all and, if so, how it should be integrated into a clinical decision.
So, good doctors use both individual clinical expertise and the best available external evidence, and neither alone is enough. The practice of EBM means integrating individual clinical expertise with the best available external clinical evidence from systematic research. Doctors practising EBM will identify and apply the most effective interventions to maximise the quality and quantity of life for individual patients.
In 1972, British epidemiologist Archie Cochrane highlighted that most treatment-related decisions were based on an ad hoc selection of information from the vast and variable quality scientific literature, on expert opinion, or on trial and error. Cochrane proposed that researchers and practitioners should collaborate internationally to systematically review all the best clinical trials (especially randomised controlled – RCTs). In the early 1990s, funds were provided by the UK National Health Care to establish a Cochrane Center in Oxford. The approach was further outlined at an international meeting organized by the New York Academy of Sciences and at first Cochrane Colloquium in 1993, when ”The Cochrane Collaboration” was founded. http://www.cochrane.org
The total number of RCTs published has increased exponentially since the 1940s. A total of 20 000 trials are published each year (with over 300 000 in total) and approximately 50 new trials - every day. So, to keep up-to-date with RCTs alone, we have to read 1 study report every half hour, day and night !!! P.Glasziou, C.Del Mar, 2003
Epidemiological evidence is the cornerstone of EBM “The goal of any therapeutic intervention is to increase the probability of improvement or to reduce the probability of harm. These probabilities are often reported as ratios, rates, or risks, in which the probability of an event occurring in one group, usually an experimental or treatment group, is compared with the probability of the event occurring in another treatment or placebo control group”. Lang T.A., Secic M. How to report statistics in medicine. - American College of physicians, 2006. – P.19.
Differences between Clinical and Statistical Significance (1) SS essentially reflects the influence of chance on the outcomes; CS reflects the biological value of the outcome. Statistics are derived from groups of individuals; medicine is practiced on specific individuals. Statistical conclusions require adequate amount of data to be valid; medical decisions must often be made with insufficient data.
Differences between Clinical and Statistical Significance (2) Statistical answers are probabilistic; medical treatment requires committed decision. Statistical analysis always requires measurement; medicine sometimes requires intuition. The statistical and clinical applications of the term “normal” are often confused and vague. Lang T.A., Secic M. How to report statistics in medicine.- American College of physicians, 2006. – P.19.
The 4 steps of EBM / EBP Formulate an answerable question Search for appropriate epidemiological evidence to help answer question - “track down the best evidence of outcomes available” Appraise (critically) the evidence - “find out how good is it” Apply the evidence - “integrate the results with clinical expertise and patient values” Assess/ evaluate practice – to improve next time (clinical audit).
Generalizable knowledge for better clinical practice and healthcare knowledge from research (‘research evidence’) knowledge from the analysis of routinely collected and audit data (‘statistics’) knowledge from the experience of providers and patients (‘know how’)
What is the “evidence” of EBM / EBP? ‘clinically relevant evidence, sometimes from basic science, but especially from patient-centred clinical research into the accuracy of diagnostic tests*, the power of prognostic markers*, and the efficacy & safety of interventions*’ Modified from Sackett et al. EBM 2nd Edition 2000 * from clinical epidemiological studies
GATE: Graphic Appraisal Tool for Epidemiology The GATE Approach: every epidemiological study hangs on the GATE frame There is only one study design: RCT - interventions Cohort studies - prognosis / interventions / aetiology Cross-sectional studies - diagnosis Case-control studies -interventions /aetiology
P E C O GATE: Graphic Appraisal Tool for Epidemiology T GATE Frame: PECOT Population Exposure Comparison Outcome Time
Population P Comparison E C Outcomes (O) O yes no Exposure a b c d © GATE: epi study design (O)
Population P Outcomes: CHD O Randomised Controlled Trial (RCT): HERS Time T Randomised allocation EG CG Exposure (intervention): HRT Comparison (control): Placebo Hulley et al. JAMA1998;280:605-13
Types of studies (1)
Types of studies (2)
Population P Exposure Group Comparison Group non-randomised allocation Outcomes O “Life” is a cohort study: a “natural experiment” Cohort (Follow-up) study: archetypal epidemiological study Time T Real life time EG CG cohort ill-health
Population P Cohort of women diagnosed with breast cancer childbirth <2yrs before diagnosis Non-randomised allocation to prognostic groups yesdeath .no O Cohort study (prognosis): Danish Breast Cancer Cooperative 10 years T EG CG No childbirth <2 yrs before diagnosis © Kroman et al. BMJ 1997;315:851-5
Population P Cohort of British Doctors established in 1951 Smokers Non-smokers Non-randomised allocation (self-reported smoking) yesLung Cancer .no O Cohort study (aetiology): British Doctors Study T EG CG 50 years Doll et al. BMJ 2004;328:1519
Population P Cohort of US Nurses HRT No HRT Non-randomised allocation (self-reported use) yesCHD .no O Cohort study (intervention): Nurses Health Study T EG CG 10 years Stampfer et al. NEJM 1991;325:756-762
Population P EG C No one allocated to Comparison Group Outcomes O Time Case series T Exposure Grp ©
Descriptions of a clinical case or a case series are brief reports about successful treatment or threatening side-effects of pharmacothrapy (-) – poor evidence of information (+) – energy and efficiency of information !
What are your clinical questions? A 35 year old man says his brother recently died of a ruptured cerebral aneurysm. He is worried about whether he might have one and what the chances are that it would rupture.
Risk Factors Cause(s) Symptoms Signs, Tests Prognosis Treatment Effect Past current future Types of question and relevant studies : stroke Frequency Cohort Study Survey Inception Cohort Study Treatments Randomised Trial
Classes of evidence Class І – evidences and/or common agreement that intervention is useful and effective. Class ІІ – disputable evidences and/or differences in experts’ opinion about usefulness / efficacy of the intervention: ІІa – evidences / opinion for its efficacy; ІІb – usefulness / efficacy of the intervention is not convincingly supported by the evidences / opinion . Class ІІІ – evidences and/or common agreement that intervention is neither useful nor effective or is even harmful.
Levels of evidence А: data obtained in several RCTs. В: data obtained in isolated RCTs or in non-randomised trials. С: experts’ opinion or data from case series.
End-points of the controlled clinical trials - 5D: Death Disease - feeling of “danger” Discomfort - pain, dispnoe, dizziness, sickness… Disability - patient is not capable to activity (professional, or everyday, or self-service) Dissatisfaction - negative еmotional reaction to disease and treatment
It is usually defined as “a probability of an unfavorable outcome occuring during a given period of time”. Dictionary of Epidemiology, 1988 “Risk is a combination of the probability of an event – usually an adverse event – and the nature and severity of the event”. National Academy of Sciences (In: How to report statistics in medicine. - American College of physicians, 2006. – P.21). What is Risk?
Framingham Study: Паління Артеріальна гіпертензія Гіперхолестеринемія Combination of the cardiovascular risk factors
What are “tests” used for? Diagnosis Screening Monitoring Prognosis Treatment planning
Diagnostic Test Accuracymeasurements Sensitivity is the probability of a positive test in a diseased person Specificity is the probability of a negative test in a non-diseased person
Sensitivity and specificity of diagnostic and screening tests are revealed in cross-sectional studies Sensitivity = the proportion of people with disease who have a positive test: Se = a / (a + c) Specificity = the proportion of people free of disease who have a negative test: Sp = d / (b + d)
Predictive value (PV) of the test – the probability of presence or absence of the disease in case that result of the test is known. For test result interpretation this index is considered as the best !!! PV of the test depends on its sensitivity and specificity, as well as on prevalence of the disease. Prevalence - the baseline risk of a disorder in the population of interest: P = (a+c) / (a+b+c+d) Incidence: the number of new cases of illness commencing, or of persons falling ill, during a specified time period in a given population.
Positive predictive value (+PV) = the proportion of people with a positive test who have disease. + PV = a / (a + b) Negative predictive value (-PV) = the proportion of people with a negative test who are free of disease. - PV = d / (с + d)
2 by 2 table Disease Test + - + - Sensitivity Positive predictive value a b c
Examples Computerized decision support Evidence-based textbooks Evidence-based journal abstracts Systematic reviews Original journal articles The evolution of information resources for evidence-based decisions
Premier evidence resources Systems: EMR with decision support Summaries: Clinical Evidence, PIER, UpToDate, Dynamed Synopses: ACP Journal Club, EBM Syntheses: via BMJUpdates+ Studies: via BMJUpdates+, PubMed Clinical Queries
The best evidence for different types of question (1)
The best evidence for different types of question (2)
Systematic review - an article in which the authors have systematically searched for, appraised, and summarised all of the medical literature for a specific topic (for one structured clinical question). Meta-analysis - a systematic review which uses quantitative methods to summarise the results.
What is a meta-analysis? Optional part of a systematic review Systematic review Meta-analyses
What makes a Review “Systematic”? Based on a clearly formulated question Identifies relevant studies Appraises quality of studies Summarizes evidence by use of explicit methodology Comments based on evidence gathered
STEP 1: Ask focussed 5-part questions: 1. Participants (patient/population group) 2. Exposure (Intervention if about therapy) 3. Comparison (there is always an alternative!) 4. Outcome 5. Timeframe
STEP 2: Search for best evidence using 3/4-part search terms: 1. Participants (patient/population group) 2. Exposure (intervention if about therapy) 3. Comparison (e.g placebo) 4. Outcome
Searching for best evidence 2 main databases of primary information origins we can use to search the evidence are Pub Med and Cochrane library. PubMed - the National Library of Medicine – is electronic on-line free database in Medline. http://www.ncbi.nlm.nih.gov/entrex/query.fcgi Cochrane collaboration - a worldwide association of groups who create and maintain systematic reviews of the literature for specific topic areas. http://www.cochrane.org
Medline Embase Cochrane Trials Registry Comparing Databases Medline = Pubmed, Webspirs, OVID, …
Using the question to guide searching You are interested in checking the hearing of elderly patients, and have heard that the ‘whispered voice test’ is good. Question Population Indicator (Intervention, test, etc) Comparator Outcome Underline the key terms Number the order of importance from 1-4 Think of alternate spellings, synonyms, & truncations
You are interested in checking the hearing of elderly patients, and have heard that the ‘whispered voice test’ is good. Question Population – in elderly patients does Indicator – a ‘poor’ whispered voice test Comparator – a ‘normal’ whispered voice test Outcome – predict poor hearing (audiogram) 1 2 3
Start up Start Explorer and enter www.pubmed.gov Put on CAPS lock So that AND and OR are in CAPITALS Start 2nd Explorer window, enter www.tripdatabase.com
* Means any other letters AND means both terms required Check the question type Check the emphasis
Structured Question Do adults (aged > 18) using oral anticoagulation therapy have fewer episodes of thrombotic events if they are self-managed than those that are managed by doctor/health practitioner? population outcome intervention control
EBM / EBP Step 4: Applying to the individual What do the results mean on average? What do they mean for this individual?
In 2002 the Guidelines International Network (GIN) was founded www.g-i-n.net www.osdm.org
Basic principles of giving medical help Clinical practice guidelines - systematically developed statements designed to assist health care professionals and patients make decisions about appropriate health care for specific clinical circumstances. Medical standarts – normative documents defining a list of norms and requests to medical care and recognizing an accordance to indicators National (local) clinical reports (statements) normative documents, detailed instructions
So, today we see… intensive development of medical sciences, increase of cost of researches, growth of biomedical information (scientific and popular), ageing of population, changes in morbidity structure (growth of chronic non-infectious diseases, co-existing of them in individual pts), increase of cost of treatment while the resources of health care systems are restricted, humanisation of world community, attention to pts’ rights, development and implementation of clinical recommenda-tions and medical standarts - to increase quality of medical care.
“Knowledge is the enemy of disease” The application of what we know will have a bigger impact on health and disease than any single drug or technology likely to be introduced in the next decade. Sir Muir Gray, Director,NHS National Knowledge Service