Why read papers at all?
“Evidence-based medicine is the enhancement of a clinician’s traditional skills in diagnosis, treatment, prevention and related areas through the systematic framing of relevant and answerable questions and the use of mathematical estimates of probability and risk”.
Professor Dave Sackett, in the opening editorial of the very first issue of the journal evidence based medicine, summarised the essential steps in the emerging science of evidence based medicine:
- To convert our information needs into answerable questions (i.e. to formulate the problem).
- To track down, with maximum efficiency, the best evidence with which to answer these questions – which may come from the clinical examination, the diagnostic laboratory, the published literature or other sources.
- To appraise the evidence critically (i.e. weigh it up) to assess its validity (closeness to the truth) and usefulness (clinical applicability).
- To implement the results of this appraisal in our clinical practice.
- To evaluate our performance.
Let’s take a look at the various approaches which health professionals use to reach their decisions in reality, all of which are examples of what evidence based medicine isn’t:
- Decision making by anecdote
- Decision making by press cutting
- Decision making by expert opinion (eminence based medicine)
- Decision making by cost minimisation.
Sackett and colleagues have recently helped us by dissecting the parts of a good clinical question:
- First, define precisely whom the question is about (i.e. ask “How would I describe a group of patients similar to this one?”).
- Next, define which manoeuvre you are considering in this patient or population (for example, a drug treatment) and, if necessary, a comparison manoeuvre (for example, placebo or current standard therapy).
- Finally, define the desired (or undesired) outcome (for example, reduced mortality, better quality of life, overall cost savings to the health service, and so on).
Searching the literature
Dr David Jewell, writing in the excellent book Critical Reading for Primary Care, reminds us that there are three levels of reading:
- Browsing, in which we flick through books and journals looking for anything that might interest us.
- Reading for information, in which we approach the literature looking for answers to a specific question, usually related to a problem we have met in real life.
- Reading for research, in which we seek to gain a comprehensive view of the existing state of knowledge, ignorance, and uncertainty in a defined area.
Getting your bearings (what is this paper about?)
It usually comes as a surprise to students to learn that some (the purists would say up to 99% of) published articles belong in the bin and should certainly not be used to inform practice.
Most papers appearing in medical journals these days are presented more or less in standard IMRAD format: Introduction (why the authors decided to do this particular piece of research), Methods (how they did it and how they chose to analyse their results), Results (what they found), and Discussion (what they think the results mean).
If you are deciding whether a paper is worth reading, you should do so on the design of the methods section and not on the interest value of the hypothesis, the nature or potential impact of the results or the speculation in the discussion. Strictly speaking, if you are going to trash a paper, you should do so before you even look at the results.
Common reasons why papers are rejected for publication
- The study did not examine an important scientific issue
- The study was not original – that is, someone else has already done the same or a similar study
- The study did not actually test the authors’ hypothesis
- A different study design should have been used
- Practical difficulties (for example, in recruiting subjects) led the authors to compromise on the original study protocol
- The sample size was too small
- The study was uncontrolled or inadequately controlled
- The statistical analysis was incorrect or inappropriate
- The authors have drawn unjustified conclusions from their data
- There is a considerable conflict of interest (for example, one of the authors or a sponsor might benefit financially from the publication of the paper and insufficient safeguards were seen to be in place to guard against bias)
- The paper is so badly written that it is incomprehensible.
Three preliminary questions to get your bearings:
- Why was the study done and what hypothesis were the authors testing?
- What type of study was done?
- Was this design appropriate to the broad field of research addressed?
Primary studies, the stuff of most published research in medical journals, usually fall into one of three categories:
- Experiments, in which a manoeuvre is performed on an animal or a volunteer in artificial and controlled surroundings.
- Clinical trials, in which an intervention, such as a drug treatment, is offered to a group of patients who are then followed up to see what happens to them.
- Surveys, in which something is measured in a group of patients, health professionals, or some other sample of individuals.
Secondary research is composed of:
- overviews, which may be divided into: (a) (non-systematic) reviews, which summarise primary studies; (b) systematic reviews, which do this according to a rigorous and predefined methodology; (c) meta-analyses, which integrate the numerical data from more than one study
- guidelines, which draw conclusions from primary studies about how clinicians should be behaving
- decision analyses, these use the results of primary studies to generate probability trees to be used by both health professionals and patients in making choices about clinical management or resource allocation
- economic analyses, which use the results of primary studies to say whether a particular course of action is a good use of resources.
Terms used to describe design features of clinical research studies
- Parallel group: Each group receives a different treatment, comparison with both groups being entered at the same time. In this case, results are analysed by comparing groups
- Paired (or matched): Subjects receiving different treatments are comparison matched to balance potential confounding variables such as age and sex. Results are analysed in terms of differences between subject pairs
- Within subject: Subjects are assessed before and after an comparison intervention and results analysed in terms of within subject changes
- Single blind: Subjects did not know which treatment they were receiving
- Double blind: Neither investigators nor subjects knew who was receiving which treatment
- Crossover: Each subject received both the intervention and control treatments (in random order) often separated by a washout period of no treatment
- Placebo controlled: Control subjects receive a placebo (inactive pill), which should look and taste the same as the active pill. Placebo (sham) operations may also be used in trials of surgery
- Factorial design: A study that permits investigation of the effects (both separately and combined) of more than one independent variable on a given outcome (for example, a 2 x 2 factorial design tested the effects of placebo, aspirin alone, streptokinase alone or aspirin plus streptokinase in acute heart attack).
Broad topics of research
Most research studies are concerned with one or more of the following:
- Therapy – testing the efficacy of drug treatments, surgical procedures, alternative methods of patient education or other interventions. Preferred study design is randomised controlled trial
- Diagnosis – demonstrating whether a new diagnostic test is valid (can we trust it?) and reliable (would we get the same results every time?). Preferred study design is crosssectional survey in which both the new test and the gold standard test are performed
- Screening – demonstrating the value of tests that can be applied to large populations and that pick up disease at a presymptomatic stage. Preferred study design is crosssectional survey
- Prognosis – determining what is likely to happen to someone whose disease is picked up at an early stage. Preferred study design is longitudinal cohort study
- Causation – determining whether a putative harmful agent, such as environmental pollution, is related to the development of illness. Preferred study design is cohort or case-control study, depending on how rare the disease is, but case reports may also provide crucial information.
Greenhalgh, T. (2001). How to Read a Paper (2nd ed). London: BMJ Publishing Group.