Ladies and gentlemen, it’s time to give you a break from my ponderings and let someone who knows what they are talking about have a say.
This week was a good one for vaping news. The smoking toolkit showed 18000 extra smokers quit by switching to e-cigs (yeah, it sounds low but it’s a conservative estimate based on complex math and stuff and it’s still over and above what would have quit if vaping had not figured). An update to the Cochrane review showed e-cigs can help people quit lit tobacco and showed no evidence of harm short to medium term. Irish Health reported the story here E-cigarettes do help smokers quit All good! I have however heard some commentators when speaking about this review say the quality of the studies are low which seems to imply that the Cochrane review itself should be taken with the proverbial quantity of salt. So to explain this issue I got someone far better able to do so than me. Sarah Jakes, @ on twitter if you want to follow her. Sarah is a trustee of the UK charity New Nicotine Alliance (UK). She knows her stuff!
Over to Sarah;
Cochrane reviews are one of the most highly regarded review systems in the academic world, but sometimes they can leave you feeling a little, well, flat. Let’s face it, ‘E-cigarettes may help smokers quit but the quality of the evidence is low’ is not exactly the advocacy message of the century.
To understand why this finding is kinda better than it seems you have to first understand the purpose of a review. As anyone following the vaping debate knows, practically every day brings new studies and the conclusions of those studies are sometimes problematic. This could be down to the methodology used, or it could be due to bias. It would be an impossible task for decision makers who need a question answered to wade through literally decades of conflicting studies in order to come to the correct conclusion about what the evidence currently says. That is where reviews come in. Reviews will start with a question, find all of the research that addresses the question, assess not only the quality of the research itself but also, and importantly for our purposes, how well it answers the question and how much confidence the reviewers have that the resulting effect size will not change once more research is undertaken. The primary question asked in the latest Cochrane Review was – do e-cigarettes help people who smoke achieve long term abstinence from smoking?. The conclusion was a muted ‘yes’, with confidence in the accuracy of the estimates of the size of the effect being graded from low to very low. What this means is that the reviewers believe that with more research the effect size will change. In theory it could change in either direction. Both Cochrane Reviews on e-cigarettes found only two relevant randomised control trials and both were undertaken using now obsolete cigalike type devices.
Of course the effect size is going to change! Saying otherwise would be like testing a model T Ford for fuel efficiency and then assuming that result will apply to all cars forever more. There was absolutely nothing wrong with the two RCTs themselves, but they don’t tell us much about the effect that modern devices would have. At the time the review was written Cochrane had come across a further 15 ongoing RCTs which they say appear to be eligible for inclusion in future reviews. I wouldn’t mind betting there are more in the pipeline than that, although many could be excluded for various reasons. Let’s hope that the new studies incorporate factors missed so far such as improved nicotine delivery from modern tank systems, and study designs which take into account real world use patterns. Of course RCTs will never fully quantify the efficacy of e-cigarettes in real world use, but they’ll come close enough for those who need such answers, and the rest of us really don’t care.
So there you have it, next time you hear someone try to dismiss the Cochrane review by claiming its low quality you will know they are engaging in wibble. Take what they say and apply salt generously.