Echinacea cold study claims analysed
Echinacea “can prevent colds”, reports The Daily Telegraph, while the Daily Mail reports that the “largest ever clinical study into Echinacea finds herbal remedy can protect against colds”.
These headlines were based on a study which found that giving healthy adults three doses of the herbal remedy Echinacea daily for four months reduced the combined number and duration of cold episodes by an average of 26% compared to placebo.
What was not widely reported in the news was that the study also reported finding no significant difference between the groups when they looked at the number of colds each group caught. So, the difference seems to have been related to how long a cold lasted, rather than the frequency of cold.
This randomised control trial was well designed and had a good sample size (755 participants), however, there are a number of oddities in the reporting of the study findings that cast a shadow of doubt over the results, such as:
- no declaration of funding and only partial disclosure of conflict of interests
- no results table
- limited reporting of unpleasant side effects
- no estimates of error around the results reported
- selective reporting of results
- the applicability of the results to the general population
Many of these basic problems would usually be picked up by the peer review process or journal editors. The lack of such quality standards may leave the reporters and editors a little red-faced. This news story should stand as a warning to journalists of the dangers of taking research at face value without bringing any critical faculties to bear.
In conclusion, based on this study alone, it is not clear whether taking Echinacea prevents cold episodes, though it does suggest it may reduce their duration. Further research is needed to confirm or refute these findings and also to see if they also apply to people with long-term health conditions such as asthma.
Where did the story come from?
The study was carried out by researchers from the Common Cold Centre in Cardiff University. The study was published in the open access, peer-reviewed medical journal Evidence-Based Complementary and Alternative Medicine.
No funding source was reported in the published research article, but three of the five authors declared no conflict of interest. Information on the other two authors’ potential conflicts of interest was absent.
The lack of funding information and incomplete declaration of conflicts of interest by all the study authors is unusual. The standard practice in all good science or medical journals is to clearly state all funding sources and conflicts of interest, or how the funder was involved in the research design or write-up. This article falls short of this standard, and this should arouse a healthy scepticism in readers.
Interestingly, both the Mail and the Telegraph reported that the study was part funded by A. Vogel, a Swiss manufacturer of herbal remedies, including Echinacea products (such as Echinacea toothpaste). This could not be confirmed from the research article alone, although the Echinacea used in the research was provided by this company, and the study is highlighted on the A. Vogel website’s blog.
The media reporting was generally focused on the finding that “taking three daily doses of the common remedy for four months reduced the number of colds and duration of the illness by an average of 26%”. This finding is the result of combining the number of colds and their duration into one variable.
The media chose not to report the finding that the number of colds alone was not significantly different between the two groups, which is also informative.
Finally, the media did not highlight the numerous and significant limitations of the study discussed below.
What kind of research was this?
This was a was a randomised, double blind, placebo-controlled clinical trial designed to assess the safety and benefit of Echinacea purpurea (Echinacea) extract in the prevention of common colds.
Common colds are caused by a range of viruses that result in the familiar symptoms of a runny nose, cough, and sore throat and sometimes complaints of headache and fever. The authors report that the common cold is the most prevalent disease in Western civilisation, with substantial related healthcare costs, so a medication to reduce this disease burden would be welcomed.
What did the research involve?
A total of 755 healthy subjects were randomly assigned to receive an extract of Echinacea purpurea (a flowering plant found in North America, supposed to have immune-boosting properties), or a placebo for a period of four months.
The Echinacea administered was a commercially available product called “Echinaforce drops”, supplied by A. Vogel Bioforce. The placebo drops were similar in shape, colour, consistency, odour and flavour. Participants and study investigators were ‘blinded’ to which treatment was given to which participant.
Participants took three doses of 0.9ml of the drops each day for four months in an attempt to prevent colds. This corresponded to 2,400mg of Echinacea extract per day. During a cold (cold episode), participants were instructed to increase the dose to five doses of 0.9ml per day (4,000mg per day). Each dose was diluted in water and held in the mouth for 10 seconds, a method described by researchers as having “maximum local anti-viral effects”, although why this would be the case is unclear.
Throughout the investigation period participants were required to keep a diary to record adverse events by answering questions, such as “did you have any unusual or unexpected symptoms today?”.
Researchers considered whether any reported adverse effects could be related to the study medication, documenting this as “unlikely”, “possible”, “probable/likely”. Participants were also asked to record and rate cold-related issues and the use of any medications in a diary.
When participants got a cold they were asked to collect nasal secretions using home kits which were then screened for viruses.
The statistical analysis employed by the researchers was basic and potentially incomplete.
No summary table of the comparisons that the researchers tested was provided in the write-up.
This lack of clarity in reporting makes it harder for the reader to see exactly what statistical comparisons have been tested, and which were significant.
The researchers combined individual measures of the number of colds caught (cold episodes) with cold duration (episode days) to create a single variable of “cumulative events” (cold episodes and episode days combined).
Combining the results in this manner falls far from the standards of transparency you would expect to see from a well-conducted randomised controlled trial.
What were the basic results?
Of the 755 participants randomised, 673 (89%) completed the study. The researchers aimed to examine two main topics, safety and effectiveness. Unfortunately, due to the way the findings were reported (and the lack of a clear results table) the exact comparison and findings of this study were difficult to pin down.
Safety
There were no significant differences in the number of adverse events reported in the Echinacea group compared to the placebo group. As mentioned, very little information was given on adverse events, in both the Echinacea and placebo group.
This is somewhat surprising as the researchers write that one of the main reasons they are conducting the study is to establish whether Echinacea has ‘a good safety profile’.
There were also no significant differences between the two groups on key blood and biochemical measures.
Effectiveness
The authors report that the placebo group had 188 cold episodes, lasting 850 episode days, compared to the Echinacea group which had a total of 149 episodes, lasting 672 episode days. They report that the combined “cumulative events” variable (number of episodes and their duration) was 26% lower in participants given Echinacea, compared to placebo, and that this was statistically significant, but without giving a 95% confidence interval level – the usual measure of statistical validity. The researchers also found that there was a 59% reduction in recurring cold infections in the Echinacea group compared with placebo (again, no 95% confidence interval level was reported). Not reporting confidence intervals for results is unusual for a randomised controlled trial.
The authors found that significantly more people (52% more, without 95% confidence interval) in the placebo group used aspirin, paracetamol, or ibuprofen to treat their cold while in the study, than in the Echinacea group. There were 58 cold episodes treated with pain medication in the Echinacea group compared to 88 in the placebo group.
The authors then did a subgroup analysis on those that had followed the study medication protocol for the entire four months of the treatment (84 people who took all doses in the Echinacea group; the number in the placebo group not reported). Those who had deviated in any way from the medication, or who had dropped out, were excluded. This sub-analysis showed that the Echinacea group had 53% fewer cold episode days, which was statistically significant (no 95% confidence level reported). The results of a comparison of cold episodes, the other key variable, were not reported.
How did the researchers interpret the results?
The authors concluded that “prophylactic intake of E. purpurea over a four-month period appeared to provide a positive risk to benefit ratio”.
Conclusion
This study appears to show that giving healthy adults Echinacea every day for four months may result in an average 26% reduction in the combined number and duration of cold episodes compared to placebo over the same period. Combining outcomes in this way is a potentially useful way of reporting outcomes. However, in some cases it is used to “fish” for statistically significant findings when the individual outcomes fail to reach significance on their own. Indeed, when this combined variable was ‘disaggregated’ the researchers found no significant difference between the numbers of colds that occurred in the two groups.
This study appeared to be well designed, and may have collected important results. However, the way in which it was written up meant that it was difficult to assess the findings. These basic problems should really be picked up by the peer review process or journal editors.
The following issues with the study's reporting muddied the water of interpreting these findings fully and clearly:
No declaration of funding and only partial disclosure of conflict of interest
The lack of funding information and incomplete declaration of conflicts of interest by all the study authors is highly unusual. The standard practice in all good science or medical journals is to clearly state all funding sources and conflicts of interest. This article falls short of this standard. While it is not unusual for commercial companies to fund, or provide drugs, for research purposes (which is not in itself a bad thing), it is unusual when this is not declared in the publication.
No results table
Unconventionally, this study did not contain a results table showing which statistical comparisons were made, for example, the number of colds in the placebo group versus the control group (and whether this was statistically significant). Describing only a few significant results in the results section makes it unclear whether other comparisons were performed and whether they were left out because they turned out to be non-significant. The discussion section of the article makes suggestions of non-significant findings that were not mentioned in the results section. The study also failed to list any adverse effects or side effects in a meaningful way.
No estimates of error around measures of effect reported
For the results that were reported, there were no 95% confidence intervals. The research does include the probability (p values), which confirms the significance of these calculations, but the 95% confidence intervals would be valuable. These can demonstrate, for example, if risk reductions only just reached significance, or, if the confidence intervals were wide, it would mean we could have less confidence in the accuracy of the estimated effect of Echinacea.
Selective reporting of results
In their discussion section, the authors state that statistically significant findings between the Echinacea group and placebo group were only found for “cumulative cold episode days” and on the use of pain medications to treat cold episodes. They also mention (in the discussion only) that the number of cold episodes alone was not statistically significant between the Echinacea and placebo group. Only raising important non-significant findings in the discussion without first mentioning them in the results section is a further unconventional practice. It also raises questions over whether the authors have reported only significant findings in the results section, which would give a biased view of their findings.
Statistical analysis
The authors highlight that the Echinacea group had a higher susceptibility to colds than those in the placebo group at the start of the study (measured by assessing the number of colds in the past). They also report that participants in the Echinacea group reported less frequent use of common pain medications. They point out that adjusting for these covariates would most likely have resulted in an even higher beneficial effect of Echinacea.
The researchers employed the double-blind randomised control trial method as this is rightly seen as the gold standard in judging whether a treatment is both effective and safe. However, they left out many of details you would expect to see in the reporting of a well-conducted randomised control trial – details that would ultimately add credibility to the results.
It could well be the case that Echinacea may have a role in the prevention or treatment of colds. But, based on the results of this study, it is very hard to state that with any certainty*.
*
