Are drug trials stopped too early?

Trials of anti-cancer drugs are being stopped too soon, says The Guardian . The “real benefit of some cancer drugs may be exaggerated because of a growing tendency for firms and investigators to call a premature halt to trials the moment a benefit appears,” the newspaper says. The Daily Telegraph says that this could mean that drugs “hailed as breakthroughs could be of less benefit or even cause more harm”.

The news stories are based on research carried out by a team in Milan which found that results of 11 of 14 trials that terminated early were used to support drug-licence applications. Carefully controlled cancer trials are essential for the development of new, beneficial treatments and technologies for serious and potentially life threatening diseases. Trials can be terminated early because there is a lack of clear benefit or because there is evidence of harm. However, terminating a trial early because there is evidence that the treatment is effective will always cause a difficult ethical dilemma: Should the trial be stopped so that all affected patients, including those who are in the trial control group, can receive the new treatment? Or should the trial continue, to the detriment of some patients, to avoid premature claims being made about the drug’s effects? This study has added some fuel to this debate.

The researchers found only 28 cancer trials that had been stopped because of evidence that the treatments were effective. In the context of the hundreds of studies of cancer drugs that take place and run to completion, this is a very small number of studies.

Where did the story come from?

Francesco Trotta of the Italian Medicines Agency, Rome, and colleagues of Mario Negri Institute for Pharmacological Research, Milan, and Utrecht University, The Netherlands, carried out this research. The authors received no funding for the research and they all work for not-for-profit organisations. It was published in the Annals of Oncology , a peer-reviewed medical journal.

What kind of scientific study was this?

This descriptive review aimed to assess the use of ‘interim analysis’ to demonstrate the effectiveness of new cancer treatments. It used all published clinical trials that were terminated early because of evidence that the anti-cancer drug was beneficial.

The researchers searched the Medline database of studies to identify all randomised controlled trials of cancer treatments, published between January 1997 and October 2007, that contained an ‘interim analysis’ of the data. An interim analysis would suggest that the results were examined before the planned end of the study. They also looked through three major medical journals to identify extra trials that their search database may have missed. Initially, 233 trials were identified. Of these, the researchers excluded all studies not relevant to their question, including trials of surgery, radiotherapy, palliative treatments, those examining different dose regimens and those terminated early due to lack of drug effect or due to harm.

In all, the researchers looked at 25 trials that were terminated early due to evidence that the treatment was beneficial. They then looked at the duration of the trial, disease studied, sample size, presence of a ‘data and safety monitoring committee’, reason for early end of the study and type of analysis carried out. The researchers used a standardised form for their data extraction.

What were the results of the study?

Of the 93 identified trials that were terminated early and had interim analysis performed, 30% were stopped due to drug benefit. Another 30% were terminated due to lack of benefit. The 25 randomised controlled trials that were analysed looked at a wide range of cancers and cancer treatments, including those for kidney, lung, gastrointestinal, breast, bladder, ovarian, pancreatic and liver cancers. Most trials compared the drug being tested with another drug treatment, while a few used an inactive placebo or no treatment. Half of the trials were published in the past three years and 11 were used in support of an application for marketing at the US Food and Drug Administration or European Medicines Agency.

Nearly half of the trials identified overall survival as the primary endpoint, and 95% of trials used the same endpoint for their interim analysis as would have been used at the end of the study. The reasons for carrying out an interim analysis included a planned cut-off date, when a number of observed events of the outcome had been examined or when a predetermined number of patients had been involved in the study. In 60% of the trials, analysis was carried out when over 50% of the sample size that was needed to demonstrate final efficacy had been reached. However, five of the trials had less than 43% of the target sample size. The studies were stopped either by switching those on the comparison drug onto the trial treatment, ending enrolment into the study or releasing study results. There were approximately two years between the end of the studies and the publishing of results.

The researchers say that, of all of the studies, 8,000 patients or treatment outcomes were planned for, but early termination meant that 3,300 potential patients/treatment outcomes were not examined.

What interpretations did the researchers draw from these results?

The researchers say that there has been a high increase in the number of clinical cancer trials being terminated early over the past three years. They say that there is concern over the early termination of cancer trials and that “the relation between sparing patients and saving time and trial costs indicates that there is a market driven intent”. Only trials that are completed can provide full evidence of efficacy.

What does the NHS Knowledge Service make of this study?

Carefully controlled cancer trials are essential for the development of new, beneficial treatments and technologies for serious and potentially life threatening diseases. Terminating a trial early because there is a clear lack of benefit or there is evidence of harm is acceptable. Stopping such a trial would allow unwell patients to receive treatments that are more effective. Patients would also be spared from receiving drugs that would not treat their condition and may even cause them harm. However, terminating a trial early because there is evidence of benefit will always cause a difficult ethical dilemma: Should the trial be stopped so that all affected patients, including those who are in the trial control group, can receive the new treatment? Or should the trial continue, to the detriment of some patients, to avoid premature claims being made about the drug’s effects? This study has added some fuel to this debate.

In this research, the search for clinical trials was carried out in only one medical database, so some trials may have been missed. The researchers identified relevant studies by using a search engine to find the term ‘interim analysis’ in clinical trials. They may have missed potential studies that did not contain this term. Non-published studies were also excluded. The identified trials are very different to each other, had different methodologies and statistical analyses, and were not directly comparable with each other. 

Importantly, the researchers only included trials that had terminated prematurely. They did not investigate this as a proportion of the total number of trials that run to completion. There are hundreds, if not thousands, of cancer drug trials that run to completion. The 25 studies that terminated early due to evidence of benefit represent a small proportion of the total research in this area. It is important to note that this study has only examined the reasons for early termination and looked at final sample sizes; it has not examined whether any of the treatments subsequently licensed went on to demonstrate lack of benefit or even harm. Based on this study, it cannot be concluded that cancer treatments currently being used are ineffective or unsafe. Much further research and debate will be needed on the subject of when to terminate cancer, or any drug, trials.

Sir Muir Gray adds...

Most ‘breakthroughs’ are not breakthroughs; scientific journals give an overoptimistic impression of progress. We should rely only on systematic reviews of all the research reports; not single studies, acting like snails and not evangelists.