Distinguishing clinical from statistical significance

[CindyA]

Distinguishing clinical from statistical significance: Are cancer trials aiming high enough to make a real difference?

January 2nd, 2014 - by Dr. Jack West

Back in August, a press release came out about a new monoclonal antibody against EGFR, called necitumumab, which is a “humanized” antibody that is expected to lead to fewer infusion/hypersensitivity reactions than Erbitux (cetuximab), which is another antibody against EGFR. This press release from Lilly, the makers of necitumumab, was about the SQUIRE trial, a randomized phase III trial in previously untreated patients with advanced squamous cell NSCLC, in which patients were randomized between cisplatin/gemcitabine with this new agent or placebo. The press release noted that the trial was positive and demonstrated a statistically significant survival benefit. GREAT!

So why hasn’t there been any discussion of it? First, this agent is remarkably similar to Erbitux, which has been heavily tested and hasn’t found its way into treatment of lung cancer. But the bigger issue may be related also to Erbitux, ,and that is that it highlighted the distinction between statistical and clinical significance.

Specifically, a trial called FLEX came out several years ago that compared standard chemo with Erbitux to standard chemo with placebo and was actually technically a positive study, showing a survival benefit that was statistically significant. This means that the difference of the two arms in the primary endpoint, the main variable of the trial, was great enough that there is a less than 5% chance that this is due to random chance alone.

But there is a difference between statistical significance and clinical significance. If a trial is large enough, there can be a statistically significant but actually quite unimpressive in actual terms. Conversely, a smaller trial may fair to be statistically significant but still show results that are extremely promising. In the case of the FLEX trial, the study showed an improvement in median overall survival of about 5 weeks, which is arguably meaningful, but when you factor in the weekly visits for IV Erbitux, the side effects, and the cost of over $50,000/year for the drug, this agent has failed to lead the world to be sufficiently impressed enough to have Erbitux become a leading consideration in advanced lung cancer.

Now this press report states that there is a significant improvement in overall survival, but the result is as conspicuous for what it omits as for what it includes. As a general rule, companies don’t sit on good results — if anything, they provide preliminary insinuations of their newest agent being a “game-changer” (the most nauseatingly over-used term in the world today, I think), but they don’t withhold favorable findings. So when a press report alludes to a positive trial but includes NO information about the actual results in the trial, it’s as much of a red flag as a profile for an online dating site that doesn’t include a photo.

The press release also says that the full results will be disclosed at a meeting in 2014. I hope I’m wrong, but I fear that there’s a good reason we didn’t see the results in 2013. Today, a new drug is judged by the intersection of three criteria: efficacy, side effect profile, and cost, to produce a sense of the value of a drug. We don’t know the cost yet, but with new cancer drugs averaging a cost of more than $10,000/month now, they need to do better than demonstrate results that are only significant based on statistical analysis.

http://expertblog.lungevity.org/