Outcome measures are among the most critical decisions in clinical trial design. After all, this is what determines whether or not your product was "successful." In the case of drugs, established disease etiology dictate what constitutes a successful outcome. In the case of supplements, however, "success" is open to interpretation.
A drug to treat hypertension is effective if blood pressure levels return to safe, healthy ranges. But what constitutes better sleep? Falling asleep faster? Spending more time asleep? Having more daytime energy? Supplement claims are often less specific, leaving many options for operationalization of outcome variables.
Which begs the question, just how many outcome measures should you have? The more the merrier? Should you include a few extra just in case significance isn't found on the first choice? Does adding a few extra outcome measures provide some inexpensive protection against a failed study?
Not quite.
Increasing the number of outcome measures not only dramatically reduces the quality of your trial, it can also render your trial unusable for the purpose of claim substantiation and open the door to regulatory actions.
More Outcomes = REDUCED QUALITY
The more outcomes you evaluate, the greater the chance of finding one that is significant, by chance alone. For example, if you run 20 tests, and your p-value is the standard .05, the likelihood of getting at least one significant result is 64%. This doesn't make the trial more effective at showing that your product works; it makes the trial more likely to produce a false positive.
And while this might seem like an easy shortcut to claim substantiation, the biostatisticians at the FDA and FTC recognize it for what it is: unethical study design that leads to inaccurate conclusions. The consequences of trying to substantiate a claim with a trial that suffers from multiplicity issues can be severe. To see the extent to which this can destroy a company, look no further than Prevagen. The makers of this brain-boosting nutraceutical have lost millions to legal battles for trying to substantiate their "clinically proven" claim with a clinical trial in which significance was only found after increasing the number of statistical tests. The FTC called these claims "false and unsubstantiated," because "This methodology greatly increases the probability that some statistically significant differences would occur by chance alone."
What's the Solution?
This doesn't mean you have to risk your entire study budget on a single research question. The solution is to use measures that are fit for purpose. Rather than throwing everything at the wall, hoping something sticks, create a trial that is more efficient. Intelligent, industry-specific outcomes which target your specific goals make it possible to run clinical trials on natural products that are more likely to produce true positives.
Not only does precise selection of outcome measures increase the chance of success, it enhances the overall quality of your trial as it is no longer subject to multiplicity issues. And as an added bonus, targeting fewer but more specific outcomes reduces the total cost of your clinical trial. Higher quality trials don't require a greater investment; they require intelligent solutions customized for your unique needs.
