Conducting clinical trials in the supplement industry comes with a unique set of challenges. Under the current regulatory structure, the nutrition industry has to be cognizant in bearing the burden of proof for marketing claims. FDA uses an evidence-based review system to evaluate the strength of the scientific evidence supporting a proposed claim about a substance/disease relationship. The review system is heavily based on evidence based medicine (EBM)—a concept that governs scientific research in the pharmaceutical industry. The underlying hypothesis in EBM is “the intervention ameliorates the condition."

This is not the case for nutrients, which are necessary for health; thus, the underlying hypothesis is “low or inadequate intake of nutrients causes or contributes to disease." With nutrients, the question is always not “whether" but “how much?" (Heaney, 2011). Unlike the pharmaceutical concept of “one drug—one disease," a simplistic view of “one nutrient—one disease" is not applicable (Heaney, 2006, 2008). Nutrients affect multiple cells and organs because they tend to work in complex systems in concert with other nutrients (Shao and Mackay, 2010). Nutrients are also homeostatically controlled; therefore, the body’s baseline status will affect the response to a nutrient intervention.

Hence, the design of studies in the context of EBM will differ greatly from evidence based nutrition (EBN). For EBM, a no-intake control group is appropriate because the hypothesis is that “adding" an intervention treats the disease, while the “absence" of the intervention does not cause disease. Nutrient interventions are in total contrast to this hypothesis. There is no true placebo, which makes it challenging to clarify the nutrient-disease relationship.

To further compound the challenges in designing dietary supplement trials, the current set of FDA-approved surrogate endpoints were developed for drug trials. The Biomarker Definitions Working Group defines a surrogate endpoint as “a biomarker that is intended to substitute for a clinical endpoint." Surrogate endpoints are expected to respond to therapeutic interventions more quickly and/or with a smaller sample size than clinical endpoints. While they may suffice for pharmaco-therapeutic interventions for these indications, they are inadequate and are not true representations of endpoints that meaningfully reflect changes in dietary supplement studies.

Dietary supplement companies are also expected to show statistically significant movement in endpoints in healthy populations of subjects. There are two problems with this. First, the question as to why it is important to show an intervention exerts treatment effects on an already healthy person is puzzling. Healthy people have normal clinical values, by definition, and changes to normal clinical values are not desirable. At worst, the intervention may lead to healthy subjects having deteriorated clinical values. For example, a dietary supplement that is effective in lowering blood pressure may cause hypotension in populations with normal blood pressure. At best, the intervention keeps healthy people healthier by normalizing or improving clinical values that are already normal, though improving low-density lipoprotein (LDL) cholesterol levels in subjects categorized as having optimal or near optimal levels is certainly a feat to accomplish. This leads to a high percentage of studies that fail to achieve between-group significance in their results. Therapeutic drugs treat a disease while supplement interventions are designed to decrease the progression or more ubiquitously prevent a disease. In this context, is it still possible to expect statistically significant heterogeneous group differences? Does inability to prove statistical between-group significance suggest that the investigational product has “no effect"?

Second, the treatment effect of an intervention is determined by aggregating study data, and then the differences pre- and post-treatment are compared and assessed using statistical models and equations such as the paired t-test, analyses of variance (ANOVA) or covariance (ANCOVA), as appropriate (Eisen, 2007). For larger studies, it is possible that small differences can reach statistical significance, even though the improvement is not clinically meaningful. As well, the analysis of aggregated data may be of limited clinical value and of little help for clinical practice.

There is also a tendency for scientists and researchers to rely and focus on P-values as the “gold standards of statistical validity," above and beyond other frameworks for data analysis. P-values measure the probability of an observed result being attributed to chance. This was criticized by Nuzzo in 2014, who stated the P-value does not answer the question as to the probability of the study hypothesis being correct in the first place, and does not account for the actual size of the effect. Nuzzo argued “researchers need to realize the limits of conventional statistics" and “bring into their analysis elements of scientific judgment about the plausibility of a hypothesis and study limitation."

Evidence, based on good science, is important, and standards of proof should not be relaxed. However, the question is “whether we need as much proof of efficacy for a nutrition policy decision as we do for approval of powerful, expensive and potentially dangerous pharmaceutical agents" (Heaney, 2011). The healthy volunteer restriction for clinical studies only accentuates the small effect size decreasing the gap between the placebo/control vs. the intervention. An additional challenge with nutrient intervention studies is that a decrease in intake of a nutrient actually leads to an increase in the risk of disease or may cause disease. It is also clearly unethical to lower nutrient levels in order to study efficacy endpoints.

Therefore, the application of EBM principles should not be applied to EBN to meet the standards of acceptable evidence for policy makers.

Malkanthi Evans, D.V.M., Ph.D., scientific director/CRS division, KGK Synergize Inc., earned her master's and doctorate in physiology from the University of Guelph, Ontario, Canada, and also received her D.V.M with distinction in physiology from the University of Sri Lanka. As KGK's scientific director,  Evans contributes her expertise to research, design and the development of clinical trial protocols and writing final reports and manuscripts for publication.

Looking for more information on Breaking Away from the Pharma Model?

Malkanthi Evans, D.V.M., Ph.D., will speak on “Breaking Away from the Pharma Model" as part of the Natural Products INSIDER track in the SupplySide West Education Program. Her session will take place on Tuesday, Oct. 6, from 9 to 9:50 a.m. at Mandalay Bay in Las Vegas. Visit http://west.supplysideshow.com/edu-more.aspx for more information and to get registered.