Competent Clinical Research Supports Product Marketability

Manufacturers and marketers of supplements, nutraceuticals, and functional foods have become aware that consumers and government regulators are expecting solid evidence that their products are safe, contain the ingredients listed, and have been shown to have the effects claimed.

Food and food products can qualify for health claims in the United States only if they meet FDA requirements. Unsubstantiated claims can cost a manufacturer or marketer millions in penalties. Clinical trials provide credible scientific evidence to support "qualified health claims" that a product has a developing relationship between its dietary components and the risk of a disease. Clinical trials are also the basis for "significant scientific agreement" that confirms a relationship between dietary components and improvement in the risk of disease or maintenance of health.

In 1994, Congress passed the Dietary Supplement Health and Education Act (DSHEA), which defines FDA's authority to regulate dietary supplements. DSHEA has driven FDA to develop a strategy to establish a science-based regulatory program that is fully implemented by 2010. Over the last several years, FDA and FTC have enforced product recalls or withdrawal of claims on many companies that did not have sufficient research evidence to support the claims for their products. FDA and FTC have accelerated their emphasis on assuring safe and effective supplements, functional foods, and other complementary and alternative treatments so that the demand of rigorous clinical research has come to the arena of the industry. Anecdotal reports, testimonials, and poorly conceived clinical research no longer suffice to convince consumers to buy or stick with a product, and they certainly don't provide regulators incentive to approve one.

Experimental clinical trials are the accepted means for manufacturers to establish the safety and efficacy of their product. Pharmaceuticals are subjected to rigorous clinical trial research before they are approved and marketed as drugs. There is more flexibility of the research requirements for a nondrug supplement done without FDA regulation than for an investigational new drug (IND) with FDA regulation. Supplement manufacturers usually do not need to provide research to support the product's safety or efficacy and there is no required premarket approval for dietary supplements for human consumption. However, a substance that will be added to food is subject to premarket approval by FDA unless its use is generally recognized as safe (GRAS) by qualified experts.

In 1997, FDA issued a proposed rule (the GRAS proposal; 62 FR 18938) to create a procedure to allow anyone to notify FDA that a particular use of a substance is safe. Although the proposed notification procedure is not yet final, FDA started accepting GRAS notices in 1998. Based on a notification letter detailing the data for the safety of the product, FDA determines if it provides a sufficient basis for a GRAS determination or whether information in the notice or otherwise available to FDA raises doubt that the substance is generally recognized as safe. Clinical trials are used to support GRAS determination.

Despite the lack of direct oversight of supplement research by FDA, it is essential that sponsors meet the standard of quality that produces credible results. A poorly conceived or inaccurately implemented clinical research trial will not be accepted by the regulators or by the increasingly sophisticated consumer, so the money spent will be wasted.

The supplement industry now needs to know the good, the bad, and the ugly about clinical trial research. When the research shows efficacy, the product enjoys approval and a sustained marketing life. If partly effective, the product still has some viability and the sponsor may be able to take it in another direction that is better suited to its efficacy. When the results are negative, the product needs to be replaced with another formulation or dropped. To get the best possible results, the product's capability must be well defined and the clinical trial must be carefully designed to avoid errors that will wrongly deny a good product's efficacy.

General Principles of Clinical Research
The design of a clinical trial begins with establishing a research hypothesis, framing it as a question, and then using the question to define the plan for the research. The result revealed by competent clinical research may or may not support the hypothesis or expectations of the researcher, so it is crucial that the hypothesis be specific and based on a product's strengths. One cannot test a panacea: a specific dose, a specific indication, and a specific primary outcome measure must be targeted. The more delimited the question and the more concrete the endpoints, the more likely the research will be able to validate the use of a good product. Ancillary questions may be included in the trial by using independent secondary outcome measures. The possible questions are nearly limitless, so the sponsor must assess the necessity and feasibility, testing each one in the research. Once the question is framed, the sponsor can choose the trial type and design that will provide objective evidence to support the hypothesis.

A research project can only evaluate a subset of the total universe of subjects with a target condition, for example diabetes, so the result can never be absolute. The research will produce a statistical probability that the hypothesis is true or false within the study situation, providing precision or internal validity. Ideally the result also should reflect the situation in the universe of all target subjects, providing accuracy or external validity. The choice and design of the clinical trial is devoted to optimizing the probability of internal and external validity for the hypothesis.

To improve both precision and accuracy in a clinical trial, the sponsor should specify the operational definitions of the measurements with specific instructions on the timing and the technique so they are standardized. The observers who are making the measurements should be trained in the standardized methods so they do it consistently. Mechanical and electronic instruments should provide the sensitivity of measurement required to support the outcome variables. Automated testing should be used when it is more likely to give reproducible results. For the key data points, repeated measurements under specific circumstances can yield a mean value that will improve precision.

Accuracy is improved by making unobtrusive measurements when possible, by blinding the treatment groups, and by careful calibration of the instruments to reduce systemic biases that may interfere. Basing the endpoint outcome on measurements that give concrete numerical values provides the best way to ensure precision and accuracy in the trial results. Products that affect overall health or well-being or affect symptoms that may be subjective are hard to quantitate with numerical data. When abstract variables (e.g., quality of life) are measured, special techniques may be needed to objectify the data as much as possible. Symptom response scales should be validated before use in the trial and questionnaires should be chosen from previously validated instruments when available. Standardizing the method of obtaining subjective data and attempting to categorize it numerically helps to establish an analyzable endpoint.

Phases of Experimental Clinical Trials
The experimental clinical trial, which applies an intervention to the subjects and measures its effect during a treatment period, is the best type of trial for identifying a cause effect or treatment-response relationship. There are several types of interdependent trials that are used sequentially to get to the ultimate goal of establishing the safety and efficacy of a treatment.

The Phase I trial tests the product in normal subjects to prove it is safe and to expose any potential toxicity. The Phase I trial is usually an open-label trial, sometimes uses varying doses, and has few if any efficacy measures. Phase II trials also stress safety but are done to explore the possible effects in subjects that have the target condition. The trial attempts to demonstrate the proof of concept that is behind the hypothesized efficacy as well as in some cases establish the best dose or duration of treatment. Phase III trials test the product's efficacy specifically in a large group of subjects against the targeted condition. In Phase IV trials, a known or approved product is used under research conditions to expand the information on its efficacy and safety.

The Research Protocol
The research protocol is the heart and brain of the clinical trial. It defines in detail the purpose of the research, the outcomes expected for success, and the methods used to obtain and analyze the results. It specifies the criteria by which the subjects are included or excluded from the trial and it stipulates the application, timing, and method of every test or observation that is used in the trial. The protocol designates the rationale of the experimental intervention as well as the dose, duration, and form of the treatment. Very importantly, the protocol defines the number of subjects considered to be necessary to reach a statistically valid conclusion. The protocol defines the randomization, the measures for compliance with the study treatment, the monitoring of the trial, and the method for protecting subjects' rights and privacy. It sets out stereotyped procedures to ensure that all the data can be collected homogeneously on each subject and at each study site. The protocol parameters are the essential rules for the conduct of the trial and are inviolate. If there is any deviation from the stipulated methods set out in the protocol, the subject is invalid and dropped from the trial. The strict adherence to the protocol reduces variance that can invalidate the results. All the other documents used to conduct the trial are derived from the protocol.

Once the protocol has been finalized and approved by an institutional review board (IRB)-a requirement for all human research-it cannot be changed during the trial except by a written amendment that must also be approved by the IRB. Changes to the protocol are costly in time and money and may invalidate the data collected before the change. The sponsor's goal is to think of every contingency before finalizing the protocol.

Varieties of the Research Design
The specific design of the research protocol depends on the goals of the trial, the nature of the product, and the available budget. There are two broad categories of experimental clinical trial: the uncontrolled and the controlled trial. Uncontrolled trials simply treat a group of subjects and measure the effects on lab values or other endpoints without comparison to an untreated control subject. This type of trial is used in Phase I safety trials.

Controlled trials used in Phase II and III are required to establish efficacy. The control subject undergoes all the tests and evaluations in the trial but gets a placebo rather than the active study intervention. Rather than placebo, there can be an "active" control group that receives another known-effective product that is compared with the study product. That design is useful when denial of treatment (e.g., for hypertension) is not medically wise. A "historical" control can be used when a condition has a known natural history (e.g., death in two years) so that the treatment effect is compared with the expected outcome already established in general medical practice.

If the treatment status is known to the investigator, the trial is an "open label" trial. This type of design may be used when there is an absolute objective endpoint (e.g., platelet count, calcium level, etc.), but even then it is less credible than a trial in which the treatment status is not known. When neither the investigator nor the subject knows which product has been dispensed, the trial is called "double blind." Sometimes only the observers making the measurements are blind to the treatment if there is no need for the whole trial to be blinded. The blinding choice depends on the endpoints used and the nature of the product. Single-blind trials are easier and cheaper than double-blind trials.

The randomized, parallel, placebo-controlled, double-blind clinical trial is the most widely used in establishing efficacy and safety in medical treatments and is a good choice for most supplement trials. The parallel design takes the subjects as they are enrolled and adds them either to a treated or an untreated group, randomly. There are then two parallel groups, one treated and one not treated, which will be compared to detect a difference, if any. The subjects can be randomized to one of the two treatment cohorts after enrollment based on random number schemes or other paradigms. In blinded product- efficacy trials, it is easier to randomize the treatments rather than the subjects. The investigator dispenses the blinded study product to a subject without knowing whether it is the active or the inactive product. The ratio of treated to untreated (e.g., 2:1) can be set by the number of treatment and placebo doses that are delivered in each randomization lot. Other randomized blinded designs (run in, factorial, matched pairs) are also used in some trials.

Trials done within a single group of subjects are another option that is sometimes used in product efficacy trials. The double-blind crossover trial is the best known and uses a single group of subjects divided into treated and untreated cohorts which, at a specified time, are "crossed over" so the treated becomes untreated and vice versa. The crossover design has an advantage of reducing variables since each subject becomes his own control after crossover. It also delivers more data with fewer enrolled subjects since each subject provides both control and experimental observations. The flaw in the design is time-dependent confounding either because of an external change (e.g., cold season is over in a flu trial) or the product efficacy carries over into the untreated arm of the trial after crossover. The crossover design is good for short-acting, reversible treatments that do not rely on high value of the outcome variable for inclusion.

There are no absolute criteria that dictate the exact design for a clinical trial, but all decisions are aimed at optimizing the data collection to ensure the validity of the results. At the end of the trial, all the measurements will be analyzed using statistical methods beyond this discussion.

Conclusion
It seems clear now that supplement manufacturers must enter their products into credible clinical trials to support their claims of safety and efficacy. Although trials are expensive and take months to complete, their value in preventing liability and adverse regulatory enforcement far outweighs the cost and tribulation. Products that are not effective will never be put into trials because they will not perform. Consumers will gradually realize that silence about supportive data or reliance on feeble claims, are in fact, the signal of a product's inadequacy.

Thomas Walshe, MD, is a neurologist at the Brigham and Women's Hospital and at Harvard Medical School (Boston). He has been involved in pharmaceutical research for 25 years. He now designs clinical research trials for the supplement industry as the research director at Academic Research Associates. His e-mail is twalshe@partners.org.