Capturing the effects of natural products can be complex. The wide range of nutraceuticals and supplements available deliver an even wider range of effects on human health. Yet, many of these effects are far too nuanced to be reflected with simple lab tests or documented through standard medical tools.
For some supplements, lab tests or medical tools are the best way to measure an outcome. If measuring a supplement's ability to maintain healthy cardiovascular function, our team prioritizes heart rate, blood pressure, cholesterol, and related outcomes such as A1C.
But for a majority of natural product effects, the best way to measure an outcome is to collect reports directly from the patient. In these cases, lab tests not only add bloat to the trial budget; they place the entire trial at risk of failure.
So why don't all research organizations prioritize self-report outcomes? One primary reason is confusion about the difference between objective outcomes, subjective outcomes, and the role of self reporting of symptoms. Another is a lack of high quality, validated instruments from which to choose.
Many companies, and even research organizations, mistakenly classify all self-reported outcomes as subjective. This is based on an archaic medical system in which patients are not trustworthy sources of information about their own bodies.
Ironically; this same attitude is what drives many patients to the natural products industry. The notion that patients are not the best sources of information about their own symptoms is incompatible with patient-centered care.
Avoidance of self reported outcomes not only neglects this reality, it results in reliance on proxy measures that are less reliable indicators. Salivary cortisol is often used to quantify stress levels. Heart rate are used to measure pain levels. Yet, study after study reveals that these lab or machine-generated outcomes are poor indicators. More often than not, substitution of proxy lab procedures for patient-sourced information results in sloppy data, and sloppy data can destroy your entire trial.
Patients are the best source of data about their own symptoms
A subjective outcome such as pain, fear, stress, or anxiety is best assessed by going directly to the source: the patient who experiences these symptoms. When a high quality validated instrument is used for self-report, the data it produces can far exceed the quality of data available through lab proxy measures.
Patient reported outcomes (PROs) include “any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else.” (National Quality Forum)
PROs are specifically beneficial for natural product trials because one of their greatest features is the ability to track changes in clinical symptoms over time. They provide more comprehensive, holistic data about how a trial participant's life improved with the test supplement. Because the primary goal of a clinical trial is to document these changes, the ability to accurately and comprehensively track these changes is critical for success.
Another key benefit of using PROs in natural product research is that this approach is patient-centered. It captures the outcomes that actually matter to your target market. While it is definitely exciting and impressive to learn that a supplement can influence key biomarkers, this isn't what your customers care about when standing in the health food store considering your product. What your customers really want to know is whether or not your product will help them remember all of the little things they need to finish in a day. Or whether or not your product will help them to have enough energy to keep up with their grandchildren this weekend. PROs deliver quantifiable results on the metrics that really matter to your customers.
Self Report vs Subjective
While PROs capture subjective outcomes such as pain scores or fear, they can also capture objective information. The definition of self-report simply involves the source of the information, it does not dictate the type of information.
For example, If you ask me for my pain level on a scale of 1-10, the number I provide is subjective. It is my estimate about my feeling. Conversely, if you asked me to report how many times I vomited yesterday, the number I provide is objective. Vomiting is not a subjective experience; the number of times your body ejected its stomach contents is an objectively clear figure.
This also doesn't mean self-report data are flawless. If you asked me how often I vomited yesterday, I could easily report that the number is zero. If you asked me how often I vomited during my first pregnancy, my self-report number may be the best estimate of the factual number, but it is also likely to be inaccurate because my memory no longer holds that information. Self reported outcomes can be misremembered or forgotten, they can be exaggerated or minimized, or they can be inaccurate due to misunderstandings about the question. They are subject to what methodology experts refer to as measurement error.
Every outcome measurement is prone to error. If you've ever had an odd reading on a blood pressure cuff or had to adjust your bathroom scale, you've encountered measurement error. Lab errors occur in up to 3% of tests. The notion that something obtained from a source other than the patient is more accurate is a false, outdated concept.
Valid and Reliable: The Gold Standard
The key to accurate outcome measures, whether self reported, from a lab test, or from a medical device, is validity and reliability. In measurement science, validity refers to the ability of the tool to accurately capture a measurement, and reliability refers to the consistency of the tool to capture a measurement. In other words, reliability = consistency and validity = accurate. We want tools to be consistently accurate. The same scientific and statistical tests to capture reliability and validity are used whether the measurement tool is a device, a lab test, or a patient questionnaire.
In many instances, the most accurate tool to measure an outcome is patient reported questionnaire.
How do you choose a quality self report measure?
The key to scientific quality is not whether or not the source of data comes from the patient; the key is how accurately it is measured. For something like a blood pressure cuff, confirming accuracy is rather straight-forward. For something like measuring stress, the process is far more complex. Simple visual analog scales (i.e. On a scale of 1-10, how bad is your...) are the most commonly used self report measure but they are neither comprehensive nor objective.
Capturing concepts such as sleep quality or brain health requires highly technical measurement instruments which were created for your target market and have undergone multiple rigorous statistical tests to confirm accuracy. Creating a measurement instrument is a task that is neither quick nor simple, but with the right measurement tool, PROs can unlock an entire new range of substantiated claims.
