Tuesday, July 22, 2008

Bad Study. Bad, Bad Study.

This study is what you call junk science. Upon seeing the title of this article I said to myself - "I bet this is a very small study, published in an obscure alternative medicine journal, measuring some subjective variable, and funded by a juice company". Well...

"Published in the Journal of Alternative and Complementary Medicine, the randomized, placebo-controlled, double-blind clinical trial claims to be the first study reported from outside China that has examined the general effects of consuming goji berry juice."

and...

"The current study was funded by Arizona-based FreeLife International, a goji berry juice supplier.  The study was also conducted by FreeLife employees."

and...

"A total of 35 people were recruited for the study.  All participants were healthy adults, and were randomly assigned to either a goji berry group or a placebo group."

and...

"Participants were given a questionnaire at the start and finish of the test period, where they were asked to give ratings (0-5) of general feelings of well-being, neurologic/psychologic traits, gastrointestinal, musculoskeletal, and cardiovascular complaints as well as any adverse effects."

Pretty convincing, no? Here are the "results":

"According to the study authors, "significant differences" between day one and day 15 were found in the goji berry group."

Awsome!! Significant results?! That's almost unbelievable!!! But, how significant are we talking here?

"Around 50-60 percent of people in the goji group reported feelings of good health, contentment, and happiness."

Wow! 50% of the people feel good? Wait, how many people in the normal population would say they feel good?

Here's the clincher:

"The placebo group reported two statistically significant changes during the 14-day period - heartburn and feelings of happiness."

Actually, I'm not sure what this statement means. The control group had increased heartburn? Was that from not drinking goji juice, or was it from the crazy placebo juice they gave them? And I'm going to go ahead and assume they meant that the placebo group had decreased happiness. Is this study even ethical? Giving people heartburn and sadness seems kind of mean.

DS

2 comments:

Science Bear said...

My favorite part is attempting to explain why this isn't a good study and what I do is actually considered science (though of course not in this field).

People believe what they want to believe when it comes to their health, so I've decided to simply wait and let them kill themselves off.

Ketan said...

Hi!

Nice post! Unfortunately, not many with 'natural' and 'organic' bent of mind would be able to see the flaws.

Take care.