nos-trum. pronunciation: \nos'-trum\. noun. Etymology: Latin, neuter of noster our, ours.
1. a medicine of secret composition recommended by its preparer but usually without scientific proof of its effectiveness.
2. a usually questionable remedy or scheme.
See here for more discussion.

Thursday, August 19, 2010

STUDY: Tai Chi and Fibromyalgia. GIGO

Touching openness by one of our premiere medical journals.  They want to prove it so bad.

In keeping with its new trend in exploring dubious therapies, the New England Journal (Aug 19) publishes a randomized, if single-blinded and poorly controlled, study of 60 people with fibromyalgia who underwent 12 weeks of Tai Chi versus education and stretching (approx 30 in each group).

The main outcomes were subjective assessments of pain reduction.

For Onion Peelers,
Mean (±SD) baseline and 12-week FIQ scores for the tai chi group were 62.9±15.5 and 35.1±18.8, respectively, versus 68.0±11 and 58.6±17.6, respectively, for the control group (change from baseline in the tai chi group vs. change from baseline in the control group, −18.4 points; P<0.001).
(FIQ is the Fibromyalgia Impact Questionnaire).

Statistically significant results.  Tai Chi works better to reduce pain.  And here's the title of the media article:  Tai chi eases fibromyalgia symptoms, study finds (Assoc Press, Aug 18).

Has the Holy Grail of alternative therapy been found?

No.  Despite the hype, despite a degree of openness on the part of the New England Journal that risks their brains falling out, despite the randomized design, and despite the statistical significance, this study represents almost every aspect of why benefit from alternative medicine therapies continues to be unproven.

The authors "dutifully" report (as the NEJM editorial says) that it would have been better if the study had a valid "sham" tai chi control.  (The subtitle of the editorial is "Are we there yet?"  Like it's assumed we will get there.)

At this point you're probably asking yourself, "What's the big deal?  They showed it helped, right?"

No they didn't.  Here are some of the reasons why not:
1.  As stated above, they didn't have a comparable control.  You can tell which subject group you're in.  This introduces a positive bias for the tai chi group.
2.  There was no equivalent group that "looked" like tai chi, but wash't .  You can't compare apples and oranges (tai chi v. stretching).  The researchers believe they met this requirement by just saying that it was a test of two different types of exercise regimen.  Right.  I wonder if ANY of the 60 people were fooled by that.  Maybe you haven't done tai chi, but can you tell the difference between it and a lecture with stretching?
3.  The study was only single-blinded.  Double blinded studies are best because neither the test subjects nor the investigators know which group they are dealing with.
4.  Measurement of pain is notoriously subject to external influences (like knowing which study group you're in).  My severe pain may be your mild pain, and tomorrow our positions may be reversed.  Also influencing subjective assessment of pain is:  how much sleep you got, your attitude, financial stress, personality type, etc.
5.  Thirty people in a test group is peanuts.  We not only need better controls, we need larger groups.  Small groups are subject to influence, and selection bias.
6.  The tai chi group were taught by a single teacher, who was a committed tai chi master.  Hundreds of studies have shown that if you have an instructor who clearly believes what he's doing, and exhibits great confidence in his ability, it influences the outcome...but it's still a placebo effect.  In the "stretching" group there were a number of different leaders of the sessions.
7.  Finally, fibromyalgia is a tough nut to crack.  Nobody can show what causes it, so patients are desperate for something that will help.  My guess is that tai chi's placebo effect will wear off over time (well beyond the 24 week follow-up that this study used).

There are other limitations, but it's clear we aren't there yet.  The first thing alternative therapies need to show is that they have some biological plausibility before we go spending a lot of money looking at them.  Then they need to meet the same requirements for benefit that all therapies do.

And before somebody starts claiming that my mind is closed, it aint.  I welcome well-designed and conducted research.  If tai chi and some other (even less plausible) therapies have one thing going for them, it's that there are few complications and side effects...unless a more serious disease is overlooked in the process.

This study doesn't hack it.   And shame on the NE Journal for failing to exhibit the same caution they would express for a new drug treatment of cancer.

Doc D


evilrobotxoxo said...

I agree that the evidence is less that compelling, but I don't think that the notion that Tai Chi would be effective for fibromyalgia is intrinsically implausible or even pseudoscientific, and I think it's dangerous to discount these results reflexively without thinking rigorously about what they might mean. Tai Chi is basically a combination of physical exercise with meditation and relaxation techniques. Since all pain syndromes have a significant psychological component, and fibromyalgia in particular, I don't think it's crazy to think that exercise + relaxation would be more effective than exercise alone. I don't even think there's anything intrinsically "alternative" about that.

As a psychiatrist, I can tell you that there is high comorbidity between somatizing/psychogenic pain syndromes and cluster B personality traits. I don't know if there's published data to support this, but it's something that is pretty obvious to anyone in medicine. A lot of people with strong cluster B personality traits respond very well to meditation and mindfulness exercises such as those in yoga or tai chi, and that's been a big part of the success of dialectical behavioral therapy in borderline personality disorder, which has a respectable evidence base. So I guess what I'm saying is that I don't see this as the promotion of woo so much as (admittedly quite weak) evidence that addressing a certain psychiatric component of the pain syndrome is helpful.

Doc D said...

Thanks for your comment. I don't think we disagree much. I wouldn't hesitate to tell a patient "There's a program of exercise and relaxation that might help your pain. Some people give it a name." But making claims about harmonizing energies and such is deceitful in the same way intentional use of placebo is.

My main complaint however is the poor quality of CAM research. Alternative therapies need to survive the same scrutiny we apply to devices, radiation, pharmaceuticals, etc., for all the reasons I listed. At this point we cannot preclude Hawthorne effect, regression to the mean, good-patient syndrome, natural history overlay, selection bias, ascertainment error and all the other gamuts treatments must run to show a benefit over and above placebo.

Again, thanks.

evilrobotxoxo said...

I agree, the quality of the study is poor, but there are lots of poor studies across the board. My main problem is not with the study results themselves, but the way the press uses the results to imply that this is some sort of victory for woo, when even if the results turn out to be completely correct, it really isn't.

Doc D said...

If they don't imply a victory for woo, only the woo merchants will.

Sorry...that was kind of a joke.

Great comments. I enjoyed it.

Post a Comment


What I'm Reading - Updated 3 May

Blog Archive