A Poorly Conceived Study Fails to Prove Ayurveda Works | Science-Based Medicine (original) (raw)

_[Editor’s note: Here is a guest post from Dr. Apoorva Chandra, an emergency medicine physician from India, currently working and training in the UK. Welcome!_]

As COVID-19 spread across the world, another significant problem came to light: the spread of pseudoscience and misinformation.

Although this isn’t a new problem, the pandemic-struck world has become a fertile ground for the proponents of pseudoscience to spread pseudoscience and misinformation in the name of so-called “alternative” medicines and magic cures.

What’s more worrying is that science literacy is so uncommon that even some scientists and doctors are promoting absolutely baseless claims and alternative therapies in the name of science! The failure of professionals to understand science and scientific methodology facilitates the spread of pseudoscientific information because their voices are considered by the masses to be scientific and authoritative.

Pseudoscience in India

Pseudoscience is extremely common in India and we even have a government Ministry of AYUSH (Ayurveda, Unani, Siddha and Homeopathy…Yes! HOMEOPATHY) dedicated to peddling pseudoscience. There are even government-run homeopathic medical colleges churning out “doctors”. Imagine someone being trained as a “doctor” in an imaginary “science”! In short, pseudoscience is legitimized by the government itself in India. I don’t mean to personally attack the “doctors” trained by these colleges and universities. They are merely victims of a system which does not understand science. If only they could realize this and make the leap to real science!

Apart from this, pseudoscience is visible everywhere – newspapers, TVs, banners, and at small talks. There’s more pseudoscience in the public sphere in India than there is real science.

Since the COVID-19 pandemic began, there have been dozens of claims of cures and effective treatments – most of them complete nonsense. One recent entry is Baba Ramdev’s “Coronil”, an “Ayurvedic medicine” proclaimed to have a 100% success rate. It is the epitome of pseudoscience, false claims, and misinformation. This has even turned political. Many doctors are supporting this in the name of “open mindedness“, clearly demonstrating that they don’t understand what science is.

What triggered my concern was an argument/debate with a doctor who was justifying alternative medicines and was also making multiple vague claims about how “Ayurveda” works and is genuine (without providing any evidence), which resulted in a series of tweets. He claimed that Ayurveda was scientific, but he demanded differential treatment for Ayurveda and less scientific scrutiny for Ayurvedic medicines. When I questioned this, his answers were once again vague, claiming it is a “different system of medicine”. He did not give any rational answer when asked “What exactly does that mean in a scientific sense?” (Is he claiming that there’s an alternative physics, chemistry, biology/physiology? He doesn’t say.) My interlocutor claimed that evidence of the efficacy of Ayurveda for stroke has been published; he provided a link to that “evidence”. I read the study, which in itself raises some serious questions about the state of science in India.

The study

The study in question is “Modulation of Cardiac Autonomic Dysfunction in Ischemic Stroke following Ayurveda (Indian System of Medicine) Treatment” in Evidence-Based Complementary and Alternative Medicine.

After reading it, I hardly know where to begin; but I’ll try to list some of my concerns.

• First, let me clarify that this is not a post against Ayurveda. Ayurveda potentially may have a few medicines that may work, because it uses substances (herbs) that may contain active components. Human beings have evolved to be good at pattern recognition, but they also recognize false patterns. The only way to know if something works is to follow a strict and disciplined scientific process with specific definitions of what we study. Saying something works just because it’s Ayurveda, ancient, or based on anecdotes is not evidence, and there can’t be a different yardstick for the assessment of efficacy, safety, or toxicity of Ayurvedic drugs.
• The study is classified under “Evidence based complementary and alternative medicine.” But there is nothing called “alternative medicine” or “complementary medicine” in science. As Tim Minchin famously said, if there was scientific evidence proving that an alternative medicine worked, it would no longer be called alternative but would just be called “medicine”. When people claim something is alternative medicine, they are usually looking for a way to bypass scientific methodology and claim fake scientific legitimacy.
• The title mentions Ayurveda as if it were a single medicine or an intervention, but the study uses a mishmash of multiple interventions and treatments whose efficacy and safety have not been established. Further, there is no rationale for that particular combination of interventions.
• It was a small, single center study with a sample size of 50. Multiple dropouts and technical difficulties resulted in analysis of only 24 patients. Heart rate variability, one of the main treatment measures, could not be analyzed in 12 patients due to the presence of artifacts in the ECG. The study has not been replicated or corroborated elsewhere. The patients were admitted to the Advanced Ayurvedic Research Institute for Mental Health and Neurosciences, creating a conflict of interest as researchers were obviously biased in favor of Ayurveda.
• Although patients were said to be randomized, there was no blinding, which makes the entire study meaningless because there are so many confounding factors and biases in play. This is a typical A vs A+B design where new interventions are added to conventional treatment and the combination is compared to conventional treatment alone. As pointed out by Edzard Ernst, this is a recipe for guaranteeing positive results that are almost certainly spurious. Subjects know they are getting special, extra treatments, so placebo responses and other psychological factors come into play.
• The introduction says “From an Ayurveda perspective, stroke is recognized as pakshaghata (hemiplegia), a disease attributed to an aberration of vatadosha (a physiologic entity)”. What physiologic entity is that? The reference cited is Charaka Samhita“. It is bizarre to make a claim based only on a book written centuries ago and call that evidence!
• “The diverse treatments advocated in Ayurveda for this disease primarily harmonize the aberrant physiology”. Wait! Wait! What is the “aberrant physiology”? And what might “harmonizing” it mean? Science demands precise terminology.
• Patients with stroke were assessed by a neurologist and by an Ayurvedic physician who made a diagnosis of pakshaghata (hemiplegia). How did they manage to agree? Were there any cases where their diagnoses differed? We aren’t told.
• What Group 2 got in the name of ‘Ayurveda’ is beyond belief. There are so many medications mentioned and so many interventions done, how on the earth would you know which one worked and which did not (if something even worked in the first place)? What is the justification or evidence for using all these interventions in those patients? This is problematic on so many different levels!
• The three main outcome measures, heart rate variability, blood pressure variability, and baroreflex sensitivity, are not routinely measured in stroke patients. Their clinical significance remains questionable.
• I’m not a statistician, but isn’t it useless to apply statistical measures to data from an ill-conceived study that couldn’t possibly produce meaningful results?
• A requisite quantity of Niramisha Mahamasha taila (medicated oil) was warmed on a water bath and applied on the patient’s body. Fun fact: The ingredients of this oil include more than 30 herbs. How did they know which herbs to mix?
• Perspiration was induced by putting patients in wooden boxes for 10–15 minutes depending on the patients’ “tolerance level”. What determined their tolerance? What is the effect of perspiration on physiology?
• Enemas were administered with a mixture of 19 herbs. Why enemas? Why that mixture of herbs? Enemas are known to potentially cause harm; is this even ethical?
• Patients were also given some medicines orally. Nobody knows what they are or how they work but they are called Ashtavarga Kashaya and Ksheerabala, made up of 8 and 3 herbs respectively! Evidence for their use? Justification?
• They used combinations of almost 60 different herbs given rectally, orally, and by massage, and they performed 3 interventions: massage, sitting in a box to induce perspiration, and enemas! They give no justification other than “It’s Ayurveda.”
• The discussion section is problematic. “The science of Ayurveda has evolved by in-depth observation and experience of scholars. However, the potential benefits of this science need to be evaluated with objectivity.” Isn’t that an admission that Ayurveda is not a science?
• “The current study demonstrated that cardiac autonomic functions can be used as a marker to detect one facet of aberrant physiology in pakshaghata (hemiplegia) and that a whole system Ayurveda treatment protocol as used here was able to modulate the same.” But they did not test whether that “modulation” produced any clinical benefits.
• “The study has used a whole system treatment protocol, adapted from Ayurveda classics keeping in mind convenience for clinical application.” This is pure meaningless jargon.
• They vaguely talk about antioxidant and neuroprotective actions of these herbs. “Antioxidant” is a word used so often in pseudoscience and so carelessly thrown around I get suspicious when someone uses it!
• They conclude that Ayurveda could be useful in stroke as an “adjuvant” therapy! This is speculation, not a conclusion supported by the data.

I’m tired and speechless! They say when assessing or appraising a study we should comment about its good points. The only positive thing I can think of is that somebody thought of doing a study and all the patients got standard therapy. Unfortunately, the second group were bombarded with a lot of different things that didn’t make sense and were probably unethical.

My larger concerns

This study was poorly designed and should never have been done. It is filled with logical fallacies, baseless assumptions, and treatments that are not supported by scientific evidence. It is full of red flags. How it got approval from the ethics committee is beyond me.

It assumes the value of Ayurveda and calls it a science, which it clearly is not.

It was done in NIMHANS (National Institute of Mental Health and Neurosciences), a premier institute of national importance, and funded by the Department of AYUSH (Ayurveda, Unani, Siddha and Homeopathy) Ministry of Health & Family Welfare. These government ministries legitimize quackery in India by legally promoting homeopathy and other alternative medicines.

Pseudoscience has already gained access to reputed academic institutions and we are on a slippery slope! It’s time we take note of this and stand up for science!

• Apoorva Chandra, MBBS, MRCEM (UK) is an Indian Emergency Medicine doctor who currently works as a Registrar in a UK emergency department while pursuing higher specialty training in EM
  View all posts

Posted by Apoorva Chandra

Apoorva Chandra, MBBS, MRCEM (UK) is an Indian Emergency Medicine doctor who currently works as a Registrar in a UK emergency department while pursuing higher specialty training in EM