On February 19, Ramdev of Patanjali Ayurved announced the results of what the company claimed was a groundbreaking new research trial of its supposed Covid cure, Coronil. Union health minister Harsh Vardhan and transport minister Nitin Gadkari spoke at the event in Delhi and endorsed both the trial and the treatment.

The results of the trial were detailed in a paper by Ganpat Devpura and colleagues that was published in a scientific journal most doctors might not have heard of – Phytomedicine. Owned by the scientific publishing company Elsevier, Phytomedicine describes itself as a “therapy-oriented journal” that publishes research on the “efficacy, safety, quality and mechanism of action of specified plant extracts”.

In his talk at the launch, the health minister said there was no reason now for any doctor to hesitate about prescribing Coronil for patients infected by the novel coronavirus. The research trial had been done and published, he declared, and the results showed a clear positive result.

Several top newspapers ran with Patanjali’s claim that Coronil was the first “evidence-based medicine” for coronavirus and that the World Health Organisation had recognised it. Pooja Chaudhuri of Alt News quickly showed the claim of WHO approval to be false, but not before it had been carried by many more news outlets, such as Mint, ABP, and Business Standard. Acharya Balkrishna, Patanjali’s managing director and one of the authors of the research paper, later clarified on Twitter that it wasn’t WHO but the Drugs Controller General of India that had provided the Certificate of Pharmaceutical Product to Coronil.

So is all the hype around Coronil justified?

The Phytomedicine paper leaves much to be desired in its scientific quality, making the trial results highly suspect. But let’s start with the clinical trial protocol submitted to the Clinical Trials Registry of India where its entry was last updated on December 25. It was proposed to be a single-centre placebo-controlled randomised clinical trial of an Ayurvedic treatment in 120 patients, aged 15-80, with coronavirus infection and experiencing none, mild, or moderate symptoms. The primary endpoint was virus clearance to be measured at three, seven, and 14 days after randomisation. The treatment was a combination of three tablets – Ashwagandha, Giloy extract, and Tulsi extract; powder of Swasari Ras; Anu Taila nasal drops.

There was no account of how the sample size was determined nor a clear statement about the size of the treatment effect that was considered to be clinically worth detecting. One of the secondary endpoints, besides measurements of blood markers of immunological activity, was symptomatic relief, that is, conversion of symptomatic status to asymptomatic at days three, seven, and 14. This wasn’t reported in the paper, presumably because, as events turned out, the patients recruited were largely asymptomatic.

The introduction to the paper tells us that “Ayurvedic medicines were effective against the 2006 Chikungunya outbreak in India”, quoting a reference that turns out to be a single report about “the first known case of a Covid positive patient treated entirely with Ayurveda”. He was an investment banker in New York who received Ayurvedic treatments prescribed remotely from Chennai, and recovered at home.

In the event, between May and June 2020, the trial recruited a total of 95 mostly younger age group patients with a positive RT-PCR test, with 45 randomised to the treatment arm and 50 to the placebo arm. There is no clear description of the symptomatic stage of the patients recruited other than the statement that “patients who had no or mild symptoms…were referred to the test site”. We have to assume, therefore, that the patients studied were mostly asymptomatic. This may be significant in interpreting the main finding of the study.

What did the study find?

In summary, the study reports this main result: of the 50 patients in the placebo arm 25, or 50 percent, were RT-PCR negative on day three, compared to 32 of the 45 (71 percent) treatment arm patients. By day seven, all of the 45 treatment arm patients (100 percent) and 30 of the 50 (60 percent) placebo arm patients were virus-negative. The authors argue that this faster clearance of the virus is the main takeaway of the trial.

The paper presents the same data to claim a relative risk reduction of 62 percent at day three and 100 percent at day seven. However, when the primary endpoint is time-to-an-event – “event” in this case being virus clearance – it’s inappropriate to artificially dichotomise the outcome at arbitrary cutoff time points. For example, there is no data presented on the viral status of the remaining 20 placebo patients at day 14. Presumably, since no deaths were reported and these were all asymptomatic patients they too would have become virus-negative at some point. Since the main benefit claimed is the faster viral clearance, the Kaplan-Meir curves and the associated odds ratio test is the appropriate analytical test.

But do these results stack up?

On the face of it faster viral clearance is a potentially valuable outcome. However the results in this study are at odds with what we know about the natural history of coronavirus infection. Consider for a moment the 50 patients on the placebo arm of the study. Half of them were RT-PCR negative on day three and another five had turned negative by day seven. It’s one thing to attribute to the drug high rates of viral clearance in the treatment arm. But in the placebo arm? We know from previous studies that patients remain infected even if asymptomatic for upto two or even three weeks, sometimes longer.

One logical explanation, indeed the obvious one, is that it is by no means certain that all, or even most, of the patients were enrolled as soon as they were infected. On day one in the trial, many of the patients may well have been at an advanced stage of the course of their asymptomatic infection. It is not unlikely that the 45 patients in the treatment arm may have been, by pure chance alone, further along the natural history of asymptomatic infection than those in the placebo arm. This alone would explain findings that have been described by some commentators as too good to be true. This goes to the crux of the question: why do a randomised controlled trial on asymptomatic patients in the first place? It’s bound to lead to a bias known as the lead time bias.

This bias is a trap for the unwary investigator. In prospective studies the starting point for the study is trial recruitment. Day zero is the date the patient was enrolled in the study but the infection could have occurred sometime before. We don’t know how long before day zero the infection started, but let’s say it was a week earlier. At day zero, we give some placebo and observe that the infection cleared on day seven. The investigator concludes that the infection lasted seven days. This would obviously be a mistaken or “biased” conclusion. The infection ran its usual natural course of two weeks, and because we first observed it midway we are misled into thinking it lasted only seven days.

In studies of cancer screening, lead time bias is well known and usually overestimates the benefits of early detection. What we may have observed here is the opposite. If the RT-PCR test was done towards the fag end of the patient’s infection then we observe apparently quick viral clearance not because our drug does any good but simply because we did not start to count from the true start of the infection but from when we first observed it. This infographic explains the idea of lead time bias.

Is there evidence of lead time bias in the Patanjali trial?

Yes. One piece of evidence is the remarkable and almost incredible rapidity of viral clearance in the placebo group.

But for lead time bias to give a false positive result in favour of the treatment in this trial, the extent of the lead time bias must be different in the two groups of patients. Random allocation would be expected to correct for this, but did it in this case?

This is where the data presented in Table 4 of the paper might help us. The authors measured blood levels of an immune mediator known as Interleukin-6 (IL-6, for short) on days one and seven in each of the two groups of patients.

They use this to argue that there was a smaller percentage change in IL-6 between day one and day seven in the treatment group than in the placebo group. Bizarrely, the argument seems to be that the treatment reduces the risk of cytokine storm. This flies in the face of the asymptomatic nature of all the patients and the rapid viral clearance even in the placebo group. This analysis, ignoring the paired nature of the data, is statistically flawed, but leave that to one side for now.

Table 4 gives us the raw data on a representative sample of 10 patients in each group.

I plotted this data to show that the day one levels of IL-6 in the treatment group are uniformly higher than in the placebo group. It is possible therefore that the two groups, despite the randomisation, were very different in respect of the stage of the disease process. The treatment group may have been further advanced in the course of the infection and, therefore, their immune response had peaked, and they were also further along the natural process of viral clearance. We can’t know for sure but it's a plausible explanation of these unusual findings.

The small sample size

Compared to many other studies of Covid treatments where efficacy has been convincingly demonstrated in clinical trials involving thousands of patients, a sample size of 95 is tiny. The likelihood of a false positive finding is high. The authors acknowledge this, but then argue that “the robustness of the observations obtained were verified through rigorous statistical analysis to make up for the small sample size”. This is both a logical fallacy and a factual inaccuracy. Some of their statistical analysis is flawed as in their use of intergroup comparisons that ignore the paired nature of the data. But more tellingly, no amount of “rigorous statistical analysis can make up for...small samples”. That would amount to falling victim to the aphorism that “if you torture a data set enough it will confess to almost anything”.

Patanjali’s previous claims

This isn’t the first time Patanjali has made claims about the effectiveness of Coronil. In the summer of 2020, Patanjali claimed a breakthrough for the same product. Abhishek Sharma, one of the authors of the new paper, and a doctor at the National Institute of Medical Sciences in Jaipur, where this trial was done, is quoted as saying on June 23, 2020:

“As many as 45 patients were given the inactive form of the drug and 45, the active form. The recovery rate at the end of three days was 69 per cent in active form and 50 per cent in inactive. At the end of seven days of administration of the drug, we found 100 per cent recovery rate in those who were given active form and 65 per cent in inactive form.”

Those results are almost identical to the numbers in this paper. Clearly, it has to be the case that the bulk of the enrollment was done by June 23. It is also strange that Sharma had said it was only a pilot study at one centre, Jaipur. At the time, the AYUSH ministry ordered Patanjali not to advertise Coronil as a cure for Covid. Curiously, while the document that carries this order is featured on the homepage of the AYUSH ministry, the link itself is broken.

Be that as it may, the components of the treatment used in Patanjali’s study were already included in the government’s clinical management protocol for the use of Ayurveda and Yoga in Covid, updated last in October 2020.

Where does this trial leave us?

In the debate over the place of traditional systems of medicine, Ayurveda in particular, this trial takes us no further forward. Publishing a trial is but one step in the iterative scientific process of development and evaluation of treatments. This trial is not the coming-of-age scientific breakthrough for Ayurveda that some are claiming it is. Just as the robustness of a criminal justice system is judged not by its conviction rate but by the acquittal rate, Ayurveda would have come of age when they publish the results of a randomised controlled trial rejecting a drug as “no better than a placebo”. Until then, I stand by my earlier assessment that Ayurveda should market its claims on faith, not science.