Number game: How India is using statistics to hide the truth about Covid

Whatever one may say about the health and family welfare ministry, it has certainly used the coronavirus pandemic to polish its skills in using statistics to mislead and misinform the country. The mandarins in Harsh Vardhan’s department are either themselves innumerate or they must think the rest of the world is.

The daily press releases put out by the government’s Press Information Bureau, on behalf of the health ministry, is a study in how to mislead with numbers. See this, for example, from the press release on May 26. In large font, it tells us:

By using the expressions “more than” and “exceeds” it clearly seeks to convey the message that India’s vaccination programme is performing well beyond expectation. But if there was a target that has been surpassed then it remains unspecified.

Another press release on the same day is clearly an attempt to absolve the government of any real or imagined criticism. This one proclaims:

The opening sentence of this statement bizarrely claims that the government “has been supporting the States and UTs by providing them Covid vaccines free of cost”. One could be forgiven for imagining that the Indian government thinks of itself as a wholesale vaccine merchant that could have turned a neat profit by selling its vaccine stocks on the world market if it had not, in a regrettable fit of generous philanthropy, decided to give them away to ungrateful and tardy states and union territories “free of cost”.

When it comes to releasing data about Covid cases and deaths, the health ministry's press releases strike a different, positively optimistic tone. This is from May 27:

Why is this misleading?

While none of the statistics and statements is factually incorrect, what they do not give us is an accurate and comprehensive picture of the course of the epidemic and the vaccination programme that holds the promise of arresting it.

Yes, total recoveries so far are in excess of 24.6 million but that is because the total number of infections now stands at 27 million. And that is just the officially reported number. Estimates of the extent of underreporting and undercounting vary. Toronto University’s Dr Prabhat Jha estimates that India’s actual cases could be 10 times higher than what is being reported. But the official press releases fail to even mention the number of infections, never mind give an estimate of the true number. They focus on the recoveries which few other countries emphasise because unless the patient dies, recovery, at least from acute illness, is the default outcome in the natural history of Covid.

Yes, test positivity rates may be coming down and that is a movement in the right direction but it is misleading not to mention how high it still is. According to Our World in Data, India’s testing rate has dropped from a high of 23 tests per confirmed case in mid-May 2020 to 4.5 on May 26, 2021. To set this in context, experts caution that a testing rate much below 30 tests per confirmed case runs a risk of missing many cases. Experience on the ground in India is that while testing may be accessed without too much difficulty in big cities and large towns, the picture in smaller towns and in villages is quite desperate.

Yes, India is vaccinating vast numbers of people but the trouble with self-congratulatory press releases claiming to have exceeded some selectively chosen and arbitrary milestone is this: data on the numbers vaccinated makes sense only when it is set against the population to be vaccinated. Very few countries can possibly vaccinate even a quarter of the population that India has for the simple reason that most countries have far smaller populations. So, while it is a statistical fact that India is second only to the United States when it comes to the number of doses administered (198 million doses as on May 26 as against 289 million in the US), it is a statistic that isn’t amenable to meaningful interpretation because it doesn’t relate this “floating numerator” to the underlying population that needs to be vaccinated.

US vice president Kamala Harris was on the ball when she tweeted that 50 percent of adult Americans were now fully vaccinated. By contrast, 43.85 million Indians were fully vaccinated as of May 27, and if we assume an adult population of about 800 million, the comparable statistic for India is 5.5 percent.

Presented with the obvious point that vaccination performance makes sense only as a proportion of the target population, the riposte from the government’s apologists is that the critics are merrily presenting the absolute numbers of infections and deaths in India and not relating those numbers to the vastly bigger population.

Manmohan Singh wrote to Narendra Modi precisely about this, asking his government to “resist the temptation to look at the absolute numbers being vaccinated, and focus instead on the percentage of the population vaccinated”.

Responding to the former prime minister, Harsh Vardhan, the health minister, sent a sharply worded and ill-tempered letter accusing him of selectively choosing, “when it suited him”, to use the absolute numbers of infections and deaths to show India in bad light.

The health minister was either ignorant or he was being disingenuous. The fact is that if the purpose of statistics is to inform our decisions then it is epidemiologically the right approach to use the absolute numbers for new infections and deaths and the population proportions or coverage for vaccination.

Absolute numbers or population proportions?

In an epidemic, new infections and deaths each day give us useful and actionable intelligence. Those who seek to present these numbers as deaths per million population – because it shows India in a good light compared to, say, America or Britain – make the obvious mistake of presenting it as some kind of international competition. India is not battling America to have a lower death rate; India is fighting the novel coronavirus to limit its spread.

In any case, inter-country comparisons depend, for whatever information they might convey, on the selection of the comparator countries. Choose the US, Britain, Italy, and France, and India’s deaths per million look good. Choose Bangladesh, Pakistan, Sri Lanka, and Thailand, and India’s figures look bad.

The right number to present, if the objective is to convey an accurate picture of the progress or control of the epidemic, is the absolute number of daily cases and deaths, suitably averaged over a seven-day period to avoid random day-to-day fluctuation because of weekend or holiday effects in collation and reporting.

When it comes to the vaccination programme, the only way to judge its performance is to monitor progress in terms of the proportion of the population covered. This is because the programme can only succeed when upwards of 70 percent of the population is protected by vaccination. Telling people that thus far we have protected about one in 20 would act as a spur to action by everyone involved because it makes it clear that there is much still to do. Proclaiming that 44 million Indians are protected sounds like a premature declaration of victory.

It is a sobering thought that India was put under the most draconian and rigorous lockdown on March 24, 2020 when the total recorded infections were a mere 536 (Pakistan then had reported 1,063 cases) and recorded deaths stood at 10 – yes, you read that right, 10! Nearly 27 million cases and 3,15,000 deaths later, we should at least have learned how to present the statistics with the intention to illuminate rather than obfuscate.