Exit polls are the enfant terrible of an Indian election, more so of the Lok Sabha ones. Given the emotional, political and financial resources that both voters and politicians invest into a country-wide election, exit poll seat predictions tend to pocket controversy in one quarter or the other.

For polling companies, an enormous amount of time and capital goes into earning this notoriety: it includes marketing the polling agency, training or hiring its surveyors, sending them on the ground, creating and maintaining in-house tech and collecting and analysing the humongous data. Even so, once multiple polling agencies spend multiple hundred crores and produce their seat projections in public domain, it is only one among them hitting the bullseye.

This election season saw eight major polling players in the fray: Ipsos, CVoter, Axis My India, Neta, VMR, CNX, Today’s Chanakya and Jan ki Baat. In previous elections, the trophy of precise predictions was usually bagged by a single pollster. This year, however, this distinction went to two pollsters with one other on a close trail: Axis MyIndia, Today’s Chanakya and Ipsos respectively.

Axis My India partnered with India Today and predicted the highest seats for the National Democratic Alliance: an impressive bracket of 339-365 seats, or 352 (±13). The United Progressive Alliance led by the Indian National Congress, it claimed, would be locked between 77-108 seats. Samajwadi Party-Bahujan Samaj Party and its regional combine were projected to win 10-16 seats and other parties 69-95 seats.

With a sample size of 7.5 lakh voters, the Axis poll redeemed its prediction not only at the Centre, where the NDA won 350 seats, but also foresaw the Bharatiya Janata Party’s tally of 63 in Uttar Pradesh, where it forecasted a bracket of 62-68 seats. Its predicted range of 19-23 seats for the BJP in West Bengal also came very close to the actual figure of 18.

On May 19, however, the Axis exit poll came under widespread scrutiny for putting out these figures. A methodology lacking clarity coupled with other glitches led many to suspect the agency’s easy sweep for the NDA. “Pollsters, one expects, may be overcorrecting in favour of the BJP, particularly with the memory of underestimating its 2014 victory in mind,” said one article.

Following the brickbats, an article detailing Axis’s methodology soon appeared on the India Today website. It claimed that the survey was conducted face-to-face with voters, but was technically a post-poll survey, where the agency sent 500 surveyors to meet voters a day after the voting and not as they were exiting the booths, which is what exit poll surveys are.

It further claimed that even though the collected data might not have been adequately representative, this was fixed through statistical methods: “In the survey process, however, some communities might be over-represented, some under-represented. So, after the raw data was collected, Axis My India used statistical methods to weigh the data and factor in the representativeness.”

When the results revealed that over 340 seats poured into NDA’s pockets on May 23, Axis My India could claim victory. Its Managing Director, Pradeep Gupta, looked visibly emotional on live television.

Anchor Rahul Kanwal remarked that Gupta’s tears were “tears of joy, tears of accomplishment”. He added: “We have been trolled mercilessly over the last several days by people who have said all kinds of things … hats off to every single member of that [Axis My India] team.”

Today’s Chanakya

Today’s Chanakya, which teamed up with News24, ran another victory lap this year. The polling company earned its winning touch in 2014 when it came closest to predicting the NDA’s 336 seats tally, coupled with the BJP’s 282 seats (it predicted 340 for the NDA and 291 seats for the BJP, with a 14 seat margin).

Three hundred and fifty (±14) is the number Chanakya put on NDA’s fortunes this year, and 300 (± 14) on BJP’s. The numbers were correct. This performance was replicated in the difficult state of Uttar Pradesh, where it gave 65 seats to the BJP. In West Bengal too, the prediction was fixed at 18 with pin-point precision.

In a profile published in The Indian Express in 2013, Chanakya’s CEO VK Bajaj had claimed that the mantra behind the pollster’s stellar performance during the 2013 Delhi assembly elections was careful planning at different stages, be it data collection, sampling, tabulations, analysis and records. “We have developed our own election model which has checks and counter checks at every level. We also try to avoid any interaction with the client and hosts to avoid biases,” Bajaj had claimed.  

Ipsos

It was a debut Lok Sabha exit poll for Ipsos this year, a France-based market research company founded in 1975. The company’s India branch collaborated with CNN-News18 and projected 336 seats for the Bharatiya Janata Party-led National Democratic Alliance (NDA). The United Progressive Alliance (UPA) led by the Indian National Congress (INC) were given 82 seats and other regional parties 124 seats.

After Axis and Chanakya, Ipsos’s figures came closest to the NDA’s actual tally of 350 seats. More so, it predicted the UPA’s tally of 83 seats with greater precision than Axis and Chanakya. The pollster’s Uttar Pradesh forecast landed very near to the actual figure of 63—it had predicted 61 seats for the BJP and its allies. However, Ipsos’s silver cloud did have a dark lining: West Bengal. Their exit poll highly underestimated the BJP and gave them 3-5 seats, with 36-38 seats to the All India Trinamool Congress (TMC). The saffron party ended up bagging 18 seats in the state, with 23 going to the TMC.

The News18 website had laid out Ipsos’s methodology: the company conducted exit polls in 199 constituencies selected on the basis of past electoral results. Four to six Assembly constituencies were selected in every constituency, and six booths were studied in each of these. “Every third voter coming out after casting vote were selected,” says the article. The sample size of the whole exercise was 1,21,542 voters.

Parijat Chakraborty, the Country Service Lines Group Leader for Public Affairs at Ipsos India, told Newslaundry that the agency’s polling was representative and its analysis had “high statistical rigour”: “We chose constituencies and polling stations systematically to ensure geographical diversities of the country. Large random footprint of polling stations ensured true representation of all castes and communities. The surveys too were carried out at different points of the day to include all kinds of voters, for example, the working class voters generally come out at a certain point, the housewives at another.”  

Chakraborty was fairly confident that the Ipsos analysis would turn out to be correct: “Even if the results deviate, we made sure that our analyses was air-tight. It won’t be fair to comment on other agency’s analyses, but apparently some of those were not exit poll as the definition goes and used unreliable alternative methodologies.” So would Ipsos snatch victory away from them? “We are confident, but one can’t deny that there are misleading undercurrents. The mistakes usually creep in because of them.”

Election polling forms only a portion of Ipsos’s research. The company produces research for brands, universities and think tanks, operating in the arena of public opinion and social media across multiple countries. They’ve also had political parties as clients, which Chakraborty hesitated to disclose.

The exit poll exercise that Ipsos undertook must’ve involved a huge cost—Chakraborty agreed but refused to put a figure on it. When this correspondent asked him about who paid for it, he reiterated that he was uncomfortable to put that on record.

Neta

The most conservative exit poll tally came from the Neta camp: 242 for the NDA, 162 for the UPA and 136 for other parties. These results were also the farthest from the results.

Speaking to Newslaundry a day before the results were declared, Neta’s co-founder Pratham Mittal said that he was pretty confident of exit poll’s predictions. “When other polls predicted a landslide victory for the NDA, we wondered that either we had done something wrong, or that we are right and all others wrong,” he said.

For Mittal, the consolation was the fact that outlying exit polls that are usually correct. “We are the outlier this time,” he said, adding, “but even if we get it wrong, it’ll be something to learn.”

Of all polling companies, Neta had no face-to-face interaction with the voters it surveyed. Claiming a participation of 47 lakh people, the company reached out to 5.5 lakh voters through its app and the rest through automated calls. “For every 10 automated calls, we had one manual call as a double check,” Mittal revealed.

The survey’s core sample came down to 3-4 lakh voters. Mittal claimed that representation among this sample was ensured through algorithms that could discern a voter’s community, gender and caste to a significant degree. Interestingly, class representation was ensured thanks to the information the app collected about the phone that it was installed on.

Mittal claims that the whole project cost $2-2.5 million. It was funded by Neta’s own capital and through the capital generated through Outgrow, a polling technology company founded by Mittal. “Since we were self-funded, we could take the pressure of vested interests who weren’t impressed by our predictions,” Mittal says. He further claims that the app had no official collaboration with the TV channel News X, where Mittal was brought in as a panelist the day Neta’s exit polls were revealed. “I was on CNN-News18 too, but that didn’t imply any collaboration,” he said.

CVoter

The Center For Voting Opinions and Trends in Election Research, or CVoter, is the oldest player in the field of exit polls. On May 19, it put out its seventh Lok Sabha exit poll. The poll predicted 287 seats for NDA, 128 for UPA and 127 seats for other parties—thus missing the NDA’s mandate by more than 60 seats.

Yashwant Deshmukh is the Managing Director and Chief Editor at CVoter. He claims that he sends surveyors to every Assembly segment of every constituency in India. In every segment, these surveyors visit 2-5 booths (depending on the state) and collect 60-70 samples at every booth. Its “random probability sample” was around five lakh people.

“I’m a very conservative pollster,” Deshmukh told Newslaundry, “and I’m absolutely confident of our numbers. Our confidence has gone up because of one thing: that we have the most conservative tally. Everybody has given many more seats than what we have given. So that means the direction is right. We never give hyper seat predictions.”

He added that CVoter surveys are entirely face-to-face interactions, and surveyors are often chosen keeping in mind the demography of the assembly segment or constituency. “If a constituency is majority Muslim and the popular dialect is Braj, we’ll try to send a Braj-speaking surveyor who is Muslim. It elicits trust and helps extract the right answers from the voter,” Deshmukh said.

Besides exit polls, CVoter’s profile involves academic and social research. According to Deshmukh, the party has never had a political party as a client and its exit polls are 100 per cent funded by TV channels with which it collaborates. “We’re transparent and all our records are in the public domain,” he said. In 2019, CVoter’s exit poll clients included TV9, Republic TV, and ABP News and a string of regional channels.

When this correspondent asked Deshmukh about the total expenditure on the exit poll, he said that given polling is a year round activity for CVoter, calculating its cost as a single operation would be wrong. “Our surveys don’t begin and end around election time. If you ask me when the work for our 2019 exit poll began, I’d say May 17, 2014. We train students to be surveyors through workshops on a large scale. There is a  huge roster of people we create which are often poached by others,” Deshmukh said. He laughed and added: “And even after doing all this we tend to get it wrong.”

Deshmukh also opened up about getting caught in political slugfests for generating inconvenient poll predictions. “If you Google my name, you won’t see search predictions for ‘Yashwant Deshmukh research’ or ‘Yashwant Deshmukh articles’, it’ll be ‘Yashwant Deshmukh wife’ or ‘Yashwant Deshmukh RSS’. One side will say you took money from the Congress, the other side will say iska baap sanghi tha and so on.”

On the probability of an inaccurate prediction, Deshmukh said: “If our results turn out to be wrong as they did in 2004, we’ll simply go back to the basics and try to find out why it happened.”

It should be noted that with the exception of the 2004 polls, the accuracy rate for exit polls for Lok Sabha elections has been 97 per cent since 1980. While many journalists in the run-up to the results dissed Exit Poll predictions, it would be prudent to take these numbers a little more seriously. Transparency from pollsters on methodology and owning up when you get it wrong would help build more credibility.