Aakar Patel: There’s an uncertainty over opinion polls in India, can they be trusted?

Opinion polls are uncertain about who is going to win the Assembly elections in Madhya Pradesh and Chhattisgarh. Out of nine opinion polls published on Friday December 8, eight said Congress would win Rajasthan and similarly the TRS is expected to win in Telangana. I was surprised to see that there was uncertainty over MP and Chhattisgarh. In fact the average of the polls suggests that the Bharatiya Janata Party will lose its majority in both Chhattisgarh and MP. These two states have looked like they are following the Gujarat model. These are all two-party states where the Congress seemed to be permanently in the opposition. If the opinion polls are right, this could be ending on Tuesday, December 11 .

However we do not know for sure, and because there is a problem with opinion polling in India. In the United States, presidential elections can be predicted by an opinion poll with a sample size of less than 1,000. This is done over the telephone and often by a robocall, meaning a recorded message which asks the voter to select options from a menu. In the 2012 election, one psephologist, Nate Silver, predicted accurately the result in all 50 of America’s states.

America has only two parties, the Republicans and the Democrats. They have other divisions like blue collar and white collar workers, and rural and urban residents. Lastly they have race, the biggest minorities being Black and Latinos (meaning the Spanish speakers). Opinion polls factor in these differences and are able to accurately predict who will win. On election day in America, news networks have exit polls that predict the results even before counting begins and these predictions are almost always right.

In India we have the most complex political society of any democratic nation. We are divided by language, by economics, by cultural differences, particularly comparing the north to the south and the northeast. We have the division, which has become increasingly pronounced, of religion. And we have the unusual and unique division of caste. This makes it difficult to predict elections, especially those which are close. Very large sample sizes are needed in India and this costs a lot of money because the exit polling agencies must send out field agents to interview people.

Some agencies get around this by unique solutions. There is a successful agency called Today’s Chanakya that says it has a different way of assessing each constituency. It sometimes does not reveal to respondents that it is conducting a opinion poll.

One of the biggest problems pollsters have is that people often lie in India. Not because we are devious people but because we have no confidence in how this information will be used and whether revealing it will harm us in some way.

The agency CSDS does a post poll survey in which it sends its staff to people’s homes and conducts in-depth 30-minute interviews. It seeks to know why people have voted in the way that they have and what are the issues concerning them. They have a relatively small sample size because of the focus on depth and quality rather than quantity.

Often the polling agencies seem to be merely guessing. I can remember one general election about 15 years ago in which the agency was headed by my friend, the pollster GVL Narasimha Rao (who is these days a leader of the BJP). The show was being hosted by another friend, the columnist Swaminathan S Anklesaria Aiyar. As the results started coming in, it turned out that Narasimha Rao’s agency had got almost every state wrong. Aiyar asked him about it, and Rao said that his national total was more or less right. This is of course not acceptable and it’s just guesswork pretending to be science.

One way of getting around the unpredictability of polling in India is to give a band or a range. If we look at the Madhya Pradesh polls, four agencies have predicted not a specific number but a range, sometimes as wide as 20 seats. This means that they are not confident of their polling figures and not even sure about who will win. The data is confusing or the numbers are too close and one thing this indicates is that the dominance of the BJP in these states is waning though the prime minister campaigned hard.

The results are very important nationally because in the Lok Sabha elections of 2014, the BJP had won 62 out of the 65 Lok Sabha seats in Rajasthan, MP and Chhattisgarh. The confidence of the opposition will go up and alliances will be easier to form if it becomes apparent that the BJP is losing ground in the north of India. In that sense of all the Assembly elections that we have seen since 2014, these are the most important.