Irish Polling Indicator, update May 2014

Note: the last Red C poll does not provide separate estimates for the Green Party and Others, therefore the OTH and GP line end in April.

Pooling Irish Opinion Polls: Introducing the Irish Polling Indicator

Opinion polls are a source of information for voters, parties and the media alike. Especially polls into the popularity of parties are usually quite popular. Polls can be a powerful instrument: if done well, a sample of just 1000 voters can give great insights into the (political) views of the entire Irish electorate. But polls are often misunderstood and incorrectly reported. A new tool brings together all Irish opinion polls and aggregates them into one estimate of the support for Irish parties.

The ‘Poll of polls’ has become popular in many countries over the last few years. Besides the popular electoral forecasts by the US website FiveThirtyEight, several sites have been aggregating opinion polls in Canada, the United Kingdom, Germany and the Netherlands, to name a few. The basic logic is simple: if you pool together all available information, you get the best estimate of parties’ electoral support. At the same time, we can remedy some of the problems with the reporting of opinion polls.

One problem with media reports on opinion polls is that they usually ignore the margin of error. As polls are usually based on only 1000 interviews, results can vary by chance just because of the selection of the sample for that particular poll. If you would repeat the exact same poll on the exact same date, you would get slightly different results. This is called the ‘margin of error’.

While most Irish pollsters report this margin, it often does not get mentioned at all in media reports. What is even worse is that all too often small changes in parties’ support are presented as major shifts in their electoral fortunes, while most likely this is just statistical noise. If a party moves up one or two percentage points in a poll, this is probably just sampling error. Media need to present these margins more clearly.

There is another issue: different polling companies use slightly different methods to select respondents and to interview them. Therefore, there can be structural differences in the electoral fortunes of parties between different polling companies. For example, on average Red C research estimates Labour about 1 point higher than the average polling company, while Behaviour & Attitudes estimates them almost a percentage lower. This is most likely related to how these firms account for undecided and the probability that voters will actually turn out on the election day. If we want to compare polls from different companies, we should take these ‘house effects’ into account.

The Irish Polling Indicator aggregates the opinion polls done by the four main polling companies in Ireland: Behaviour & Attitudes, Ipsos MRBI, Millward Brown and Red C. Using a method based on the work of political scientist Simon Jackman, I combine those polls into one estimate of parties’ support. Previously I have used the same model in the Netherlands. I take the house effects into account as well as the error margins. I will elaborate on the method in a future post - it is not just an average of polls - but first have a look at the results.

The figure above presents the Irish Polling Indicator estimates for the Irish parties since the 2011 General Election. For each party the dark line is the ‘best estimate’ while the shaded area indicates the uncertainty of the estimate. We see a decline of Labour and Fine Gael since the election, while Fianna Fáil has gained ground until about a year ago and lost slightly since. Sinn Féin and Others/Independents have mainly seen an increase in support.

We can see that in the most recent polls Fine Gael has dropped to a level close to Fianna Fáil: the error margins of the two parties now overlap slightly. If the difference becomes even smaller, we would not be able to tell from the polls which party has currently more electoral support. This is true, for example, for Sinn Féin and Independent/Others, which have been very close in terms of support for at least a year now.

The Irish Polling Indicator arguably provides the best summary of what election polls tell us about the support for parties. At the same time, it invites us to take limitations of polls, for example their error margins, more seriously. Polls are very helpful if used as an indicator for trends in political support, not small short-term changes.

I will update the figures regularly. In upcoming posts, I will discuss the method to calculate the Polling Indicator in more detail.

This post was posted earlier on politicalreform.ie

Technical note (for statistical nerds): the Irish Polling Indicator uses a Bayesian state-space model to estimate the support for each political party on a given day. Each poll is taken as a (noisy) indicator of party support on the day it was taken, taking into account the ‘house effect’ from the company that conducted the poll. The reported intervals are 95% Bayesian credible intervals from the posterior distribution.