Times Talk Campaign Watch: Can we trust election opinion polls?

With just over a week before voters head to the polls to choose Malta's next government, the latest episode of Times Talk delves into the science and controversy behind political polling, as current surveys continue to indicate a significant lead for the Labour Party.

Over the last weekend, a Times of Malta poll and a separate poll by Professor Vince Marmara indicated Labour was on the way to win by around 28,000 or 29,000 votes, while another Malta Today poll said the margin would be around 17,000 votes.

Times of Malta fact-checker Neville Borg and Esprimi CEO Morgan Parnis joined the podcast.

During the episode, Borg gives an overview of what the polls are showing while Parnis explained the results and why some polls differ from others, even when they are carried out during the same period.

Esprimi is the firm commissioned for Times of Malta's surveys.

If you have questions or doubts about the polls, this abridged Q&A version of the podcast might give you some answers.

For a more detailed analysis and explanation, however, we suggest you watch or listen to the entire episode.

Q: How can pollsters predict the election result so confidently if they only speak to 600 voters?

A: They do it by following a scientific methodology centred on representativeness rather than sheer volume. 

In polling terms, this means ensuring the 600 people are randomly chosen and represent a perfect cross-section of the population, covering all ages, genders, occupations, income levels, and educational backgrounds.

And people with similar backgrounds and life experiences tend to think alike, which explains why pollsters can reliably predict that one respondent represents thousands of others who will never be contacted but who most likely think, feel and plan to vote like that one respondent.

This is clear from last weekend's polls, Parnis said. Esprimi's poll arrived at the same conclusion as the Marmara poll, even though it had half Marmara's sample size.

It is true that a bigger sample would reduce room for error, or what is known as the margin of error, but not by much. For a poll that is merely trying to understand who will win the election, 600 respondents are more than enough to get a clear picture.

But beyond the sample size, the confidence comes from analytical models, Parnis added. The pollsters don't just count the raw answers. They use machine learning and back-testing to compare current data against past election results.

This allows them to account for people who refuse to answer or those who say they are undecided, further refining their predictions to a high level of accuracy.

Q: How can pollsters accurately predict which way undecided voters are actually going to vote?

A: There are several techniques of sophisticated predictive modelling known as analytical models, and the method has already worked in past elections.

Parnis explained that in their most basic forms, these models work in a similar way to how Netflix recommentswhat a user will want to watch next. The algorithm analyses past behaviour and the behaviour of thousands of other users with a similar profile. If you fit a certain demographic and behavioural pattern, the model can compute your likely choices even if you have not stated them.

The key difference is that unlike Netflix, which learns without knowing outcomes in advance, the analytical models used in polling are supervised. They are trained on known data: the actual results of past elections. "The models learn from what actually happened, and they are tested against it," Parnis said.

In polling terms, pollsters are looking at how respondents voted in previous elections, their age, education, income and other factors, and how people with nearly identical profiles are currently saying they will vote, and that is how they can narrow down the actual margin of victory.

Q: Thousands of people who are contacted refuse to take the survey. All pollsters run into this problem. Sometimes researchers get four or five refusals before reaching one person who is willing to answer the questions. Could the types of people who actually agree to take surveys lean more toward one specific party? And if this is the case, doesn't that skew the result?

A: Parnis said this is a justified concern, but they have specific scientific methods to prevent this from skewing the final result.

He explains they do not just take the first 600 people who answer. They must ensure the final 600 people match the national demographic profile (age, gender, education, etc.).

Each demographic profile must be filled precisely, once enough respondents from one group have been reached, interviewing for that group closes and the work shifts to the remaining open quotas.

The sample is only complete when every demographic segment reflects the national profile. 

Also, to combat the fear factor or the possibility that supporters of one party are more willing to speak on the phone, the pollsters use a mix of data collection methods.

Parnis notes that certain demographics who might be uncomfortable or refuse to answer a survey over the phone are often much more comfortable and willing to respond honestly through an online survey. By comparing phone data with online data, they can identify and correct biases. Esprimi is not using this method for the election campaign surveys, but Parnis said they are still collecting online data to learn from it after the real election result is officially out.

Finally, they use analytical models to interpret the data rather than just taking it at face value. These models are designed to identify patterns among those who refuse to speak or those who might be hiding their true preference.

Q: Why is one poll showing a significantly lower victory margin than the others?

A: Parnis said he cannot speak for other pollsters' methods, but the divergence does not necessarily mean that some polls are getting it wrong. It is likely down to how different pollsters interpret the data.

They could all be painting an accurate picture of the truth, but the discrepancy could result from the way they interpret the respondents who say they are undecided or are hiding their voting intentions.

Marmara, on the other hand, has spoken publicly about his methods, Parnis said, and it seems that while he uses different methods, he still uses analytical models which have provided the same results as the Esprimi poll - which is another piece of evidence which proves that the science of polling works.

Neville Borg said the science works so well that the Esprimi and Marmara polls perfectly predicted the 2022 general election result.

Q: So, what happened in the MEP election of 2024?

A: This was a significant learning moment for pollsters because the actual result was vastly different from what the surveys had predicted. And the biggest lesson was: voters behave differently in MEP elections.

Parnis attributed this to the phenomenon of protest voting. He argued that voters feel more comfortable sending a message to the government during European elections because the stakes are lower.

He said their data modelling at the time did not properly factor in this specific type of protest behaviour. Esprimi has since refined its back-testing and machine learning models to better account for this type of voter behaviour come next MEP election.

This means that the MEP election phenomenon will not necessarily repeat itself this time round.

Q: Does the PN have a chance at winning?

A: Nothing is impossible, but it is highly unlikely, according to the polls.

Parnis said that while the absolute number of the gap might vary due to the margin of error and the voter turnout, the overall trend and trajectory are completely in favour of the Labour Party.

The final countdown

As the campaign enters its final stretch, Times of Malta will be publishing one final Esprimi poll next week. Fieldwork for this poll is currently underway, capturing the final shifts in public sentiment before the nation heads to the ballot box.