This text is taken from the American Election Mail. To subscribe, click here.
Between national polls, regional surveys and averages of all kinds, indicators are flying when it comes to predicting who will win the American presidential election on November 5. But can we trust them? An American professor remains critical.
As recently as Monday, the renowned New York Times released the results of its latest poll. The American daily claims, among other things, that Republican Donald Trump has a lead of nearly five percentage points over his Democratic opponent, Kamala Harris, in Arizona, one of the key states in the November election.
On the same date, the firm Redfield & Wilton, in partnership with the British daily The Telegraphhowever, placed the two presidential candidates tied in the same state, with each about 47% of the voting intentions. A Morning Consult poll published a few days earlier gave Harris a lead. So who is right?
“We can’t claim to know what will happen on election night,” he summed up in an interview with Duty Don A. Moore, a professor at the Haas School of Management at the University of California, Berkeley. “This election is going to be very close, and [les résultats] are impossible to predict with certainty.”
The professor studies cases of overconfidence in statistical contexts related to electoral polls. Because the science of polls, despite all the good intentions of the pollsters, is undermined by sources of errors that can distort in one way or another the resulting portrait.
One of the major problems remains the samples of voters providing the basis for the data collected. Often conducted by telephone with people whose number is dialed randomly and from whom one hopes to answer a few questions, these surveys do not always prove representative of the population they claim to describe.
Of course, an “honest” pollster adjusts his results for demographic shifts, Moore points out. But even so, there are differences that can’t be predicted or quantified.
These confidence intervals ignore systemic differences that exist between the sample and the population. We should all be prepared to question the very act of trying to predict our future.
A confidence interval… too “confident”
And yet, a very large proportion—if not all—of these election barometers have a 95% confidence interval. This means, roughly, that the poll has a 95% chance of correctly predicting the outcome of an election within the margin of error it presents.
To test the accuracy of these intervals, Professor Moore co-led a study with his then-student Aditya Kotak on the accuracy of election polls in the United States. In their analysis published in the journal Behavioral Science & Policy In 2022, they compared the poll results and final scores from different elections.
The result: only 60% of the surveys studied that were published a week before the election were proven to be conclusive, falling well within the 95% confidence interval. But the further the date of publication of the survey is from that of the election, the more this proportion decreases or, in other words, the less accurate the surveys are… until they fall to a meager percentage of 40% a year before the vote. The margin of error of the data would have to be doubled, or even tripled in some cases, to make them more representative.
“These confidence intervals ignore systemic differences that exist between the sample and the population,” Moore says. In light of these results, “we should all be prepared to question the very idea of trying to predict our future.”
Averages to see more clearly
So what is the solution? “We can combine surveys, as many aggregators do,” the professor argues. These specialized sites establish averages by grouping together all the surveys conducted across the country and weighting them according to their methodology or the size of their sample, among other things. This method gives an overall view of the results.
Currently, the national average weighted by the specialist site FiveThirtyEight gives Kamala Harris a lead of just under three percentage points over Donald Trump. The gap between the Democratic candidate and her Republican opponent has been widening since the former’s nomination last August, which came after Joe Biden abandoned the idea of running for a second term.
Looking at state-weighted averages reveals, among other things, where races are the tightest. This allows us to keep an eye on key states, that is, where the results were particularly close in the 2020 presidential election.
North Carolina, for example, is proving to be a fierce battleground between the two candidates. The gap between Trump and Harris has been within a few decimal places for more than a month, according to FiveThirtyEight data.
It is worth noting that the margins of error of the polls analyzed have been constantly overlapping for months. It is therefore impossible to say with certainty that one or the other has a strong chance of winning in the state.
Aggregators are also not immune to false results. In 2016, FiveThirtyEight predicted that Hillary Clinton had a 71% chance of winning. In the end, Trump won by more than 300 electoral votes, which gave him victory. There are many theories that could explain this discrepancy: a poor analysis of the Republican electorate by pollsters and the media, an impression of too great a success among Democrats, etc.
“We have to abandon this desire to be certain” of results or voting intentions, Don A. Moore reiterates. “It would be better to accept the deep flaws and the great deal of uncertainty that come with these polls. And as voters, we would do much better to make our decisions based on certain issues or on how candidates would govern.”
The professor also denounces the immense place that these polls take in the American media landscape. “Voters deserve to have useful information on the candidates’ programs, not on who is ahead of whom. They want to know what their plans are to improve the economy and deal with our budget deficits, how to manage Medicare, what to do about the war in Ukraine or Gaza…”