Monthly Archives: October 2024

Presidential Polls are Flawed

This was written Sunday October 27, 2024 and will not be revised.

  1. The president is not elected by popular vote.

I remember watching Headline News on CNN a few days before the 2012 election and hearing that the election between Obama and Romney was a statistical dead heat. Each candidate was predicted to have 47% of the vote. However, the results in the Electoral College were not close Obama 332, Romney 206. People who forecasted based on a state by state basis (Nate Silver and a talented amateur at Princeton) concluded on the days before the election that the probability Obama would win was 99%.

  1. Even when the difference between the candidates is less than the margin of error of the poll it still contains information.

Let’s start with the question: what is the margin of error? It is often reported, but rarely with the sample size of the poll which is needed to be able to check the calculation. Some examples with both pieces of data are Franklin-Marshall for Penn 794/4.3; Emerson College for North Carolina 1226/3.6, and for Wisconsin 800/3.4; Marist for Arizona and Georgia 1193/3.9.

If Ɵ is the estimated frequency of votes for Harris then using standard formulas from an intro states class the radius of the 95% confidence interval the margin of error is 1.96(Ɵ(1-Ɵ)/n)0.5.  Since Ɵ is usually in (0.4,0.6) then 0.24 < Ɵ(1-Ɵ) < 0.25 and replacing 1.96 by 2 we can simplify this to 1/n0.5 and our data points become

n,error               800,  0.0353       1200, 0.288

somewhat smaller than the reported margins of error.

  1. Frequencies should be reported to one decimal place and should be recalculated to remove responses that are not one of the two major candidates to make it easier to compare predcitions.

As some readers have noticed I have not gotten to #2 yet. That is coming soon. Bloomberg, the one poll that I have seen which reports frequencies with one decimal place had on October 23:

Harris/Trump    H/(H+T)           P(H win)   538 ave.

Michigan            49.6/46.9           51.4                     79.2         H 0.6

Penn.                  50/48.2               50.9                     67.7         H 0.3

Nevada               48.8/48.3           50.3                     59.2         H 0.2

Wisconsin          48.3/48               50.2                     51.6         H 0.1

Arizona               49.1/46               50.2                     51.6         T 1.8

  1. Carolina 48.5/48.8 49.4                     43.9         T 1.2

Georgia               48.4/49.9           49.3                     35.2         T 1.5

If the uncertainties look large, it is because they are based on one poll. Once one has the average of several polls then the uncertainties decrease. Of course comparing Bloomberg with 538 average shows that there are sampling biases. The Washington Post is even more discordant. They have Harris leading in all states except Arizona. I got these figures from an article in Forbes but when I went to check them the Post wanted $2 to read the article.

Combining these projections to calculate is difficult without a computer and a good programmer. 538 uses simulations to conclude that, when this was written: Trump wills with probability 0.54 and Harris with probability 0.45.

https://projects.fivethirtyeight.com/2024-election-forecast/

One final selling point for people who publish polls, if you make predictions like this you can’t be wrong.