Exit polls, comparison with results

Jump to: Elections main page | Simulations | Home

Summary

Shortly after the elections several commentators complained about the accuracy of the exit polls released on 2/11. Early exit polls releases in the morning and afternoon of 2/11 from various internet sources were more inaccurate than statistical analysis would imply. Even final polls reported at the time precincts closed were widely inaccurate. Following standard practice, exit polls were later corrected to reflect actual results.

I will show below some details of (1) early polls posted in the afternoon on slate.com (2) revised polls posted by cnn.com on 3/11 after corrections made to reflect actual results.  Unfortunately I do not have accurate details of the exit polls as reported by the various media organizations at the time precincts closed.

newExit polls posted in the afternoon of 2/11 on slate.com

These are polls reported by this article on slate.com on 11/2 at 4:28PM PT. The polls failed to pick the right winner of OH and FL, two crucial states. More importantly, are they significantly off the mark in a statistical sense? Even assuming that these were random samples of the population, It is hard to say anything without knowing the number of respondents. So, let's make a couple of different hypothesis:

N
 Margin of error
 States off the Moe
500 respondents
 ~4.4%
 NY,NH
1000 respondents
 ~3.1%
 NY,NH,FL
  1. If exit polls were collected from random samples of the population, statistical analysis says that 95% of the states should report vote shares closer to the actual result than the margin of error. The rest, 5%, or 1 in 20, would report vote shares that are outside the bounds defined by the margin of error. We have early polls from 16 states, and certainly 2 or 3 are outside the margin of error.
  2. The second problem is that sampling errors alone would imply that the difference between the vote share in the exit poll and in the actual results should be about 50% of the times in favor of Kerry. As it turns out, all of the exit polls were biased towards Kerry. How can we explain the discrepancy? The most likely answer is that the numbers were not coming from random samples of the voting population.
State Bush
 Kerry
 Vote shares (Bush/(Bush+Kerry)
Exit poll Results Exit poll Results Exit Poll Results Diff
Florida
49
 52.1 51
 47.1 49.0 52.5 -3.5
Pennsylvania
46 48.6 53 50.8 46.5 48.9 -2.4
Ohio 49
 51 51 48.5 49.0 51.3 -2.3
Michigan 46 47.8 52 51.2 46.9 48.3 -1.3
New Jersey
44 46.5 54 52.7 44.9 46.9 -2
Missouri 52 53.4 47 46.1 52.5 53.7 -1.1
Wisconsin 48 49.4 51 49.8 48.5 49.8 -1.3
Minnesota 46 47.6 52 51.1 46.9 48.2 -1.3
Colorado 50 52.4 49 46.3 50.5 53.1 -2.6
Iowa
 49 50.1 50
 49.2 49.5 50.5 -1.0
Arkansas
 53 54.3 45 44.5 54.1 55.0 -0.9
New Mexico 48 50 50 48.9 49.0 50.6 -1.6
Nevada
48 50.5 49 47.9 49.5 51.3 -1.8
New Hampshire
 44
 49 54
 50.3 44.9 49.3 -4.4
West Virginia
 54 56.1 45 43.2 54.5 56.5 -2.0
New York 36 40.5 62 57.8 36.7 41.2 -4.5

Final exit polls, at the time precincts closed

All I know is that they are different from the exit polls available now. A reader sent me  this screenshot of an OH exit poll  at  12:21am on 3/11.  Around 1am all polls were revised to reflect official results more accurately.

Revised Exit Polls vs Results

Final exit polls were later revised to better reflect election results. This is standar practice, as illustreaded in  this interesting article. As you can see, the revised polls reflect the results very well. All polls predict the state winner correctly. All poll predictions differ from the actual results by less than the margin of error, despite the margins of error being very small (around 2% in many cases). The implied probability that Bush wins the election computed using my probabilistic method using these polls is above 92%.
See sources below

State (respondents) Bush Kerry Votes
Exit poll Results Diff Exit poll Results Diff Bush Kerry
Battleground states
Florida (2862 ) 51.38 52.1 -0.72 47.62 47.1 0.52 3,911,825 3,534,609
Pennsylvania (2107 ) 48.35 48.6 -0.25 51.18 50.8 0.38 2,756,361 2,883,833
Ohio (2020 ) 50.94 51 -0.06 48.59 48.5 0.09 2,796,147 2,659,664
Michigan (2555 ) 47.96 47.8 0.16 50.55 51.2 -0.65 2,306,292 2,471,905
New Jersey (1520 ) 46.22 46.5 -0.28 52.78 52.7 0.08 1,587,494 1,799,320
Missouri (2264 ) 53.06 53.4 -0.34 45.94 46.1 -0.16 1,452,715 1,253,879
Wisconsin (2321 ) 48.82 49.4 -0.58 49.71 49.8 -0.09 1,477,122 1,488,935
Minnesota (2190 ) 46.96 47.6 -0.64 51.52 51.1 0.42 1,345,168 1,443,564
Colorado (2534 ) 51.88 52.4 -0.52 46.68 46.3 0.38 1,045,920 924,540
Iowa (2512 ) 50.38 50.1 0.28 49.16 49.2 -0.04 745,970 732,737
Oregon (1064 ) 47.75 47.5 0.25 51.8 51.5 0.3 819,207 889,005
Arkansas (1459 ) 53.96 54.3 -0.34 45.04 44.5 0.54 566,678 464,157
New Mexico (2006 ) 49.9 50 -0.1 48.55 48.9 -0.35 370,475 362,239
Nevada (2189 ) 50.36 50.5 -0.14 48.16 47.9 0.26 414,939 393,372
New Hampshire (1883 ) 48.57 49 -0.43 50.43 50.3 0.13 330,848 340,019
Maine (1991 ) 45.29 45 0.29 52.77 53.1 -0.33 309,667 365,089
West Virginia (1728 ) 55.53 56.1 -0.57 43 43.2 -0.2 418,151 321,641
Hawaii (622 ) 45.7 45.3 0.4 54.3 54 0.3 194,109 231,318
Other states
Alabama (736 ) 62.52 62.5 0.02 37.02 36.8 0.22 1,174,348 691,993
Alaska (1177 ) 61.9 61.8 0.1 35.12 35 0.12 151,876 86,064
Arizona (1907 ) 54.82 54.9 -0.08 44.18 44.5 -0.32 908,211 735,327
California (2390 ) 43.94 44.3 -0.36 53.57 54.6 -1.03 4,403,495 5,427,055
Connecticut (872 ) 44.35 44 0.35 54.65 54.3 0.35 686,923 847,666
Delaware (772 ) 45.73 45.8 -0.07 53.7 53.3 0.4 171,531 199,887
District of Columbia (795 ) 9.2 9.3 -0.1 89.36 89.5 -0.14 19,007 183,876
Georgia (1618 ) 58.64 58.1 0.54 41.36 41.4 -0.04 1,889,832 1,345,198
Idaho (801 ) 68.38 68.5 -0.12 30.16 30.4 -0.24 408,254 180,920
Illinois (1434 ) 44.38 44.7 -0.32 55.08 54.7 0.38 2,313,415 2,826,757
Indiana (941 ) 59.92 60.1 -0.18 39.08 39.2 -0.12 1,474,475 960,899
Kansas (667 ) 62.9 62.2
 0.7
 36.2 36.5
 -0.3 566,678 464,157
Kentucky (1050 ) 59.38 59.5 -0.12 39.62 39.7 -0.08 1,069,451 712,903
Louisiana (1683 ) 56.7 56.8 -0.1 42.3 42.2 0.1 1,101,710 818,211
Maryland (1065 ) 42.84 43.3 -0.46 56.16 55.7 0.46 951,892 1,223,813
Massachusetts (889 ) 36.84 37 -0.16 61.68 62.1 -0.42 1,067,163 1,793,916
Mississippi (799 ) 59 59.8 -0.8 40 39.5 0.5 666,396 440,255
Montana (650 ) 57.49 59.1 -1.61 39.53 38.6 0.93 265,473 173,363
Nebraska (785 ) 66 66.6 -0.6 33.52 32.1 1.42 485,766 234,236
New York (1452 ) 40.9 40.5 0.4 58.2 57.8 0.4 2,782,457 3,971,059
North Carolina (2167 ) 56.46 56.1 0.36 42.72 43.5 -0.78 1,910,936 1,484,158
North Dakota (687 ) 62.92 62.9 0.02 35.08 35.5 -0.42 195,998 110,662
Oklahoma (1577 ) 65.44 65.6 -0.16 34.56 34.4 0.16 959,655 504,077
Rhode Island (809 ) 38.88 38.9 -0.02 59.65 59.5 0.15 161,654 247,407
South Carolina (1782 ) 58.44 58 0.44 41.13 40.8 0.33 924,170 650,350
South Dakota (1550 ) 60 59.9 0.1 38.5 38.4 0.1 232,545 149,225
Tennessee (1783 ) 57.9 56.8 1.1 41.1 42.5 -1.4 1,381,937 1,033,176
Texas (1794 ) 61.16 61.2 -0.04 37.84 38.3 -0.46 4,519,023 2,827,756
Utah (816 ) 71.46 71.1 0.36 26.62 26.4 0.22 612,623 227,286
Vermont (698 ) 38.8 38.9 -0.1 59.1 59.1 0 120,710 183,621
Virginia (1431 ) 54.14 54 0.14 45.4 45.3 0.1 1,662,484 1,396,269
Washington (2178 ) 45.87 45.7 0.17 52.7 52.9 -0.2 930,056 1,077,499
Wyoming (761 ) 69.04 69 0.04 28.94 29.1 -0.16 167,129 70,620

Sources