Polls and Polling

Some links on polls and polling

A story about polling and sampling variation

  • One week before the election, the CNN/USA Today/Gallup daily tracking poll had the Clinton-Dole gap at 12 points, its lowest in weeks, although the fine print said
    . . . the difference between today's results and previous findings is only 2 to 3 percentage points and it is too early to tell whether this is the start of a trend toward [Dole]. . . . It is inaccurate to say the race has definitely tightened until additional interviews over several days clearly establish a trend. Keep in mind that Dole has posted gains in the past but been unable to sustain them.
    Pretty good wording, although it doesn't really explain that Dole ``gains'' in the past may have been mere statistical fluctuations of no real significance.

    On the very same day, the ABC News tracking poll had the Clinton-Dole gap at 19 points, its highest since mid-August. Although the fine print at the bottom of the page does say the margin of error is 4.5 points, ABC News showed no caution about overinterpreting the number.

    Over the next several days the numbers were

    CNN/USA Today
    Gallup    		ABC News
    Oct 26-27 12 19
    Oct 27-28 16 19
    Oct 28-29 14 17
    Oct 29-30 16 12
    Oct 30-31 18 11

    Over the week the two polls reversed. It's obviously all sampling variability. None of the differences between any of these polls are statistically significant except for the original 7 point difference at the beginning and the 7 point difference with the polls reversed at the end, and those are not significant when data snooping is taken into account. But ABC News again showed no caution in overinterpreting the numbers

    There has been modest change this week: Dole gained five points, moving up from his low of the fall campaign, and Clinton lost four, slipping from his season high. Clinton's lead fell from 19 points to 11 - closer, if not especially close.

    IMHO, this lesson about the silliness of overinterpretation of random noise is much more important for students to learn than the lesson that bad design ruins a poll (the Literary Digest poll lesson), though, of course, that's important too.

    Addendum to the story

    Just for the record, the results up to the election were
    CNN/USA Today
    Gallup    		ABC News
    Oct 31-01 13 10
    Nov 01-02 13 11
    Nov 02-03 16 12
    The actual gap in the election was 7 points -- Clinton 49%, Dole 42%, Perot 9%. These two polls (other polls too) overestimated the actual result and by more than the ``margin of error'' of the polls. The possible reasons are many, voters may have actually changed their minds in the last few days, the turnout (49%) was somewhat lower than the polls predicted, and some people may not have known their own minds when they talked to the pollsters or may not have wanted to admit their preferences.

    Back to my home page.