Post-N.H.: Explaining the Unexpected

Frank Newport

Pre-election polls in New Hampshire -- including our USA Today/Gallup poll -- did not predict Hillary Clinton's narrow win over Barack Obama in Tuesday's New Hampshire primary. The Jan. 4-6 USA Today/Gallup poll, conducted Friday, Saturday, and through about 4:00 p.m. Eastern on Sunday, showed Obama with a double-digit lead. Hillary Clinton won by 2 points.

It's a reasonable question for observers (and poll critics) to inquire as to why there was such a significant discrepancy between the candidate preferences voters expressed in the pre-election polls and how they actually voted on Tuesday in the case of the Democratic New Hampshire primary.

We are examining several explanations at Gallup (and other professional pollsters involved in New Hampshire polling are examining these, as well). Generally, we can group these explanations into two broad questions.

The first question: Was there something wrong with the sampling or methodology of the poll? Polls are estimates of the views of an underlying population, and given the many technically challenging aspects of refining a sample to reflect the pool of those who actually turn out and vote, those estimates can be off. This year's New Hampshire primaries provided an unusual challenge, given the fact that voters not registered with one or the other major party could -- up to the last minute -- choose to vote in one or the other party's primary. Plus, a certain percentage of the time, estimates can be wrong just by random chance.

With the 2008 New Hampshire primary situation, there are two arguments against the hypothesis that the USA Today/Gallup poll had some type of sampling or methods problem.

First, almost every other poll of New Hampshire Democrats showed Obama leading, and most by a substantial margin. If there was an idiosyncratic sampling or methodological problem with the USA Today/Gallup poll, many other polls conducted independently and by different polling firms would not duplicate it in most circumstances.

Second, the same weekend poll that showed Obama leading Hillary Clinton by a substantial margin among Democrats was also the same poll that correctly predicted John McCain's 5-point margin of victory over Mitt Romney in the GOP voting. If there had been a sampling or methodology problem with the poll, it most likely would have affected both races. It didn't. The discrepancy was between what Democratic voters told us in the poll and what Democratic voters actually did on Tuesday. There was no discrepancy between Republican respondents and how Republicans voted on Tuesday.

Which leads us to the second category of explanations for the disconnect between the pre-election poll results on the Democratic side and the actual voting. New Hampshire Democratic voters may have confounded pollsters by actually changing their minds in the days and hours leading up to voting. This is unusual. In most pre-election environments, voter statements about their vote intentions in the days before an election are good indicators of how they will actually vote. But last minute changes do occur, which is why Gallup sometimes interviews the final Monday night before the national presidential election. But Gallup did not do this in New Hampshire this year.

There was a tremendous interest in this election among New Hampshire voters this year, as demonstrated by the record-high turnout. This high level of interest suggests that voters may have been closely attuned to election events in the final days before the election, which, in turn, may have had a greater influence on their behavior in the ballot booth.

The most obvious such event on the Democratic side was the extraordinary and frequently aired video of Hillary Clinton's personal and emotional "verge of tears" response to a female voter's question on Monday. Now, it is impossible to determine the actual impact of this event on Democratic voting behavior, but it is not unreasonable to assume that it did cause some voters to change their minds about voting for Clinton. There was also the Saturday night Democratic debate, allowing Democratic voters to contrast and compare Clinton and Obama side by side, which may have helped change voters opinions.

Plus, there may have been a natural "settling down" process in which Democratic voters continued to mull over their candidate choice after the initial impact of the highly publicized Obama win was registered in the days right after the Iowa caucuses.

Some polling observers have also opined that white voters in New Hampshire told pollsters they were going to vote for a black candidate (Obama), but when actually in the voting booth, they ended up not voting for him. This voter behavior theory has surfaced over the years in elections with black candidates, but has been difficult to prove. Discrepancies between pre-election polls and the actual vote for black candidates, in fact, are certainly not the norm. This year, as an example, the pre-election polls in Iowa were accurate in relationship to Obama's actual vote in the caucuses. No doubt other explanations will be advanced as well in the days ahead.

Pollsters are often quick to point out that polls are a "snapshot in time." That's true, but typically, the snapshot of voter preferences a few days before an election turns out to be an accurate picture of what happens on Election Day itself. Occasionally, that's not the case, as it looks like in the case of the Democratic primary in New Hampshire.

Pre-election polls are a legitimate and important part of the information flow before elections because there is great public interest in and demand for answers to the question "who's ahead." Polling also provides much valuable information about voters and their views of candidates beyond the horse race numbers.


Gallup http://www.gallup.com/poll/103663/PostNH-Explaining-Unexpected.aspx
Gallup World Headquarters, 901 F Street, Washington, D.C., 20001, U.S.A
+1 202.715.3030