Close window  |  View original article

Lies, Damned Lies, and Polls

How to get the result you want.

By Will Offensicht  |  March 31, 2008

Mark Twain used to distinguish three categories of untruth - lies, damned lies, and statistics.  Now we come to a new kind of lie - the poll.

In an article "Clinton takes hit in new poll on White House race," Reuters reports:

Democratic presidential candidate Hillary Clinton's positive rating has dropped to a new low of 37 percent in an NBC/Wall Street Journal poll released on Wednesday. (...)

The survey was taken after Obama gave a speech last week on race in America and rejected racially charged remarks by his pastor in Chicago of two decades, Rev. Jeremiah Wright.

NBC said 32 percent of respondents said Obama "sufficiently addressed the issue" and 26 percent said he needed to say more about the Wright controversy.

More than half of those surveyed -- 55 percent -- said they were "disturbed" by the videos of Wright that were widely circulated on television and the Internet, the poll found.

So far, so good (or, at least, so conventional).  The story is written so that the poll sounds like honest, objective, factual, scientific polling of voter opinions during a hard-fought campaign.

Polls are more important now than they usually are because the Democratic party establishment allocated 790 or so "superdelegate" votes at the nominating convention to party big-wigs.  The superdelegates job is to ensure that we poor, ignorant, misguided voters don't saddle the party with a loser.  The superdelegates are responsible for tipping the balance in favor of whomever they think most likely to win the general election and trounce the Republicans.

Although Mr. Obama has won more delegates than Hillary, the math suggests that neither Hillary nor Obama will win enough delegates to guarantee the nomination.  Thus, it seems that the superdelegates will decide the nominee and that all the people who voted in the primaries will have wasted their time.  So much for the people's choice.

The importance of polls

This makes polls rather vital.  The superdelegate positions were created specifically so that the party establishment had the power to override the will of the voters and nominate a winner.

Thus, the perception of who is best poised to defeat Mr. McCain will probably decide the nomination.  That makes this part of the story about this poll all the more interesting:

In head-to-head matchups, Obama and Clinton were even at 45 percent. In general election matchups, Obama led McCain by 44 percent to 42 percent and McCain led Clinton by 46 percent to 44 percent.

When asked which candidate could unite the country if elected, 60 percent said Obama, 58 percent said McCain and 46 percent said Clinton.

What was written about the results of the poll tends to indicate that Mr. Obama has put the controversy about Rev. Wright's words behind him and is sailing serenely towards victory in November.  The story is calculated to soothe the superdelegates' concerns about electability and suggest that they should go along with the people's choice and nominate Mr. Obama.

There's an old saying that goes, "The large print giveth, and the fine print taketh away."

Here, the large print was "Clinton takes hit in new poll on White House race;" relatively few people got to the fine print. The fine print is usually concealed fairly carefully on the back of the car rental form, in weensy type on medicine bottles, or buried in innocuous trivia.

This one is no different.  First came the camouflage:

The poll of 700 registered voters was conducted on Monday and Tuesday and had a margin of error of 3.7 percentage points.

That's the usual psuedo-scientific bushwah which is intended to make us feel that pollsters know what they're doing.  That error of 3.7% is more than the margin between Mr. McCain and either Democrat which means that their headline is nonsense, but no matter, the really interesting part came next:

NBC said its pollsters oversampled African-Americans to get a more reliable cross tabulation on questions regarding Obama's speech on race. [emphasis added]

It's all in the sample

Wow.  There it is, folks, they not only admitted that they lied, they admitted that they lied deliberately and told us how they lied.

There are two standard ways to lie with polls.  You can frame the questions to promote the answer you want to hear or you can choose the people whom you poll to bias the sample.

Framing the question is important.  Suppose we're asking how people feel about the welfare system.  It makes a huge difference whether a poll asks: "Do you favor increasing the money paid to a woman who refuses to work as increased rewards for each illegitimate child she has?" versus "Are you in favor of increasing spending on dependent children so that they can have better health care?"

Anybody can see that these two questions would get rather different answers, even though they're asking about exactly the same thing.  We have no idea how the NBC poll takers spun their poll because they didn't publish the questions.

In this case, however, they did tell us how they biased the sample.  If we were to ask voters whether they favored John McCain, Barack Obama, or Hillary Clinton, it makes a difference where we ask and whom we ask.  If we go to a Republican suburb, we'd probably find that McCain would come out on top.  In a black neighborhood, it would probably be Mr. Obama who's pretty much owned the black vote since the Mississippi primary.  Voters chosen from a Latino neighborhood would probably favor Hillary.

What did NBC do?  They "oversampled African-Americans" which means that they put more black people in their sample of 700 people than their presence in the population would justify.  They rigged the sample!

And what would be the effect of "oversampling" African-Americans?  What would be the effect of putting too many blacks in the sample?  That would more or less guarantee that Mr. Obama would fare better in the poll than he would in the election - not only do African-Americans overwhelming support him, but they're already quite used to the, shall we say, less than loving theology of the Rev. Wright.

We all know that Mr. Obama did less well in the New Hampshire primary than the polls had predicted.  They may not have oversampled African-American voters there, but they did something.

At least this time, they told us how they cooked the books.  What have they been doing the rest of the time?

Stand by for a poll showing Hillary far in the lead, with particular emphasis paid to the opinion of white coal-miners in West Virginia.