« Hillary's concession speech planned...early draft leaked | Yoda Urbinato's Blog | Emails prove TPM bias for Clinton »

Why polls in May (even by Survey USA) can't be trusted


This post is inspired by a couple of others that I'd suggest you check out. First, I'd recommend TPM's Eric Kleefield's post on Obama's electoral map and his possible paths to the White House. Then I'd check out two posts over at Ben Smith's blog at politico.com:

1. First, this post shows us that Survey USA and other pollsters continue to have a hard time polling African-American voters accurately. The trend in the primary polling was a general underestimating of Obama's support with African-American voters. Expect that trend to continue unless the pollsters change something or until the general election is closer.
 
2. This other post at Smith's blog shows the electoral map predicted in May 2004 based on the polling back then. You can look at this glass as half full or half empty for Obama, but really the only thing you should take away is that polls in May aren't very accurate predictions for the November election.  

How accurate it the polling happening right now? I think there's some compelling evidence to say, not very accurate. Let's focus in on that first post from Ben Smith's blog. Survey USA was one of the more accurate pollsters this year, although they all had hits and misses.  SUSA says that McCain is beating Obama by 4 points in Michigan (and of course Hillary supporters are going to pounce on that as evidence that he is a weak candidate). 

But let's look at SUSA's numbers:
- Obama is winning African American vote 62% - 26%.
- Obama is winning Wayne County 44%- 42%. (Wayne County is home to Detroit and about 1 million of Michigan's voters.

Now here are two things that are going to happen in November that are related and important: Barack Obama will win over 90% of the votes from African Americans (given that white Democrats have been getting in the 85 - 90% range of those votes) and ANY Democrat will win Wayne County, Michigan by more than 2:1. In 2004, Kerry won the county with nearly 70% of the vote.

So my word of caution is to avoid reading too much into any poll released for any election to be held in months rather than weeks. Even from respected pollsters like SUSA. They are just relaying the data as it comes to them - but when we dig a little deeper I think we can see how flawed opinion polls are this far out. Generally, I think the polls are pretty close a week or two from an election (minus New Hampshire) but months before an election they are going to be way off. Hell, remember how just a couple of months before Iowa and New Hampshire the polls stated that Rudy Guiliani and Hillary Clinton were sure to be the nominees?

But please do make plenty of guesses and predictions about what will happen in November - that's the fun part.  

8 Comments

| Leave a comment

(I) Hillary Clinton. Your future president, after reviewing all the available math, have determined that by any metric though I am losing this nomination race, nevertheless remain the most likely person to be president.

As such, I am hereby announcing my dual candidacy for president under the Democratic ticket in 2012, as well as my candidacy as an Independent in 2008.

Now, I know this may seem a bit unconventional or sexist-racist, so in a return to normalcy, I would also like to announce I have selected multiple running mates: Senators (R - Republican, as far as I know) Barack Obama and (CinC) John McCain, former VP Al Gore, the president formerly known as Bill C., and film-maker Michael Moore.

The voices of 349 trillion Americans from Florida and Michigan must be heard, and I would like to be the president of those two states.

Much Love,
Hillary - I'll be seeing you in Denver.

user-pic

PJ:

Good points all!

And who can forget the numerous polls in August of 1988 that had Michael Dukakis ahead of George bush by 17 points?

And yet - I can't remember ever using the phrase "President Dukakis".

:)

These polls now are basically worthless except in the rare cases when the margins between the candidates are so large and history so consistent, that the polls are merely a trace of what we already know.

To wit, 6 months ago they told us that this would be a Guliani-Clinton election.

The fascination with the polls and the belief that they tell us anything about the future is a kind of sickness. We are looking for certainty when there is none. We are looking for an edge, when all they provide is a crutch for any argument that is convenient to you at the time. Worst, they distract us from the fact that a campaign evolves as candidates and voters engage in the process. Polls have no ability to capture that evolution.

Just say no to polls for at least 3 months. It will be much healthier for you and for all of us.

Young voters tend to be under represented in polls too because many of them have cell phones and not land lines so cannot be polled by phone.

Polls in May are worthless in terms of predicting what will happen in November. They're mighty useful in telling you where you need to get boots on the ground and candidates on the tarmac.

user-pic

I swear that people should put more stock in their common sense than in some of these early polls. And I'm sure the Obama campaign does - and knows that he will win Wayne County big and they'll focus on turning out as many people as possible there.

user-pic

Polls are based on math. Math lies. Ergo, polls lie.

Also, polls this early are historically way off, especially in the swing states that people care about. Check out Ohio in 2004:
http://cgi1.usatoday.com/news/politicselections/nation/polls/2004-10-21-ohio-poll.htm

Leaving aside the registered-voter/likely-voter variance, you can see the polls fluctuating from 9 point advantages to one side or the other and virtual ties between August and Ohio.

So listen to Yoda, people.

Polls are based on math. Math lies. Ergo, polls lie.

Math doesn't lie, Genghis. Ever. And I know you know that since you deal with computer software.

To put it into perspective: physics doesn't lie, yet you can get cockamamie results from a poorly defined experiment.

I agree that polls need to be carefully designed, including polling group selection and question placement and wording, but the underlying statistical basis is always sound. If a poll fails to represent reality, it is the fault of the pollster, not on math.

Why do I belabor this point?

It's because it's easy to feel the deck is stacked against you if "math lies" -- why even bother learning it? So I make it a point to end these types of cutesy comments as they become unfortunate memes going forward.

Leave a comment

Yoda Urbinato

user-pic

Following:
Followers:

Posts
Comments & Recommends


Favorites

All Reader Posts
How to use myTPM

Advertise Liberally
Share
Close Social Web Email

"To" Email Address

Your Name

Your Email Address