As Suspected, Rasmussen Polls Heavily Skewed Republican

As Suspected, Rasmussen Polls Heavily Skewed Republican


Nate Silver reveals the ugly truth:

The 105 polls released in Senate and gubernatorial races by Rasmussen Reports and its subsidiary, Pulse Opinion Research, missed the final margin between the candidates by 5.8 points, a considerably higher figure than that achieved by most other pollsters. Some 13 of its polls missed by 10 or more points, including one in the Hawaii Senate race that missed the final margin between the candidates by 40 points, the largest error ever recorded in a general election in FiveThirtyEight’s database, which includes all polls conducted since 1998.

Moreover, Rasmussen’s polls were quite biased, overestimating the standing of the Republican candidate by almost 4 points on average. In just 12 cases, Rasmussen’s polls overestimated the margin for the Democrat by 3 or more points. But it did so for the Republican candidate in 55 cases — that is, in more than half of the polls that it issued.

If one focused solely on the final poll issued by Rasmussen Reports or Pulse Opinion Research in each state — rather than including all polls within the three-week interval — it would not have made much difference. Their average error would be 5.7 points rather than 5.8, and their average bias 3.8 points rather than 3.9.

As many of you know, I was very pro-Rasmussen during the 2008 election because they produced reliable daily results, but we can’t ignore the analysis. Somewhere along the way something went wrong.

How did the poll get steered into the ditch?


Rasmussen, for instance, generally conducts all of its interviews during a single, 4-hour window; speaks with the first person it reaches on the phone rather than using a random selection process; does not call cellphones; does not call back respondents whom it misses initially; and uses a computer script rather than live interviewers to conduct its surveys. These are cost-saving measures which contribute to very low response rates and may lead to biased samples.

Rasmussen also weights their surveys based on preordained assumptions about the party identification of voters in each state, a relatively unusual practice that many polling firms consider dubious since party identification (unlike characteristics like age and gender) is often quite fluid.

Also, it wasn’t like the writing wasn’t on the wall early on…

The discrepancies between Rasmussen Reports polls and those issued by other companies were apparent from virtually the first day that Barack Obama took office. Rasmussen showed Barack Obama’s disapproval rating at 36 percent, for instance, just a week after his inauguration, at a point when no other pollster had that figure higher than 20 percent.

Just saying…

  • Rich Horton

    Actually, statistically speaking this “research” is nonsense. What Silver is doing is using outliers (like missing notoriously difficult to predict blowouts in Hawaii by wide margins – after all Democratic pollster PPP missed the same race by almost 20 points) and then using the outliers to bring the averages down.

    A better measure would be to look at how well they did in races that were close.

    Besides, Silver is just proving to me that he is a spendthrift. From a cost/benefit point of view there is no reason to spend a fortune on a race like the Hawaii senate if it is highly unlikely it is close. After all the Rasmussen result said the Democrat had a lead in the double digits (i.e. the race wasn’t close.) But Silver is going to reccommend spending 3 to 4 times as much on a poll that will wind up telling you the same thing (i.e. the race isn’t close.) Silver is lost in the private sector! He seems destined for bureaucratic career of distinction!

    Believe it or not, in polling money IS an object.

    So lets look at the close races (data off of RCP), and see who called them right between the four main pollster (Ras, PPP, Quin, and SUSA):

    CA: Ras, PPP, ANd SUSA correct. No Quin poll
    WA: Ras correct; PPP and SUSA wrong. No quin poll
    PN: Ras, PPP and Quin correct; no SUSA poll
    NV: Ras and PPP wrong. No Quin or SUSA poll
    WV: Ras and PPP correct. No Quin or SUSA poll
    CO: Ras and PPP wrong. SUSA correct (closest); no Quin poll
    IL: Ras and PPP correct. No Quin or SUSA poll
    CT: Ras, PPP and Quin correct. No SUSA poll
    AK: Ras and PPP wrong; no Quin or SUSA poll.

    I begin to see why SUSA and Quin did so well. They didn’t do a lot of polling in close races.

    Just using the hard number the results for the Senate races are:

    Ras: Correct in 6 of 9.
    PPP: Correct in 5 of 9
    Quin: Correct in 2 of 2
    SUSA: Correct in 2 of 3.

    Giving them a +/- of 4 points the results are:

    Ras: Correct in 9 of 9
    PPP: Correct in 8 of 9
    Quin Correct in 2 of 2
    SUSA: correct in 3 of 3

    Also, because Ras (and PPP) do a lot more polling in general, the chances they will generate an outlier increase, particularly in races that are not close. But Silver would have us believe it is important that in the New York race between Gillibrand and DioGuardi that Rasmussen said the spread was 21 points while Quin said it was 23.

    (The unspoken criticism here is really that Rasmussen works with the forces of evil – you know, like Juan Williams.)

  • Tully

    Now now, Rich, don’t go confusing people with applied real-world methodology. It gets in the way of the truthy.

  • Jim Satterfield

    Sorry, Tully, but Silver knows more about this than Rich or you. Go back to listening to Limbaugh now.

  • Rich Horton

    Yeah Jim, I’m only a professor of Political Science – what the hell could I know.


  • Parnell

    No pollster is going to nail everything. The issue that many have seen is that in the last cycle Rasmussen seemed to consistenly be overestimating the GOP side. Any cursory glance at Rasmussen polling shows a GOP bias. Looking at the actual votes vs. the last polls done by Rasmussen, they had the GOP winning 4 more Senate and 4 more Governor races, with no polls showing a Dem winning that ended up with the GOP. Plus, almost every single poll Senate and Governor poll overestimated the GOP. And in a wave year like this, if anything there should have been a bias towards dems with undecideds breaking for the GOP.

    Over-estimated GOP in 21 out of 23, called 4 wrong GOP wins (including Miller in AK)

    state actual Ras dif
    HI D+54 D+13 R+41
    GA R+40 R+19 R+21
    DE D+26 D+11 R+15
    MD D+28 D+18 R+10
    NV D+5 R+4 R+9*
    AK W+7 R+1 R+8* R(w)=Murkowski
    WV D+11 D+4 R+7
    OR D+17 D+11 R+6
    CA D+9 D+3 R+6
    OH R+18 R+24

    NY2 D+26 D+21 R+5
    CT D+12 D+7 R+5
    CO D+1 R+4 R+5*
    NY1 D+32 D+28 R+4
    WA D+2 R+1 R+3*
    IN R+15 R+18 R+3
    IL R+2 R+4 R+2
    WI R+5 R+7 R+2
    NC R+12 R+14 R+2
    PA R+2 R+4 R+2
    FL R+19 R+20 R+1

    KY R+12 R+12 –

    MO R+13 R+9 D+4

    Over-estimated GOP in 20 out of 26, plus called 4 wrong GOP wins.

    state actual Ras dif
    HI D+18 D+2 R+16
    NY D+27 D+14 R+13
    ME R+1 R+14 R+13
    VT D+2 R+10 R+12*
    NV R+12 R+23 R+9
    CA D+13 D+3 R+9
    CO D+14 D+5 R+9
    IL D+0.1 R+8 R+8*
    MA D+7 D+2 R+5
    WI R+5 R+10 R+5
    SC R+4 R+9 R+5
    OR D+1 R+3 R+4
    MD D+14 D+10 R+4*
    CT D+1 R+2 R+3*
    NH D+8 D+6 R+2
    MI R+18 R+20 R+2
    FL R+1 R+3 R+2
    MN R+8 R+10 R+2
    OH R+2 R+4 R+2
    AZ R+14 R+13 R+1

    GA R+10 R+10 –

    PA R+10 R+9 D+1
    MN D+1 D+3 D+2
    TX R+13 R+9 D+4
    SD R+24 R+19 D+5
    RI I+2 I+7 I+6

    This kind of dramatic house effect towards the GOP is hard to understand in 2010, where it is clear that the turnout for GOP voters vs. Dem voters was much more in favor of the GOP than than in 2006 and 2008. It seems that the Rasmussen likely voter model has even more radical assumptions regarding GOP turnout or lack of Dem turnout, at least for this midterm. The question is, will this GOP house effect carry forward towards the 2012 likely voter model.

    The bigger question is if this GOP bias is due to political leanings of Rasmussen and part of some kind of agenda. The notion being that Ras poll are more to help drive naratives. Certainly when Republicans and FOX talkers talk about polls, they more often cite Rasmussen to make their points.

    Personally whenever I see a Ras poll i just subract 4 from the GOP-Dem margin.