Obviously our polling on the 10th Congressional District Democratic primary in Illinois was way off, one of our worst misses ever. We apologize to our clients, the Progressive Change Campaign Committee and MoveOn, for letting them down on this race.
PCCC and MoveOn have been transparent with their polling on the race and have posted all the polls we did for them over the last 3 months here. This error is on PPP, not them.
We did see the race tightening significantly over the final week. The tracking poll we did on Sunday, March 11th found Sheyman leading by 21 points, 46-25. The tracking we did on Wednesday, March 14th found Sheyman's lead down to 14 points at 44-30. The numbers that were released publicly last Thursday combined the interviews from Sunday and Wednesday, for the purposes of a larger sample size.
The final night of tracking we did on Sunday, March 18th continued to show Schneider gaining ground while Sheyman stayed in place, and that the lead was down to 11 points at 44-33. Still we never would have expected this.
We have a few theories on what went wrong:
I think the bigger possibility though and a lesson learned for our work with all of our clients is this: we may have overpolled the district:
-The pool of likely Democratic primary voters in this district is small enough that we had to call everyone for both a message testing poll we did at the end of February and for one of the 3 nights of tracking we did over the last week. Calling people twice in such a short period may cause respondent fatigue.
There's 2 reasons I think this may have benefited Sheyman. The first is that on the message testing poll a much higher percentage of Schneider supporters hung up in the middle of the call than Sheyman ones, presumably because they thought the poll was being too negative to him, even though the poll tested negatives against both candidates and was nothing out of the ordinary for the polls we conduct. I think it's possible that when we called again for the tracking over the last week Schneider voters may have been more reluctant to participate. We actually had a different person record the tracking poll than the message poll so people previously called would hear it with a fresh ear but there may have still been some aspect of that.
The 2nd reason I think respondent fatigue would have helped Sheyman's numbers is that we did continuously find that his supporters were more excited than Schneider's were, and that may have extended to their being more excited to answer polls than Schneider's were too.
One thing I know is that it didn't have anything to do with the way the data was weighted. Weighting had very little impact on the final results.
For PPP I think the lesson is that a Congressional primary electorate is too small to call for multiple polls in less than a month, especially if there's some aspect of negative message testing involved. It needs to be a big enough pool of voters that you can call completely different universes every time.
We apologize that our polling skewed perceptions of where this race stood. When you poll as many races as we do you're going to badly screw one up every now and then. But we strive to be as close to perfect as possible and in this instance we fell badly short.