The Marquette-Franklin poll…What wasn’t reported in the media

The recent report of the Marquette Law School Poll conducted by Charles Franklin in the Milwaukee Journal Sentinel was headlined “Walker leads potential Democratic opponents in new statewide poll.” Like lemmings into the sea, statewide mainstream media followed the Journal Sentinel’s lead on this story; including the much improved 51% approval rating of Scott Walker, and the lead Walker enjoys over several presumed Democratic opponents. As is often the case, the real story of this poll was missed – primarily for a lack of due diligence on the part of reporters to dig just below the surface of the poll, in favor of a three-paragraph story that grabs a headline.

To dig deeper, Badger Democracy enlisted the consultation of two experts in the field to examine the methodology of the poll, and interpret the results. One expert is a professional pollster in Wisconsin, the other an Assistant Professor of Epidemiology (that’s biostatistics in lay terms) at the University of Illinois – Chicago. The results were not only surprising, but eye-opening. Both were contacted and interviewed independently, and both arrived at the same conclusions about the poll.

Both experts verified the legitimacy of the poll. There was no “hidden bias” or deliberate attempt at a conservative bias in the way the poll was conducted, the samples taken, questions, or formulas used to arrive at the poll results. One of the experts remarked at the thoroughness of information available as a result of this poll. So, what does it REALLY say?

The first point of interest is the Margin of Error (MoE). The margin of error for the overall poll is 3.8%, based on the sample size and respondents. What the  Journal Sentinel’s Craig Gilbert, and other media, failed to report is the difference in the margin of error for the head-to-head presumed Gubernatorial race. That MoE is higher – anywhere from 7.0% to 7.2%. This difference was cited in the Marquette University press release, but ignored by media and conservative pundits. According to Professor Franklin, as explained in an email to Badger Democracy, this detail is not uncommon, and leads to “confusion” in poll reporting:

“…if each candidate has a 3.8 moe, then one could be up and the other down by 3.8 each, suggesting a 2×3.8 MOE for the DIFFERENCE between the two. The actual formula makes a result a little less than that because there are also other candidates or “don’t know” or undecided and taking them into account reduces the moe for a difference to a bit less than twice the moe for individual percentages. Most pollsters don’t report the MOE for a difference, no doubt because it is hard to explain. But I felt that since the Walker lead over the top 3 dems was just a bit inside the MOE it was important to point that out to clarify that while he led it wasn’t quite statistically significant. That was the last sentence of the 2nd paragraph of the release but hasn’t been much noticed.”

(For you stats geeks, Franklin attached the formula explanation which you can download here: DiffPropAmStat1983)

The next point of interest also affects the outcome of the head-to-head races polled. The Methodology Statement, page 3, shows that the political leaning of the respondents (reached at random) in total is 4% higher than state average Republican (based on Gallup polling). The Gallup poll utilized is the most recently available (January-June 2011), but does not reflect any shift in the statewide difference over the summer. One expert pointed out that there are many reasons for this – 3% more males responded than in the general population (males tend more conservative), cell phone respondents in polls tend to be more conservative (due to the nature of cell phone numbers that reach pollsters), etc.

While this is not intentional, it is important. The respondents tended to be registered voters, leaning Republican, and that 4% higher Republican respondent came directly out of the Independent voter classification, according to the methodology. The independent, non-registered voter lacked representation in this poll – by chance of the random dialing and who responded to the poll. United Wisconsin has reported that nearly 30% of recall signatures came from voters who were not registered at their current address, and likely considered themselves “Independent” voters. Indeed, Franklin’s email states that the “not eligible” …”includes 65 who said they were not registered and did not intend to register. If so we didn’t interview them” That 65 represents nearly 10% of those that actually completed the survey. The poll does not reflect current “who MAY vote,” only “who HAS and WILL vote.” With that in mind, the head-to-head results and Walker’s approval rating look VERY different.

Here are highlights to take away from the Marquette-Franklin full poll results:

Questions 8, 15 – The Scott Walker approval and favorability ratings, as mentioned above, is reported as 51%-46% (favorable), 50%-45 (approval). The 3.8% MoE, coupled with a 4% Republican lean in the poll eliminates any positive for Walker.

Questions 16-20 – Opinion of potential Democratic  Candidates – Only Tom Barrett has a “haven’t heard enough about” rating under 50%. All potential Democratic candidates “favorable” ratings are within the MoE save one – Kathleen Falk at 19%-25%, 6% being outside the 3.8% Moe; although 51% of respondents felt they haven’t heard of Falk enough to respond.

Questions 26-28b – Head-to-head candidates – Remember, the MoE in this question is about 7%, and the poll has 4% higher Republican respondents than the general population. First, Walker 49%, Falk 42% – within the MoE. Second, Walker 50%, Barrett 44% – again, within the MoE. Third, Walker 50%, Cullen (the least known candidate) 40% – outside the MoE, but recall the 4% higher Republican response. Finally, Walker 49%, Obey 43% – again, within the margin of error. Noteworthy in a Republican-leaning poll (by 4%), Walker does not garner over 50% of the “vote.”

Question F11 – Income distribution is extremely close to the state’s average.

Question F12 – Only 32% of respondents stated the recession had a major effect, finances not recovered in their personal economy. A full 38% stated the recession has had NO effect on their personal financial situation. This indicates a significant level of economic insulation found in many Walker supporters.

The “crosstabs” regarding frequency of newspaper reading provide tremendous insight into who supports Scott Walker, and their level of engagement in current state politics.

Among those who read a newspaper 7 days a week, Scott Walker’s approval rating is only 45.5%, while his disapproval rating is 50.2% (3.8% MoE). Among those who NEVER read a newspaper, Walker’s approval rating jumps to 62.8%, with only 35.3% disapproving. Amongst all statewide candidates listed in the poll running for Herb Kohl’s soon-to-be-vacant Senate seat, the only candidate with an unfavorable rating outside the MoE is Jeff Fitzgerald, at 16.8% favorable, 26.3% unfavorable, with over 50% not knowing enough to respond. In addition, people who do not read a newspaper have a higher favorable rating of the Tea Party.

All these factors, taken together as a whole, give us a good snapshot of Wisconsin politics. The state remains a purple battleground state. Despite the rhetoric and multi-million dollar ad campaign, Scott Walker cannot garner more than 50% against any lesser known Democratic challenger, and according to the poll, even highly conservative voters support funding public education and Badger Care. Walker’s approval rating continues to be below 50%, even in a moderately conservative poll. According to the crosstabs, the most likely Walker supporter is a high income ($75,000+), white male who does not read a newspaper, but rather gets their news from local television newscasts.

The other picture that is clear is the impact the Independent, unregistered, new voter will have on any future election. Those are the voters we, as Progressives, need to focus on. Mobilizing, educating, and connecting at the grassroots level will be able to overcome any amount of money spent in future elections. The people are more powerful than corporate money in Wisconsin politics.

That is the big story missed by the mainstream media. Solidarity.