The pollster who got it wrong

Posted by on Oct 21, 2021 in Brian's Blog, Politics | Comments Off on The pollster who got it wrong

The headlines in the Postmedia-owned Calgary Herald on Sept. 30 – two weeks before the Calgary civic election – came as a surprise. “Mayor’s job is on the line,” said one headline. “Smith holds lead over Nenshi,” said another. The press release headline from Mainstreet Research – the outfit Postmedia commissioned to do the poll – also didn’t mince words: “Nenshi faces defeat.”

So, how did this happen? How did Calgary’s popular mayor, Naheed Nenshi, go from being the 2014 winner of the World Mayor Prize from the international City Mayors Foundation to a failing municipal politician in danger of losing his job? Where was the corroborating evidence? Where were the other pollsters?

There were none. And that was the problem with this story right from the very beginning.

Naheed Nenshi

Mainstreet’s Sept. 30 poll showed Nenshi trailing political unknown Bill Smith by nine percentage points. A second Mainstreet poll, conducted a week later, showed Nenshi trailing Smith by 17 points. “Nenshi’s path to victory is growing thinner,” said Mainstreet president Quito Maggi. At that point, there was still no corroborating evidence. But that didn’t stop such other media organizations as the CBC, Canadian Press, Toronto Star, Globe and Mail and Global TV from running with the story.

With the Smith campaign gathering traction, there came from political scientists and other pundits widespread criticism of what Maggi now admits were the “odd results” of the two Mainstream polls. His first response to the criticism was to double down: “The math is the math. The only thing I can do is take the sample, weigh it accordingly, test it as many different ways as I possibly can, and then put out our number. This is no different than what we’ve done before, time and time again.”

Then, on Oct. 13, came a third and final poll from Mainstreet. It too showed Nenshi trailing Smith, now by 13 percentage points. But by then there were other factors in play, including what Maggi called Smith’s “self-inflicted wounds.” One wound came from a CBC story saying lawyer Smith’s firm had briefly defaulted on a $24,545.60 loan before paying it back. Another came from a CBC story saying Smith had been sued for $2.2 million for allegedly failing to “exercise the care and skill to be expected of a reasonably competent solicitor” in a case involving a mortgage acquired for renovations to a Calgary commercial property. That suit was settled out of court.

Bill Smith

A third factor in play on Oct. 13 was the release of another firm’s poll showing Calgarians supporting the first phase of the $4.65 billion Green Line public transit project that Smith had said he would delay and re-evaluate. Tacked onto that survey was a poll showing Nenshi with a 15-point lead over Smith in the mayoralty race. Maggi says, however, this poll “gave us no reason to doubt the accuracy of our results.” Neither did a subsequent Canadian Municipal Election Study poll showing Nenshi leading with a 17-point edge. Maggi says it would have taken a poll by a “professional organization with a measurable track record” at that point to convince Mainstreet to pause and re-examine its data.

We know, of course, what happened next. On Oct. 16, Nenshi scored an eight-point victory over Smith in the mayoralty race and Maggi acknowledged publicly that Mainstreet failed to get it right. “It was a catastrophic failure,” he told the Herald. “We have called (nearly) every election for the past five years correctly across Canada, across Alberta, but we got this one wrong.” Later, on the Mainstreet website, he made reference to some of the more accurate predictions he had made during the previous seven years, including his projection of a Liberal majority government in the 2015 Canadian federal election.

This claim of Maggi’s seven-year track record bears examining.

Quito Maggi

Seven years ago, Maggi was better known as a political campaign manager and strategist than as a pollster. He had been involved for a time with the campaign of Martha Hall Findlay when she ran unsuccessfully for the leadership of the federal Liberal party in 2006. Maggi had also quarterbacked election campaigns for various Liberal candidates in Ontario at both the provincial and federal levels.

He did, according to the Mainstreet website, accurately predict victory for Alison Redford in the 2012 Alberta provincial election. But, the site posting adds, his prediction didn’t stem from any prior public opinion polling. “Our polling was not released to the public at any time during this election.” In other words, if Maggi had been active as a pollster up to that point, he was flying under the radar. During that same period he also served as campaign manager for former Alberta Conservative MLA Mike Allen, who later pleaded guilty to a misdemeanor charge following a prostitution sting in Minnesota.

It wasn’t until late 2014 that Maggi emerged publicly as a pollster, when he accurately predicted victory for former Toronto mayor Rob Ford in his run for a city council seat in Etobicoke. The Postmedia-owned National Post referred to Mainstreet then as an “upstart pollster.” At around the same time, Mainstreet began polling for all the Postmedia newspapers, including the Calgary Herald and Edmonton Journal.

Perhaps Mainstreet’s most spectacular success came from its confident prediction in May 2015 that Rachel Notley’s New Democrats would defeat the Progressive Conservatives and Wildrose Party to win the Alberta provincial election. Other pollsters were being cautious about the possible outcome of the tight three-way race but Maggi said, “the NDP surge is staggering. It’s going to be an NDP government.”

He was right. It was an NDP government. But Maggi isn’t always right. And that was manifest most vividly a week ago when he declared, “our expectation is that Smith will be elected mayor of Calgary on Monday.” Postmedia has since put all further polling from Mainstreet on hold until the research firm concludes an internal review of its methodology

So what to take away from all of this? First, that one should look for a second opinion from a credible pollster with a proven track record before accepting a single poll as gospel. Second, that one should take all advance polling during an election campaign with a grain of salt. Callers to CBC Radio’s alberta@noon program said yesterday that when a polling firm phones them using Mainstreet’s automated IVR platform – otherwise known as robocalls – they either hang up or misinform.

Maggi says that in his past life as a campaign manager and strategist he used to tell his candidates and staff that the only poll that counted was the one on election day. Perhaps he should include this as a caveat the next time he feels moved to call the outcome of an election race. If he got it wrong once, he could get it wrong again.

Click here for reuse options!
Copyright 2017 Brian Brennan - Author