Andrew Scheer is the new leader of the Conservative Party. His victory is a relatively big surprise. In terms of odds, I had him with around 20% chances (actually 25% the last time I ran the simulations but I never published these odds, so that's not fair). So that's more than what the BC Liberals had in BC in 2013 (15%) and less than what Nate Silver was giving to Trump (around 30%). This should put things in perspective.
As usual after a mistake or error of the projections, the question is: what could I have done differently? In this case, honestly, not much. Some people have called this race unpollable and while they will feel vindicated by the fact Bernier lost, I think this is an overstatement. The fact is that polls (and fundraising data) were able to predict this race fairly well. Except in one province.
So, why the surprise? The short version is that the farmers in rural Quebec managed to gather against Bernier (who wanted to end supply management). I was aware of the movement against him but data was still showing a large and comfortable lead in Quebec for Bernier. We can also mention the social conservatives (So-Cons) whose second choices ultimately helped Scheer. But the impact was much smaller than the under-performance of Bernier in Quebec. Plus, the model was actually factoring in the fact that Trost and Lemieux supporters would go more towards Scheer than Bernier.
Here below you have the results by province (the main ones) for Scheer and Bernier for the first round, compared to the latest Mainstreet polls as well as my own numbers (which were a mix of fundraising and polling data).
Mainstreet was closer (which makes sense given that they had access to data I didn't, like the number of members per riding). Still, you can see that Scheer beat the polls/projections in many places, including Ontario and BC but especially Quebec (and Bernier did worse). By the way, it's surprising that Mainstreet was closer overall but was giving lower chances of winning to Scheer than me (they were giving him 15%). I think their model simply had less uncertainty than mine. Also, remember, my model was free when their cost $100 a month just to access (at the same time, I thank them for at least providing some polls).
Let's focus on Quebec alone. In terms of points, the polls and projections (averaged) were predicting Bernier to get 3826 points and Scheer 1275, a lead of over 2500 points (in the first round and in Quebec alone). What actually happened is Bernier got only 3073 points and Scheer got 2161, a lead of of only 912!
This, right here, is why Andrew Scheer is now the leader and not Bernier. As a matter of fact, Bernier could have won with around 160 more votes in some key rural ridings in Quebec (in some, each vote represented around 2 points given the low number of members!). Things get worse for Bernier when you realize that he actually lost his own riding in Beauce to Scheer! It's literally possible that a 100 votes in his riding (plus 100 elsewhere in Quebec) could have decided this race!
So again, was it predictable? Hindsight is 20/20 but we simply did not have any indication that Bernier would win Quebec by so little. Not the polls, not the fundraising. I'm sure there are people out there saying "I called it". Well, good for them. But allow me to be skeptical. Yes we knew the farmers didn't like Bernier's plan but to think that this would be enough to cut his lead (in points) by more than half? Never.
So Bernier did worse than expected in the first round and Scheer much better. Overall however, I think the polls and projections did fairly well. Look at the last update I posted on Twitter on Saturday morning:
I think it's pretty close overall. Lemieux is too high and I knew it (I mentioned it) and Trost too low. Scheer is slightly outside of the confidence interval but remember that intervals are at 95%, not 100%.
The second part of the story is how Scheer got more 2nd choices (or 3rd, 4th, etc; I'll use the term 2nd choices or votes from now on). But as predicted, there wasn't a massive and systematic transfers towards Scheer and against Bernier. Look at this table from Andrew Coyne:
There are only really 2-3 cases where the transfers really helped Scheer: from Lemieux and Trost (and O'Toole but at this point, some of his votes were coming from Lemieux and Trost). I tweeted that Bernier's lead was stable at 6-7 points for like 6 rounds.
The model was predicting this. For instance we had that Lemieux supporters had Trost and Scheer as main second choices. Qualitatively the model was sound. Quantitatively? It was closer than the transfers observed in the actual results.
Maybe the one mistake I made was not to update enough the second choices. For instance, once Lemieux was eliminated and a lot of his supporters were with Trost, I could have made is such that even more Trost supporters would have Scheer as second choice. I didn't do that. Why? Well, first of all, these second choices were tricky to estimate and I had very limited data. Secondly, Bernier's lead was supposed to be big enough (thanks to Quebec) to survive a transfer of vote going 60% for Scheer and 40% for Bernier.
The 60-40 split is exactly the split observed yesterday after the first round until the 13th. Some of these transfers can be due to attrition (some ballots were dropped since they only had 1 or 2 choices). My simulations, in average, where increasing Sheer's points total by 9489 points between the 1st and the 13th round while Bernier's total was only increasing by 6243. In percentage, this gives 60% to Scheer and 40% to Bernier! So really, if you are looking for a reason as to why my projections failed, the redistribution of votes isn't where it happened.
At the end of the day, I think the comparison to Trump is an appropriate one as far as the level of surprise that was the Scheer victory (Note: do not read in this statement that I'm comparing Scheer to Trump. I'm not. Because I'm not a moron). Like Trump, the numbers were showing that Scheer could win, it just required a number of things to happen. A number of unlikely things based on the available information. The predictions failed to see the Trump victory because he needed to win a couple of States that were unlikely based on the polls. For Scheer, the fact that he did so well in Quebec (and Bernier so bad, relatively speaking) is by far the biggest reason the upset happened.
Also, let's remember that when my model gives a 20% chance of something happening, it means that, well, such an event should happen 1 out of 5 times. In other words, there is something wrong with a model where the favourite always win. But obviously it doesn't look (or feel) good when the upset happens.
Also, let's remember that when my model gives a 20% chance of something happening, it means that, well, such an event should happen 1 out of 5 times. In other words, there is something wrong with a model where the favourite always win. But obviously it doesn't look (or feel) good when the upset happens.
