This is the question I asked myself when I saw that we could order individualized Google Surveys.
Some of you probably know what I'm talking about. You can install the Google Opinion Rewards app on your phone and once in a while you'll receive surveys. Usually this is to ask you what you thought of a store you went to (or Google thinks you went to through tracking). You answer and you get a magical 10-20 cents that you can use to buy apps, music and movies on the Play Store.
Ok, I know what some of you will say: this isn't a valid poll! This won't work! This is the same as those non-scientific polls on websites.
Not quite. Google has a robust database (not shocking) and can provide a good sample. In the 2012 US election, the second best poll was actually from Google Survey! There is a page about using it for election. There is a template that is recommended for best results (using a two question system where you first ask people how likely they are to vote).
So look, I'm not trying to argue that a Google Survey is equivalent to a well designed poll by another firm. But if you think this is equivalent to those Sun Media polls on their websites for instance, you are dead wrong.
Ok so as an experiment I decided, last week, to order a poll. I only asked one question because if you ask two, the cost increases significantly (instead of 20 cents per answer, it was over $1!). I asked the following question: "If the Ontario election were held today. Which party would you vote for or are you currently leaning towards?"
And I offered a randomize choice of the parties with the leader's names as well as the "undecided" and "will not vote" options. The poll took 3 days, from May 22nd to 25th to collect the request 1000 observations. It was quite slow at first, which was disappointing, but picked up quickly at the end. I asked Google to ask the question in English to residents of Ontario aged 18 and over.
Ok, enough already, what were the results? Here they are below, in raw, unadjusted form:
Remember that this poll was done between May 22nd to May 25th. Unfortunately for me, I picked the few days where the NDP surged. I could actually see it by looking at the questionnaires coming in. As reference, the polling average around that time was (excluding the Forum poll as I think it's an outlier and I don't have enough polls conducted between the 22nd and 25th to compensate for its crazy results) 21% OLP, 37% PC and 35% NDP (and around 5% Green).
It says 1000 respondents but Google actually indicates that I only have 631respondents once weighted. Once the undecided and wouldn't vote are removed, we have roughly 400 respondents. A survey of that size would have margins of error of 4.8% 95% of the time. So the polling average actually falls within the results of this poll once we account for the margins of error. Except for the "another party". There is an obvious overestimation here. This makes no sense and I can't explain it right now.
My results are quite similar to the tracker from Mainstreet published on the 23rd (it was around 21-38-34 at that date)!
At 28%, this survey also got way more undecided than most polls (between 5% and 15% depending on the firm). To be fair, I only asked one question and even though it said "leaning towards", it might explain partially why I got so many undecided.
So, was the experiment a success? Kinda. I haven't played with the raw file, trying to reweigh the data (which is possible since I have age and gender data from most respondents; I, however, do not have the region or city) to see if I could "correct" the weights to get better results.
I also lost a lot of observations to the wouldn't vote or undecided, so a more serious experiment would require a higher budget to increase the sample size.
Still, I find it pretty interesting that I was able to go online and order a poll for only $200 and this poll's results can actually be reconciled with the polling average. Plus, as I said before, my poll was done during a period of rapid growth for the NDP. The PC had a much bigger lead for the first day and the NDP slowly but surely caught up.
Would it replace a true poll from a known Canadian pollster? Of course not! But there is some potential there. I wish Google would allow regional breakdowns though.
I'll gladly share the raw data file with whoever wants it. Just reach me on Twitter or by email and tell me why you'd want the data. I only asked that you link to my site and mention my name if you'll use the data for a blog post or anything.
