Quantcast

Bing: We Don’t Track The ‘Bing It On’ Results Because Of Privacy, Have No Idea If People Are More Likely To Select Bing

Get the WebProNews Newsletter:


Bing: We Don’t Track The ‘Bing It On’ Results Because Of Privacy, Have No Idea If People Are More Likely To Select Bing
[ Search]

Bing has no idea if people are more likely to select Bing results when using its Bing It On challenge, and has admitted as much in a blog post defending its campaign.

Despite being over a year old, the Bing It On challenge has been in the news this week, due to a study from Freakonomics’ Ian Ayres and a a group of Yale law students.

When it launched the campaign last year, Bing said that people prefer its search engine 2:1 over Google. Earlier this year, they updated the wording to say that people simply prefer Bing to Google for “the web’s top searches”. The Bing It On site currently says, “Wherever we go, people prefer Bing over Google for the web’s top searches.”

And they are indeed still going around offering the challenge. They were recently spotted presenting it at a Seattle Seahawks game.

Ayres wrote this week, “When I looked into the claim a bit more, I was slightly annoyed to learn that the ‘nearly 2:1′ claim is based on a study of just 1,000 participants. To be sure, I’ve often published studies with similarly small data sets, but it’s a little cheeky for Microsoft to base what might be a multi-million dollar advertising campaign on what I’m guessing is a low-six-figure study. To make matters worse, Microsoft has refused to release the results of its comparison website, BingItOn.com.”

He said, “So together with four Yale Law students, I set up a similar-sized experiment using Microsoft’s own BingItOn.com site to see which search engine users prefer. We found that, to the contrary of Microsoft’s claim, 53 percent of subjects preferred Google and 41 percent Bing (6 percent of results were ‘ties’). This is not even close to the advertised claim that people prefer Bing “nearly two-to-one.” It is misleading to have advertisements that say people prefer Bing 2:1 and also say join the millions of people who’ve taken the Bing-It-On challenge, if, as in our study, the millions of people haven’t preferred Bing at a nearly a 2:1 rate. Microsoft might have realized this and has more recently altered its advertising to back off their original claim to just say that people ‘prefer’ Bing.”

Bing’s Matt Wallaert defended the challenge in a blog post, criticizing Ayres’ claims and study. You can read the entire post to see all of his jabs, but here’s what he said about Bing not tracking the results of Bing It On:

Ayres is bothered that we don’t release the data from the Bing It On site on how many times people choose Bing over Google. The answer here is pretty simple: we don’t release it because we don’t track it. Microsoft takes a pretty strong stance on privacy and unlike in an experiment, where people give informed consent to having their results tracked and used, people who come to BingItOn.com are not agreeing to participate in research; they’re coming for a fun challenge. It isn’t conducted in a controlled environment, people are free to try and game it one way or another, and it has Bing branding all over it.

So we simply don’t track their results, because the tracking itself would be incredibly unethical. And we aren’t basing the claim on the results of a wildly uncontrolled website, because that would also be incredibly unethical (and entirely unscientific).

Ayres’ final issue is the fact that the Bing It On site suggests queries you can use to take the challenge. He contends that these queries inappropriately bias visitors towards queries that are likely to result in Bing favorability.

First, I think it is important to note: I have no idea if he is right. Because as noted in the previous answer, we don’t track the results from the Bing It On challenge. So I have no idea if people are more likely to select Bing when they use the suggested queries or not. Emphasis added.

He goes on to explain some differences between Bing’s controlled studies and the Bing It On site.

More from the Bing It On saga here.

Image: BingItOn.com

Bing: We Don’t Track The ‘Bing It On’ Results Because Of Privacy, Have No Idea If People Are More Likely To Select Bing
Top Rated White Papers and Resources
  • http://robertjduncan.me Robert J. Duncan

    Yup, “tracking itself would be incredibly unethical” …. http://www.bingiton.com/sa/1203190233/Challenge/media041213/omniture_bingiton.js

  • http://www.mattwallaert.com Matt Wallaert

    Actually, we have a really good idea if people prefer our results, because a 3rd party conducted not one but two studies showing that they did. Heck, even SurveyMonkey showed it (though in a small sample study): http://blog.surveymonkey.com/blog/2013/06/26/google-bing-seo-surveys/.

    Just because we don’t track the marketing website that is BingItOn.com doesn’t mean we have no idea what our quality is, just that we have no idea the results of the site itself.

    (Note: I work for Bing)

    • http://www.webpronews.com/author/chris-crum Chris Crum

      So the actual Bing It On Challenge that people can actually take is not proving anything, right?

  • http://www.infusion-labs.com Ian

    Well now, this is a cleverly manipulative press release by MicroSoft. We’re not interested in MS tracking their Bing It On comparison toy. It’s the actual Bing search engine people want to know about.

    They wrote “Because Of Privacy” making you think subconsciously they’re talking about the Bing Search Engine.

    Way to manipulate the prescription drug zombies out there. Not fooling me though.

  • Join for Access to Our Exclusive Web Tools
  • Sidebar Top
  • Sidebar Middle
  • Sign Up For The Free Newsletter
  • Sidebar Bottom