Bing Does Another, Slightly Different Blind Search Test
As you may have read, Microsoft is attacking Google again. This time, they’ve extended the “Scroogled” campaign to call out Google over ad targeting in Gmail, based on a feature that Google has openly implemented since Gmail came out.
But that’s not all, Bing is also pushing the “Bing It On” campaign again as well, with some updated testing. Microsoft commissioned another study by Answers Research using 1,000 people 18 and older from across the United States, choosing them from a random survey panel. These people were required to have used a major search engine in the past month, and according to Microsoft, had no idea that Bing and Google were specifically being tested, nor were they told the study had been commissioned by Microsoft.
Like on the Bing It On site, people were shown results without ads, Bing’s social features or Google’s Knowledge graph. Just raw organic search results. Instead of being able to search for whatever they wanted, they had to choose from a list of five queries at a time. The list could be refreshed if they didn’t like any of the options.
“We wanted queries that matched what people typically searched for, so we finally settled on using terms from Google’s Zeitgeist 2012, because while we could have used our own Top Searches of 2012, we figured the right thing to do was to go with our competitor’s terms,” explains Bing behavioral scientist Matt Wallaert in a blog post. “After all, you’d think Google would be better at their own top queries, right?”
“Wrong,” he adds. “In a blind test, people preferred Bing to Google for the web’s top searches. And that is just based on pure web results, so no ads, no Bing Snapshot and Social Search, no Google Knowledge Graph. Even taking away some of our most innovative features and with the handicap of using Google’s top search queries, Bing still comes out on top, with 52% of people preferring Bing’s results over Google’s, 36% preferring Google’s, and 12% choosing Bing and Google equally (for those that favor discarding ties, that’s 60% Bing, 40% Google when people had a clear preference). For the especially geeky, all those numbers are +/- 3% at a 95% confidence level.”
As is the case with Bing’s previous test, and with the concept of the Bing It On site, the test is still hardly representative of the true search experience, given the absence of the key features that were left out (Bing’s social features and Google’s Knowledge Graph). These features have both become important parts of each search engine’s respective user experience. The trend in search is moving away from the “ten blue links”.