Microsoft unveiled its big “Bing It On” campaign this week. Part of that is a site, which allows users to perform a search query, and choose which results they prefer. After five rounds, the tool reveals whether your picked Bing or Google for each one. The one you picked the most, must be your search engine of choice.
It’s an interesting comparison of organic search results between the two rivals, but it strips out large parts of the user experience for both search engines. It doesn’t include Google’s Knowledge Graph or BIng’s social bar, for example. It’s hardly an accurate representation of today’s search experience for either engine.
Still, Bing says people prefer Bing to Google two to one:
On Thursday, after the Bing It On site was launched, Google’s Matt Cutts tweeted:
Tried searching for bingiton on http://t.co/RArZYm0D. Bing had trouble finding itself: http://t.co/l7s3SK3k
I tested the query out myself, and found similar results to Cutts’.
It must be embarrassing for Bing to have Google beat Bing on a query for the very tool that Bing is promoting to show that its results are better for Google. However, it’s unclear how many users actually had these results. In the comments section of an article we did on that, some users said BingItOn.com was the top result on both. Perhaps some personalization signals come into play, though I can’t imagine why Bing would associate the cheerleading movie “Bring It On” or its musical counterpart with anything from my personal life. Honest.
Despite the differences in results in this example, there are other queries that provide much more similar results. Cutts had something to say about that too.
In a comment thread on Hacker News (via Barry Schwartz), Cutts said, “Last time I checked, it looked like Bing was still using clicks on Google search results as a signal in Bing’s rankings.”
It’s funny he should mention that, because I couldn’t help but be reminded about that whole ordeal as I was playing around with Bing It On.
Early last year, Google ran a “sting operation,” as Danny Sullivan who first reported on the story called it, that appeared to show Bing “stealing” at least top results from Google, by monitoring how Internet Explorer and Bing toolbar users use Google.
Google created some test search results pages returning results for queries that nobody would ever search for, and results that wouldn’t make sense for such queries. For example, a query for “hiybbprqag” would return a top result from TeamOneTickets. A query for “mbzrxpgjys” would return RIM’s homepage. A query for ” indoswiftjobinproduction” would return a result for Sandra Lee Recipes at FoodNetwork.com.
“The only reason these pages appeared on Google was because Google forced them to be there,” explained Sullivan at the time. “There was nothing that made them naturally relevant for these searches. If they started to appear at Bing after Google, that would mean that Bing took Google’s bait and copied its results.”
Bing’s results were mirroring each of these examples, though Google found that only a handful of the pages tested proved the point.
In response, Bing’s Harry Shum wrote in a blog post, “We use over 1,000 different signals and features in our ranking algorithm. A small piece of that is clickstream data we get from some of our customers, who opt-in to sharing anonymous data as they navigate the web in order to help us improve the experience for all users. To be clear, we learn from all of our customers. What we saw in today’s story was a spy-novelesque stunt to generate extreme outliers in tail query ranking. It was a creative tactic by a competitor, and we’ll take it as a back-handed compliment. But it doesn’t accurately portray how we use opt-in customer data as one of many inputs to help improve our user experience.”
“The history of the web and the improvement of a broad array of consumer and business experiences is actually the story of collective intelligence, from sharing HTML documents to hypertext links to click data and beyond. Many companies across the Internet use this collective intelligence to make their products better every day,” Shum continued. “We all learn from our collective customers, and we all should.”
“From its inception, we have had what we believe is a distinct approach to search, and the features and innovation in Bing – from our new user experience and visual organization approach to our focus on inferring user intent and helping customers complete complex tasks, Bing has added a new voice and new experiences to search,” he added. “We never set out to build another version of an existing search engine.”
After that post, Cutts and Shum (as well as Blekko’s Rich Skrenta) had a discussion about the whole thing at the Farsight Summit. Throughout that, Cutts maintained the position Bing was basically cheating, and Shum echoed the sentiments of his post, adding, “My view is that we just discovered a new form of spam or click fraud and the Google engineers helped us to figure it out. He said that he wished people could share things like that with them before taking it to the press and getting a “wow effect”. He also said that it would be great if he and Matt could compare signals that they could use.
At one point, Shum played the “Google has a toolbar too” card, but Cutts said users see “big red capital letters” letting them know about the data sharing as soon as they install it. Cutts also said at the time, “We don’t use clicks from Bing’s users in Google’s rankings.”
In the new Hacker News thread, when asked how Microsoft has access to Google’s algorithm data, he replied, “IE and Windows, I believe.” He then points to a section in Microsoft’s IE 8 privacy policy, which says:
“When Suggested Sites is turned on, the addresses of websites you visit are sent to Microsoft, together with standard computer information. … Information associated with the web address, such as search terms or data you entered in forms might be included. For example, if you visited the Microsoft.com search website at http://search.microsoft.com and entered “Seattle” as the search term, the full address http://search.microsoft.com/results.aspx?q=Seattle&qsc0=… will be sent.”
“Most people have little idea that allowing a feature called ‘Suggested Sites’ will result in their Google searches and clicks being sent to Microsoft, or that Microsoft will use clicks on Google search results in Bing’s ranking,” said Cutts. “MSFT also uses something called the Microsoft CEIP (Customer Experience Improvement Program), and I think that’s either opt-out already or they’re making it opt-out in Windows 8–it’s built into the ‘Use Express Settings,’ I believe.”
“Again, I haven’t looked at this very recently, but if you’re using a recent version of Windows and IE, you’re probably sending your searches and clicks to Microsoft unless you’ve been very careful about how you configured your computer,” he concluded.