Matt Cutts Responds To Hitwise

SearchNewsLeave a Comment

Share this Post

Whether or not Bing’s search results are more accurate than Google’s isn’t up for discussion here.  Aside from the inherent sampling errors when comparing the actions of either group -- a 65 percent market share and a 27 percent market share are not equal sample sizes, leading to large amounts of variability -- Hitwise’s conclusion seemed more like wishful thinking.

Matt Cutts Talks About Google's Lastest Web Spam EffortsAnd now, the Googler Supreme, Matt Cutts, has thrown his two cents into the fray.  Needless to say, he didn’t agree with Hitwise’s “conclusion” either. 

In his Google Buzz post, Cutts asks:

Are you able to determine whether the user clicked on a search result vs. just left the search engine to go to another site? There's a difference between an abandoned search and clicking on a search result, but both result in the user searching and then going to a different site. By Hitwise's definition, wouldn't doing a query on Bing and then going to Google or Yahoo count as a "successful search" on Bing?

Not only that, but when asked, Hitwise couldn’t even define what they considered a successful search:

Hitwise later confirmed to me that they don't know whether the user actually clicked on a search result or just went to a completely unrelated site. Given all that, I'm surprised to see Hitwise is still pushing this metric and still calling it "search success rate."

So here comes Hitwise, armed with incomplete data that’s rife with variability biases, and they can’t even define what they consider a successful search engine query?  If that’s the case, their entire “Bing is more accurate than Google” thesis should be thrown out the window.

Now, this is not an attack on Bing.  If you use Microsoft's search engine and you like the results it gives, more power to you.  That doesn’t mean, however, Hitwise should be given a free pass when they deliver erroneous conclusions that ignore each engine’s market share.

Are Bing’s results more accurate than Google’s?  I really don’t know and without extensive, side-by-side comparisons of numerous queries performed by equal-sized groups, neither do the rest of us.

Ultimately, it all comes down to preference.

Leave a Reply