Google Launched 665 Search ‘Improvements’ In 2012

    March 1, 2013
    Chris Crum
    Comments are off for this post.

Google says on its new “How Search Works” site that it launched 665 “improvements to search” in 2012.

In a graphic looking at data from 2012, Google explains that it had 118,812 “precision evaluations,” which are described as “The first phase is to get feedback from evaluators, people who evaluate search quality based on our guidelines. We show evaluators search results and ask them to rate the usefulness of the results for a given search.”

From there, it had 10,391 side-by-side experiments.

“In a side-by-side experiment, we show evaluators two different sets of search results: one from the old algorithm and one from the new, and we ask them for details about which results they prefer,” Google explains.

You can see Google’s Search Quality Raters Guidelines (at least a “cliffs notes version“) here (pdf).

Google ran 7,018 live traffic experiments in 2012. “If the evaluators’ feedback looks good, we move forward with a ‘live traffic experiment,'” Google explains. “In these experiments, we change search for a small percentage of real Google users and see how it changes the way they interact with the results. We carefully analyze the results to understand whether the change is an improvement to the search results. For example, do searchers click the new first result more often? If so, that’s generally a good sign.”

From there, came the 665 actual launches.

“Finally, our most experienced search engineers carefully review the data from all the different experiments and decide if the change is approved to launch,” says Google. “It sounds like a lot, but the process is well refined, so an engineer can go from idea to live on Google for a percentage of users in 24 hours. Based on all of this experimentation, evaluation and analysis, we launched 665 improvements to search in 2012.”

Typically, Google has been providing monthly lists of “search quality highlights” showing some of the tweaks they’ve made, but they haven’t done it in months. Despite today’s effort in transparency, it remains to be seen whether we’ll see exactly what Google has been up to since October.

  • http://www.rankwatch.com Rank Watch

    Google has been the only search company which has been very transparent about their algorithm updates. Almost all the algorithm updates on Google happen with their users in mind not generating more revenue and hence Google as a company has been very successful. With the web having so much of spam its has been very tough for search engines to separate the wheat from the chaff but they have been equal to the task.

  • John

    And 665 of them helped major sites like Overstaock, Amazon etc…. Search anything in my niche and they now dominate page after page. This is what they said would happen if we let the Feds pass net legislation. Lying jerks you did it on your own….I now actively support anything the Fed’s come up with that Google says is bad.

  • http://www.aqueous-seo.co.uk Jonathan

    Those of us who watch the SERPS regularly know that ‘what they have been up to since October’ is more significant and fundamental changes to the algorithm. John’s point above about big names dominating is true in some sectors but the more worrying issue is that Google alone are deciding what gets seen, by whom and when.

    Whilst this isn’t quite censorship, when you have 95% of the UK search market it is a very powerful position which you have to be very careful not to abuse.

    The key here is that we have handed the power to Google to decide these things, but we have never actually said that we believe that they are the best people to do this job.

    Have you or anyone here ever met one of their ‘evaluators’ and if so what are their qualifications for doing this job of evaluation? How can we be sure that they are doing the best job, or even the right job? Who checks? What if the results aren’t right, what then?

    Despite all these ‘improvements’ Google are a long way from providing great search results in many sectors. Just try Googling ‘mortgages Leeds’ and see how one firm can dominate the SERPS.

    If 2012 was volatile 2013 has started out even more so with wild swings for many businesses in their keyword positions. Building a sustainable competitive advantage for many businesses just became a whole lot harder.