Google Posts Big ‘Search Quality Rating Guidelines’ Document, Says It’s Just The ‘Cliffs Notes’ Version Of The Real Thing

    March 2, 2013
    Chris Crum
    Comments are off for this post.

We’ve seen Google’s search quality raters referenced numerous times, but now Google has made available the whole set of guidelines in one giant PDF for your perusal. The document is called “Search Quality Rating Guidelines,” and interestingly, it’s labeled version 1.0, and is dated November 2012. It was released as part of Google’s new “How Search Works” site.

“Google relies on raters, working in countries and languages around the world, to help us measure the quality of our search results, ranking, and search experience,” Google explains. “These raters perform a variety of different kinds of “rating tasks” designed to give us information about the quality of different kinds of results in response to different kinds of queries. The data they generate is rolled up statistically to give us within the Google search team a view of the quality of our search results and search experience over time, as well as an ability to measure the effect of proposed changes to Google’s search algorithms. Raters’ judgments do not directly impact Google’s search result rankings. While a rater may give a particular URL a score, that score does not directly increase or decrease a given website’s ranking. Instead these scores are used in aggregate to evaluate search quality and make decisions about changes.”

In the preface of the document, Google notes that the document itself is not the entire version that raters actually use on a daily basis, but rather a “Cliffs Note” version.

“The raters’ version includes instruction on using the rating interface, additional rating examples, etc.,” Google explains. “These guidelines are used as rating specifications for search raters, and this document in particular focuses on a core type of rating task called ‘URL rating.’ In a URL rating task, a rater is shown a search query from their locale (country + language) and a URL that could be returned by a search engine for that query. The raters ‘rate’ the quality of that result for that query, on a scale described within the document. Sounds simple, right? As you’ll see, there are many cases to think through, and this document is used to guide raters on some of those cases and how to look at them.”

In a Webmaster Help video released this past October, Matt Cutts also discussed the quality raters’ “impact” on algorithms.

Here’s another one they put out in May talking about how Google uses the human raters:

  • Mike 71

    My question is not about how the algorithms are changed as a result of human raters, but one particular competitor argues, the algorithms are skewed for commercial purpose to steer searchers to particular advertisers. This is an issue Google will have to address in light of media attacks, which question its objectivity.

  • http://www.lots0cash.com lots0

    Human Powered…. The BETTER Search Result.

  • BlokeToys

    Hmm, it seems Google is struggling against a backlash with all this PR. lol

    It doesn’t matter how they come to their conclusions for developing algo changes, the public now knows for a fact that they support massive corporations over small business for profit, they manipulate the search results for money, they dictate their rules to businesses all over the world without a mandate to do so, and they are hypocrites.

    I will not be happy until anti-competition laws are applied to this corporation to stop them from destroying small businesses all over the world. As long as Google is allowed to manipulate and control global business in the way that it does, millions of small employers, retailers and sites will suffer.

  • http://acrylicadvertisingdisplays.com Bob Shrewsburg

    It seems there are now rules to be followed or applied. Specific search terms used by searchers seem to go largely by the wayside in favor of their paid search companies. It is very difficult to use their engine. They stack the same result which were not asked for on the first few pages. Thus small companies. Get lost and relevant results not given equal weight.Is it pay or die on google?

    • http://signature.eu.com Steve, Web Marketing Consultant

      Totally agree with Bob’s comment. There is so much noise in the top third of the page (with priority to local listings and PPC that the top 3 sites in the organic search can be about a third way down the page. I’m not surprised this the way things are going. When your the only pub open in town, everyone who wants to drink has to pay your prices. The absence of a genuine contender for generating leads (not pay-per-lead) only strenghens their position.

  • John

    So sick of Google and their self important, assclown of search.

  • http://www.rioseo.com Marcos

    Yes its more about human interaction, just released a white paper The Future of International SEO which compliments of the Search Quality Rating Guidelines.

    International marketers now have the ability to talk to the newest emerging markets in their own language.

  • http://www.businessrefinery.com/ BizCode Barcode Solutions

    Looks like I’d better start reading this documents to see what’s going on :) I hope it’ll be usefull

  • http://marketerseoonfiverr.com marketerseo

    is very good for search engine user looks exactly what she wants when looking for specific keywords!
    I liked the article, good editor!