Matt Cutts Talks Quality Raters’ Impact On Algorithms, Says Guidelines May Be Made Public

Get the WebProNews Newsletter:

[ Search]

While it has been known that Google’s “quality raters” (the people who judge sets of search results behind the scenes) don’t directly influence Google’s algorithms, there is still a misconception out there to the contrary.

Nobody at Google (as far as we know) is looking at these sets of search results and voting sites up and down as if they were browsing reddit.

Google’s Matt Cutts talked about this in a new Webmaster Help video released today. He responds to the user-submitted question:

If you have human ‘quality raters’ evaluating the SERPs and influencing which sites may be impacted by Panda, how do you confirm that consumers are more satisfied with the results?

“There’s a problem with this question…the word ‘influencing,’” says Cutts. “So, we have evaluation raters who look at the quality of pages, using their own judgment, as well as guidelines that we give them on when things are navigational, when things are vital, which things are off topic, which things are spam…all that sort of stuff. But those folks don’t influence our algorithm in any direct sense.”

“When an engineer has…an idea for an algorithm – call it “panda” – he’ll come up with an algorithm, and it will rank the results 1-10, so you’ll have a side by side (left side and right side), so you’ll actually have the results right there,” he continues. “That goes out to the evaluation team and these human quality raters, and as a blind taste test, they say, ‘I prefer the left side of the search results’ or ‘the right side of the search results’…and then we’ll get that feedback back, but that evaluation where the search quality evaluators say, ‘I prefer this side’ or ‘I prefer that side’ does not directly affect the algorithm. It doesn’t affect Panda.”

Cutts does suggest that we might see the actual guidelines Google gives to the quality raters made public. They have been leaked in the past, as he notes, but Google may sometime soon post those for anyone to see anytime.

“We might be able to make those human quality rater guidelines that we make available to people at Google available to the larger world, and I think that would be a good thing because then people would be able to read through it,” says Cutts. “It leaked a few years ago, and what someone said was, ‘The biggest surprise is that there weren’t really that many surprises. All the guidelines that we provide are pretty much common sense, and would match with what I think just about anybody would sort of say about…’Yeah, it does make sense that this is a navigational page or that this is pages off topic.’”

For more on what Cutts has said about Google’s Quality Raters process in the past, read this.

Matt Cutts Talks Quality Raters’ Impact On Algorithms, Says Guidelines May Be Made Public
Top Rated White Papers and Resources
  • serbon

    Matt Cutts, GWT and google today broken again. Lot of queries missing, traffic stats (even old stats) is incorrect, no traffic from google on estabilished sites today (where it was everything fine yesterday).
    It also for best user experience?

    • John

      “Lot of queries missing, traffic stats (even old stats) is incorrect”

      I noticed this as well this morning. Only about half the queries and impressions that were showing up yesterday in GWT for the past month are showing today.

      Did they recalculate the data from the previous month or is this just a bug?

  • http://www.udisystem.com Madhukar S V

    Are these raters industry specific ?

  • http://ephedrinewheretobuy.com Mike Budd

    Honestly I wouldn’t mind reading these “human quality rater guidelines” but I don’t need them to know what is quality and what’s not, for a website, as writer or as reader of other sites, and quality in general: give me 2 minutes in topics that I know well and I’ll tell you if the content is original (meaning created by the person or copied) and original (meaning new, fresh, interesting, smart, because writing your own stuff doesn’t mean quality in all cases ;)

  • Join for Access to Our Exclusive Web Tools
  • Sidebar Top
  • Sidebar Middle
  • Sign Up For The Free Newsletter
  • Sidebar Bottom