While it has been known that Google’s “quality raters” (the people who judge sets of search results behind the scenes) don’t directly influence Google’s algorithms, there is still a misconception out there to the contrary.

Nobody at Google (as far as we know) is looking at these sets of search results and voting sites up and down as if they were browsing reddit.

Google’s Matt Cutts talked about this in a new Webmaster Help video released today. He responds to the user-submitted question:

If you have human ‘quality raters’ evaluating the SERPs and influencing which sites may be impacted by Panda, how do you confirm that consumers are more satisfied with the results?

“There’s a problem with this question…the word ‘influencing,'” says Cutts. “So, we have evaluation raters who look at the quality of pages, using their own judgment, as well as guidelines that we give them on when things are navigational, when things are vital, which things are off topic, which things are spam…all that sort of stuff. But those folks don’t influence our algorithm in any direct sense.”

“When an engineer has…an idea for an algorithm – call it “panda” – he’ll come up with an algorithm, and it will rank the results 1-10, so you’ll have a side by side (left side and right side), so you’ll actually have the results right there,” he continues. “That goes out to the evaluation team and these human quality raters, and as a blind taste test, they say, ‘I prefer the left side of the search results’ or ‘the right side of the search results’…and then we’ll get that feedback back, but that evaluation where the search quality evaluators say, ‘I prefer this side’ or ‘I prefer that side’ does not directly affect the algorithm. It doesn’t affect Panda.”

Cutts does suggest that we might see the actual guidelines Google gives to the quality raters made public. They have been leaked in the past, as he notes, but Google may sometime soon post those for anyone to see anytime.

“We might be able to make those human quality rater guidelines that we make available to people at Google available to the larger world, and I think that would be a good thing because then people would be able to read through it,” says Cutts. “It leaked a few years ago, and what someone said was, ‘The biggest surprise is that there weren’t really that many surprises. All the guidelines that we provide are pretty much common sense, and would match with what I think just about anybody would sort of say about…’Yeah, it does make sense that this is a navigational page or that this is pages off topic.'”

For more on what Cutts has said about Google’s Quality Raters process in the past, read this.