Google tweaks its search algorithms over 500 times a year. You may have already known that, but Google is sharing a new video today designed to give people a “deeper look” into how Google makes “improvements” to its algorithms.

I don’t think everyone would agree that they’ve all been improvements, as we see complaints about this every day, but Google makes a lot of changes aimed at improving search results, nevertheless.

“There are almost always a set of motivating searches, and these searches are not performing as well as we’d like,” says Engineering Director Scott Huffman. “Ranking engineers then come up with a hypothesis about what signal, what data could we integrate into our algorithm.”

The video does provide some unique behind the scenes footage of Google engineers plugging away on their computers, presumably working on the algorithms.

Google briefly talks about the process of raters. “These are external people that have been trained to judge whether one ranking is more relevant and higher quality than another,” says software engineer Mark Paskin.

“We show these raters a side-by-side for queries that the engineer’s experiment might be affecting,” explains Google Search Scientist Rajan Patel. “We also confirm these changes with live experiments on real users.”

“We do this in something called a sandbox. We send a very small fraction of actual Google traffic to the sandbox. We compute lots of different metrics,” says Paskin.

“In 2010, we ran over 20,000 different experiments. All the data from the human evaluation and the live experiment are then rolled out by a search analyst,” says Huffman.

Sangeeta Das, a quantitative analyst says, “For each project, it’s usually one analyst assigned from the moment that we’re talking to the engineers, trying to learn about their change.”

“We then have a launch decision meeting where the leadership of the search team then looks at that data and makes a decision,” says Huffman.

“Ultimately, the goal of the search eval analyst team is to provide informed, data-driven decision, and present an unbiased view” says Das.

“If our scientific testing says this is a good idea for Google users, we will launch it on Google,” says Google Fellow Amit Signhal.

The video then looks at the “did you mean” and “showing results for” features as an example.