Google Explains Recrawling for Updated News

Google on Recrawl Feature for Staying Up to Date

Get the WebProNews Newsletter:

[ Search]

Google has a post up on the Google News blog today talking a little bit about how it recrawls news content in order to provide the most up to date content and eliminate dead links.

"How do you balance looking for new content against the need to update older content? How can you make sure the content is fresh, doesn’t link to dead pages or display headlines that have been changed by the publisher?" asks Google.

Dewey Defeats TrumanGoogle’s answer is that it has implemented a recrawl feature that lets it focus on getting the newest content, while displaying the most current version of older content. After Google News discovers an article, it will continue to crawl it repeatedly to look for changes. In the first day, it will actually recrawl it more frequently, because as the company says, the most changes are usually made to news stories soon after they’re published.

"In some cases, we’ll even revisit articles we had trouble crawling the first time around," says Google. "After that, we visit them less often. Either way, we try hard to present users with the freshest news. (We bet whoever wrote "Dewey Defeats Truman" wishes they had recrawl!)."

Google says the feature is intended to reduce the number of outdated headlines and dead links, and for publishers, it will provide assurance that Google will index the latest stories and updates as soon as possible.

Related Articles:

Google Changes How it Handles Paid Content

> Minds of the Media Gather to Discuss Future of News

> Google Okay With Blocking News Corp.

Google Explains Recrawling for Updated News
Top Rated White Papers and Resources
  • soccersher

    my customized google news page doesn’t update many sections on a regular basis. instead of continually showing newer articles in each section, it shows some articles for days at a time but replaces others. it’s really annoying. what is causing this?

    also, gnews seems to have an overabundance of crap from the New York Times. Is there any way to filter out NYT?

    Thanks in advance.

  • Join for Access to Our Exclusive Web Tools
  • Sidebar Top
  • Sidebar Middle
  • Sign Up For The Free Newsletter
  • Sidebar Bottom