Google Slows Down Crawling So SOPA-Protesting Sites Don’t Get Hurt

A lot of sites have gone dark today in protest of SOPA, and many others are protesting in other ways, with banners, and whatnot. Google, protesting SOPA itself, is slowing down crawling for today to h...
Google Slows Down Crawling So SOPA-Protesting Sites Don’t Get Hurt
Written by Chris Crum
  • A lot of sites have gone dark today in protest of SOPA, and many others are protesting in other ways, with banners, and whatnot. Google, protesting SOPA itself, is slowing down crawling for today to help out those protesting.

    “We realize many webmasters are concerned about the medium-term effects of today’s blackout,” said Google’s Pierre Far on Google+. “As a precaution, the crawl team at Google has configured Googlebot to crawl at a much lower rate for today only so that the Google results of websites participating in the blackout are less likely to be affected.”

    A couple days ago, Far provided a how-to guide for webmasters looking to join the protest without hurting how Google sees their sites.

    The most important of these tips, he said, is: “Webmasters should return a 503 HTTP header for all the URLs participating in the blackout (parts of a site or the whole site).”

    Get the WebProNews newsletter delivered to your inbox

    Get the free daily newsletter read by decision makers

    Subscribe
    Advertise with Us

    Ready to get started?

    Get our media kit