How Many Spiders Does Google Have?

Google Talks Number of Machines Crawling the Web

Get the WebProNews Newsletter:

[ Search]

Google has posted a short but interesting video to its Webmaster Central YouTube channel. A user asked the question, "How many bots/spiders does Google currently have crawling the web?" and Google’s Matt Cutts gave his answer.

"It’s important to realize that it’s not really actual robots or actual spiders out there…instead, it’s banks of machines …at Google’s data centers who open up an HTTP connection and request a page and then get it back," he says. "So any bank of machines (even 50 machines) could easily be requesting a bunch of different content."

"We try to refresh a large fraction of the web every few days," he adds. "So it turns out you really don’t need a ton of machines. Even a relatively small amount of machines operating in parallel and fetching pages in parallel can really be able to crawl of find new pages on the web in a very quick way."

Matt says that Google doesn’t give out the exact number, but that it’s somewhere between 25 and 1,000. I’m not sure what you can really do with that information, but it’s worth hearing a quick rundown of how it works for those who aren’t real familiar with how Google indexes content.

Related Articles:

> Google Rolls Out Breadcrumb Display in SERPs

> Google Makes it Easier to Tell Where Results Originate From

> Get More Links in Your Actual Google Results


How Many Spiders Does Google Have?
Top Rated White Papers and Resources
  • http://www.nhile.net/ Guest

    now a days, spiders is SEO best friend.. 😀 “rather than Dog?” hehe

  • http://www.autorii.com Autorii

    thanks Chris Crum for bringigng up this video – is really interesting tahat Matt tells – 25><100 spiders for google.

    i always beleived that they have mouch more.

    • Chris Crum

      He actually said more than 25, less than 1,000, so it’s a bit more vague.

  • Guest

    Up to a thousand spiders? I can just see the arachnophobes fleeing to Bing and Yahoo.

  • Join for Access to Our Exclusive Web Tools
  • Sidebar Top
  • Sidebar Middle
  • Sign Up For The Free Newsletter
  • Sidebar Bottom