iEntry 10th Anniversary RSS Newsletter Advertising
Join the WebProWorld Forum!
Text: Decrease Font Size Increase Font Size | Print Print Article | Share: Delicious Digg StumbleUpon Post to Twitter Post to Facebook
3 commentsWednesday, August 6, 2008

Give The Googlebot A Break Already

He hates being an infinite space cadet

Got one of those nifty calendars on your site that expands month after month with the click of a mouse? Be kind to the Googlebot and slap a nofollow tag on the link—he might just follow it forever!

Well, Torrey Hoffman says the Googlebot isn't quite that stupid, but he's close. The calendar conundrum is a classic one the bot should be already familiar with and knows not to keep clicking that button like a heroin-addicted rat. It's also a classic example of what the search engineers at Google call an "infinite space."

If you don't know what else qualifies as an infinite space, the Webmaster Central team has already started notifying webmasters about the problem when it occurs. The team recommends verifying your site at Webmaster Tools and checking the Message Center.

Hoffman mentions another scenario where a website will provide filtering for a set of product search results, dividing items according to color, brand, price, etc, resulting in thousands of automatically generated URLs for the bot to follow. Again, think of a nofollow tag to give the robot a break, or follow other guidelines provided by the Webmaster Help Center.
 

News Tags: Search, Google, SEO

Aww, poor google bot!!

this is brilliant!! With all the intelligence google and their bot has, could it really be stupid enough to crawl a calander?!?

Kinda makes me wanna set up a test page just to see!!!

Lol, Love It!!

webmaster tool

webmaster tool is great, usefull tool between you and crawler

Publish A Comment

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.
CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
2 + 15 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
SEARCH
Popular WPN Business Resources












Subscribe to WebProNews


Send me relevant info