Spiders And Widgets And Feet… Oh My

    January 12, 2007

As I was looking around the blogosphere this morning, I came across a number of good search engine articles I’d like to share with you. Most of these are of the helpful, SEO variety, but the last one is just weird.

First off, there’s some news concerning Google’s spidering habits that you might find interesting – the Google spider does not necessarily pay attention to meta tags and robot.txt files; a concept that makes me a little uncomfortable. On the Digital Inspiration blog, Amit Agarwal discovered a couple of popular sites (del.icio.us and Google Finance of all sites) whose robot.txt files were being ignored.

del.icio.us’ robot.txt includes the “noarchive,nofollow,noindex” directives for all search engine bots, a request Google’s apparently ignored. As for Google Finance, file was setup to disallow agents from the /finance pages, a request that the Google bot obviously ignored (Amit reports of 44000 pages with the /finance path are in Google’s index).

Considering what we’ve just learned, if I were you, at the very least I’d check to make sure the directives of my robot.txt file is being acknowledged by Google. If they are not, try posting something about it at the Google Webmaster Help group forum.

In case you haven’t heard, some of the more effective tools that can assist non-junk link building campaigns are widgets. If you aren’t sure of what a widget is, think of the all the blog chicklets visible around the Internet. You know, those little images in the left and right columns of blogs inviting users to click here or submit this or read that.

A fantastic example is the Flickr Zeitgeist widget that shows recent images from the Flickr index.These little devices can be incredibly beneficial when it comes to link building campaigns, just ask Abhilash.

The brown sugar SEO king wrote an extremely helpful post about these little suckers, complete with testimonials of just how successful they can be when it comes to link building (you can also ask Lee Odden* who helped confirm Abhilash’s thesis). If you’d like to give the widget strategy a shot, I would suggest reading his post thoroughly and follow the links he provides.

*Odden indicates his RSS button tool (yes these are considered widgets) has generated over 40000 unique backlinks alone – something I’m sure many would do a great deal for.

The feet portion of this post comes courtesy of the Yahoo Buzz blog. Apparently, the search for celebrity feet is increasing a great deal… so much so Yahoo blogged about it. An example of this behavior – searches for Kylie Minogue’s feet were up 123% yesterday. In true Yahoo Buzz fashion, they have a list of the most popular celebrity feet searches that’s topped by ex-Black-Eyed Peas singer Fergie.

I’m pretty much at a loss for words after that, so I’ll leave you with this thought – people are weird.