Yahoo Unleashes New Search Crawler

    July 28, 2006
    WebProNews Staff

Yahoo Slurp may be showing up heavily in access logs around the web, as the company gave the new spider a workout alongside the old spider before retiring the previous Slurp.

Yahoo Unleashes New Search Crawler
Yahoo Unleashes New Search Crawler

It has taken a few weeks for Yahoo to ensure its new version of Slurp will work properly, which explains why sites have probably seen a lot more Yahoo requests trawling their sites.

On the Yahoo! Search blog entry about the new Slurp crawler, David Simpson and Priyank Garg discussed the switch in their Weather Report post:

In addition to crawling the Internet faster, our new crawler is more efficient at visiting websites. As a result, site owners will notice as much as a 25% reduction in the number of requests and bandwidth consumed by the crawler.

Now, with the new crawler in full production, we have turned off the old crawler and site owners should see a much lower crawl load without a loss in content coverage.

As always, ranking changes and page shuffling around the index may have happened for websites, so publishers will want to take a look at where they show up now in Yahoo! Search.

The efficiency as reported by Yahoo should be welcome by publishers when their new bandwidth bills arrive.

Discuss this article with your fellow WebPros at WebProWorld.


Add to | Digg | Yahoo! My Web | Furl

Bookmark WebProNews:

David Utter is a staff writer for WebProNews covering technology and business.