<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>WebProNews &#187; Slurp</title>
	<atom:link href="http://www.webpronews.com/tag/slurp/feed" rel="self" type="application/rss+xml" />
	<link>http://www.webpronews.com</link>
	<description>Breaking News in Tech, Search, Social, &#38; Business</description>
	<lastBuildDate>Sun, 12 Feb 2012 17:01:07 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.3.1</generator>
		<item>
		<title>Yahoo Slurp 3.0: Crawler Revised And Rolling Out</title>
		<link>http://www.webpronews.com/yahoo-slurp-30-crawler-revised-and-rolling-out-2008-04</link>
		<comments>http://www.webpronews.com/yahoo-slurp-30-crawler-revised-and-rolling-out-2008-04#comments</comments>
		<pubDate>Mon, 14 Apr 2008 23:37:19 +0000</pubDate>
		<dc:creator>WebProNews Staff</dc:creator>
				<category><![CDATA[Search]]></category>
		<category><![CDATA[Crawler]]></category>
		<category><![CDATA[Slurp]]></category>
		<category><![CDATA[Yahoo]]></category>

		<guid isPermaLink="false">http://www.webpronews.com/?p=45022</guid>
		<description><![CDATA[Site publishers will see a new Yahoo bot crawling through their pages, and webmasters should be aware of a couple of little issues that may affect them.
]]></description>
			<content:encoded><![CDATA[<p>Site publishers will see a new Yahoo bot crawling through their pages, and webmasters should be aware of a couple of little issues that may affect them.<br />
<span id="more-45022"></span>
<p>
Yahoo&#8217;s search team needed the past few weeks to prep their latest crawlers for its sojourns onto the World Wide Web. This new mutli-legged beast began its official rollout, as noted on the <a href=http://www.ysearchblog.com/archives/000531.html>Yahoo Search blog</a>.</p>
<p>
&#8220;The new Yahoo! Slurp 3.0 recognizes the same user-agent and all robots.txt directives for &#8216;Yahoo! Slurp,&#8217; though it&#8217;ll identify itself as Slurp 3.0 in your web logs,&#8221; the blog said. A handful of webmasters may notice a couple of changes.</p>
<p>
&#8220;Please note that if you&#8217;re using IP-based recognition of our crawlers, you might see a drop in crawl/coverage from Yahoo! We strongly recommend that you move to reverse DNS-based identification of Yahoo! Slurp if you&#8217;re using any other method to avoid this problem,&#8221; they said. The IP ranges for Yahoo&#8217;s crawlers will change to a smaller set, too.</p>
<p>
The other concept applies to robots.txt files and directives. Yahoo Slurp 3.0 will recognize existing entries for &#8216;Slurp&#8217; or &#8216;Yahoo Slurp&#8217;, but not a reference to &#8216;Slurp/2.0.&#8217; Yahoo thinks those entries are rare enough anyway, but just in case, it was prudent for them to reference it.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.webpronews.com/yahoo-slurp-30-crawler-revised-and-rolling-out-2008-04/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>

<!-- Performance optimized by W3 Total Cache. Learn more: http://www.w3-edge.com/wordpress-plugins/

Page Caching using memcached
Database Caching 1/9 queries in 0.008 seconds using memcached
Object Caching 192/203 objects using memcached

Served from: webpronews.com @ 2012-02-12 12:15:11 -->
