Live Search Tweaks Crawler

Crawler screams

Get the WebProNews Newsletter:

[ Business]

The Live Search Webmaster Center team announced earlier this week that Microsoft made some improvements to its Live Search webcrawler to improve indexing efficiency.

The two main improvements, blogs, Fabrice Canel, are with HTTP Compression and with Conditional Get:

HTTP Compression: HTTP compression allows faster transmission time by compressing static files and application responses, reducing network load between servers and Microsoft’s crawler. Microsoft says it supports the most common compression methods: gzip and deflate as defined by RFC 2616.

Conditional Get: Live Search now supports conditional get as defined by RFC 2616, meaning that Live Search will not download a page unless it has changed since the last time it was crawled. The crawler includes the "If-Modified-Since" header and time of last download in the GET request and when available includes the "If-None-Match" header and the ETag value in the GET request.

More information and links can be found at the Live Search Webmaster Center blog .

Live Search Tweaks Crawler
Comments Off on Live Search Tweaks Crawler
Top Rated White Papers and Resources

Comments are closed.

  • Join for Access to Our Exclusive Web Tools
  • Sidebar Top
  • Sidebar Middle
  • Sign Up For The Free Newsletter
  • Sidebar Bottom