New “Bingbot” Will Crawl Non-optimized Sites More Easily

Microsoft has announced that it will be bringing the Bing web crawler out of beta on October 1st. It will be rebranded as "the Bingbot" and replace the existing msnbot. "It will stil...
New “Bingbot” Will Crawl Non-optimized Sites More Easily
Written by Chris Crum
  • Microsoft has announced that it will be bringing the Bing web crawler out of beta on October 1st. It will be rebranded as "the Bingbot" and replace the existing msnbot. "It will still honor robots.txt directives written for msnbot, so no change is required to robots.txt file(s)," a Bing representative tells WebProNews.

    "Improvements to the bot enable more efficient crawling, and increase the ability to crawl content on sites not optimized for search," he says.
     
    Robot - This is not the real Bingbot, but it will be here in October.Rick DeJarnette has more about the change on the Bing Webmaster Blog:

    Instead of the old msnbot 2.0b showing up in your server logs, the updated user agent will be:

    Mozilla/5.0 (compatible; bingbot/2.0 +http://www.bing.com/bingbot.htm)

    The HTTP header From field will also change as shown below:

    From: msnbot(at)microsoft.com

    will become

    From: bingbot(at)microsoft.com

    If Bing finds separate sets of directives for Bingbot and for other crawlers, directives for bingbot will take precedence, the company says.

    I find the part about increasing the ability to crawl content on sites not optimized for search to be particularly interesting. I wouldn’t exactly call this an invitation to ignore SEO. Obviously Google is still the biggest search engine anyway, but even as far as Bing is concerned, good SEO practices will likely still help your rankings.

    Also keep in mind that optimizing for Bing is becoming increasingly important. Not only is Facebook giving more reason for people to search (where Bing provides the web results), but the Yahoo/Bing integration will be here (likely) before the holidays.

    Get the WebProNews newsletter delivered to your inbox

    Get the free daily newsletter read by decision makers

    Subscribe
    Advertise with Us

    Ready to get started?

    Get our media kit