Got Crawl Issues? Live Search Can Help

Webmaster Tools Gets Major Update

Get the WebProNews Newsletter:

[ Search]

Microsoft launched its first major update to Live Search Webmaster Tools since November, adding crawl issue feedback, backlink data, improved filtering, and the ability to export data in CSV format to Excel for offline analysis. Update: Five minutes after publishing, the Live Search Webmaster Team let us know they were delayed a little, and that the update should go live at 5:00 EDT.

WebProNews got a sneak peak of the update from Live Search’s Jeremiah Andrick and Nathan Buggia, who say, "search has become mission critical" for websites. This update to the Webmaster Tools service is intended to help webmasters achieve better rankings in Live Search, and to provide a two-way dialog between webmasters and Live Search.

In other words, it answers a lot of FAQs, provides a lot of data, and eliminates a lot of theorizing in terms of penalties and/or achieving higher placement. "No other search engine provides this data," Buggia assured us.

The update revolves around a few frequently posed questions about improving a website’s relationship with a search engine. Is my site being crawled and indexed? Did the Live Search bot encounter any issues while crawling my site? Have I incurred any penalties? Who’s linking to my site? Is my robots.txt file configured properly?

Naturally, in response to questions, Live Search identified four common "crawl issues"

  • Broken links/File Not Found (404): Webmaster Tools will list URLs Live Search tried to crawl but was unable to due to removed content or misspelled link.
  • Robots Exclusion Protocol: Obviously, if robots.txt, a meta tag, or HTTP header attribute is used to block a search crawler, it’s generally on purpose. On the off chance a URL is blocked inadvertently, Webmaster Tools will provide a list of links not indexed for this reason.
  • Your link’s too long: Live Search flags URLs with "exceptionally long query strings" for fear of an infinite loop scenario (Google calls them "infinite spaces"). The "Long Dynamic URLs" report helps webmasters identify links they should consider shortening. How long is too long? The team suggests that if it won’t fit into an email subject line, it’s too long.
  • Unsupported Content-Types: Live Search likes content types like HTML, XML, Powerpoint, etc., and maintains a list of acceptable content types. Unsupported content-types, like .exe files, are not supported and will not be indexed by Live Search.

Buggia says that, unlike Google, Live Search does not penalize for backlinks coming from shady neighborhoods. Nevertheless, a webmaster likes to know where his or her traffic originates. For that reason, Webmaster Tools now has a filter functionality that allows webmasters to track backlinks from specific domains or subdomains (digg.com for example), and from top level domains (.fr, .uk, etc.).

Sometimes that kind of data can be overwhelming for websites with a lot of pages, which is why the filtering functionality allows webmasters to filter results by up to two subdomains or subfolders.

An interesting new feature is the ability to export data in CSV (comma separated values) form to Excel or other reporting tool for examination offline. You list of crawl issues or backlinks, for example, can be put directly into Excel.

The Live Search team also said they now provide penalty notifications and descriptions.

Listen to the audio

Got Crawl Issues? Live Search Can Help
Top Rated White Papers and Resources
  • http://www.brugbart.com/ BlueBoden

    Google dose not punish websites who is getting links from bad neighborhoods, it wouldn’t make sense if they did, because this would obviously be abused by some webmasters.

  • http://www.threemoneymethods.com Three Money Methods

    I’m not sure if I’m doing something wrong, but I signed in to the webmaster tools, and typed in my URL and my XML feed… and it went into a clicking/reloading pattern and then just brought me back to the same screen…

    After typing this, I tried it without my XML feed…  I didn’t understand that it didn’t want an actual XML feed, but rather a MSN feed…  When I just put in my web address, it gave me the code for adding LiveSearchSiteAuth.xml to my web site(s), so now I’m going to go through the process of adding this to several sites, to see if it helps MSN in any way.

    I still find Google’s interface much more intuitive, but this is a good improvement for MSN.


  • http://www.LeadsLander.com Ed Shea

    Webmaster tools does not indicate if your sitemap is uploaded, or valid, or even read.  The text on the sitemaps page indicated that you would submit your sitemap each time it changes. I beginning to question the value of publishing sitemap.xml files.  Are they being used?

    I also see that files that are disallowed in my robots.txt file are being reported as indexed. Other directories that are allowed have zero pages reported as indexed.  If the tools are now reporting reliable information what value can they have?

  • http://moneysurf.angelfire.com sajjad

    It’s a welcome move and should offer some convinience to the web masters.

  • http://www.greekvillasforsale.com greeksteve

    Google appears to be ‘punishing’ my site ‘greekvillasforsale.com’ as it has’nt crawled it since 12/7/08 would the msn item give me a clue a to Googles’ actions ( or inactions)?



  • http://www.sun-herald.com/elections/monroe.html Peter Monroe

    Thanks for the tips, Jason.  As always, a nice article with some solid information.

  • Join for Access to Our Exclusive Web Tools
  • Sidebar Top
  • Sidebar Middle
  • Sign Up For The Free Newsletter
  • Sidebar Bottom