Bing Indexing NoIndex/Nofollow Content

Bing’s webmaster forum has a number of posts where people are complaining that Bing will not index their content. Some people, however, are having the opposite problem. Content that they do not want indexed by Bing is being indexed by Bing, despite the webmaster’s efforts to keep it out of the search engine. Are search engines ignoring your NoIndex/Nofollow attributes? Let us know.

One person posted about this, and a Microsoft employee responded, admitting that this is an issue on Bing’s side. Here’s how the conversation goes:

Webmaster:

I have a site containing pages that I don’t want their content being indexed by search engines so all of those pages have

<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW" />

in their header and it has been like this for a while now.

Both Google and Yahoo respected the tag and those pages are not being indexed by them but today I checked and found all those pages even recent ones are indexed and cached on Bing.

Program Manager, Bing Webmaster Center:

This is a known issue we are working quickly to resolve. If you have pages you would like permanently removed from our index, please send me a mail to bwmc@microsoft.com with your domain name and "MSNBot ignoring robots tags" in the subject line. Please also include the URLs in the body of the message. You may use an * wildcard for any directories such as:

http:example.com/ wrongdirectory/*

Normally, I would request that you fill out a content removal request, however, since this is a problem on our side, I’ll do the leg work for you.

Others have complained recently that Bing and (in some cases) even Yahoo have been ignoring NoIndex and Nofollow in meta tags. Have you experienced any similar issues? Please share.

There are 48 Comments. Add Yours.

Steve
August 10, 2009 at 4:59 pm

How about blocking Bing’s class B or class C IP range(s) from your server instead of worrying about robots.txt or meta tags? It’s not hard to find it’s ip range.

I think that will send the high and mighty Microsoft a very clear message with each webmaster that blocks Bing’s access that we’re not going to put up with the stupid search engines making up the rules as they see fit. If we let this start with a small(er) fry, by the time the big guy starts dictating how things are going to be, it will be a lot harder to get them to realize we’re the customers and they are the here to serve us, not the other way around.

What if Google came out and said “We don’t care about robots.txt or tags, we’re going to do how and what we please” ?????

With some sites reporting over 80% of their traffic coming from search engines like Google (ours is like 91%, of which 70% of that is Google), how can they justify cutting a search engine entirely if it decides to not follow the standards of the rest of the world? If we don’t stop this now, it will just get more difficult later on.

If somebody were to write up a petition site for meta tag/robot.txt reform, I’d gladly add my name to that list.
- Guest--Carl A. Helsing
  August 11, 2009 at 10:10 am
  
  To;The Webmaster;Coinidence in”time”.Over the previous year or so I have been able to find my own web site thru Yahoo-Google-FireFox and so on.
  Today,none of those access it either by url or by title(SKYTRAINZASTRON)–But,Bing is now accessing it beautifully by the title.I’m thrilled with that.I just got a new Hewlett_Packard PC and it come’s with a Bing browser in windows vista.
  So with me,everything is fine.If it can’t be the”Robot Of Fate”,is it I who may be accused of cheating—Thank you—Carl A,. Helsing–
rip blu-ray
August 15, 2009 at 11:19 am

Enjoy the software and enjoy your life.blu-ray to mp4 converter, blu-ray burner
Spheric
August 25, 2009 at 6:17 pm

I don’t give a darn if they crawl any page I have publicized on the internet. But, I would like them to honor their promise to not index pages I have asked them not to index.

I have an ecommerce site and Bing is ignoring both the robot.txt file and the robots meta tag.

As others have tried to point out, there is some content that is only relevent in context to the site. I would prefer not to have searches that return my site cluttered up with this stuff.

For example, why on earth would I want them to index empty cart pages, checkout pages (that redirect to the empty cart pages), ‘email a friend’ pages that don’t know what product to send, customer pages that all redirect to the login page, or even the login page itself, etc. etc. etc. And, why in God’s name would they want to waste their own resources indexing all this crap.

They have indexed dozens of pages like this that I have asked them not to index (nearly half of the pages that occur in the index). This is garbage and they know it.

Now, if they will just get off the dime and rectify the situation, I’ll calm down a little.

< /rant >
mobile-phone-lcd.com
August 29, 2009 at 4:44 am

I submitted my info regarding the pages I wanted removed to the listed email address for the Bing Webmaster Center as above. I even got a reply back requesting more details and how they could find the pages in their own index. I told them use site:MyWebsite.com and you will see lots of indexed popup pages.

Since then nothing. No removal of the Popup image pages.

I sent a couple of follow up emails. Got no reply.

So i think the above reply and email address as far as I can see is just a public relations stunt to suggest they are on top of this.

How difficult can it be to remove pages that I have requested.

So far the pages that have been dropped out the Bing index are genuine product pages I want listed. The number of product pages listed actually dropped after I contacted them.

So what is going on with these guys at Bing?
Murray
September 10, 2009 at 1:35 am

This issue has not gone away as far as my site is concerned. What is more worrying is the popup image links are for the most popular products on my site. The products with the best back links from other sites that have recommended or linked to them.

It seems very convenient that the best selling products link to popup images.

Is this some kind of strategy by Bing to eliminate competition for their best paying corporate customers who have big advertising budgets?
blog webmarketing
November 25, 2011 at 8:37 pm

I used to be suggested this web site by way of my cousin. I am not sure whether this publish is written by way of him as no one else know such specific approximately my problem. You’re incredible! Thanks!
Whaaaat?
November 26, 2011 at 7:56 pm

I am not positive the place you’re getting your information, however good topic. I needs to spend some time learning much more or figuring out more. Thank you for great information I used to be in search of this information for my mission.
коттедж| строительство| материал| сооружение| здание| стена| технология| участок| проект коттеджа| загородный коттедж
December 25, 2011 at 6:27 am

Hi there, I discovered your website by means of Google even as searching for a related topic, your web site came up, it seems great. I’ve added to favourites|added to my bookmarks.