Blogspot’s Content Warning Blocking Spiders

    December 2, 2008
    WebProNews Staff

Edgy bloggers using Google’s Blogspot might have been miffed already—but likely also understood—at Google slapping content warnings over their adult-oriented material and demanding age verification. But what might deter (poorly) minors from viewing is also preventing search crawlers from accessing the content at all.

Blogspot's Content Warning Blocking Spiders

Erotica blogger Rebecca Molay broaches the subject on Google Groups amid fears her blog will disappear from search engines entirely:

My blog has erotic content, which have led Blogger to put up a content warning page. Users will have to click on a link on that page to get to the blog.

 This is fair, and the content warning page itself has actually had no effect on the number of visitors.

The reason I hadn’t put up one myself was that I was afraid of the effect it would have on search engine visibility. Unfortunately my fear turned out to be based on a real problem.

The content warning page uses a DIV-tag/javascript for the relevant button. There is no regular link from the Content Warning page to the blog itself.

Even if Blogger is a part of Google, no one has apparently told the Google spiders how to find away around this javascript. Nor has anyone told the Blogger people to change the code of the content warning page
(or to give the spiders another route around this hurdle).

Blogspot's Content Warning Blocking Spiders


The theme was picked up in another thread by “Grey,” who explains the problem this way:

Google Blogger blogs with Content Warning splash pages ( aren’t friendly with Google Webmaster Tools. The forced splash page is evidently to confusing for the Webmasters Tool to bypass. Attempted verification causes errors. Blogspot blogs without the warning work perfectly fine.

John Mueller, Webmaster Trends Analyst at Google Zurich, responded with a less than reassuring post in that thread:

This is something the teams here are already looking into, but I don’t have more information than that at the moment. I agree that it would be great if we could get all websites at least verified in Webmaster Tools, as well as being indexed in the normal web search index.

It seems reasonable no one would object to a content warning, but if Google is true to its commitment to index the world’s information, erotic content on its own blogging service should be on the list to include.

More on this topic if and when Google addresses it.

Hat tip to Search Engine Roundtable.