Coverage of SMX Advanced continues at WebProNews Videos.  Stay with WebProNews for more updates and videos from the event this week.

Cutts started by giving a rather witty answer to the question of “how can you tell if your site is in the penalty box?”  He replied, “One really good way is if it disappears completely from Google.  That’s what we call a leading indicator.”

After some laughter subsided, Cutts then shared additional details.  He said, “We make the penalties public where we think it can give the most help.  So if you’re a small mom and pop, and you didn’t even realize (this happens a lot) – I hired a webmaster, and he put some hidden text on the page in 2003, and it was like 3 sentences, and they didn’t know – that’s the sort of thing where you want to tell them [through official channels].

“But if you see a sustained drop in ranks, or if it drops completely out, that’s the sort of thing where . . . there’s lots of forums on the web, including the Google Webmaster Forum, where you can go and ask for some help.”

Cutts noted that Google’s employees will often respond with some “very, very blatant hints” about what’s gone wrong on such forums.  And here’s one more important detail: unlike the LeMons judges, Google isn’t trying to toy with anyone.  Cutts said that the only reason all of this isn’t conducted out in the open is because scammers and black hats would use the info to their advantage.

Maybe it’s getting to be time they do so. The days of one reverse-deciphering the algorithm seem long gone – the best at Yahoo and MSN don’t seem to have done it in all this time – as are the days when Google could be easily gamed with mere keywords and links. At this point, what’s the harm in letting webmasters know exactly why their sites incur penalties, giving them ways to atone, and taking away their wild speculations?

Then again, what would they blog about?

Google is in an enviable position. First, they don’t have to tell webmasters squat as long as they’re clamoring and clawing each other just to get listed there. Secondly, Google only loses if searchers take off, not search engine optimizers. But it seems they could eliminate some headaches by being more forthcoming about how not to incur the rankings wrath.

Not that they don’t fire warning shots. Last fall, much like how the Boston Massacre was conducted, Google felled a few as a warning to others to abandon the hope of paid link schemes. The speculation about minus-whatever penalties suggest the gradual increase in ranking penalties are similar warnings.

The leading theory is that these penalties are enacted by humans, not algorithms. If so, Google should think about being more forthcoming about their reasoning. We know that penalties are not exclusively enforced by humans. Last December a glitch in the system caused many to see their rankings drop by six places. Google acknowledged the –6 penalty as an error.

Recently the –60 penalty has been the focus, not just because the number of webmasters reporting drops in that range, but also because Googlers sort of back-alley nodding the penalty in certain forums.

In a Google Groups thread, Swiss Googler John Mueller replies to the pining post of one who took a –60 penalty. While this penalty and others had been explained away by webmasters as having to do with bad linking practices, this one had more to do with the template used, and the hidden content/links popping up in the code.

I’m still seeing templates being distributed with them; not only that, the links are disguised in a way that the average webmaster cannot find them.

 Personally, I think having footer links are fine if they are relevant to the site or template and nofollow’ed. However, hiding them in this style is – in my opinion – not ok at all. Just this week I helped a friend with his template: it had a block of code similar to this in it. It turned out that not only was it hiding links, it was also spreading malware. It really upsets me to find code like this in a template: it shows that whoever made the template not only knew the contents were not ok, but also wanted to prevent the user from finding or editing it.

We learn a few things from that statement: 1.) Be careful with your templates; 2.) Footer links should be nofollow links; 3.) There is a bit of manual interpretation when it comes to ranking. Barry Schwartz says it also means Google admits a –60 penalty exists.

Mueller suggested the webmaster clean up the code and submit a reconsideration request. Others suggested, under their breaths, that Google stop dictating every last detail of their sites.

A –30 speculator mentioned the use of a similar footer as well as buying some links, which he doesn’t think helped. Another, who got hit with a 950 rank-busting, declares no black hat tactics while admitting non-perfection. It could be that a –950 penalty is the last warning shot before getting booted from the index altogether.

One explanation for getting the –950 penalty was the use of interlinked sites with content that was "too thin" or irrelevant. That has a simple solution, really, and an old one: provide great, valuable content and only link to sites or pages that do the same.

Until we get more clear-cut do’s and don’ts and explanations of penalties, we’ll be free to speculate. If that never happens, here’s what we know:

1.    Nofollow links that could be construed as spammy or are bought and sold
2.    Be careful with templates, and pay close attention to footers to make sure there’s nothing shady there like hidden content or links
3.    Content is still the most important thing
4.    Pay attention to the warning shots

It’s not clear why it was a reality, other than a Google glitch.

Webmasters reported worries about sites that historically had ranked in the top two results for certain keywords had dropped to the sixth position. Dropping to below the top five, or below the fold (below where a person may have to scroll), could mean sudden death in the exposure game.

Though it sounded kind of silly that Google would invoke a four or five-place penalty (for what, a broken link and saying something mean about Sergey maybe?), the number of webmasters reporting the problem added a bit of an urban myth weight to it.

Search Engine Roundtable is reporting that Matt Cutts confirmed a glitch with Google, and that the database is in the process of resetting everything to normal.

There wasn’t much explanation other than that. Guess even Google sometimes makes mistakes. 
 

(click to enlarge)

They also noted that 76% of this traffic comes in as referrals from other search engines.

This is slightly ironic, since Jason Calacanis, founder of Mahalo, has historically been very critical of the worth of search engine optimization. I’m not the only one who sees the irony in this, since Allen Stern also noted it, saying “Mahalo is an SEO Play“. As Allen notes, if Mahalo didn’t want this traffic it would be easy for them to block the spiders thru their robots.txt file.

I’ll go further: it appears to me that Mahalo was built with optimization in mind from the outset, since it was built with a clear navigation structure and non-query-stringed, keyworded links — something that rarely occurs without site designers intentionally desiring to bring in search spiders.

The unfortunate side of this is that Mahalo appears very vulnerable to me now, since they’ve designed themselves in direct opposition to Google’s Webmaster guidelines. Mahalo has marketed themselves as “a human powered search engine”, and Google’s guidelines state that they don’t want to be indexing search results pages:

“Use robots.txt to prevent crawling of search results pages or other auto-generated pages that don’t add much value for users coming from search engines.”

I’ve seen Google drop search results pages from large sites due to this very thing before, and Mahalo is now prone to receive this same treatment. It’s a bit sad in a way, because startups like this need traffic to be successful, and Mahalo’s recent traffic growth is in danger of getting completely reversed if Google now chooses to apply their editorial controls as they have done with others.

Mahalo appears to have around 145,000 pages indexed in Google at the moment, and I’d say they are in danger of losing most of that indexation.

Of course, Google’s definition of what constitutes “search results pages” may be a bit semantic at times. There are many catalogs, guides and directory sites which expose their “search results pages” for indexing, and Google appears to accept those as good quality content sites.

It’s not always clear what definition Google is using for “search results pages”. It may be that they’re wanting to reduce/prevent the indexing of “search results pages” which are merely other indices of webpage links under the concept that Google SERPs are already a sort of meta-content, and indexing other similar meta-content is of fairly low value to endusers. In this case, Mahalo’s traffic certainly is endangered, regardless of the fact that their results pages are human-generated as opposed to the more traditional, algorithmically-generated variety. A rose is a rose and search results are search results.

Jason might want to bite the bullet and consult with some SEO experts now to get recommendations on how to offset the risk of losing his newly-found traffic. He’s one SEO misstep away from taking a bump on the nose.

Comments

They damage rankings and ultimately may result in banishment from the search engines. However, before panic sets in, let us guide you through the basics of penalties and filters.

A penalty is caused by significant violations of a search engine’s website guidelines, such as:

Penalties can be issued after a person reviews a website or after it has been crawled and processed by search engines. They result in a website being heavily held back in the rankings, or removed from the search engine’s index entirely. A penalty is called a ban when a website is completely removed from a search engine.

To remove a penalty, a site needs to first correct the problem that caused it, then contact the search engine and request a reinclusion. It is important to note penalties are not common for business websites, particularly in Google.

1. In Google, this is done through a Webmaster Tools account, which can be set up for free.
    
2. MSN has just created a Webmaster Tools system like Google’s.
    
3. Yahoo does not have a defined reinclusion process.
    
4. A filter is caused by passing a search engine’s threshold setting for one or more optimization/link building elements, such as:
    
   1. Too many keyword mentions on a page’s body content (over-optimization).
       
   2. Too much keyword blurring between a site’s pages.
       
   3. Too many links with the same anchor text.
       
   4. Too much keyword-rich internal linking (can cause 950 filter).
       
   5. Too much link building in a short period of time.
       
   6. Too much link building using the same anchor text.
       
   7. Link building in bad neighborhoods.

Filters are common. They are issued automatically after the site is crawled and processed by search engines and result in a site being held back in the rankings.

1. Filters can be keyword-based or site-based.
    
2. Filters can be mild (held back a few positions) or heavy (held back hundreds of positions).
    
3. Filters can have a time element (like the normal Google Sandbox process, where a site is initially held back many positions and over time gets held back less and less until eventually it ranks near its allinanchor rankings).

In general, to remove a filter a site needs to first correct the problem that caused it, then wait for the search engine (s) to crawl the site again and find the corrections. The next time the search engine updates its rankings with the corrected data, the filter will be lifted automatically.

1. In some cases, like the Google Sandbox, you can simply outwait a filter. Websites commonly spend anywhere between a few months and a year or so in the Google Sandbox. The time websites spend in the Sandbox has significantly decreased over the last few years.
    
2. In some cases, you can remove a filter on a site by doing things that search engines like (such as getting quality links to the site from other respected websites in the same sector) to outweigh the things about the site or optimization they don’t like.

Filters are common, especially in Google.

1. The Google Sandbox is technically a filter. Google closely examines new websites for over-optimization to try and minimize spammy websites filling its SERPS. As a result, newer websites often trip filters when they start an optimization campaign using traditional SEO (lots of keywords on the page, keyword-heavy titles and description, keyword heavy anchor text in incoming links). This pattern of new websites getting filtered and eventually getting released from the filter is called the Google Sandbox. Websites can speed up their release date from the Sandbox by getting quality internal links and not going overboard with on-site optimization.
    
2. New websites are not the only targets. Older websites can trip filters when they go overboard with over-optimization

Being aware of how search engines assess penalties and filters is essential to avoiding them. Do not try to cheat the system, over-optimize or trick search engines. They are savvy to these tactics and punish those who attempt to take advantage of them. What may help in the short term will only end up hurting in the long term.

Being competitive in the SERP’s is important, and sitting idly is not a good strategy. If you must build links or optimize your website try to stick to the guidelines and use common sense.

Comment

The chilling effect of the nonstop lawsuits and crackdowns on people for downloading movies and songs concerns University of Arkansas law professor Ned Snow. His recent abstract on the issue, called Copytraps, received mention on PhysOrg for what it has done to web users.

Those who download music, for example, and believe they are doing so legally, does not matter to the courts. Copyright infringement has occurred, and the accused has to pay.

The problem as Snow sees it comes from the broader impact of "copytraps," his word for sites that indicate downloading is legal. Anyone busted for dong this comes away with reluctance to do any other downloading.

That’s where Snow sees a problem. He contends in his analysis that considering the normal First Amendment protections for exercise of speech, penalizing downloaders is unconstitutional:

That certain forms of copy-speech receive First Amendment protection implies that copyright’s strict liability punishment is unconstitutional. It is well established that the First Amendment precludes strict punishment of unprotected speech, for a possible effect of strict punishment is chilling protected speech.

Speakers may refrain from engaging in protected speech for fear that they might mistake whether the speech actually is protected. The mere possibility of such chilling, courts have held, is sufficient to rule unconstitutional strict liability punishment of unprotected speech.

Snow’s essential argument seems to be that if people are prosecuted and penalized for downloading from a "copytrap," they are unlikely to engage in any legal downloading in the future. The effect of the punishment creates a violation of the First Amendment.

We will be interested to see someone try Snow’s opinion of prior restraint as a legal defense. "Strict liability punishment of copying makes no sense in a world where copying is the architecture of being," Snow said of the Internet.

The House of Representatives Committee on Foreign Affairs voted Tuesday to pass the Global Online Freedom Act, a bill designed to penalize U.S. companies up to $2 million if they cooperate with the technological surveillance of political dissidents or share technology and information used for “Internet-restricting” purposes.

“Dictatorships need two pillars to survive: propaganda and secret police. The Internet, if misused, gives them both in spades,” said Rep. Chris Smith of New Jersey. “Both wittingly and unwittingly, companies operating in places like China have discovered they’re a part of these regimes.”

Tuesday’s bill must still pass several hurdles before reaching the House or Senate floor.

$2 million? Somehow I can’t imagine Google loses any sleep over such sums (even when it may hurt their image to be punished in such ways). Though why not directly convert this into prize money handed out as “best censorship circumvention tool or service” award for smaller start-ups (the kind of company to which that sum is meaningful)? The “Chinese knowledge workers, fighting with one hand tied behind their back” – to quote Tim Bray, and to name just one group – might appreciate it.

[Via Search Engine Land.]

Comments

The only commonality among the sites appeared to be that they either sold links or were part of a network of sites with internal linking structures. The immediate response was fear that traffic would plummet. However, as noted by many traffic remained the same, or in at least one case, increased.

Barry Scwartz at SERoundtable saw a two percent increase in traffic following a 3-point PageRank drop, from PR 7 to PR 4. The lack of real impact has produced the theory that the PR hit was more of a Google head fake. If the search engine company meant business, these sites would have dropped in the organic search results or would have been delisted altogether.

Following that train of thought away from conspiracy theories that Google was penalizing sites and blogs that are often critical of company, Schwartz comes up with quite the level-headed conclusion:

If Google delisted all of those sites, then that would hurt their relevancy on some queries. With Google, they want to deliver the best possible results. How can they do that and also send a message to link sellers and link buyers? The safest method is to take this route and lower their PageRank. Link buyers, although not recommended, look at PageRank as a measurement for buying links. Google lowering the PageRank of some of these sites should make it harder for some of those sites to sell links.

Robert Scoble has his own, somewhat less controversial theory, from a post entitled "Google PageRank is Dead and Has Been For Some Time":

[B]loggers were showing up too high in searches anyway. In comparing to my friends we got lots of traffic from Google that we didn’t deserve. The problem is that traffic isn’t good anyway. Put it this way, let’s say I showed up high in a search for Saturn Cars (since I’ve written about them). Most people wouldn’t have found much value in that post and even if they did they wouldn’t have stuck around to be a regular reader.

So whether the PageRank update was a message or a relevancy tweak remains a bit of mystery, but these two theories are as good as any.

This was the tone of things that would happen to anyone who spammed Wikipedia as Jonathan Hochman, Founder/President, Hochman Consultants said during the Wikipedia, Yahoo Answers & Answer Sharing session at the New York leg of SMX Social Media.

Speaking to Forbes’ Andy Greenberg, Jonathan Hochman said: "The blacklist is public, so search engines can read it. You don’t want to get on that list."

"Basically it’s ‘shoot on sight.’ You’re guilty until proven innocent."

Jonathan also writes in Search Engine Journal that:

"If you only end up on the Wiki blacklist, that might be ignored. If you do a bunch of other shady things, that signal starts to look consistent.

I know some of the people who run the blacklist. Being hardcore geeks, they enjoy watching what happens to the search rankings of the sites they add. Their impression is that being added to the list isn’t a good thing for your search marketing campaign. Maybe that’s wishful thinking on their part, or maybe not."

Comments

Tag:

Responding to request for comment as to why the directories in question had vanished from the search results and whether Google was penalizing paid link directories especially, a Webmaster Central source replies:

There’s no "outright penalty" for being a directory, but we do value, as I’m sure you’ve heard, "unique, compelling content."

Directories can run into the problem of not containing original information. Should the webmaster believe their site has fallen in search results, there’s no reason they can’t add more "unique, compelling content" to help their site rise.

So there you have it, though it’s still not an answer (and we really wouldn’t expect one) as to whether the directories affected most were the result of the paid links reporting form.

But we can assume from that answer that Google is treating link directories much the same way it treats splogs and spammy websites. As long as the directory has some value to the end user (i.e., the content is king philosophy), then there shouldn’t be any beef.

Further, as is true with any site, if penalized by Google (and if Google search results are a major goal), directory owners can add content and resubmit for indexing.