One major thing you need to avoid (or in hindsight, should have avoided) is cloaking, which is basically just showing Google something different than you’re showing users. Google’s Matt Cutts did a nice, big video about cloaking last summer. He calls it the “definitive cloaking video,” so if you have any concern that you may be in the wrong on this, you’d better watch this. It’s nearly 9 minutes long, so he packs in a lot of info.

Cloaking is “definitely high risk,” Cutts says in the video.

With Penguin, there’s been a lot more talk about bad links costing sites. Link schemes are specifically mentioned in Cutts’ announcement of the Penguin update, and Google has been sending webmasters a lot of messages about questionable links recently. That’s definitely something you don’t want to ignore.

But while Google didn’t mention cloaking specifically in the announcement, it did say the update “will decrease rankings for sites that we believe are violating Google’s existing quality guidelines.” Cloaking fits that bill. Google divides its quality guidelines into basic principles and specific guidelines. Cloaking appears in both sections.

“Make pages primarily for users, not for search engines,’ Google says in the Basic Principles section. “Don’t deceive your users or present different content to search engines than you display to users, which is commonly referred to as ‘cloaking.’”

In the Specific Guidelines section, Google says, “Don’t use cloaking or sneaky redirects.” This has its own page in Google’s help center. Specific examples mentioned include: serving a page of HTML text to search engines, while showing a page of images or flash to users, and serving different content to search engines than to users.

“If your site contains elements that aren’t crawlable by search engines (such as rich media files other than Flash, JavaScript, or images), you shouldn’t provide cloaked content to search engines,” Google says in the help center. “Rather, you should consider visitors to your site who are unable to view these elements as well.”

Google suggests using alt text that describes images for users with screen readers or images turned off in their browsers, and providing textual contents of JavaScript in a noscript tag. “Ensure that you provide the same content in both elements (for instance, provide the same text in the JavaScript as in the noscript tag),” Google notes. “Including substantially different content in the alternate element may cause Google to take action on the site.”

Also discussed in this section of Google’s help center are sneaky JavaScript redirects and doorway pages.

In a statement from Ashley S. Huston, Vice President, Corporate Communications, for the Wall Street Journal, she explains that “Google First Click Free is a way to introduce our content to new readers and broaden our audience. As a strategy, we hold back a few of our top stories by not having the full story crawled, which limits select articles from being available via First Click Free. We have been doing this since last summer as a strategy to encourage subscriptions.”

Google’s First Click Free program allows users to see full-text content that is typically hidden behind a subscription barrier. With certain articles remaining “unlocked,” Google can understand them better as it runs a query, which in turn affords more visibility to the website of the publication. Also, Google doesn’t have to worry about users getting mad that they can’t fully read articles that show up in a search. Basically, if a user searches for something in Google, the full article comes up, though subsequent clicks within the site are blocked per subscription status, as detailed at the bottom let of the image below:

Google allows 5 First Clicks per day, to limit the amount of free content one can access, but publishers are still pulling back on this. So far, news outlets have been all or nothing on the matter, with the New York Times offering all of their content available to be first-clicked. Some outlets allow none. The Wall Street Journal has been implementing a sort of hybrid model where only some content is available. Google wants to be able to search and index all of the content of an entire website, which can make subscription news sites tricky, as an outlet might mistakenly (or purposely) cloak some of their content, to maintain site traffic, while still locking their stories. The WSJ’s hybrid system looks to be mutually beneficial to its website and Google.

Quality guidelines are on the minds of advertisers in particular this month, as Google continues to tweak what it thinks is and is not a quality experience for the arriving visitor. Much has been made of the advertiser side of quality guidelines, but they certainly apply in other ways on Google.

The company’s Maile Ohye talked about serving pages and their indexing by Google’s crawler as part of her post at Google Webmaster Central. The post comes in response to webmasters interested in a discussion of certain webserving techniques.

Webmasters who use geolocation or IP tracking to deliver content based on location need to be careful not to accidentally “cloak” their pages. In cloaking, a site delivers one type of content to the Googlebot crawler, and by extension to people finding the site in Google Search, and much different content to an arriving visitor.

Google doesn’t approve of the practice. “This is a violation of our webmaster guidelines. If the file that Googlebot sees is not identical to the file that a typical user sees, then you’re in a high-risk category,” said Ohye.

Google likes the first-click free approach to web content, just as it does for content showing up in news results. Someone arriving at a page from Google expecting to read a story, and slamming into a login screen instead, won’t care for the experience.

Worse, since Google sent the person to the page in question, the person will more likely blame Google for the poor experience, rather than the cagey webmaster hoping to pull a conversion this way.

They damage rankings and ultimately may result in banishment from the search engines. However, before panic sets in, let us guide you through the basics of penalties and filters.

A penalty is caused by significant violations of a search engine’s website guidelines, such as:

Penalties can be issued after a person reviews a website or after it has been crawled and processed by search engines. They result in a website being heavily held back in the rankings, or removed from the search engine’s index entirely. A penalty is called a ban when a website is completely removed from a search engine.

To remove a penalty, a site needs to first correct the problem that caused it, then contact the search engine and request a reinclusion. It is important to note penalties are not common for business websites, particularly in Google.

1. In Google, this is done through a Webmaster Tools account, which can be set up for free.
    
2. MSN has just created a Webmaster Tools system like Google’s.
    
3. Yahoo does not have a defined reinclusion process.
    
4. A filter is caused by passing a search engine’s threshold setting for one or more optimization/link building elements, such as:
    
   1. Too many keyword mentions on a page’s body content (over-optimization).
       
   2. Too much keyword blurring between a site’s pages.
       
   3. Too many links with the same anchor text.
       
   4. Too much keyword-rich internal linking (can cause 950 filter).
       
   5. Too much link building in a short period of time.
       
   6. Too much link building using the same anchor text.
       
   7. Link building in bad neighborhoods.

Filters are common. They are issued automatically after the site is crawled and processed by search engines and result in a site being held back in the rankings.

1. Filters can be keyword-based or site-based.
    
2. Filters can be mild (held back a few positions) or heavy (held back hundreds of positions).
    
3. Filters can have a time element (like the normal Google Sandbox process, where a site is initially held back many positions and over time gets held back less and less until eventually it ranks near its allinanchor rankings).

In general, to remove a filter a site needs to first correct the problem that caused it, then wait for the search engine (s) to crawl the site again and find the corrections. The next time the search engine updates its rankings with the corrected data, the filter will be lifted automatically.

1. In some cases, like the Google Sandbox, you can simply outwait a filter. Websites commonly spend anywhere between a few months and a year or so in the Google Sandbox. The time websites spend in the Sandbox has significantly decreased over the last few years.
    
2. In some cases, you can remove a filter on a site by doing things that search engines like (such as getting quality links to the site from other respected websites in the same sector) to outweigh the things about the site or optimization they don’t like.

Filters are common, especially in Google.

1. The Google Sandbox is technically a filter. Google closely examines new websites for over-optimization to try and minimize spammy websites filling its SERPS. As a result, newer websites often trip filters when they start an optimization campaign using traditional SEO (lots of keywords on the page, keyword-heavy titles and description, keyword heavy anchor text in incoming links). This pattern of new websites getting filtered and eventually getting released from the filter is called the Google Sandbox. Websites can speed up their release date from the Sandbox by getting quality internal links and not going overboard with on-site optimization.
    
2. New websites are not the only targets. Older websites can trip filters when they go overboard with over-optimization

Being aware of how search engines assess penalties and filters is essential to avoiding them. Do not try to cheat the system, over-optimize or trick search engines. They are savvy to these tactics and punish those who attempt to take advantage of them. What may help in the short term will only end up hurting in the long term.

Being competitive in the SERP’s is important, and sitting idly is not a good strategy. If you must build links or optimize your website try to stick to the guidelines and use common sense.

Comment

>>> What are your thoughts on paid links? Do you agree with Google’s hard line? Comment here…

Well as it turns out, not ALL search engines are as hard line on these issues as some claim to be. MSN Live specifically has now gone on the record that none of the above are necessarily taboo.

Over the holiday break, Jeremiah Andrick, product manager for MSN Live Webmaster Tools, stopped by our offices. We did a lengthy video interview where we chatted about all kinds of good searchy things. Highlights of our chat included some of the emerging differences between Live Search and engines like Google on subjects like cloaking and paid links.

Google doesn’t like cloaking because it can be used to spam and fool their crawler.

Google doesn’t like paid links because their algorithm places such a high emphasis on links as a quality indicator, and paid linking subverts their system’s integrity.

Google’s solution to these problems has basically been to disallow paid links and cloaking and punish the sites that disregard their rules by penalizing or even booting them from the index.

Much to my surprise, MSN isn’t exactly following Google in this regard. They obviously aren’t going to recommend anyone use cloaking and paid links, but they don’t discount either practice as forbidden.

MSN Live Search is becoming a lot more interesting. Initially, Microsoft’s revamped search product drew some criticism for being basically a Google spin off in appearance, with less than stellar results.

The folks in Redmond haven’t just been sitting around though. The quality of their results has shown some nice progress, but more impressive (and promising) has been the quantum leaps they’ve made in terms of communicating with the webmaster community.

Paid Links:

In our video interview, you might notice that Jeremiah says to avoid paid links. Apparently, the Live Search crew has reevaluated their stance on paid links. Live Search’s Nathan Buggia, in an email on the subject of paid linking said the following:

"Paid links are a gray area. Are they of value to the end user? Sometimes they are. Often they’re less valuable and less relevant than the organic links on a page. We reserve the right to treat them that way."

The operative phrase here is "a gray area". That’s not saying paid links are forbidden or evil or a bannable offense. Some paid links are crap, some… not so much so. Live Search is working on methods to evaluate and qualify links – paid or otherwise – before they ascribe authority to them.

When Jeremiah says in the video to avoid paid links, it would be more appropriate to rephrase that as ‘avoid bad (irrelevant/junk) linking’.

Indexing issues:

Webmasters have issues with Live Search indexing sites properly. I asked Jeremiah what he thought people were running into when Live Search wasn’t properly indexing them.

Jeremiah said most indexing issues fall into one of three categories:

1. Problems with design

2. Problems with content

3. Technological issues

According to Jeremiah, "It tends to be technological issues, or the content itself, that’s the problem," when people are having trouble being indexed correctly. "Most people are kind of hip to using better structure in their site," so design isn’t as often the culprit.

Jeremiah cited a mix between design and content as a specific problem. "Most publishing systems and CMSs a lot of people are using were designed 3 or 4 years ago," he said, before the now widespread recognition of the importance of crawlability and SEO in site design. As a result, some of these systems aren’t exactly the most efficient or effective in making a site’s content crawlable.

In terms of being totally optimized, Jeremiah said, "People do what they can, but I don’t think they always do enough or that they are not necessarily doing the right things".

But even under ideal circumstances, Jeremiah admits "you’re not going to get everybody and you’re not going to get everything out of everybody." As proof, Jeremiah offered a poignant example: "MSDN TechNet is a tier 1 site with eight or nine million documents in 42 different languages… I can tell you that it is not 100% indexed."

Their goal is likely common to all search engines… 100% indexability of everything. But that is just not a reality at this stage of search evolution.

Cloaking:

We’re all pretty clear on Google’s position on cloaking by now. It’s a pretty simple and straightforward ‘no’. Never. Under no circumstances. It’s evil, it’s bad, it’s spammy. Don’t do it.

While MSN Live doesn’t exactly endorse cloaking, they do seem to have a slightly softer stance on the issue. Jeremiah and I talked about companies who make heavy use of Flash and other non indexable graphics on their sites.

At one point, we talked about Nike, whose site is made completely out of Flash. Jeremiah was quite up front about it, saying, "They break some of our rules just to get to the point where they can get all of their content indexed. They do a bit of cloaking and things like that."

Google of course, still indexes Nike. I’m pretty sure if MSN knows they cloak, Google can probably figure it out as well. I don’t look for Nike to suffer any Google penalties – much less be thrown out of the index for it though. However, if you aren’t Nike but maybe just a smaller webmaster, I don’t know that you’d be afforded the same considerations from Google. I’d just about bet on it.

On one hand you have Google, who apparently selectively enforces their strict no-cloaking policy depending on who you are. On the other hand, you have MSN saying sure, it goes on and while we don’t encourage it, we aren’t necessarily going to boot you from the index for it. Interesting, no?

Conclusions:

At the end of the day, we didn’t hear too much in our video that we haven’t heard a few times before from every other search engine. Jeremiah told us in terms of SEO, "It’s always the basics. Keep it clean, let’s try to be natural, and as Live Search grows we’re going to try to provide better results for people when we’re able to do that algorithmically."

All pretty standard party line stuff. With the exception of their obvious difference in their stance on cloaking, and paid links, it could have been a Matt Cutts interview in many respects.

It’s not so much what was said, as it was who was saying it. Keep in mind, this is Microsoft. Typically, they don’t say diddly – or didn’t used to, at least.

I think this interview may be emblematic of an interesting movement going on at Microsoft. Traditionally, MS has been perceived as a closed empire – operating secretly behind closed doors, dispensing information strictly on a ‘need to know’ (or as subpoenaed) basis. Recently though, they seem to be trying to reach out a lot more.

Take for example, some of Jeremiah’s quotes: "The thing that Nathan and I are trying to do, and that is to bring more transparency… We want to endear ourselves to you and want you to want to work with us." From Microsoft? Really?

Then on GameSpy I see an email from their Xbox division publicly apologizing for holiday problems with the Xbox Live network. Could this be a kinder, gentler Microsoft emerging in 2008? And more importantly, will it lead to more marketshare for Live Search and more traffic for site owners?

Do you agree with Microsoft? Do you agree with Google? What are your experiences with paid links? Comment here…

Matt Cutts Provides More Cloaking Giggles

As the clich

On June 5, Google Webmaster Central blog announced its plans to restructure Webmaster Tools help and on the next day, it released new Webmaster guidelines like hidden text or hidden links, cloaking or sneaky redirects and automated queries to Google, to name a few.

Complaints and suggestions from webmasters and user for what they would expect in the latest Webmaster Tools Help Center include:

1. "It’s not as easy as it should be to find the information you’re looking for. You’d like Google to do a better job of surfacing the answers to the most common questions.
2. The browse structure doesn’t make it easy for users to find help, and sometimes search depends on users knowing exactly the right term to search for.
3. You like the idea of context-sensitive help – on-the-spot assistance (often shown in a tooltip that appears when you hover over an item) that doesn’t require you to click to a different Help page.
4. Right now, it’s not clear when new Help information – or new features – are added, and you’d like Google to look at calling these out.

You want Help to be more useful:

1. You’d like Google to look at adding videos and graphics
2. You’d like us to providing the kind of information that’s relevant to the average webmaster, who may not have a deep knowledge of SEO techniques. You’re looking for good and understandable answers to common questions.
3. You’d like us to expand the actual content, and do a much better job in explaining potential reasons why sites may have dropped the rankings."

Some of our personal feedback include:

1. If Google Webmaster Central can show the status of crawl of sitemap in the dashboard. Basically date when Google last crawled sitemap.xml, this will be handy for webmasters who have more than one site in Google Webmaster Tools.
2. Regular mail to be sent if some error occur in the xml file will be helpful as not all the Webmasters login to check the Webmaster Tools daily.
3. Also some way of integrating it with Google Analytics will really be helpful.

Comments

Andrew Gerhart first noticed the problem on Monday.  Gerhart collected screenshots to prove his point, and as he notes, Yahoo Autos is clearly in violation of Yahoo’s own policy regarding “search content quality.”

“Normally, I could care less if someone is spamming or using shady techniques,” Gerhart writes.  “BUT, I don’t feel that this applies to the search engines.  They are the ones placing the ‘quality guidelines’, penalizing websites, banning websites, and trying to enforce the rules that they’ve made up.  And they penalize and ban websites for less than what Yahoo! is doing above.”

Gerhart continues by asking, “How is that fair?  With one hand you’re going to ban a site and in effect reduce their revenue and with your other hand you employ the same strategies (or worse)?  Come on now.”

I should pause here to say that every fault Gerhart found with Yahoo Autos is a real one – a discussion in the WebmasterWorld Forum corroborates everything, and Search Engine Roundtable adds, “Tim Mayer of Yahoo! has confirmed on [the] May 22nd edition of The Daily Search Cast that Yahoo Autos has changed the page since this has been reported.”

But there’s one other interesting angle to this story, and it relates to Gerhart’s position as Director of SEO for Primedia Automotive.  As WebProNews reported, Primedia started a “Ride of the Month” contest last week, and that contest will involve a game for Microsoft’s Xbox 360.  So it seems that someone with ties to Microsoft is pointing out problems with Yahoo.

Interesting, yes, but again, it doesn’t in any way counterbalance the fact that Yahoo Autos was cloaking.