"The big shocker is that MSN is so bad at following 301 redirects that they now send the site 0 traffic. The old site is listed URL only, while the new site is not listed," he writes.

The conclusions of Wall’s case study:

1.    Move in the off-season
2.    Increase ad spend
3.    Block indexing of low-information pages
4.    Bring profitable pages forward, emphasizing them with position and internal link structure.
5.    Move it page by page for speed testing.
6.    Issue press releases, submit to directories, build links.

Wall goes into way more detail at his site, complete with stats.

Eytan Seidman, Lead Program Manager, Live Search, Microsoft said you should be concerned with duplicate content on your site because it fragments your pages in some way. In order to avoid duplicate content, he said to keep session parameters simple.

 For local sites he recommended having unique content on them and if you don’t have different content, you most likely don’t need multiple pages.

 To avoid people copying your content, he said to tell people that are going to use your content to attribute.

Peter Linsley, Senior Product Manager for Search,  at Ask.com gave a standard definition of duplicate content saying it was the same content on multiple urls. The reason it is bad for search engines is because users do not want to see the same content result.

As for avoiding duplicate content you should use copyright or creative commons notice. Also make the content unique and difficult to be taken out of context.

Amit Kumar, Senior Engineering Manager, Yahoo Search, said that Yahoo tries to crawl links from duplicate, less; so fewer visits from the crawler could mean that the content may be considered duplicated.

He added if you get content from somewhere else you should attribute.

Vanessa Fox, Google, Product Manager weighed in on duplicate content saying they want to show unique information so they want to avoid duplicated information.

If you have pages that are fairly similar, you may want to consider combining the two. In some cases, pages that are similar just need to be distinguished from one another a bit more.

On client site redirects, she said a 301 is considered a client side redirect because the client is actually taken from one page to another and the crawler can see the message.

On search engines reaching out to Word Press and blog software types to avoid duplicate content, she said there is a lot they can do as far as working with the CMS software people to come up with better solutions so end users won’t have to worry about duplicate content.

When you remove a webpage, or move it to a new domain, it’s nice for your visitors if you notify them and/or automatically redirect them to the new location. A quick Google search brings up lots of advice on 301 and 302 redirects.

While all of this technical code Web jargon mumbo jumbo may seem to have little to do with the polished rhetoric of public relations, it comes down, at it’s heart, to image. An image takes a lot of work to maintain, and it’s attention to detail that makes the difference.

An event planner, a good one, knows that before a big dinner, somebody needs to check the salt and pepper shakers to make sure not only that the lids are on tight, but also that the spices therein haven’t swelled too big for the holes overnight.

Maybe I’m cranky on Monday morning. Maybe I’m making a mountain out of a molehill.

But this is where AOL’s media relations page was. Now, it’s a sea of white with an island of Not Found in the corner. There’s no notice of where the page has gone, no redirection to Time Warner’s press release page, where it seems AOL announcements will be handled in the future.

It’s a small annoyance, hardly more than an itch on your tongue perhaps. But it does nothing for the image of an already turbulent company coming off an extraordinarily bad year.

Last year, when AOL’s 2006 PR nightmare reached its peak, when it appeared that the PR department, not because of their own faults but because of the faults of the company itself, was snapping under blunder after blunder, I poured salt in the wound.

The last quarter of the 2006 saw company shakeups and corporate communication heads roll. After the Christmas holiday, the aforementioned PR webpage just fell apart. Images stopped working, and appeared as those unsightly x’s.

New announcements, when they finally resumed after a week or two, showed up below an announcement that was dated December 22nd. That December press release wasn’t about anything super important, but it stayed put as datelines January 17, 22, and 31 popped up beneath it.

And I thought to myself then: has AOL really just let their media page go un-maintained? That’s like showing up to a job interview with a five o’clock shadow and a lit cigarette.

It’s the little things like a sharp pair of cufflinks, or shined shoes, or creased trousers, that convey a message – especially in times of trouble – that a person has it all together. A person in PR, even when his office is on fire, shows up outside looking sharp.

So what are we to think after AOL loses $140 billion in value, bleeds subscribers, has to overhaul their business model, has to tap dance all year around one crisis after another, if their public face appears out of the rubble dirty and broken?

We think that if AOL really has its act together, it should look like it has its act together by sweating the small stuff first – especially its appearance on the Web where its bread is buttered.     

Just my two cents, for what it’s worth.

Not all webmasters are aware that search engines view each URL as a different page, even it has the same content or is different from the other URL by ending with a slash or if it preceeded by a www or not. To serve only one version of the page to get the deserved traffic, you’ll need to remember a couple of things about fixing URL issues.

www and non-www

The simplest matter is having both versions of a domain, the one with www and the one without (smart developers even call this URL canonicalization issue, but I believe – or hope – there has to be an easier term for this). The search engines view them as two separate sites with identical content and regard links pointing to them accordingly. Though they each get their own share of traffic, you can get more traffic by having just one version, because the links would be pointing to one site, thus boosting its search engine rankings.

To redirect a site from the non-www version to the www version, you can insert the following code in your .htaccess file:

RewriteEngine On
RewriteCond %{HTTP_HOST} ^yoursite\.com [nc]
RewriteRule (.*) http://www.yoursite.com/$1 [R=301,L]


• Replace yoursite.com with your domain
• copy the code to your .htaccess file to your root (main) folder on your server
• refresh your page (press the “Refresh” button, F5, Ctrl+R or anything that works for you)

It’ll only work for Unix platforms with Apache and mod_rewrite enabled. If it doesn’t work after refreshing the page, make sure you do have Unix hosting and contact your support to check if they have mod_rewrite enabled. If they don’t, I’d suggest switching hosting.

Read more about using 301 (permanent) redirects with .htaccess.

URLs with and without slashes

Though it is technically easy to understand that www and non-www sites are different, how about understanding that pages with and without slashes in the end of the URL are different, too, according to the search engines. That’s why you only need to have one version of those.

If you have a WordPress blog, then a simple Permalink Redirect plugin will help. I have recently installed it on my site and it works like a charm.

For other websites, you’ll need to check your Apache version with your hosting and go to the according section of the Apache manual. Here are the links:

• Apache 1.3
• Apache 2.0
• Apache 2.2

To me, the code samples in the guide look the same. I also didn’t manage to make them work – probably due to conflicts with WordPress, or something. You can discuss the topic or ask questions about the trailing slash issue at Cre8asite Forums.

Having unique, accessible pages

Suppose you have pages with the same content on your article and you want to only have one page, but don’t want to lose the link weight or decrease the experience on your website. Then you need to 301 redirect (probably a PHP one) one page to another.

A 301 redirect (as well as .htaccess redirect) can be very helpful in solving duplicate content and broken links (if you have links pointing to a page that doesn’t exist) and is mostly the only cure to the problem.

The feeds

If you have a blog or just serve RSS feeds to your visitors, you should consider blocking them from the robots through the robots.txt file, because it’ll help filter out duplicate content from their indices (feeds have the same content as your main site, remember?).

A good start could be inserting the following code in the robots.txt file:

User-agent: *
Disallow: /feed/
Disallow: /feed/atom/
Disallow: /feed/rss/
Disallow: /wp-
Disallow: /#comment


Alternatively, you can try using the following code to block spiders from indexing feeds for all pages:

Disallow: /*/feed/$
Disallow: /*/feed/rss/$
Disallow: /*/trackback/$


Read more about duplicate content and feeds at The Van Blog – be sure to check the comments, too.

Note: the above code seems to work for GoogleBot only (or for other bots, supporting the * wildcard.

Let me know if you have a better solution for blocking duplicate content with robots.txt.

How to avoid duplicate content

When developing your website, you need to remember a couple of things to keep your content more unique (both to the people and the search engines):

• it is better to have less pages, but of more quality, because it’ll interest more people, attract links to the same place and drive more visitors from the search engines
• have clean URLs: no dynamic parameters, less folders (it doesn’t matter much, whether you have words in your URLs or not, though, but it helps – a tiny, weeny bitty bit)
• always write unique content, don’t copy it from others: it doesn’t give you credibility, won’t drive visitors to your site, and without a credit link it is also a theft of copyrighted material
• when you link to pages, pick one URL for a page, and use it to link to it wherever you can (for example, always link to domain.com/, instead of also linking to domain.com, domain.com/index.php)

Read more on how you can get rid of duplicate content on the Google’s Official Blog.

Keeping your site content unique, you will get more visitors from the search engines, because all your incoming link weight will be associated with a single page, thus making it more visible. Partly, correct indexing will help get your real content pages out of the supplemental index. That being said, don’t be obsessed with supplemental results. Focus on your customers.

Conclusion

By knowing how you can keep your site unique, you can easily increase your chances of getting more search engine visitors. Granted, this process is fairly easy and can be done within a day with the right education. So, you are still not looking at your site? Here’s what you should look at:

• whether your site has feeds
• whether your site has pages with same content, but different URLs
• whether accessing your non-www version gets you to the www version (or visa versa), or not – if not, check the .htaccess code above
• your .htaccess file (in the root folder)
• your robots.txt file (in the root folder as well)

Enjoy, and may Google be with you.

Comments

Several top level domains have come along in recent years to complement the .com, .net, and .org that have been available to the public. Dot com started life as the domain for commercial organizations, and over the years has become the must-have domain for one’s business name.

Topix has been Topix.net for quite some time. Skrenta wants the dot com credibility, not to mention making it easier for people to find the news aggregator site. When it comes to the Web, people tend to think in terms of dot com.

Skrenta told the Wall Street Journal about the concerns he has with the change. Sites that make name changes without accounting for the ways search engines look for them could make a website invisible to searchers.

Redirection, particularly the issues with using 302 redirects, has been an issue of contention between webmasters and Google in particular. As the dominant search engine, Google can determine a website’s success. Skrenta fears what a mishap with redirecting people from Topix.net to Topix.com could do for the site:

Even if traffic to Topix, which gets about 10 million visitors a month, dropped just 10%, that would essentially be a 10% loss in ad revenue, Mr. Skrenta says. "Because of this little mechanical issue, it could be a catastrophe for us," he says.

Skrenta also experienced the palpable frustration shared by many site publishers when dealing with Google:

Further frustrating him is that Google’s response to Topix’s plea for help was an email recommending that, if the switchover were to go badly, the company should post a message on an online user-support forum; a Google engineer might come along to help out. "This can’t be the process," Mr. Skrenta says. "You’re cast into this amusing, Kafkaesque world to run your business."

Amusing, maybe, but no one at Topix will be laughing if Topix vanishes from the Google realm. In the same article, Google’s Matt Cutts said the post-and-wait support strategy ‘is more reliable than it sounds.’

Fortunately, the issue of pleasing the search engines with a redirect can be addressed. Search engines like 301 redirects, which unlike the 302 redirect tells the search engine crawler that the redirect is permanent.

WebProNews Blog Talk contributor Eric Enge discussed 301 redirects in a recent article. In Google, a site Enge moved managed to be recognized properly within two to three weeks.

Skrenta probably will choke on a Pop Tart at the prospect of two to three weeks of uneven traffic. But we see no reason to think Matt isn’t giving Topix good advice, so perhaps Skrenta should just drop in on the relevant Google Group and send out the SOS.

An Example of a Domain Waiting to Fall:

In spite of making 5 figures a month, one of my unappealingly named domain names has cost itself significant credibility and links. Since it is an invisible cost it is hard to estimate how much it has cost, but I have a perfect example of showing how much it hurts.

One time I tried sponsoring an event and they said sure. They got my credit card details and then asked for the domain name. Once they saw the domain name they said sorry they couldn’t accept my money. And this is a reputable content site in a field that is easy to like, but on a junky sounding domain name. Ouch.

Being Honest With Yourself:

If you have a quality legitimate content site, and people who typically sell reviews or links are unwilling to take your money you know it is time for a change.

Other Signs of Trust:

If people who need sponsorship are unwilling to take my money imagine how much a bad domain name suppresses my click-through rate in the search results, and how many other links it cost me. If and when relevancy moves toward an attention based metric I am screwed if my house is built on a cheesy domain name that looks spammy.

Domain Buyer’s Pricing Tips:

You can sometimes capture emerging field names cheaply, but you are probably going to have to spend at least a few grand to get a good name if you are in an established field.

If you are new to domaining, and can’t afford a great .com there are still a lot of great .org and .net names out there available for $1,000 to $10,000.

Is a 301 Redirect Risky?

I will eventually 301 redirect my high ranking ugly domain name to the undeveloped MyKeywords.org domain that I just spent $8,000 buying. Short term I will probably see some drop in traffic, but long-term it is going to be far less risky to create an leverage what looks like a real resource and a real brand.

If you do something like this, make sure you have enough other passive income streams to afford the risk, and keep developing links to the new domain name. Keep that old trusted domain registered and redirecting for many years into the future. If you lose it you will probably lose a large portion of your link authority.

There is No Value in Being Anonymous:

You can spend the money your site is making as a passive income source, but if you believe in what you are doing, and have money in the bank, there is no reason to use a bad domain name. It is like writing nameless.

You can get a decent design for few thousand dollars, or a design modification for a few hundred. You can get good content for $50 a page or less. You can move a CMS for a low price too. The cost of moving and re-branding a non-brand site are negligible compared to the potential upside.

If you ever decide to sell, you are not going to get much out of my-ugly-do-mainz.biz, but if you create a real brand on an undeveloped strong domain name you will be able to sell it for a premium far in excess of the domain name cost.

Comments

Furl

Bookmark WebProNews: