Duplicate Content Owners Catch a New Break from Google

Google Now Supporting rel="canonical" Across Domains

By Chris Crum · December 29, 2009 · 60 Comments

Get the WebProNews Newsletter:

Google announced that it now offering cross-domain support of the rel="canonical" link element. If you are unfamiliar with this link element, Google’s Matt Cutts discussed it with us here. Basically, it’s a way to avoid duplicate content issues, but until now, you couldn’t use it across domains.

"For some sites, there are legitimate reasons to [have] duplicate content across different websites — for instance, to migrate to a new domain name using a web server that cannot create server-side redirects," says John Mueller, Webmaster Trends Analyst with Google Zürich.

Do you have legitimate reasons for having duplicate content? Tell us about them.

"There are situations where it’s not easily possible to set up redirects," he says. "This could be the case when you need to move your website from a server that does not feature server-side redirects. In a situation like this, you can use the rel=’canonical’ link element across domains to specify the exact URL of whichever domain is preferred for indexing. While the rel=’canonical’ link element is seen as a hint and not an absolute directive, we do try to follow it where possible."

Mueller gives the following ways of handling cross-domain content duplication:

- Choose your preferred domain
- Reduce in-site duplication
- Enable crawling and use 301 (permanent) redirects where possible
- Use the cross-domain rel="canonical" link element

Barry Schwartz at Search Engine Roundtable gives three reasons why the addition of cross-domain support for the rel="canonical" link element is really important:

1. Some hosts don’t allow webmasters to deploy 301 redirects
2. Some site owners aren’t technical enough to implement a 301 redirect
3. In some cases, webmasters do not want to redirect users but rather only search engines (i.e. pagination, weird filtering, tracking parameters added to URLs, etc).

To use the link element, pages don’t have to be identical, but they should be similar. According to Google, slight differences are fine. You should not point point rel="canonical" to the home page of the preferred site. Google says this can result in problems, and that a mapping from an old URL to a new URL for each URL on the old site is the best way to go.

You should not use a nonindex robots meta tag on pages with a rel="canonical" link element because those pages would not be equivalent with regards to indexing, Google says. One would be allowed while the other would be blocked. Google also says it’s important that these pages aren’t disallowed from crawling through a robots.txt file, because search engine crawlers won’t be able to discover the rel="canonical" link element.

Will you find cross-domain support of the rel=”canonical” link element useful? Comment here.

Related Articles:

> Duplicate Content on Google, Bing & Yahoo

> Google Busts the Duplicate Content Myth

> Internationalizing Without Duplicate Content Worries

Top Rated White Papers and Resources

There are 60 Comments. Add Yours.

Like (0) Dislike (0)
agency collection
December 18, 2009 at 1:58 pm

Great information thanks for the information. Google is making a lot of changes and its hard to keep up on things. You guys are always on top of things.

Reply
Like (0) Dislike (0)
shady massalha
December 20, 2009 at 7:32 am

I built a new website called http://arab-baby.com and 2 weeks alter decided to call it http://muslim-names.us .. I used Redirect 301 and all the necessary steps according to google… but i still see arab-baby.com results in Google search.

Reply
Like (0) Dislike (0)
Denis Hebert
December 20, 2009 at 8:16 am

We are a florist and we use a database that FTD give us for promotion. The description as well. We need to use that database and it’s the same for 23000 florist across North America. We are ranked number 24 and we are base in Canada. Does it will affect us ? Denis

Reply
Like (0) Dislike (0)
Dan
December 20, 2009 at 8:59 am

We have a church that is transitioning brand, even identity. We are Victory Christian Fellowship at http://getvictory.net . We are transitioning to 422CHURCH.COM at http://422church.com which directs to http://getvictory.net . We are advertising featuring the 422CHURCH.COM brand. That change of identity also tells the prospective church visitor where to find out more about us.

Reply
Like (0) Dislike (0)
Jiffy oil Change
December 20, 2009 at 9:04 am

Google is making some awesome changes some people like to have there auto blogs with plr articles all over them

Reply
Like (0) Dislike (0)
Dan
December 20, 2009 at 9:34 am

Anyone who thinks most people know what the heck this article is talking about is smoking crack. I’ve been building websites for 8 years and don’t have the slightest idea what you are talking about.

Reply
- Like (0) Dislike (0)
  Doc
  December 20, 2009 at 11:48 am
  
  Dan-
  If you’ve been building websites for 8 years, and haven’t the slightest idea what rel=”canonical” is, it would seem you’re not trying very hard to stay abreast of developments in your field.
  
  Ever wonder what ELSE you may have missed?
  
  Reply
Like (0) Dislike (0)
Elaine McCool
December 20, 2009 at 9:36 am

This year I’ve been in the process of moving some pages to a new domain since I decided I had too many categories of products on the original website. The older site is hosted on Yahoo and they do not allow 301 redirects. I used a “noindex” in the robots metatag and hoped for the best, but I think that this issue is still hurting my newer website.

Reply
Like (0) Dislike (0)
newbie
December 20, 2009 at 9:48 am

Great article, great information…. Thanks sir for this article

Reply
Like (0) Dislike (0)
Mark
December 20, 2009 at 9:50 am

Awesome Information Many Thanks

Have a awesome xmas and new year

Reply
Like (0) Dislike (0)
medlaw
December 20, 2009 at 10:30 am

Thanks for the info. Very handy when moving content from one sister website to another. There can be reasons to leave the content up and active on the old site where it was first shown but now wish for Google to see that page as duplicate but list the content on the new site as the original.

Reply
Like (0) Dislike (0)
Guest
December 20, 2009 at 10:34 am

Great information thanks you

Reply
Like (0) Dislike (0)
Kok Siong Chen
December 20, 2009 at 10:38 am

May i know what is canonical url? I still not very understand about this. I’m a new blogger. Hope to learn from you all. Thanks for information!

Reply
Like (0) Dislike (0)
Steve
December 20, 2009 at 10:53 am

Great article any idea what the point of sidewiki is. Is this all one big idea mix together. I’m not sure why they even have sidewiki i see it now but don’t really get it.

Reply
Like (0) Dislike (0)
David T
December 20, 2009 at 10:54 am

Your article came just in time. I’m presently rewriting several of my web sites, which were built up from daily notes. Now I am reorganizing the information with the reader in mind. I’m also moving up from static html to Drupal, Joomla, and Mediawiki. All of my content will necessarily be duplicated, but reorganized and this had me worrying how much time I would lose in the reindexing transition.

I will be sure to use the canonical link tags.

Thanks for the article.

Reply
Like (0) Dislike (0)
2012 Consciousness Shift Resources
December 20, 2009 at 11:04 am

I use excerpts from documents on my web pages in order to promote the various ideals on other websites for my readers to consider. I realize that Google Bot must be penalizing the site for this, but hopefully just does not give weight to these excerpts, but rather just overlooks the promotional excerpts.

Reply
Like (0) Dislike (0)
Technology News
December 20, 2009 at 11:18 am

Great information. Google is trying to overcome of the things that webmasters seen as shortcomings.

Reply
Like (0) Dislike (0)
nick
December 20, 2009 at 11:19 am

Hi,

My question is our most of the website pages generated through RSS FEEDS. We index RSS feeds and generate the news. So in my case what you advise.

Thanks

Reply
Like (0) Dislike (0)
eBay Dude
December 20, 2009 at 11:29 am

On a few of my sites i am pretty sure i am taking a hit for duplicate content and i hope it is just on those pages.

I have a few eBay reseller sites and i have the same article on each site on what to do in a eBay dispute.

http://www.viewcoloradoonline.com/ebay-dispute-and-what-to-do.html

http://www.harleysite.biz/ebay-dispute-and-what-to-do.html

I hope this just hurts those pages and not the whole site. The information however is important.

Reply
Like (0) Dislike (0)
Guest
December 20, 2009 at 11:43 am

I run a jewelry website,
they design jewelry in stainless steel and gold.

Now for example, I have to create 12 pages for 12 rings that
are made of the same material but different design and model number.

So the pages will have, same title, same keywords, same design structure, same page load speed, the only difference will be:
Obviously the URL and the SKU model number!

You think I will be penalized for it?

You all have a nice day

Reply
- Like (0) Dislike (0)
 Lynne Gordon
 December 20, 2009 at 3:26 pm
 
 I have been spending a lot of time at the GOOGLE Webmaster Tools page and they suggest there that in your situation, you show the difference in your 12 pages in your page . Title 1 – 14 carat gold spring flowers ring – 830062 Title 2 – 14 carat gold spring flowers ring – 830061 etc. I hope I understood you correctly and I hope this information helps. Lynne
 
 Reply
 - Like (0) Dislike (0)
 Guest
 December 23, 2009 at 3:14 pm
 
 Yes you understood correctly.
 
 So you are seing to add some kind of progressive numbers, perhaps the actual sku or model # of the actual item right into the title?
 
 Matter of fact I did try as much as I could to vary the titles, and description,
 I hope did help.
 
 tnx for your help
 
 Reply
Like (0) Dislike (0)
stefan
December 20, 2009 at 11:50 am

“You should not point point rel=”canonical” to the home page of the preferred site. Google says this can result in problems, and that a mapping from an old URL to a new URL for each URL on the old site is the best way to go.”

What can I do if i have duplicate content because my website is available with and without www?
In this case I have to place the rel=”canonical” to the index.html page but it is the preferred and the not prefered in one and the same file??????
My host does not support 301 redirect.
Could somebody help me?

Reply
- Like (0) Dislike (0)
  Lynne Gordon
  December 20, 2009 at 2:57 pm
  
  If I understand your dilemma, you should be able to go to GOOGLE Webmaster Tools and enter your domain with and without the www and it will be all the same.
  
  Lynne
  
  Reply
Like (0) Dislike (0)
Mona
December 20, 2009 at 12:30 pm

I started with a free web-publishing site to build my website and didn’t decide on a working domain name until later. The web-publishing site offers the templates to make publishing easy and when I registered my domain name, the result was the original name that was free and the new name that I registered. Two sites. The one I manipulate and change is still the old free site, but all changes appear simulaneously on the domain named site. They are intricately woven and anyone can access either for duplicate content.

Reply
Like (0) Dislike (0)
Brian D. Hawkins
December 20, 2009 at 1:25 pm

I believe the latest WordPress update actually includes the rel=”canonical” option in it’s dashboard. That was very quick.

Reply
Like (0) Dislike (0)
WPN Fan
December 20, 2009 at 2:08 pm

That’s nice of google, but the rel=”" is already being substituted across many other applications for other purposes, so this would essentially kill those operations to use the google tag in the case discussed in this article. Can’t they come up with another unused tag to add to a link?

Reply
Like (0) Dislike (0)
Thom
December 20, 2009 at 2:45 pm

Great info, thanks for the information.

Reply
Like (0) Dislike (0)
desarrollo web
December 20, 2009 at 2:53 pm

Thanks for sharing.

I had this clean practice, only for common sense.

But now I know the resons.

Thanks.

Reply
Like (0) Dislike (0)
Lynne Gordon
December 20, 2009 at 2:54 pm

Originally I started my WordPress blog with the only domain I owned. I used this blog to chart my weekly progress for the past year on the Oxygen Therapy Program.

Now that I own the domain http://www.Oxygentherapyprogram.com and have a business set up there, I want to bring all of my blog posts that are My Self-Administered Oxygen Therapy Program – Week 1 through 52 as separate blog posts to my new domain.

I also want to do this on http://www.OxygenTherapyProgram.com/blog because it serves as additional ‘new’ content in the sight of robots.

I still do not understand the rel=canaconical thingy…but I’ll figure it out.

Anyone with any additional information, it would be greatly appreciated.

Lynne Gordon

Reply
Like (0) Dislike (0)
Guest
December 20, 2009 at 3:57 pm

Great . I would like to know more about your writing – can you cantact me
http://www.copynprofit.nbd8.com or http://www.usaonlineoutlet.ws

Reply
Like (0) Dislike (0)
Australian Bookshop
December 20, 2009 at 5:20 pm

This is great news for us as we are looking to move server and were concerned about duplicate content.

Reply
- Like (0) Dislike (0)
  Kerry Coates
  December 20, 2009 at 10:26 pm
  
  When you switch servers, you should be able to take your URL with you so there is no reason for a redirect. The new server can have your old site up and running with them within about a day using the same URL.
  
  Reply
Like (0) Dislike (0)
gugeluv
December 20, 2009 at 6:07 pm

I already have my blog get penalized by Google.
but I also see my friend’s blog that are similar with mine ranks high in Google. Both are dupe.
he’s a lucky one I guess

Reply
Like (0) Dislike (0)
china tours
December 20, 2009 at 10:02 pm

We are thinking of reshaping our sites, now it is a good solution for it. Good for Google.

Reply
Like (0) Dislike (0)
Pittsburgh Online Marketing
December 20, 2009 at 10:02 pm

Lets make it more complicated to SIMPLIFY THINGS. Sure that’s the answer. This will help however on some of our clients sites with rogue hosting companies but the simplest way is to just transfer them and their pagerank through some nifty dance maneuvers to a reliable host with better support and solid products. Cheap hosting is not cheap. It costs time and effort and dollars for the client. We try to educate our customers on this fact. Very Nice article BTW. If the new cross domain canonical is not abused (yeah right) and works as advertised (uh huh) then its great.

Reply
Like (0) Dislike (0)
Kerry Coates
December 20, 2009 at 10:38 pm

I have a website with a LOT of products on it. There are a couple of products that deserve a website all their own so I registered a new URL for each and use that URL for the product I want to feature. Most of the language on the product websites is the same as the page they are on within the master website. So is that “duplicate” content?

Reply
Like (0) Dislike (0)
Augustine
December 20, 2009 at 11:33 pm

I have a couple of questions on duplicate content.

1,I have websites for my brick & mortar services business in two states, A & B. A is the older business; B is the newer business. I decided to expand my business into state B. It’s the exact same type of business. I created a separate website for B so it would be local for residents. However, so much of my main content pages are the same. Why not? It’s the same type of business, just a different state? Is this duplicate content in the eyes of the search engines?

2. I have published articles on internet directories on subjects related to my business. I’ve decided to have these articles added to my website. Again, why not? I wrote them. They’re my brain child. But I worry. Is this duplicate content in the eyes of the search engines?

Assuming (1) and (2) are duplicate content issues, will I be penalized, and if so, how? Also, how could I possibly resolve these issues? It seems unfair to me if these situations are duplicate content. I’m not trying to spam. I’m not trying to deceive.

I’d appreciate your feedback.

Reply
- Like (0) Dislike (0)
  Dylan Whitman
  December 22, 2009 at 6:37 pm
  
  AUGUSTINE….
  
  Why not just get an article spinner, or a part time copywriter at somewhere like SupportResport.com and just have the articles rewritten?
  
  The problem here is that people confuse the word “penalty.”
  
  Perhaps your site won’t get “penalized” for having the duplicate content, however, it also won’t excel in the rankings due to it.
  
  Reply
Like (0) Dislike (0)
Down Comforter Sets
December 21, 2009 at 3:01 am

While it does seem that most search engines apply a filter of a sort, it appears that only Yahoo and MSN have the technology to analyze where the content originated. And because their filters are working like they should be, you aren

Reply
Like (0) Dislike (0)
Margi
December 21, 2009 at 6:54 am

I think many would not sleep after seeing that!

Reply
Like (0) Dislike (0)
uk vouchers
December 21, 2009 at 7:43 am

Nice article but old news

Reply
Like (0) Dislike (0)
Guest
December 21, 2009 at 10:13 am

I have multiple sites as we convert just our splash page into languages such as Simplified Chinese, Spanish, French etc. All these other sites then have a special page announcing that as the language of the international hotel industry is English, the rest of the site from here on in is in English.

This allows us to use names which are also atranslation of HotelDesigns, such as amenagement-hotels for the French address, giving us results in local search engines that we would otherwise not get.

As the industry we service is, like aviation, a fully international industry it is essential we reach designers, specifiers and hoteliers across the globe.

Our strategy has been successful in that over half our audience is outside our home market

Reply
Like (0) Dislike (0)
Chandra
December 21, 2009 at 10:58 am

Hi

I’ve creted an ecommerce site for religious product and still not get google ad sense. I’ve followed all criteria suggested by google as well as not able to find by product category or by product name in google listing. If I use by name of company it is coming on the listing but not on product name basis. I’ve also followed the image name criteria but still it is very far in listing, others have some matching are comin above.

Might be this is happining due to repeating keyword.

Reply
Like (0) Dislike (0)
Restaurant Consultants Chicago
December 21, 2009 at 4:27 pm

Yeh, we need to have this updated quickly, due to relevant content in the restaurant industry.

Reply
Like (0) Dislike (0)
Get on Google
December 21, 2009 at 5:31 pm

This is great stuff! I was looking for a way to redirect some of my old pages, but I did not want to use a 301 redirect. This method is really what I want, thanks for posting it. Keep it coming

Reply
Like (0) Dislike (0)
Colonic Irrigation
December 21, 2009 at 9:21 pm

Nice move.

Reply
Like (0) Dislike (0)
Kratom Capsules
December 21, 2009 at 9:23 pm

I’ve had people copy content and they still rank in google.

Reply
Like (0) Dislike (0)
Guest
December 22, 2009 at 7:43 am

I have website based on content from rss and this is very good source of knowledge, portal with links to full articles. Many people using websites like this because this is good set of many news, I have many returned visitors however Google do not rate good these websites. This is not fair.

Reply
- Like (0) Dislike (0)
  Guest
  December 29, 2009 at 11:12 am
  
  I have many clients setup on Blogspot.Com. The reason is they compete with each other… also with blogspot.com the content is updated via email and people with limited skills can edit their entries. From here I read their postings via RSS into a WordPress Website where similar minded or groups can read a broad spectrum of information on one site … will this apply to such domains or web sites …
  
  Reply
Like (0) Dislike (0)
freebiejeebies
December 22, 2009 at 4:19 pm

Come on Google! If people aren’t ‘technical’ enough to implement a simple 301 redirect how on Earth are they going to understand what a rel=canonical is, let alone how to implement it?

Reply
- Like (0) Dislike (0)
  Rollins
  December 24, 2009 at 3:25 am
  
  One really wonders. All the same, I feel it’s a really good development, at least for the more tech saavy people.
  
  Reply
Like (0) Dislike (0)
handa fanda
December 22, 2009 at 4:23 pm

do spamers also get break?

Reply
Like (0) Dislike (0)
Dylan Whitman
December 22, 2009 at 5:16 pm

Shady… did you go into Google webmaster tools, verify the site, and put in a “Change of Address?”

Reply
Like (0) Dislike (0)
RussianBrides
December 23, 2009 at 9:21 am

We have a few Websites with varied content, but from time to time, we syndicate a story to all of them. Will this help us there or does Google treat this scenario differently?

Reply
Like (0) Dislike (0)
SEO training
December 30, 2009 at 11:19 am

I think this is good news for only merchants and catalogers. However, it doesn’t really help from an SEO perspective. As someone focused on search engine optimization, I use techniques like article marketing to build backlinks.

Because of Google’s duplicate content rules, I have to spin articles and work really hard at making them unique. I suppose from Google’s perspective that’s the point, but it sure is a lot of work!

Reply
Like (0) Dislike (0)
Mexico dentist
December 31, 2009 at 3:57 pm

I have faced this issue, as I had to move from one host to another. and use a new domain. I appreciate this info.

Reply
Like (0) Dislike (0)
No Smoking
January 6, 2010 at 12:47 pm

Ok so maybe we should change (they are very good in other respects) , but this helps us a lot.

Reply