rel=canonical tag and Affiliate links
So here is my issue, I have a big web site with lots of pages indexed and we run an aggressive affiliate program in fact it was the affiliate program causing dupe content issues. We can’t redirect the ?afid=123 so we decided to test out the rel canonical.
http://www.mysite.com/landingpage
But, on a quite few pages, we linked to the same page using a affiliate ID:
http://www.mysite.com/landingpage?afid=123
So I know I have canonical issues between the real URLs and my affiliate ID Url.
- development team have a few issues 301ing the affiliate ID
- we can stop linking to the affiliate ID url’s unless we close the program.
So I opted for the rel=canonical tag and now I have a few months worth of data to look at.
Yahoo is the worst with over 75,000 affiliate ID indexed and ranking and growing !
Bing is just as bad 65,000 affiliate ID indexed and ranking
But Google bless them remove 45,000 indexed and ranking affiliate ID urls and replaced a high % of them with the correct Url. It seemed that the rel=canonical tag was working and working well, but then a side effect kicked in and it was awful we lost some rankings all together then started what seems to be a pattern..
Google keeps finding random affiliate ID urls and adds them to index
We lose of rankings on the targeted keywords on that original page
Google returns both pages when unique content off the master page is searched for.
Google then removes affiliate ID url cache copy but not the listing so you get a uncrawled listing
Google bounces the rankings on the master page around
Google then removes the affiliate ID url all together and rankings stabilise..
it’s been driving me crazy I thought I was the only one till Rob Kerry tweeted me this : Doesn’t work well on Google either. URLs used in inbound links can temp hijack the main URL specified in the canonical tag.
so back to the drawing board !
35 Comments
Steven Morgan
Disapointing implementation by the major engines considering it was heralded as a step in the right direction when launched.
Andre
I have implemented the tag to take care of essentially the same problem. We have unique tracking URLs that we use for PPC traffic that gets rel=canonical back to the main seo’d page. Some PPC pages are still being indexed but I havn’t noticed any dropped rankings as a result. Look forward to a solution to this problem, I was hoping it was as easy as canonical but looks like we have our doubts.
Shark SEO - http://sharkseo.com
Jesus, this is the exact kind of situation the canonical tag was built for – why announce support for it if they don’t use it (or use it badly)?
Darrell Freeman - http://www.darrellfreeman.co.uk
I’ve been seeing the same problem too! Yahoo and Bing really need to get this sorted asap. Looks like something they have implemented without trialling properly.
Rob Hughes - http://www.falconpunch.co.uk
Hey Dave –
What about an if statement based on the URL that if the URL contains “afid=” a robots noindex meta-tag is introduced into the head of the page?
Not looked into this at all, it’s just off the top of my head, I’ve also not really thought it through so it could mean I’ve forgotten something that would end in penalisation.
Might help anyway…
Aaranged - http://www.seoskeptic.com/
Perhaps impossible to determine, but is it possible that simply the influx of http://www.mysite.com/landingpage?afid=123 type URLs have been causing the issues you describe, or is it something intrinsically related to your use of the rel=canonical tag? That is, for e.g., when you say “Google keeps finding random affiliate ID urls and adds them to index” wouldn’t it do this in any case, regardless if rel link=canonical was used or not?
DaveN
@Aaranged yep they just float in and out around 7,500 at any time
Rob Woods - http://www.builddirect.com
Great article! I’m wrestling with the same issues right now. Anyone have any better solutions? Land affiliate links (or all links with a ?variable ) only on dedicated landing pages which are blocked from being crawled?
toot
You were very brave to test the canonical attribute on such a large scale. Your conclucions has put me off using it completely. The only reliable option left seems to be getting your devs to get roudn the redirect issue.
rel=canonical tag and Affiliate links | seo cloak - pingback
[…] here to read the rest: rel=canonical tag and Affiliate links This entry was posted on Friday, July 17th, 2009 at 8:35 am and is filed under davidnaylor. […]
seo boondoggle
It is impossible to set up user agent detection? When the engines hit the langingpage.html?affid=123 it is a 301 to landingpage.html – or is this not possible due to development issues?
Richard Hearne
Used URL fragments quite nicely on a fairly large site – couple million URL combinations with Aff ids – and it’s worked very well. Downside is you need to sniff the aff id on the landing page with JS, but you still cover the vast majority of users without issue.
PPCblogger - http://www.ppcblog.co.uk
@Rob URL fragment, the hash # and pick up aff ids using script. Pass juice and you can track.
Andre
Would URL fragments still work with PPC injected text though? We vary our H1’s & H2’s of our PPC traffic to match the searched term
Gary - http://www.doublespark-seo.co.uk
Would using the # as described here overcome this problem?
http://www.phoenixrealm.com/the-power-of-the-hash-in-seo/
jaamit - http://blog.freshegg.com
This is annoying, to say the least. WTF was the point of rel=canonical if it causes behaviour like this? We’ve got a similar issue with a client having thousands of aff id’s where mass 301s weren’t possible, and were hoping this would sort it. I think Rob Hughes’ suggestion of introducing a robots noindex metatag to the head of affiliate URLs only, in addition to canonical, makes sense as a double whammy thought.
Using # instead of ? is a good plan but doesnt help you with existing links being out there, you would still need 301s.
SEPo - http://www.webfinderpro.co.uk/
Great Insights and discussion guys…
Efusjon
The entire affiliate marketing world is going to change with social media. Imagine writing a review on Amazon and getting paid when someone makes a decision and gives you a positive rating or “that review helped me” thumbs up.
Andre
Have you noticed Bing doing a better job of not including the affiliate links in the search results? In my case I am Bing is not including them while Google is.
Dunncan bloor - http://Getfamous.biz
Exactly the same problem with one of my sites. Implemented the canonical as soon as it was announced as per instructions and to date, just a slight drop in rankings is the total effect!
Mark - http://digitalnotions.net
I feel your pain. I’m in a similar situation except that I’m trying to use Google Analytics parameters in my bit.ly shortened links to track the effectiveness of my Twitter and other social media campaigns. In the end, I’m getting the GA Parameters indexed into other search engines. No help from the rel=canonical tag either.
Best bet I can figure out is to use URL fragments by putting a ‘#’ in front of the parameters and then writing some JS to parse out the parameters manually. A total pain, but at least it might work. In my case, it has the added benefit of allowing me to strip off the GA parameters so if the user bookmarks my site, further hits aren’t going towards my social media campaign.
Nick
Dave,
I tried developing a couple of ideas to solve this issue – but really (as you say) it boils down to having a solid canonical url, duplication free
– 301 to affid-free url
– put affid in a cookie/session (remove from url)
I’m not sure there’s anything you can do about it without redirects. I’d take a stab and say Google/SEs may not mind you 301’ing the bot to the non-aff version? it’s helping their index by removing duplication in this instance, and isn’t sending the bot to different content – just to the correct canonical version
Remi Turcotte - http://www.remiturcotte.com
Put the aff id after a # symbol and rewrite the urls.
so
example/landing?affid=123 ===== example/landing#affid=123
SEO Moves - http://www.seomoves.com.au
Ugh, going through similar issues, got a fairly large site with 2MM pages. May have to circle back and attempt to 301 all affiliate links back to the non affiliate page. Anyone pester Matt Cutts about this issue? Interested to hear what the Google Gods have to say about their old Googlebot
Thanks
Sobiesiu
Some time ago, the idea was to generate some unique data under every aff ID, like adding to title names/IDs/etc, and some additional texts on the bottom of a page trying to not interrupt earnings.
But i have no place to test that idea right now 🙂
Bill Sebald - http://www.greenlaneseo.com/blog
I’m suspecting it’s a stand alone algo that gets run a couple times a year (like the spam algo). Not very efficient. It would help explain why they were sure to say “suggests the canonical…” instead of “adjusts the canonical…”
Michael Jon Wissekerke - http://www.storminthebox.com
oh don’t worry i have exactly the same problem with the sites i’m managing and started looking URL rewriters etc like ISAPI_rewrite (yes we running the dreaded microsoft boxes 8|) but to no avail. rel=”canon” hasn’t help sort out our duplicate content issues with affiliate codes
but when i posted the problem to a coding forum [codeplex] asking them if it was possible to rewrite the URL but maintain the session variable they suggested a solution called self forwarding.
from my understanding it works like so:
* page detects if URL has an affiliate tag
* if not proceed to landing page.
*if it does have an affiliate tag, the page grabs the session variable
*301 redirects to the canon version
*and because you’ve already grabbed the session it reflects in the affiliate program.
Affiliate Links and SEO - pingback
[…] Dave Naylors post on his blog about rel=”canon” and my subsequent commenting i thought i’d do a post about a possible solution that we are […]
Facebook duplicate url’s and rel=”canonical” - pingback
[…] albeit on a much smaller scale than Facebook’s 550 million indexed pages. I know others are questioning whether its working at the moment. Has anyone got rel=canonical to work properly on big sites […]
Reading On The Toilet | Canonical Link Tag & Duplicate Content - pingback
[…] IT DOESN’T SEEM TO WORK […]
Nick
I didn’t see this mentioned here – but there’s a blue C in firefox (address bar icons)
Links you through to the canonical version
Nick D - High Impact Seo Services - http://www.highimpact-seo.co.uk/
Hopeully google will keep working to imporve the canonical tag and get right at some point in the not too distant future, until then have you considered blocking the affiliate pages in your robots text?
GarethMc
Anyone know what the general consensus is on canonical tag use where the canonical tag is directs to the page the c-tag is on?
h2o2 - http://leturdu.com
What about the power of the Hash # ? Is it still a viable alternative?
Tim - http://cheapercosmeticsurgeryabroad.com
The canonical tag can be a very useful tool, when used correctly.
I use it quite a lot, especially where anchor text is used to jump to content later on the page, or where affiliate links are used, because as discussed, they do create an alternative url.
However, while very effective when done properly, it is critical that you take your time to do it right, as I have found out to my cost!
I simply omitted the second quotation mark at the end of the correct url, resulting in me basically telling the search engines that the correct url was in fact a custom 404 error page.
So I ended up with a large number of previously ranking pages getting de-indexed. Oops.
Luckily, I have included a correct canonical tag to the 404 error page, as well, but it has definitely affected rankings, while I’m waiting for google to re-index all pages. I have found, also, my homepage ranking better than I would expect for terms it is not optimised for.
So, a small cautionary tale, there. If you put a bad canonical tag, it is far worse than no canonical tag!