In the spirit of good content, we have provided this info in video, text format and as an mp3 download at the end of the post. Enjoy!
Embed this on your site
Video Contents – Link Removals
Today, we're going to start looking at doing some videos on different aspects of the work that we do here at Bronco. So first off, we're going to look at link removals and whether you should do it or not; it seems to be a hot topic at the moment with lots of websites running into issues. A lot of the SEO's in private Facebook groups and publicly on Twitter are going backwards and forwards on what to do and how to do it.
Let's look at it from a penalty point of view. A big argument was that if you’ve been hit with a penalty then you shouldn’t bother clearing up the links, just disavow all the links that you feel are bad and everything will be rosy. I want to run through a few steps of the way that we work here, which I believe is doing due diligence in insuring that the clients have got support for the rest of the site's life. Let's face facts; by the time you’ve picked up a client that’s been on the web for the last 10 years, they may have passed through 6 or 7 different SEO companies that have all used different types of ‘SEO practices’, that may have worked well in their day but are just really, really bad these days. I can't think of when article submissions were ever a good thing, but they did work. We look at different categories and different timelines, and we try to do that with the client's help at the very beginning; where we look at figuring out whats been done on the website.
Then what we do, while we're doing that process, is we will download Google Webmaster Tools data on a daily basis. We'll do this for as long as we feel that we need to. We’ve got little programs that will take the data, put the data together, and then tell us how many new domains were found against previous downloads, we do this until we can get that down to zero and we feel that we’ve caught all of the links that Google were actually pushing out there. The reason why we do that is because Google only shows you a sample data set. Just taking the first download; the bad links that you were really after could be in the next day's sample. Keep sampling and you should be able to get to the end of it. Don't rely on one data source but Webmaster Tools data always uncovers links that other databases miss. I'll always take data from Majestic SEO and I will always take data from Ahrefs. Bring these together, combine them, put them into one database, and then we send our python spiders out to basically look for the sites that these links are on. We'll spider them, we'll make sure that the links are still there, whether they're no-followed or followed and what the anchor text distribution is etc. etc.
We'll build this into a format that makes it very quick and easy. We categorise these links and then put them into a very large database with a simple mark of good or bad in a category selection. That goes to our SEO team. They will go through the links and look at each one individually very quickly; it's a couple of seconds per look. We did one recently that we reported at just over 7 million backlinks, that crunches down to a few thousand root domains (that are still live). Our team can go through that in about 2 days. Using the systems we have in-house, we can get through these quite quickly.
The process then is the bad links get put onto a disavow list straightaway, this doesn’t get uploaded straightaway though; it goes to an outreach team that goes through these, and then tries to email every one of those websites. The emails come back into our own systems, we can then either push them again harder at a later date or we can basically go, ‘Actually, we'll never get these removed,' or ‘We’ve never had a response, but the link has been removed.' We'll then re-spider the disavow list to check what’s still linking and then the re-inclusion request goes in with ‘These are the links that we’ve removed and are no longer active from the data sets that we're working with.' Then we upload the disavow list at the same time, send it off to Google.
The downfall of not removing the backlinks are that the disavow list is just a text file on a server in Google. People have access to that. People may decide to remove links off that and you would never know about it. We'll date-stamp it, but if they delete it by accident or they upload the wrong one by accident, then the penalty's going to come back. Matt Cutts has already said that these penalties, when they do come back, will come back harder, and harder, and harder. You try proving to Google that you didn’t remove that disavow list by accident and see how far you get.
Also, if you ever 301 the domain and you’ve got a disavow list in place, move the disavow list to the new domain because if you don't, those links will follow you onto the new domain, the disavow list won't account and you'll pick up a penalty again. If those links aren’t on the web, it's saving Google time, it's saving you time. Anyone that says to just leave them out there, they're just not doing their due diligence.
Here’s a sneak peak at the tool that’s mentioned in the video…Introducing LinkSpin!
So here is a snippet of the main dashboard, here we can quickly check the crawl status, currently it’s capable of checking 36,000 URLs an hour and not only does it check the URLs that we feed into it but it also crawls each domain to check for other links that might not have been picked up by any of the main databases that we use. The checks are allocated throughout the SEO team, so here you can also check the allocation against your name as well as the overview of that is left to check across the whole team. There are also some download options from here too.
This next screenshot shows the ‘decisions’ dashboard where we are fed the links, a lot of information is given just from this and if we want to see the next set of links for that domain we select ‘Next 10/100/all’. As you can see the anchor text is shown along with the URL. After reviewing the site we can then add any comments we like (some of these can get comical after a few thousand checks) and include a predefined ‘Tag’, these include ‘Directory’, ‘Forum’, ‘Natural’ etc. The final decision is then made as simply ‘Good’ or ‘Bad’, once selected it then disappears and the next domain expands.
All this data can then be exported for analysis.
Our tool is still very young (coming up to a year old) and constantly evolving and we have further plans for it in the coming months but so far it has increased our productivity and efficiency tenfold.
We plan on making more videos on topics of interest but if you have a burning question that you would like to ask Dave or one of the team, simply submit your questions here!!