The past two years have seen vast changes in the SEO industry. The changes, though mostly positive, included the aggressive activation of Google penalties, whether algorithmic or manual. As a result, numerous websites ended up with a manual penalty or worse, an algorithmic penalty or a filter. Unlike algorithmic penalties when we’re at the mercy of Google’s algorithm refresh mechanism, manual penalties can at least be revoked following a lengthy process of correction that culminates in a reconsideration request. Or can they?
Before we closely examine the necessity of this thorough procedure using real life examples, I’d like to address a few key terms to ensure we’re all on the same page.
Penalty:
- Can be manual or algorithmic
- Can be time limited or not
- Can sometimes be removed by filing for reconsideration
Filter:
- Solely algorithmic
- Most common (I estimate most website owners aren’t aware they’re dealing with a filter)
- Cannot be removed by filing for reconsideration
Banning:
- An extreme measure against heavy spammers
- Often irrevocable, best to desert site
Many people confuse a Penguin penalty with a Penguin filter, for instance. The difference there is that a website that receives a Penguin penalty will likely experience decreased rankings in numerous keywords and drop dozens of page rankings – something any site would struggle with. Conversely, if a site is faced with a Penguin filter, it will experience a mild decrease in rankings, generally single digits, and only for select keywords. Banning, on the other hand, is an extreme measure not taken lightly by Google. Personally, I haven’t heard of many websites that were banned altogether from search results, unless they were using explicit black hat methods and other practices that blatantly violate Google’s guidelines. Other notorious past penalties are the Sandbox and -950.
Typically, filters are applied in instances where duplicate content, slow loading time and inappropriate use of tags are found. If you’re interested in learning more about filters, refer to the full list in an old-but-great post written by Barry Schwartz.
We can only hope to remove algorithmic penalties and filters once we’ve resolved the problem and an algorithm refresh takes place. I’d like to focus on manual penalties since so much has been written about tweaking an optimal procedure that will successfully remove it. We’ve seen everyone weigh in on the topic, from superficial articles to thorough, professional guides offering a step-by-step manual to handling a manual penalty efficiently.
A number of years ago, seeing the penalty removal business flourish, I too saw the value in learning the ropes. I started to delve into the subject and after months of research and consultation with experts worldwide managed to formulate an efficient method for manual penalty removal that has enjoyed very high success rates. The system is tedious and mostly manual, but it pays off eventually. We have yet to file more than three reconsideration requests for clients before removing a manual penalty, and we’ve handled over 120 cases so far.
Is Active Link Removal Necessary?
There is one case I’d like to use as an example here, since its journey to penalty removal was unique and somewhat strange – making me question the necessity for the strict guidelines Google set forth for penalty removal.
It all started when a new client approached us about removing a manual penalty. With 15 years experience and 12 years online, his company was well established and enjoyed a positive reputation. This company collaborated with NASA, the White House, the Department of Homeland Security and other large organizations, leaving no room for doubt: we were dealing with a legitimate and honorable company.
Examine Links
After carefully inspecting the client’s website and link profile, examining a his link profile, the link profiles of the sites that linked to the client’s website, and of course analyzing the competitors, we concluded that were weren’t dealing with an overly complicated case. The website had thousands of incoming links from thousands of different domains, some high quality and some of lesser quality as expected for a website that’s been around for more than a decade. The bad links were weeded out and found not to comprise a high percentage of the overall links, making us even more confident of being able to remove this client’s penalty relatively swiftly.
The first things we did after the research outlined above, was map the existing incoming links using all available sources: Google Webmaster Tools, MajesticSEO, Ahrefs, Open Site Explorer, and Yandex Webmaster Tools. Secondly, we ran all the links through two systems to ensure they were all live. All the links that were marked as non-existent (404 errors, link not found, site is down etc.) were tagged and forwarded for manual inspection by our team, to ensure they were in fact gone. In our experience, sometimes live links and live pages can be marked as non-existent due to heavy traffic, server issues, time-out responses, and other problems. To be thorough, we always conduct a manual verification.
We opened a Google Docs file and copied the links. The reason we managed this using Google Docs instead of MS Excel, for example, is outlined in this video by Matt Cutts. Then, it was time for the longest and most rigorous phase – manually sorting the links. We sorted through all the links, deciding which to keep and which to remove. For every link we decided to remove, we wrote a reason and tagged it in red.
Screenshot taken 5/4/2014Our team manually pulled the contact details of the website owners using three different methods:
1) Listed on the page itself
2) Using WhoIs
3) Contact page found on the site
Outreach
After sifting through the links and obtaining the contact information, we moved to the outreach phase. This stage is difficult since Google won’t allow sending a large number of e-mails with the same exact wording. Those who try often deal with having their IP address flagged and blocked, preventing the delivery of identically worded e-mails in the future. Though I understand theoretically, this prevents people from sending mass e-mails and spamming the web, I feel that it greatly hinders websites owner’s ability to address penalties efficiently by contacting problematic websites.
To solve this problem, we searched for a different e-mailing platform other than Gmail, which somewhat complicates the process but is better than the alternative. We manually sent a personal e-mail to each and every one of the websites flagged in three stages, asking them to remove trouble links. Simultaneously, we submitted contact forms on all websites that offered them, with similar requests. During the documentation phase, we focused on grabbing screenshots of forms submitted and gathering copies of all the e-mails we sent.
Document Progress & Submit for Reconsideration
The process was documented further on Google Docs, including the link owners’ contact information, the decision whether to remove or not to remove, the e-mail sending dates, the sent e-mails’ source code, screenshots and form submission dates, payment details (yep, some people ask for money to remove a problematic link), and the final status of those who removed the links, refused to remove the links, and never answered. At the culmination of this process, we prepared a disavow file that we then submitted to Google, and waited at least 48 hours, precisely as Matt Cutts instructed, before filing a reconsideration request.
Resubmit
The first request was denied. We weren’t phased since we knew this was a possibility, especially on the first try. However, Google’s answer was different in that they gave us examples of bad links – which was great in itself. We took a look at the file we prepared and couldn’t find those links anywhere among those we were able to pull from our resources. Refusing to despair, we ran another scan to find links from all the biggest sources, and found an additional 40 referring domains. We added those to our list, and embarked on the process once more, up until filing for a reconsideration request once more. I feel it’s necessary to emphasize that John Muller stated numerous times that it’s often enough to pull links only from WMT, and only in cases of severe spam would other tools be needed. Here’s a video of the Google Hangout where John Muller says this explicitly (video starts at relevant section). Either way, we filed for reconsideration a second time.
Screenshot taken 5/4/2014Unfortunately, our second attempt was also met with a denial, once more giving us two examples of bad links. This time, one had appeared on our list and was removed in the first request for reconsideration, and the other one was not on our list but was found to be borderline (in terms of spam). This is odd in itself, since Google representatives tend to give indisputable examples of links that breach their guidelines.
Screenshot taken 5/4/2014Once more, let me emphasize that the link itself wasn’t that bad. Truthfully, three years ago it would have met Google’s guidelines fully, and it was actually a four year old link. For some reason, it did not appear on our list. Since we knew the link was currently being questioned, we re-ran the scan and tended to it in addition to other links similar to it. What angered us more was the link that had appeared on our list and which had been removed on the first reconsideration request.
We re-checked the page over and over again, including the code, using four different software and four different team members in order to verify that there was no link still present. Verdict: the link had been removed as we thought. Why would Google send me an example of a bad link which I was removed a month and a half earlier, which I reported? I wrote a letter asking them to explain where they were pulling these links from since I could not be expected to handle links I can’t see, and in addition I asked them to explain how they can see a link that no longer exists. Naturally, I received a generic response stating that they were sorry, but the penalty was not removed – without offering any human response to my concrete claims.
Re-Reviewing Links
We went back to square one and reviewed all the links, sifting as though we hadn’t already done it twice. We wanted to verify we hadn’t let a few bad links slip under our radar, and so we hardened our criteria and wrote off even more websites. We submitted a request for reconsideration and for the third time, and were denied. This time, the bad link examples were of the no-follow variety. This is again questionable since on a number of occasions, John Muller stated that no-follow links need not be removed. Here’s a direct quote from one of his recent interviews:
Screenshot taken 5/4/2014Needless to say, we were starting to despair. It seemed that no matter what we did and how thorough we were, we received contradictory answers that were irrelevant to our process. The information coming from Google was also apparent on the WMT, as evidenced below:
Screenshot taken 5/4/2014While they explicitly write that no action is required on our part if bad links were created outside of our control, they recommend that we remove these links and file for reconsideration. Confused? Let me spell it out for you: without removing the links and filing for reconsideration, the penalty would remain – and action is most certainly required on our part when it comes to unnatural links whether they were the result of our activities or whether they stem from competitors’ malicious attempts.
Fresh Approach
At this point, we realized we were in a never-ending cycle and needed to change direction. Since we know that the links provided through WMT are just samples, we decided to scan the web using all our tools every two days, to compile an updated list. Moreover, we halted all link building efforts from the dawn of the process so that new links were not added to our link profile. After about two weeks we managed to collect another few dozen domains and handled them as described above. We decided that as an additional step, we would physically remove a number of pages from the website, which Google pointed as being linked to from bad sources. John Muller has said in the past that in the event that we encounter a problem and want to get rid of the links referring to the page, we can simply transfer the page to a new address – returning a 404 error and eventually making Google disavow those incoming links. We took it one step further and just deleted the page, without even opening a replacement. Here’s what John Muller stated:
Screenshot taken 5/4/2014We made an updated file that contained all the links found from the first stage, made a newly updated disavow file and filed for reconsideration. In this new request, we mentioned everything that had happened, explained the process in detail and emphasized that this was a completely legitimate and established website that cooperates with some of the largest bodies in the United States. As usual, we attached all files.
Denied…Again
This time, the request was denied sooner than we expected, within just a few days. No examples were attached, nor any explanation given. I can’t even describe to you our frustration at that point, let alone the site owner’s who had given up all hope – questioning Google’s professionalism and reliability. We decided to stop all activities and scheduled a meeting to discuss the situation in one month. When we held our meeting, the site’s owner wanted to write Google a personal message, using the reconsideration request route as a direct communication channel. He did not attach any files or documentation. We were opposed to this, but he was adamant, so he started to type: “Hello,”… and then, he accidentally hit “Send”.
I asked him why he checked the check mark before he started to write the e-mail and he apologized, saying it was an old habit. We didn’t revise our message, and what do you know, four days later we received a message saying the penalty had been revoked! Just like that. No further steps were taken, no documentation provided, not even a fully written letter. Despite our vast experience in the field of penalties, and despite our impressive success record, we had never stumbled upon anything like this.
Screenshot taken 5/4/2014If you carefully examine the process we went through step by step, you can see that Google’s reported ‘procedure’ for penalty removal does not align with the way Google handles reconsideration requests. Does anyone even check these or is this a flawed automated service?
Around this time, we heard from our colleague in Chicago, who told us he never removed any links. He simply sifted through link lists and entered bad links into a disavow file. No documentation of communication with any webmasters whatsoever, and files. According to him, this method has enabled him to lift penalties from over 30 of his clients’ websites. Sounds a lot easier, doesn’t it?
We made a business decision early on to thoroughly address issues that caused penalties to be issued to our clients, and therefore did not follow in our colleague’s footsteps. Moreover, we took some of his descriptions with a grain of salt, since every professional manual out there outlines a thorough, meticulous manual process similar to ours. There may be some rounded corners here and there, but nowhere near the superficial approach he displayed.
Surprisingly, a few days ago I stumbled upon this post, describing a very similar fuss-free process for penalty removal: no active link removal, simply using the disavow function. This writer claims to have helped over 60 clients successfully have their penalties removed!
Now, we began to wonder if there was something to this much simpler way of going about penalty removal. Was Google effectively pushing an extreme route to ensure that penalized websites would undertake a lengthy manual procedure that would ‘clean up’ the internet? Perhaps none of it was ever truly necessary.
Everyone here follows Google’s publications, Matt Cutts’ videos, and John Muller’s interviews – all stressing the importance of active link removal and the ineffectiveness of submitting disavow files without said link removal. However, we’re hearing more and more accounts of instances where these statements were blatantly contradicted.
Undoubtedly, the penalties issued and the subsequent penalty removal efforts made by countless websites have helped to clean the internet considerably – but is Google using SEO as its own personal clean-up crew for the internet?
What’s been your experience with the ‘traditional’ penalty-removal process, and have you attempted to skip active link removal? Share your experience in the comments.
Featured image via Shutterstock
