Over the years I’ve been exposed to an incredible variety of preventable SEO issues – from the typical (disallows in robots.txt) to the truly bizarre (for example, a mobile-friendly site, cloaking Google and showing the old defunct dumb-phone version).
One surprisingly common problem is with URL redirects and the vulnerability which is rarely proactively monitored and detected.
URL redirects are typically configured when URLs are modified or content is removed.
The redirects are put into place to protect SEO performance for the old URLs, make sure that old URLs in Google’s index don’t 404 (or reached via following links), and to retain link equity since historic links are pointing at the old URLs.
For example, in this case, a really poorly constructed URL is redirected to good, semantic URL.
Broken redirects are redirects that generate errors (such as 404s) or are pointing to the wrong place. It is a particularly insidious issue because it is so hard to detect. Yet it can easily be prevented.
In the case above, if those redirects broke, the 50 external links coming into /women.shoes would no longer count toward that page. Page authority would drop as would the rankings of that page.
The damage would also cascade to the rest of the site as that page would have less authority to distribute via the internal links.
Many, many sites are at risk and, in most cases, the source of the problem will go undetected. The impact on organic traffic can range from minor to devastating.
In my case, eight years ago, it was devastating.
Our Story Begins With a Disturbing Organic Traffic Drop
At the time (2008) I was working for a website in the travel industry. One morning I came to work and had my typical breakfast of coffee, bagel, and Google Analytics.
What I saw that morning didn’t look good.
Organic traffic was down about 3 or 4 percentage points week-over-week. Not bad enough to ruin my bagel, but disturbing nonetheless.
I hoped it was just seasonality or one of those random fluctuations.
It wasn’t!
Day-over-day drops continued for about a week until we were down 20 percent.
At this point I was truly sick to my stomach. Like most SEO professionals, nothing wrecks my digestive system more than an organic traffic drop with no apparent reason.
I checked the usual suspects. The site seemed fine; no changes to SEO elements. No drop in Domain Authority or PageRank (back then we still looked at PageRank). No Google updates had been announced and there was no unusual chatter on the SEO boards. No new competitors were stealing out traffic.
I was stumped.
I spent the next couple of weeks trying to unearth any potential change that could have led to our downturn, but no luck.
Every morning I had a one-on-one with my CEO and every day that went by without an answer, the meeting got a little more unpleasant.
About two weeks into the drop, my lead engineer comes over to me and we have a conversation that went like this:
The Conversation
Lead Engineer: “Remember when we did the site migration last year?”
Me: “Yes, sure! Smooth as butter!”
Lead Engineer: “And remember the redirects we set up due to the URL changes?”
Me: “Yes, sure.”
Lead Engineer: “I took them down a couple of weeks ago because I figured we didn’t need them any more.”
Me: “No.”
Lead Engineer: “Yes.”
Me: “No.”
Lead Engineer: “Yes.”
Me: “NOOOO.”
Lead Engineer: “Yessss.”
The conversation degraded from there. I’ll spare you the details.
I was angry… first at my engineer for not consulting me.
Really, though, I was angry at myself. This one was on me! I was in charge of SEO and protecting vulnerabilities. This one slipped by me.
I felt guilty. Depressed. How could I let this slip by me?
I vowed never to be surprised by an unplanned change impacting SEO again.
Every Legitimate Link Is Precious
We all know how precious links are. Getting links is harder than ever. Keeping links should be a top priority.
How did this happen to us and why were we so vulnerable?
Some history:
About a year before this happened, the company had a huge migration as we moved from .Net to Rails. I did what every competent SEO would do: I put 301 redirects in place going from the old URLs to the new URLs. That would make sure that we had no immediate loss of traffic and, most importantly, it would transfer authority from the old URLs to the new URLs.
Our site migration went off without a hitch. In fact, traffic went up. Everything was golden. Until a year later when those redirects ceased to function.
Those missing links represented nearly all the deep links on the site and probably about 30 percent of the links to our domain. As Google crawled and the links scattered across the Internet started 404-ing, the links gradually got deleted from Google’s index, our authority dropped and our traffic started its decline.
What Makes Broken Redirects Particularly Dangerous
There are no visible, obvious ways to catch this issue.
- The onsite links all work.
- A typical audit tool will only find on-site URLs, it won’t try to crawl your old URLs.
- By this point, none of the old URLs are in Google index so there would be no broken links showing up in the search results.
Bottom line: If you aren’t actively testing the redirects, you will have no idea whether they are still in place.
Don’t Let It Happen to You!
Here are a few measures you can take to protect your site from the potentially devastating effects of broken redirects:
- Whenever possible, avoid changing URL structures. The easiest way to prevent broken redirects is to avoid creating the redirects in the first place.
- Create good URLs from the start! This way the discussion and decision on optimizing URLs never happens. Of course, this isn’t always possible, too often SEO pros inherit a URL structure that is sub-optimal.
- Don’t change your URLs if you don’t have to. Certainly, don’t modify URL structure for minor tweaks.
Sometimes, there is no choice and redirects are unavoidable. I can think of four clear-cut examples:
- An old URL structure was not well thought out and is confusing users.
- There’s a technology change that requires a new structure.
- A site is moving to a new domain, subdomain, or folder.
- Existing content is moving or consolidating.
Make Sure Redirects Persist
If you fall into one of those categories, as an SEO you have a responsibility to ensure redirects are done correctly and will work now and in perpetuity.
Education
Make sure everyone on the engineering team (or anyone that has the ability to make changes to the site) understands that any redirects you put in place must stay in place. Take the time to explain the significance of the redirects and what could happen if they are removed.
Feel free to use my cautionary tale. Don’t just be the pain-in-the-butt SEO dude passing down dictums. It’s much more effective if everyone involved understands the consequences.
Create URLs to Stand the Test of Time
Be thoughtful about new URL structure to avoid having to change them down the line. Create good URLs from the start!
Test and Verify Redirects
Ultimately, to protect yourself against broken redirects, you need to regularly test them to be sure they are still working.
- At a minimum, have a set of URLs that you periodically test. Create bookmarks and just click through the list periodically. Or save them in a document and click through. This may work for small websites or a short list of redirected pages, but this manual process doesn’t scale for a larger site.
- Use a backlink tool and pull a report of inbound 404s. The data may be a bit noisy with malformed links or expired content. Pay particular attention to identifying patterns that will highlight broken redirects that are representative of a large breakage. Be aware there is still a bit of manual work with this approach.
- Eliminate the manual element to identifying broken redirects with an automated process. You can create custom scripts to ping the URLs, run it on a schedule, and generate a report. Or you can use a dedicated redirect monitoring service or a general purpose URL monitoring tool.
Preserve Your Valuable Links
Links are still a critical element to Google’s algorithm and their value needs to be protected. Let my unfortunate experience with broken redirects serve as a cautionary tale. It is an example of how fragile SEO can be and how proactive those in our profession need to be.
More SEO Mistakes to Avoid Here:
- Why E-commerce Sites Must Avoid These SEO Errors
- 9 Ways to Avoid SEO Disaster During an E-Commerce Platform Migration
- The 10 Most Harmful Mobile SEO Mistakes
Image Credits
In-Post Photo: Created by Mark Munroe, November 2017.
