If you oversee (or consult on) enterprise-level websites, you’ve experienced first-hand how unruly URL growth can be. URLs often sprout like weeds on even some of the most reputable CMS systems. Sometimes they die even faster when there are several hands (users) in the cookie jar. This experience is not just on your site; if you care about understanding your backlink profile as well, you may be blind to many links that actually exist—or worse, point to dead URLs. It’s easy to get lost in a website’s labyrinth.
Truth is, sites of all sizes can present their own puzzles. In this piece, I’m going to talk about sources to track your external links. Then, I’ll discuss some ways to squeeze more value from these newly found links.
Chasing Down New Links Quickly
Release the hounds!
Wouldn’t it be amazing if Google let you download all the links they know about? Sure, they allow you to download a large portion via Search Console, but certainly not all the links in their brain. Google simply does not index or report on every URL they know about, even if they do routinely crawl them. You’ll have to chase them down.
This piece will mostly center on truly fresh links—that is, the links that are being created today. But to some degree we want to uncover the existing links you didn’t know about as well, so keep that in mind.
Let’s build one spreadsheet to rule them all!
Step one is to grab the link data from as many sources as possible. These are my favorites:
- The backlink data providers: Let’s start here because this is probably most SEO’s first destination. The biggest providers, Open Site Explorer, Ahrefs, and Majestic SEO all have a “just discovered” or “new/fresh links” type of report. And they work. My biggest issue with these reports, while powerful, they leave plenty of new links on the table. It’s understandable – each company has its own engine and crawl criteria – but they don’t have the same horsepower as Google. Nonetheless, export these links and add them to your master spreadsheet.
- Google and Bing Webmaster Tools: Both tools give you some nice link data. Not exhaustive, but a good sampling.
- Linkody: Here’s one that many people I speak to are unaware of. This tool does a great job of not only allowing you monitor links, but finding new links based on referral data in Google Analytics (among other ways). Pretty smart!
- MOZ’s Fresh Web Explorer: The Fresh Web Explorer is an engine that lets you search for keywords or brand mentions. You’ll often find pages here that are not in the “just discovered” report in OSE. It provides a 4-week snapshot and the ability to create alerts as well.
- Brand Mentions: This is a new service now in alpha (I was lucky enough to be invited in) that is quite impressive. Brand Mentions is similar to MOZ’s Fresh Web Explorer, which has been providing a larger set of results through my trial.
- Mention: Mention is another alternative that has decent results.
- Social Mention: Not to be confused with the above service, Social Mention adds a more social slant with some nice data.
- Buzzsumo: Buzzsumo is one of my favorites. It also provides alerts when you’re mentioned and has proven to be more robust than Social Mention for my campaigns. It’s not free, but provides several other features for the cost.
- Google Alerts: Saved the worst for last. Unfortunately while Google Alerts may get the credit for inspiring everything above, it really has lost its great reputation. Pity too, since the best provider of this kind of data really should be Google.
Chasing Down New Links Slowly (But More Advanced)
- Unlinked Mentions Finder: Our own Sean Malseed has created RankTank.org, a site of free tools. One of his gems is this free Unlinked Mentions Finder. Why count an unlinked mention as a link? Because Google has been known to follow unlinked “naked” URLs (even though they may not pass PageRank). This also provides the ability to reach out and request a more powerful and concrete hyperlink.
- Google Analytics Referral Report: Like Linkody above, one great way to spot links that have not otherwise been found is simply reviewing your referral data. Provided the link received at least one click, you can pull years of referral data here. You’ll need to scrub out links from email campaigns, but I guarantee you’ll find links here missed by the above tools.
- 404 Reports from Search Console: The crawl errors report in Search Console is powerful. By looking at your 404’s individually, you can click the “Linking From” tab. Often you can spot links from old sites. Restore or 301 redirect those bad boys to retain that link value. Since there’s no way to download this en masse, this process can be a nightmare (especially on enterprise level sites)./li>
Protip: Managing a disavow campaign? These are the same sources I use for that exercise as well.
Scrub the List
At this point, you’ve grabbed all the links you could find and thrown them into a spreadsheet. You’ve looked under some pretty well-hidden stones. Woo hoo—you’ll have a great list of backlinks! But the work isn’t over yet…
The truth is many of the links you have in your list are dead. It’s a common phenomenon on the web—someone posts a link, but the page or site dies for one reason or another. Some of the tools we used above have out of date indices. It’s just the nature of their services. Let’s get those dead links out of there!
Screaming Frog provides the simplest solution. Take your new list and upload into Screaming Frog’s “list mode” to get a sense of what URLs are dead. Simply exclude them from your list by filtering out of excel.
I Have the Best Possible List of Live Links—Now What?
Now it is time to append useful metrics. A giant list of links is useless without metrics to inspire action – even at a thousand foot view.
The first step is to upload and store them in your favorite link manager. I prefer Buzzstream or Linkody for the ease of use and link metrics they obtain for you, but some others SEOs might be more Excel or Google Sheets reliant. To that end, make sure you use URLprofiler to pull the link metrics into your master links list. URLprofiler can provide a ton of great metrics to each URL with only a few clicks:
Links and Link Metrics Obtained—Drive it Home!
I like to think of links as opportunities. We know that today smaller links have less value than ever before, and relevance through the link is more important than ever. We also know that relationship building and digital PR is a strong focus for the modern SEO. Now it’s time to optimize your link profile and leverage these links (or opportunities).
At this point, you’ve probably noticed many new links you didn’t realize existed, even despite years of backlink wrangling. Many of these are likely organic; thus, not a relationship you’ve created yet. Sort and prioritize your list by the most favorable metrics first and look for opportunities. I like to think of share and link counts as a proxy of popularity, as well as domain authority—so, I sort by those metrics first and work my way down the list.
- See any links from sites with high domain authority that you had no part of creating? Track down the author and get a conversation going. Maybe the author writes elsewhere? Maybe they would be willing to work with you in other capacities? You’ve already impressed them once—you should be able to do it again.
- See any new links that have some information wrong? Since the link is fresh, the author might respond better to an email suggesting a correction or enhancement. This could be your chance to add more relevance.
- See any links that are part of a conversation? You can jump in and add more context to that page, possibly making the value of that link more robust.
- See any trends on the kind of content that gets link more often? Go down that road and model some new, fresh takes on that topic.
- See republished content? Ask if the webmaster would be interested in an original piece.
- See any news sites? See if the webmaster would be interested in you as a columnist.
I would also recommend completing this exercise once a quarter if you really want to keep up with the speed of the web. It’s simple, somewhat enlightening, and the kind of nerdy fun SEOs truly enjoy.
Image credits
Screenshots taken Nov 2015 by Author
Featured image created by author on Canva.com
