Making Sure You Think About Your Search Universe

Search marketing has been a force within digital advertising since Google’s inception in 1998, and certainly since Google launched AdWords in 2000. This is when search marketing became a dominant channel, bringing with it the ability to optimize and fine-tune ad targeting and relevance in a way that we were never able to before.

In 2016, Google earned $79.3 billion from advertising revenue.

The value of the paid side of search has always been clear, but for years on the SEO side of the fence we have talked about the “Black Box,” the “Dark Mystic Art,” and the lack of visibility that means SEO professionals the world over have to test, tune, and work hard to understand the Google algorithm.

A Trip Down Memory Lane

When I first started in SEO it was all about getting keyword density on the page and in the tags for relevance. You utilized Xenu Link Sleuth for page reviews, and made directory submissions to DMOZ and Yahoo.

Google ran indexing once a month statically, and you then took your log file for the month and ran it via Web Position gold to see how you were performing, and prayed it was a positive outcome! No visibility in the back end for webmasters, and gaining any visibility in SERPs was pure chance in some ways!

Below is how Google looked in 1998 the day of launch. Only 25 million pages in the index – “soon to be much bigger” as they say – and they now currently have around 52 billion pages indexed and are aware of around 130 trillion pages on the open web… mad!

For me (thankfully) we are in changing times and the ability to understand Your Search Universe is the key to better performance data and gaining insights to supercharge your SEO capabilities. Not only will it drive performance but it will also bring you the data you need to set benchmarks, tangible ROI, and ultimately secure more organic budget in the long run.

So What Do I Mean by the Search Universe?

As SEO professionals, we understand how SEO initiatives improve organic visibility and drive revenue, but we need to take this to the CMO as well.

For years SEO professionals have battled in the boardroom to convince marketing stakeholders to spend more money on organic search and have had to try to prove the tangible value. With the growth in technology buying points, we now have an abundance of weapons at our disposal that we have never had before to make this happen.

Data layers – and the ability to make them interact – is how you succeed in this. Otherwise, you get stuck in Excel hell trying to stick together disparate data sources in a VLookup nightmare to try and find something meaningful.

Also, a key aspect in this is taking the “best of breed” solutions for each data source and stitching them together (that’s the hard part) to fully automate the single URL’s view of performance.

Like anything that you build it all has to start with the foundations, and that is a Website Crawler (of course I would recommend DeepCrawl). Crawl Data sits at the center of every Search Universe and is a crucial component of understanding how your site will be seen by search engines.

As shown above, the additional data layers break down into four key segments:

1. External Links/Backlinks

On their own, backlink tools highlight your most linked pages. Layering pages receiving links with crawl data means you’ll understand how effectively these pages are performing for your site.

It also lets you see the quality (good or bad) of your inbound links and to what pages they land on within your site architecture. This allows you to do some link clean up and remove negative links and move quality links to high authority pages and in turn supercharging your SEO value with Google.

Having this data layer with also mean you are getting data from all the best sources, such as Majestic, Ahrefs, and Moz.

2. Log Files

Adding log file data to your Search Universe shows you how Google and other search engine bots crawl your site. Log files combined with crawl data confirms whether Google is paying attention to the right pages!

I love to use log file data and recently did a webinar on how valuable this is and how underutilized it is compared to the days before analytics packages. You can now get log file data via an API from the likes of Splunk, Logz.IO and other solutions.

3. Organic Search Analytics Data

Tools like Google Search Console add information about which pages are appearing in search results.

Paired with crawl data, you can identify pages that shouldn’t be receiving clicks or impressions, and see even more data than you can within Search Console itself. Add ranking data to this from quality solutions, such as STAT Search Analytics, and you are flying.

4. Consumer Data

This is the key revenue part for your new friend, the CMO. Platforms like Google Analytics and Omniture show you which pages visitors see on your site and the level of engagement these pages receive.

Connecting Analytics tools with crawl data reveals traffic going to broken pages, and more! Bring in the revenue data on a page by page basis as well and you are able to view the page performance not only for Google but also for the bottom line revenue drive.

Thinking Beyond SEO

Don’t just think SEO at this point. Monitor your PPC pages too and make sure they are performing for you and that you are maximizing revenue from this stream as well.

Remember you can use a web crawler to also check that PPC pages are live and the content is relevant and strong from a buying point, as well as from an AdWords Quality Score perspective.

The final piece after bring all this data into one place is having it in an API. Everyone from the CMO to the SEO needs a nice dashboard – and you have the ability to push this data into numerous solutions, such as Klipfolio or your own internal data warehouse.

This approach is the future and drives organic accountability and visibility like never before. So going forward, think about building your Search Universe.