As everyone knows, the Google AdWords Keyword Tool “KWT” is one of the best free research tools available online. But I thought it would be interesting to see how its data consistency holds up if you change a few external factors and then cross check against a real campaign. I have seen variances in data previously, but never took the time to examine the issue in much detail until now.
Since Google pushes users to login when using the keyword tool I assumed that there must be a massive advantage for users to do this. My test would discover if there are any differences in the quality of the keyword suggestion data around average CPC rates, depending on whether you are logged into a Google AdWords account or not.
I assumed prior to gathering the data that there should be little difference in Google’s suggestions whether you are signed in your AdWords account or using the semi-anonymous external keyword tool. However, after seeing the results of the estimated average CPC amounts, I was surprised at the range of variance shown.
As I already had an idea of what the actual average CPC rates were, I quickly realized there was more to this and that I should conduct a larger test, this time using around 150 keywords.
Google Keyword Tool Match Bug?
One interesting aside I discovered during the research was that while the Keyword platform shows “Broad” match type as the default setting, if you also select “Exact” and “Phrase” match you get a different estimated average CPC amount.
The example below demonstrating this strange variance is using the keyword wynn las vegas. The Keyword Tool shows the estimated average CPC rate is $7.03 for the default result, but if you select to view all the match types you are now shown:
- Broad Match $0.00
- Phrase Match $8.09
- Exact Match $10.39
The $7.03 does not appear to be an average amount of the three match types. To confirm this I picked the first four related keywords which showed a variance of between 92% and 114%. It would be interesting to know where exactly the default average CPC rate data comes from, if not from averages. These small variations in the data that make it almost impossible at any scale to accurately compare Google’s AdWords data, so you have to take their average CPC data at face value. If you refine it too much for low traffic terms you will get little or no data, as you can see from the broad match result.
Actual Average CPC vs KWT Estimated Average CPC
I had to remove 13 keywords from the graph data shown below as it was skewed beyond readability. These keywords recorded average CPC differences ranging from 1420% to 13240% which were far higher than the average variance of 826%.
The variance between the estimated average CPC amounts was large enough to cause headaches and could cause campaign estimates to be scaled back dramatically.
Estimated Average CPC Logged-in vs External Access
I uncovered another interesting point: the variances did not seem to easily correlate to any other datapoints such as actual campaign clicks, impressions or actual campaign CPC amounts. This lack of ability to cross-check estimates when planning an AdWords campaign should create some concern with marketers using purely the Google KWT as it shows that is a fairly large variance.
Lastly, I discovered that you can’t assume that just because the term has a high number of clicks or impressions, it will therefore be more accurate when using the Google AdWords Keyword Tool. The only upside is that the higher the estimated average CPC rate, the more accurate the KWT estimated would be between external and logged-in data.
There seemed to be a sweet spot where if the estimated average CPC rate was around $1.20 I found it to be fairly accurate. Below $1, the estimated average CPC rate was usually incorrect. Over $1.20, my data showed that Google’s estimated average CPC rates are far higher than the actual average.
My conclusion: It’s important that you occasionally cross-check Google’s Keyword Tool estimates against real data from Google Analytics or existing AdWords campaigns data, especially if you are using it to form a business case for a new budget in 2011 for your PPC campaigns.
About the data: Because of the effort to use a larger set of keywords there was some contamination of data and the primary match type selected on the AdWords keyword tool was set to “broad” match type. This means you need to run your own tests to explore in more detail but you can assume that you will get a similar variance between Keyword tool data and actual campaigns. If you are using KWT data you should disclose this source of information to cover any variances when using it for a proposal, audit or presentation. A bulk of the AdWords campaign data used in this analysis had a fairly higher quality score of 6 and higher which does explain why the actual CPC rates are lower and the variance is greater but even when the quality score was below 4 the trends seemed fairly consistent.
