Why Do SEO Forecasting Models Keep Failing Us?

Although forecasting is a fairly routine activity in the life of an SEO agency, there are a lot of pain points associated with it. It comes with:

• The pressure to connect SEO performance with ROI and business results.
• A requirement for investing in specialized tools that need the client’s buy-in and a thorough pitching journey.

And the process is not that clear-cut all the time.

The forces that influence your SEO performance are part of a complex system that involves organic traffic, rankings, seasonality, and so on.

Your variables are constantly shifting. So where do you start?

Just like a carpenter building a wooden table.

Seems straightforward if you think about it: one top and four legs.

But, as you start building it, questions start to pile up, and a series of decisions about it along the way – the design you need for your project, the most durable or the lightest wood and its physical qualities, the quantity of wood required for your specific table, how to measure and cut the lumber, etc.

It’s not simply a table, but a system of shapes and materials that have particular properties influenced by time, space, usability, etc. Each step implies a different level of knowledge and understanding, or the material consequence will lead to failure.

You’ll first need to know what you need to know in order for the system to work for you and not against you.

It’s the same with SEO forecasting.

But, unlike the physical properties of the system, there’s another layer to consider: the capacity to understand the limits of the method and our own limited ability, as humans, to accurately predict results.

The Dunning Kruger Effect Applied to SEO Forecasting

Sometimes anecdotes can be mistaken for data and knowledge – something that the popular explanation of Dunning Kruger replicates quite often.

Although it is commonly referred to as something along the lines of “the less competent you are, the more confident you are”, the 1999 study by psychologists Justin Kruger and David Dunning set out to assess an overestimation of abilities in many social and intellectual domains, by replicating the tests in disciplines like logic, grammar, and humor.

As one of the actual graphs in their scientific article shows, the Dunning-Kruger effect refers to the people performing less (the bottom quartile) having a higher perceived ability than their real ability:

In their own words, “the authors suggest that this overestimation occurs, in part, because people who are unskilled in these domains suffer a dual burden: not only do these people reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the metacognitive ability to realize it.”

There are also the people in the top quartile that underestimate their performance, although they’re the best performers, which can be interpreted as being more aware of one’s limits.

So, what Dunning and Kruger show (with the help of four studies and logic, grammar, and humor tests applied on their Cornell students) is that the ability to recognize your competence limits is particularly important in decision making, and can be trained.

Although the study has a number of limitations and received various academic critiques throughout the years, when evaluating processes, the Dunning-Kruger effect can still be a handy tool of unearthing unknowns and working at a metacognitive level – thinking about the learning and thinking process itself.

When considering a complex task like SEO forecasting, this cognitive bias can act as an “evaluator” of shaping understanding in your agency – to know when your model is an effective tool for painting a potential growth picture.

But when is the process in-depth enough to reach the top quartile of perceived and actual performance closely correlated, like in Dunning-Kruger’s graph?

Forecasting Models & the Forces Behind Them

Looking at Organic Traffic

SEO influences organic traffic. But considering the integral organic traffic as your baseline for SEO forecasting can be misleading as it includes both brand and non-brand traffic.

What Are the Limitations?

Blending the two will make your further estimates way off.

If you want to accurately measure SEO traffic, you should filter out the brand keywords, so that you can focus on what you can actually influence: the specific keywords you want to optimize for.

There’s so much traffic you can truly have an impact on.

Also, by solving this point, you make a direct connection between your SEO efforts and your client’s results, separating other marketing activities in the process.

Estimating Rankings, Not Clicks

In the keyword research process, you can think about your targeted keywords and create a ranking estimation out of the current situation for the next 6 to 12 months.

You’ll probably include the search volumes of those specific keywords and also establish keyword difficulty based on competition.

Maybe you’ll include a comparison with your particular competitors, to further establish what it takes to get a specific rank in top 3 or top 10.

What Are the Limitations?

Rankings alone don’t paint most of the SEO picture nowadays.

Businesses, in the end, need actual outcomes that are connected to their day-to-day activities. A business could more easily forecast impact in turnover if they would have an estimate of new additional sessions/clicks instead of ranks.

By doing rank-only estimates, you’re forcing the business to create their own hypothesis of how ranks influence clicks or conversions (e.g. “How many cars do I need to add to my fleet when we reach Position 1 on a search for [car hire]?”).

Plus, consider this simple and deceiving thought jump:

Let’s say the keyword “car hire” has a monthly average search volume of 12 million. Guestimating sessions based on this information alone would imply that once it reaches TOP 3, all that traffic will be acquired, so 12 million visits guaranteed.

You’re probably smiling right now.

Estimating Sessions Based on Monthly Search Volumes

What if we go one step further and don’t simply estimate rankings, but output based on rankings – sessions and clicks? That means looking at the CTRs based on reaching a specific rank.

Again, in the process of keywords research, you’ll probably consider search volumes and keyword difficulty, this time including a topical authority evaluation – similar keywords for which a page ranks for, mapping out your current status and your competitors.

Then, to estimate sessions, you’ll take into account the CTRs baseline: if a keyword reaches number 1, it means the CTR is 30% times Search Volume in that month.

To have a more in-depth view of search volumes, you’ll add monthly distribution and take into account how seasonality affects your keywords – establishing how seasonal a keyword is and when you should target the rankings.

You can also use the year over year search data to better understand the difference in search volumes in the last 24 months, modulating your keywords evolution.

What Are the Limitations?

You have to apply a general rule to CTRs which nowadays, with all the search features, could seriously affect the outcome.

The “10 blue links” are a thing of the past.

For instance, searching for “coffee shop Paris” will include at least 2-3 ads and Google’s local pack. Also, depending on the device, there is further variance included.

Estimating Clicks & Sessions Based on Custom CTRs & Devices

Let’s go even deeper into the complexity of a potential forecasting model and say you’ll include the desktop / mobile split while estimating clicks and sessions. Now, you’ll be able to address the differences in CTRs based on device.

You might as well include an analysis of performance over time (12 months, for example), that you can calculate as linear or exponential, based on the goal you’ve set for your campaign. And, with the search volumes, seasonality, and YoY trends that we addressed, you can calculate an estimation of inertial traffic – the total number of visits included in your given timeframe, provided there are no changes in the website’s visibility.

This way of thinking gets you closer to the complex reality of forces influencing your SEO performance, yet there’s still something to address:

What Are the Limitations?

Even if you now added more sophistication to the forecast, it would still lack an important factor – how many of the clicks go to the website instead of Google’s own properties?

We return to the tricky analysis of CTRs and their variation influenced by device and SERPs.

Estimating Business Results Taking Into Account Custom CTRs

Taking all of the above into account, let’s tackle the problem of CTRs that include both device and SERPs.

You can think about a way of estimating how mobile versus desktop CTR influence your results, plus how SERP features variate for each device and keyword.

For instance, calculating a CTR curve for the Page 1 positions – an average for each individual combination of SERP features and devices – to get a more sophisticated view of what the campaign could bring.

Including the CTR curves of your targeted keywords and the historical CTRs for those combinations (SERPs and devices), you’ll be able to better estimate the actual clicks and sessions.

Furthermore, to estimate business results and paint the bigger picture for your clients, you can connect sessions to conversion rates.

Let’s remember the car hire owner questioning how many cars to add to his offering – estimating additional conversions based on the additional traffic your desired rankings would bring, once reached, can clarify what the next business steps are.

Maybe you have historical conversion data for the term “car rental”, so you’ll know what additional conversions means. If not, you can estimate conversion rates for new keywords by taking into account the channel – extrapolating the average conversion rate for “vehicles” or “cars”, for instance.

Also, as a final step, you may think to customize your performance over time. This requires even more analysis and experience, yet it’s still fragile. Estimating exactly the month where you’re going to reach a certain rank based on content development efforts is a challenge in itself.

What Are the Limitations?

In this model, conversion rates are estimated for new keywords that have no history, based on the average conversion rate of the channel as a whole – this gives you an idea of what to expect, but it’s still a hypothesis.

There’s also the limitation inherent in SERP features and their constant changes. You can’t predict what the next one will be, but the CTR curve allows you to be prepared when presenting the SEO forecast.

And, as a final point, you can’t estimate big market changes generated by global forces, such as a pandemic or an economic crash. That’s why, after all, the forecast is an indicator of how the business could look, while being mindful of all predictable risks and unpredictable events.

Conclusion

There’s a popular saying that “All models are wrong, but some are useful.”

Just like the findings of Dunning and Kruger that the limits of our knowledge affect our perceived performance and our decision-making process, so all models suffer from certain limitations.

By including some of the key forces that affect traffic, sessions, and clicks, you’ll become more aware of the implicit connections and you’ll be able to present a relevant business case and explain to your clients how their results might look with that SEO campaign.

It’s not about magic, just being thorough in your forecasting and showing how reaching a specific SEO goal can impact their business.

Going in-depth and questioning every layer in your model will shorten the distance between perceived and real.

That’s the reasoning behind SEOmonitor’s Forecasting module, which includes:

• The brand and non-brand organic traffic split.
• Search seasonality.
• CTRs based on device and SERPs, with an average CTR curve calculated for each keyword. Our team at SEOmonitor has done extensive research on this issue in order to determine how the CTRs of the top 10 positions are affected by the SERP features mix on each device.
• Year-over-year search trend for your targeted keywords.
• Inertial traffic calculated based on seasonality, provided the website’s visibility doesn’t change in your timeframe.
• The performance in time, whether linear, exponential, or custom.
• Additional traffic generated once the desired rank is achieved, taking into account the set performance in time.
• Long-tail keywords effect.

SEO forecasting with more data points is just a part of our toolkit to help SEO agencies acquire more customers. Our solutions include a Google Slides Add-on for building Sales Presentations and an intelligent assistant that allows you check-in with unresponsive leads.

Join us in our journey to bring more transparency in the industry.