Google’s John Mueller explained during a session of the Search Central Live NYC event that they do over 700,000 tests per year in order to keep up with user expectations. His explanation of why Google performs so many tests and launches thousands of changes should give SEOs an idea of the pace of change going on at Google and should inspire publishers and SEOs to consider ways that they too can take steps to anticipate user expectations and roll out changes to satisfy them.
Updates Are Not Done In Isolation
The first thing that Mueller said about updates is that they’re not done in isolation but rather they use a third party raters, a fresh pair of eyes, to evaluate their tests and new updates to their algorithms.
Mueller explained:
“So there is a lot of activity happening on the web and we kind of have to keep up with that as well.
How we look at things when it comes to updates, I think this is maybe a bit of a jarring transition here, but essentially when we work on changes with regards to search, one of the things that is core to all of the changes that we do is that we don’t do them in isolation just because we think they’re good, but rather that we find ways to actually test to make sure that they are actually good.
And one of the ways that we do that is we work together with so-called quality raters.”
Number Of Tests And Launched Updates
Google conducts a staggering number of tests every year and launches thousands of changes (updates).
Photo Showing Number of Google Updates Per Year
John Mueller said (includes paraphrasing):
“When it comes to changes that we do this number is from 2023. I imagine the number from last year is similar. We’ve made over 4,700 launches. And these launches come from over 700,000 tests and we make tests all the time. You can try to calculate like how many tests are running every day. If you assume that a test maybe runs for two weeks over the course of the day, like there are lots of tests that are happening in parallel.”
Google Says It’s All About User Expectations
Mueller offered an explanation of what motivates Google to do so many tests and launch so thousands of updates to the search results. He said it’s all about meeting user expectations.
This is what he said:
“And that also means that when people look at the search results, they see things that are sometimes a bit different.
From our point of view it’s not so much that we’re doing all of this work to keep making changes to make it hard for people to keep up, but rather because we see that users have very high expectations of the web and we want to make sure that whatever expectations they have tomorrow we can still kind of fulfill.”
Takeaway For Publishers And SEOs
Google’s not running hundreds of thousands of tests a year to confuse SEOs and publishers. They’re doing it to stay ahead of what users want before users even know they want it.
SEO has historically been reactive, which means that search marketers and publishers wait until Google announces an update and then they run back to their websites and “fix” whatever they think is broken. SEO eyes are always on Google when they should really be thinking ahead about how their consumers or site visitors are aging out or no longer reading blogs and whose habits might be changing. Do you have to wait until Google announces an accessibility update before you test if your site is usable for visitors on screen readers? Are client sites usable for people who are color blind or are you going to wait for an update? That’s reactive.
One of the reasons Google is number one in many things is because they didn’t wait for someone else to do it first. Before GMAIL all email providers gave their users email space measured in megabytes. Google killed their competition because they were offering users gigabytes of free space.
So maybe SEOs and publishers should scroll up and re-read the reasons that John Mueller gave to explain why Google does hundreds of thousands of tests and launches thousands of updates every year. If you’re not already being proactive then I really think that this is the year you start thinking about ways to do that.
Takeaways:
Google’s Testing Volume and Frequency
- Google performs over 700,000 tests annually.
- In 2023 alone, these tests led to over 4,700 changes to Search.
- Tests often run in parallel, with many active at the same time.
- This volume reflects a continuous, high-speed development cycle.
Why Google Runs So Many Tests
- Google’s motivation for running so many tests is to anticipate user expectations.
- Despite their setbacks with AI, the number of tests and changes is the reason why Google remains a formidable competitor.
Implications for SEOs and Publishers
- Search marketers and publishers who want to keep up with Google should consider emulating Google’s approach to users and look for ways to anticipate user behavior, expectations, and trends.
- Start testing and improving now rather than waiting for a Google update before accounting for shortcomings.
- Consider a site audit by a fresh pair of eyes.
