Google’s John Mueller recently advised an SEO not to manually create a sitemap for a large website, and offered more appropriate solutions.
This exchange occurred in a Reddit thread on the r/SEO subreddit. An SEO was looking for advice on creating a sitemap for a website with over 2 million products.
The thread states:
“I have a client in a very technical, industrial niche. They sell industrial parts that are similar, but very specific. They have 2 Million + SKUs in their inventory.
My challenge here is to create a sitemap. Because there are so many pages, I have to create it manually. I have a list of URLs broken up by products and categories that our dev team pulled through API. Since sitemaps can only contain 50,000 URLs, I have 37 excel sheets with 50,000 URLs on them each…”
The SEO went on to describe how they plan to go about tackling this challenge, which is all irrelevant now considering they don’t “have to” create the sitemap manually as they originally thought.
Here’s what Mueller suggests doing instead.
John Mueller’s Response
“Don’t create a sitemap manually like that,” Mueller plainly states in his response.
As the most upvoted comment, Mueller’s advice was obviously echoed by the Reddit SEO community.
Instead of creating the sitemap manually, Mueller says this is what should be done in this situation:
“You need to get [the sitemap] from the CMS or the underlying database directly.”
As another Reddit user points out, getting a sitemap from the API where the dev team originally pulled the list of URLs would be a fine solution as well.
Why is this better than manually creating a sitemap?
When creating a sitemap for a large site, or any site, you have to consider not only how it will be created but how it will be kept up to date.
A site with a product catalog of this magnitude could be adding and removing tens of thousands of products every day.
It would be logistically impossible to manually maintain an updated sitemap like that.
And that’s not even taking into account the considerable amount of work that would go into creating the sitemap in the first place.
Manually creating a sitemap is not a realistic solution when automated options exist.
The SEO Comes to Their Senses
The SEO who started the Reddit thread eventually came to their senses and agreed that manually creating a sitemap is not the right way to go about it.
“Thanks for the wake-up call guys. I agree, this was the plan of action laid out for us and it didn’t sit right with me from the beginning.”
For what it’s worth, the idea apparently came from senior staff members of the company they work for.
More Resources
For more information on creating and optimizing sitemaps, see the resources below:
