Sitemap. While this is a term you may be familiar with, what does it mean?
Do you need one? Where do you find one? How do you make one?
These are valid questions; for some, there might be more than one answer.
Today, we will take a deep dive into the sitemap world, so that you can walk away with the necessary answers and confidence around the topic!
What Is A Sitemap?
Let’s start here.
Defining a sitemap is essential for several reasons, and we are going to go through the two main types that apply to technical SEO: XML and HTML sitemaps.
XML Sitemap
An XML sitemap is a file that provides a website’s essential pages, videos, and other important files for Google to discover when crawling the site.
Not only are these listed in the file, but the sitemap can also provide details for Google to know – for instance, when the page was last updated, and if the content is available in other languages.
As I mentioned, you can also provide details about content types like videos, photos, and news-related content, specifically in your XML sitemap.
According to the Google Developers Sitemaps section, the following can be included for specific types of content in your sitemap:
- A sitemap video entry can specify the video running time, rating, and age-appropriateness rating.
- A sitemap image entry can include the location of the images included on a page.
- A sitemap news entry can include the article title and publication date.
Next, we will talk about what an HTML sitemap is and the differences between the two.
HTML Sitemap
An HTML sitemap is more targeted for users on your site than for Google.
This is a page that exists on your site and has links to the pages on your website – and in some cases, includes a little context into what those pages are.
Google mentions that you should try to establish a consistent and clear hierarchy on the HTML sitemap as, although not its purpose, it can help with indexation.
You can think of an HTML sitemap as a directory that users can leverage to navigate your site and find what they need.
An HTML sitemap should not be an attempt to replace the important pages in your site’s navigation.
XML Sitemaps Vs. HTML Sitemaps
So, what are the key differences between these two types of sitemaps? Let’s review.
XML
- The intent is for Google and other bots.
- There is no hierarchy.
- Used primarily for indexing.
- You can submit via Google Webmaster Tools.
HTML
- The intent is for users.
- A hierarchy should be used.
- No place to submit in Google Webmaster Tools.
Do You Need A Sitemap?
If you are wondering if you need a sitemap, that depends!
First, let’s discuss the XML sitemap. There are a few questions you can ask to determine if you need an XML sitemap:
- How big is your site? Is it large enough that Google may miss newly updated content when it is crawling?
- Is your site relatively new? If so, it may not have a ton of external links on the Internet that point to it to help Google discover it. Even if your site isn’t new, and you don’t have external links, your answer to this question should be yes.
- Is your site content heavy? Do you have many photos, videos, news content, etc.?
- Does your site need a better architecture that results in pages not being well linked to each other? This can also be the case with archived and orphan pages you want to be indexed.
If you answered yes to any of the questions above, then yes, it is best practice to have an XML sitemap.
Even if you answered no to all of the above, I would recommend an XML sitemap for a few reasons; If your site grows, expands its scope, and other situations may arise, having a sitemap will be beneficial!
Next, let’s review whether it makes sense for you to have an HTML sitemap. Depending on where you look, you will find that answer to be yes or no.
HTML sitemaps are known to be an older concept, but that doesn’t mean you shouldn’t have one.
The XML sitemap has the information needed for Google to crawl, index, and learn other important information about these pages. However, an XML sitemap does not show hierarchy like an HTML sitemap.
Google will crawl the links on your site, and including an HTML sitemap could allow Google to understand your site’s architecture and relationships better.
This is even more useful for sites that have an incredibly large number of pages.
So, is having an HTML sitemap critical? No, it is not.
It is also not a cure-all for a poorly architected and nested website. While it isn’t a critical element of success, it has shown benefits that make having one a best practice.
To close this topic out, I recommend you have an XML and HTML sitemap because let’s be honest, why not, when the pros outweigh the cons very clearly?
Now you may be wondering how to create these two assets and what to do with them – so, let’s jump into some ways you can create these files and where to put them on the site.
How To Create An XML Sitemap
First, we will go over how you can generate sitemaps from scratch, and then we will get into some great tools that can do it for you.
XML sitemaps have specific criteria in order to be rendered valid.
Screenshot from lowes.com, January 2023Below are a few specific requirements for XML sitemaps:
- Begin with a <urlset> tag and end with that tag closing </urlset>.
- Include the protocol you are using within the <urlset> tag.
- Each URL entry must have a <url> tag as a parent XML tag.
- Include a <loc> child entry for each <url> parent tag.
- Each sitemap can only contain up to 50,000 URLs and 50MB.
- Must be UTF-encoded.
XML Sitemap Best Practices
Now, let’s look at some key best practices when it comes to creating XML sitemaps:
- Only URLs you want to be indexed should appear in your sitemap. This means no redirected URLs, non-canonical URLs, or pages marked as no-index.
- Do not use session Ids.
- Only include the primary if you have two versions (mobile and desktop) of your site.
- Include media assets like videos, photos, and news items.
- Use hreflang to show Google that there are alternative language versions of your website.
- Google documentation notes it leverages <lastmod>, but only if it’s consistent and verifiable. If you can’t keep this accurate, don’t use it.
- Google ignores the <priority> and <changefreq> tags at this time, according to John Mueller on this Search Off the Record podcast.
- Google will not crawl your URLs in the order they are listed, nor does it guarantee indexation.
- Your sitemap should be updated regularly – automatically, or manually – or Google may not trust it.
Now, if you felt lost reading those beginning requirements, that is okay, because there are tools to help you achieve your desired outcomes! We will go over some later in this article.
Check out the refined version below:
Screenshot from lowes.com, January 2023How To Create An HTML Sitemap
When putting together an HTML sitemap, remember its purpose is to serve a user on the site and help Google understand the hierarchy of your website.
You do not want to no index this page from Google; keep it crawlable!
You will want to ensure you don’t just throw thousands of links on an HTML sitemap page with no sense of organization, as this won’t help anyone – bots included.
Screenshot from Home Depot, January 2023HTML Sitemap Best Practices
Let’s go over a few quick best practices when it comes to HTML sitemaps:
- Arrange the page’s structure to align with your website’s structure. You will want to make sure that the hierarchy is easily understood.
- The HTML sitemap should be located somewhere the user can easily find it. You will often see it in the footer links of a website.
- Use anchor text that is valuable to the user.
Need a little help getting started? No worries – there are plenty of tools to help you.
Sitemap Generator Tools
There are a number of tools to help you generate different types of sitemaps. Let’s go over a few now.
XML Sitemap Generator Tools
- Screaming Frog – This tool is a great option for generating a sitemap, especially if you want to generate one after crawling your URLs. Screaming Frog is free if you have under 1,000 URLs, but you would have to buy a license if you have more.
- XML-Sitemaps.com – This web-based application allows you to enter your website URL and it generates an XML file for you. This is a free tool for up to 500 URLs.
Depending on which CMS you are leveraging, there are also thousands of XML sitemap generator plug-ins, but be cautious as even the best generator tools have their limitations, so make sure to double-check the output.
Here are a few popular XML sitemap plugins for WordPress:
HTML Sitemap Generator Tools
- com: this is a free online tool where you can scan your website URL or upload a document to generate an HTML sitemap. As we discussed earlier, there may be better approaches than a generator if your site is poorly architected.
- Crawler: Like Eli mentions, if you have a large site and are already using a crawler like OnCrawl, DeepCrawl, Screaming Frog, or SiteBulb, you can leverage the output from a crawl to help generate your HTML sitemap.
Like XML sitemaps, there are also a variety of CMS plugins for creating HTML sitemaps. Here are a few for WordPress:
- Simple Sitemap.
- All in One SEO (can help with both XML and HTML sitemaps).
- Companion Sitemap Generator (can also generate both XML and HTML sitemaps).
In Conclusion
Sitemaps have existed in the SEO world for some time as a method for helping search engines discover and crawl websites.
And, while having a sitemap isn’t always necessary for every site, it certainly doesn’t hurt – and can be especially useful for both new and large sites.
When you are determining your next steps for creating a sitemap for your website – whether XML or HTML – I hope you can leverage this guide to decide which path makes the most sense for your site’s needs.
More resources:
- 10 Best Sitemap Generators For Faster, More Productive Crawls
- 5 Sitemap Examples That Showcase Best Practices In Action
- Advanced Technical SEO: A Complete Guide
Featured Image: Sammby/Shutterstock
