Pages can be designed to adapt to a user’s specific location, showing different content to users in different parts of the world.
While these types of pages are great for searchers in different countries, they presente crawling changes for Google. Since Googlebot uses US IP addresses, it may not be indexing all content on location-adaptive pages.
To combat this challenge Google has introduced improvements to Googlebot that will give it the ability to detect when its crawling location-adaptive pages. These improvements include geo-distributed crawling, and language-dependent crawling.
With geo-distributed crawling, Googlebot would start crawling a page using IP addresses that appear to be located both inside and outside the USA. With language-adaptive crawling, Googlebot would start to crawl with an Accept-Language HTTP header in the request.
These new crawling configurations will be applied automatically, meaning you won’t have to change your CMS or server settings.
Even with these improvements in place, Google still says the best solution for accommodating visitors from other countries is to have separate URLs with rel=alternate hreflang annotations for each location. In addition to being better for users, it’s also best for ranking and indexing of content.
To learn more about hreflang and how to use it, check out this in depth guide that will walk you through the steps.
