Google Search Advocate John Mueller has provided guidance on handling legacy AMP subdomains and their impact on crawl budget, addressing a common concern among large website owners.
A website owner managing approximately 500,000 URLs sought advice on Reddit about crawling an abandoned AMP subdomain.
He noted the crawling persists despite implementing 301 redirects three years ago.
The situation highlighted the challenges websites face when transitioning from previously implemented AMP solutions.
Key Details
The site owner reported that Googlebot continues to crawl hundreds of thousands of AMP URLs, even though:
- All AMP URLs were redirected to the main domain three years ago
- The AMP sitemap file is empty
- Both HTTP and HTTPS versions are affected
Mueller’s Response
Mueller offered two straightforward solutions:
- Maintain the current redirect setup
- Remove the hostname from DNS entirely
Notably, Mueller downplayed concerns about crawl budget impact, stating:
“If it’s a separate subdomain, it’ll generally have it’s own “crawl budget”. Also, with 500k pages, I don’t know if you’d really need to worry about crawl budget, even if it were not on a separate host name.”
Context
This guidance comes at a time when many publishers are reconsidering their AMP implementation strategies.
Mueller’s response suggests that complex technical solutions may not be necessary for managing deprecated AMP content.
For website owners facing similar challenges, Mueller pointed to Google’s “Large site owner’s guide to managing your crawl budget” for additional information.
Analysis
Mueller’s response indicates that crawl budget optimization is a valid concern for huge websites.
On the other hand, for mid-sized sites with around 500,000 pages, it’s not worth worrying about.
Next Steps
This development provides direction for technical SEO professionals managing the transition from AMP implementations.
When dealing with legacy AMP implementations, you can choose between:
- Maintaining existing 301 redirects
- Implementing a DNS-level solution
- Consulting Google’s crawl budget documentation for specific cases
