Google announced the launch of an updated version of the Google Search Console Crawl Stats report. The new Crawl Stats report has new features that offer more granular insights into how Google is crawling your website.
Google Search Console Crawl Stats Report
Google’s Crawl Stats Report offers a 90 day accounting of all files that Googlebot has downloaded, including CSS, JavaScript, PDFs and images.
The Over-time” charts report on the total requests, total download size and average response time.The Crawl Stats Report is useful for tracking changes in crawling patterns. Changes in crawl patterns can often be strong signals that something wrong (or right!) is happening.
For example, if the crawl rate suddenly drops this could be an indication of a server misconfiguration, a DDOS attack, an update that went wrong or any number of other issues.
If the crawl rate begins to drop it could also be a signal that there is a problem with with the content quality as well.
An increase in crawling may indicate that you’re doing something right, like adding useful content.
But an increase in crawl rate could also indicate a misconfiguration that’s causing the site to auto-generate duplicate or thin web pages.
New Crawl Stats Report Features
The updated Crawl Stats Report has the following new features:
- “Total number of requests grouped by response code, crawled file type, crawl purpose, and Googlebot type.
- Detailed information on host status
- URL examples to show where in your site requests occurred
- Comprehensive summary for properties with multiple hosts and support for domain properties”
Google’s announcement highlights updates to Over-time Charts, Grouped Crawl Data, and High Level & Detailed Information on Host Status Issues reports.
The updated Grouped Crawl Data report features more granular details.
“…provides data on crawl requests broken down by response, file type of the fetched URL, purpose of the crawl request, and Googlebot agent.”
Google summarizes the improvements as:
- “See Google’s crawling history in the overtime charts
- See the file types and file sizes returned by your site
- See crawl requests details in the example lists
- Track your site’s availability issues in the host status view”
This is a welcome update to the Search Console that will be useful to helping monitor websites for any issues that might come up. All publishers will benefit from making the Crawl Stats Report a part of monitoring sites for search performance.
Citation
Read Google’s official announcement:
