Barry points to a Webmaster World thread which asks how long does Google send their crawlers to a page which begins serving a 404 error not found message?
According to Search Engine Roundtable & the thread, the answer is forever.
Google will continue to try to see if the 404ed page has ever been reinstated as a live page.
Google even provides these errors in Google Webmaster Central, where you can see web crawl errors including 404 not found errors.
Instead of serving a 404 error message which makes it look like your site is full of broken pages and links, it’s much better to customize your 404 error page with:
- A site map, or link to the site map and link to the homepage which will direct the crawler to the main content rich sections of your site.
- If you run a blog try aggregating your RSS or archive feed to serve links to the 5-10 most recent posts.
- A search box which will help your users find what they are looking for on the site.