J!Extensions Store™
Forum
Welcome, Guest
Please Login to access forum.
- Joomla! Extensions Store
Re:robots.txt Crawl-delay issue in Google (1 viewing) 
Go to bottom
TOPIC: Re:robots.txt Crawl-delay issue in Google
#4399
John Dagelmore
Admin
Posts: 3873
User Online Now
Re:robots.txt Crawl-delay issue in Google Karma: 84  
Hello,

please notice that this is an argument outside the sitemap component discussed in this forum.

By the way Google wants to manage the crawl rate by itself, so remove the crawl rate directive in the robots.txt and if you have a Google Webmasters tools account ensure that the crawl rate is set as automatically managed.

Cheers, John
 
Logged Logged  
  The administrator has disabled public write access.
      Topics Author
    thread link
robots.txt Crawl-delay issue in Google
Badan
    thread link
thread linkthread link Re:robots.txt Crawl-delay issue in Google
John Dagelmore
Go to top