J!Extensions Store™
Forum
Welcome, Guest
Please Login to access forum.
- Joomla! Extensions Store
robots.txt Crawl-delay issue in Google (1 viewing) 
Go to bottom
TOPIC: robots.txt Crawl-delay issue in Google
#4398
Badan
Fresh Boarder
Posts: 4
User Offline
robots.txt Crawl-delay issue in Google Karma: 0  
Hello,

According Google there is an issue in the robots.txt file -> Crawl-delay: 10

What can I do to solve this issue ?

Thank you.
 
Logged Logged  
  The administrator has disabled public write access.
#4399
John Dagelmore
Admin
Posts: 3873
User Online Now
Re:robots.txt Crawl-delay issue in Google Karma: 84  
Hello,

please notice that this is an argument outside the sitemap component discussed in this forum.

By the way Google wants to manage the crawl rate by itself, so remove the crawl rate directive in the robots.txt and if you have a Google Webmasters tools account ensure that the crawl rate is set as automatically managed.

Cheers, John
 
Logged Logged  
  The administrator has disabled public write access.
Go to top