Quote:
Originally Posted by wankawonk
For years I just used the search console to slow their crawl rate. They have never respected crawl-delay in robots.txt.
Lately it's even worse--with the new console I can't set their crawl rate limit anymore.
|
Okay, I just realised I didn't read the OP properly. You can no longer limit the crawl rate in the webmaster console? Hmmm...