Quote:
Originally Posted by itx
And they disallow all the robots with their robots.txt
http://biggerjohnson.com/robots.txt
User-agent: *
Disallow: *
If you plan get traffic from search engines the whitelabel of chaturbate is not the best option. |
Is that really how it works? Shouldn't it be "Disallow: /"? I have no idea and too lazy to Google it.