GoFuckYourself.com - Adult Webmaster Forum - View Single Post

blinki bill · 01-07-2015, 08:58 AM

I was just looking at chaturbate's white labels and it seems they have it set with robots.txt that have disallow all! WTF!!?

I know that whitelabels don't do well with google this days, but while a lot of the programs try to program solutions that allow you to do extra customization to omit penalties, chaturbate seems to go the easy way and disable access to whitelabels to prevent penalties on their own site.

Am I missing something, is this common place these days? Why would you run a whitelabel with robots.txt file set like that?

01-07-2015, 08:58 AM
blinki bill Confirmed User Join Date: Oct 2006 Location: solar system, earth Posts: 123	whitelabel with robots.txt WTF? I was just looking at chaturbate's white labels and it seems they have it set with robots.txt that have disallow all! WTF!!? I know that whitelabels don't do well with google this days, but while a lot of the programs try to program solutions that allow you to do extra customization to omit penalties, chaturbate seems to go the easy way and disable access to whitelabels to prevent penalties on their own site. Am I missing something, is this common place these days? Why would you run a whitelabel with robots.txt file set like that?