Welcome to the GoFuckYourself.com - Adult Webmaster Forum forums.

You are currently viewing our boards as a guest which gives you limited access to view most discussions and access our other features. By joining our free community you will have access to post topics, communicate privately with other members (PM), respond to polls, upload content and access many other special features. Registration is fast, simple and absolutely free so please, join our community today!

If you have any problems with the registration process or your account login, please contact us.

Post New Thread Reply

Register GFY Rules Calendar
Go Back   GoFuckYourself.com - Adult Webmaster Forum > >
Warning with this technique (Google) Warning with this technique (Google)
Discuss what's fucking going on, and which programs are best and worst. One-time "program" announcements from "established" webmasters are allowed.

 
Thread Tools
Old 01-04-2012, 03:24 PM   #1
roganoli
Confirmed User
 
roganoli's Avatar
 
Join Date: Dec 2004
Posts: 757
:mad Warning with this technique (Google)
I'm still an inexperienced webmaster...

I was having good placements on Google for my new site... He was even making good sales!

Suddenly something happened. My website disappeared from search! Today I found out what caused it... My site was totally copied by the proxy service Google! See the link below:

https://www.google.com/#q=%22appspot...1280& bih=663

I believe this was done intentionally by another webmaster and I believe I have solved this problem using my htaccess!

But be warned for other amateurs like me!
roganoli is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 01-04-2012, 03:29 PM   #2
Klen
 
Klen's Avatar
 
Industry Role:
Join Date: Aug 2006
Location: Little Vienna
Posts: 32,235
Had same happening to me,totally copied site to three different domains.Which is why i have now in my settings protection against all kind of downloaders.
Klen is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 01-04-2012, 03:31 PM   #3
roganoli
Confirmed User
 
roganoli's Avatar
 
Join Date: Dec 2004
Posts: 757
I put it on my htaccess:

RewriteCond %{HTTP_USER_AGENT} AppEngine [NC]
RewriteRule .* - [F]

Let me know if there is better solution...
roganoli is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 01-04-2012, 03:32 PM   #4
baryl
Confirmed User
 
Industry Role:
Join Date: Aug 2011
Location: Las Vegas
Posts: 1,086
Wait..what? Can someone explain what this is?
baryl is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 01-04-2012, 03:34 PM   #5
Klen
 
Klen's Avatar
 
Industry Role:
Join Date: Aug 2006
Location: Little Vienna
Posts: 32,235
Search for article like this:
http://www.htaccess-guide.com/blocki...-and-bad-bots/
Had one with huge list of those downloaders,but dont know what was exact url.
Klen is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 01-04-2012, 03:35 PM   #6
roganoli
Confirmed User
 
roganoli's Avatar
 
Join Date: Dec 2004
Posts: 757
Quote:
Originally Posted by baryl View Post
Wait..what? Can someone explain what this is?
Sorry if my English is not helpful...
roganoli is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 01-04-2012, 03:39 PM   #7
baryl
Confirmed User
 
Industry Role:
Join Date: Aug 2011
Location: Las Vegas
Posts: 1,086
It's fine. Thanks for pointing that out and sorry about your site.
baryl is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 01-04-2012, 04:13 PM   #8
RayBonga
too cool for highschool
 
RayBonga's Avatar
 
Join Date: Nov 2005
Location: East side, West side, Worldwide!
Posts: 12,164
Quote:
Originally Posted by roganoli View Post
I put it on my htaccess:

RewriteCond %{HTTP_USER_AGENT} AppEngine [NC]
RewriteRule .* - [F]

Let me know if there is better solution...
Thanks for the warning What does this code do?
RayBonga is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 01-04-2012, 04:37 PM   #9
brassmonkey
Pay It Forward
 
brassmonkey's Avatar
 
Industry Role:
Join Date: Sep 2005
Location: Yo Mama House
Posts: 77,090
damn sorry to hear that
__________________
TRUMP 2025 KEKAW!!! - The Laken Riley Act Is Law!
DACA ENDED - SUPPORT AZ HCR 2060 52R - email: brassballz-at-techie.com
brassmonkey is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 01-04-2012, 04:44 PM   #10
redwhiteandblue
Bollocks
 
redwhiteandblue's Avatar
 
Industry Role:
Join Date: Jun 2007
Location: Bollocks
Posts: 2,792
Quote:
Originally Posted by RayBonga View Post
Thanks for the warning What does this code do?
It blocks access to any client which presents a user agent with the string "AppEngine" in it. But there are a whole load of others you need to block too, and that won't even stop all scrapers. I'd love to know the UA updowner.com is using and block that fucker.
redwhiteandblue is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 01-04-2012, 05:47 PM   #11
medicatedghost
Confirmed User
 
medicatedghost's Avatar
 
Industry Role:
Join Date: Jan 2012
Location: Los Angeles southbay
Posts: 209
check out google.com/safebrowsing/diagnostic?site=updowner.com/
interesting analysis
__________________
Just a lowly webmaster trying to start (over) and if you could see to it that I got a fair shake from your algorithms - I'd be VERY happy to sign up with you and work my ass off producing pages that sell your product...........i can make a living with only a portion getting shaved off of my checks
google voice: (four-two-four) 210-7676
gmail: harry90717 @
twitter: uses gmail account
ICQ: 643932783
medicatedghost is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 01-04-2012, 06:01 PM   #12
jakeuk
Confirmed User
 
jakeuk's Avatar
 
Industry Role:
Join Date: Jul 2009
Location: London, UK
Posts: 125
The mod rewrite you suggested offer some protection. When I had similar problem a few years ago, here are other things I did in addition to the mod_rewrite:

1. Get strong links to your site so that your site is at least stronger than the copycat.
2. Rewrite some of the contents of your site so that its not a duplicate any more but a different version of the one that was copied.
3. Update you site regularly so that the copycat cannot keep up.
4. use a more aggressive exclusion robots.txt file, allowing ONLY the big 3 crawlers.
jakeuk is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 01-04-2012, 06:31 PM   #13
rowan
Too lazy to set a custom title
 
Join Date: Mar 2002
Location: Australia
Posts: 17,393
Quote:
Originally Posted by redwhiteandblue View Post
I'd love to know the UA updowner.com is using and block that fucker.
Any half decent scraper probably uses a common UA, and possibly a random selection of common UAs. I've seen loads of multiple pages from the same IP use a different UA each time.
rowan is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Post New Thread Reply
Go Back   GoFuckYourself.com - Adult Webmaster Forum > >
Warning with this technique (Google) Warning with this technique (Google)
« Previous Thread | Next Thread »

Bookmarks



Advertising inquiries - marketing at gfy dot com

Contact Admin - Advertise - GFY Rules - Top

©2000-, AI Media Network Inc



Powered by vBulletin
Copyright © 2000- Jelsoft Enterprises Limited.