GoFuckYourself.com - Adult Webmaster Forum - Service that kills mirrors and scrapers

GoFuckYourself.com - Adult Webmaster Forum (https://gfy.com/index.php)

- Fucking Around & Business Discussion (https://gfy.com/forumdisplay.php?f=26)

- - Service that kills mirrors and scrapers (https://gfy.com/showthread.php?t=1278045)

TrafficTitan

09-01-2017 04:07 PM

Service that kills mirrors and scrapers

Is there an affordable online company that kills people who are doing complete site copies and are scraping content to displace you from google search results?

sarettah

09-01-2017 04:08 PM

Quote:

Originally Posted by TrafficTitan (Post 21982113)

Is there an affordable online company that kills people who are doing complete site copies and are scraping content to displace you from google search results?

I know a guy that says he will do that. Usually wants cash in small unmarked bills though. :helpme

.

pimpmaster9000

09-01-2017 04:10 PM

Affordable and kill is a bad idea IMO...

TrafficTitan

09-01-2017 04:15 PM

Kills the sites, not the person lol

sarettah

09-01-2017 05:39 PM

Quote:

Originally Posted by TrafficTitan (Post 21982135)

Kills the sites, not the person lol

Oh, that's a whole different story.

:pimp

.

bronco67

09-01-2017 08:18 PM

There's always going to be thieving motherfuckers and the internet just makes it 100 times easier to get away with.

Bladewire

09-01-2017 10:13 PM

Quote:

Originally Posted by TrafficTitan (Post 21982113)

Is there an affordable online company that kills people who are doing complete site copies and are scraping content to displace you from google search results?

You can protect your sites in .htaccess from scrapers. Are your sites on WP installs?

Do you want me to paste some code here for you?

Barry-xlovecam

09-02-2017 05:11 AM

It's really cat and mouse. UFW or iptables -- firewall them out -- if you have root. However, they will change IPs or AS networking so it is a never ending game.

sarettah

09-02-2017 06:31 AM

Cloudflare can protect against some of the scrapers out there but just like anything, they can be beaten too.

.

rowan

09-02-2017 06:40 AM

Quote:

Originally Posted by Barry-xlovecam (Post 21982657)

It's really cat and mouse. UFW or iptables -- firewall them out -- if you have root. However, they will change IPs or AS networking so it is a never ending game.

I have a site that's scraped to hell and back. If you exclude Googlebot and all of the scrapers, there's probably less than 2% remaining (loads by a browser).

Over the years I've added bits and pieces to log various interesting information. The big red flag that sticks out, at least for my site: scrapers use proxies, so their IPs can change without notice, but the headers they send are usually a fixed pattern that is nothing like a real browser, so they're super easy to block.

Even a simple CAPTCHA that is triggered after say 10 loads without presenting a cookie manages to block most of them. Some IPs are constantly bashing at the site, day after day, even though they are almost perpetually 403'd or firewalled.

Guess there is a market for a service like this, if one doesn't exist... but integrating it into a customer's existing site would be interesting...

Barry-xlovecam

09-02-2017 09:43 AM

Quote:

Originally Posted by sarettah (Post 21982695)

Cloudflare can protect against some of the scrapers out there but just like anything, they can be beaten too.

.

Any other questions :upsidedow

Code:



</html>barry@paragon-DS-7:/media/barry/ADATA3$ dig NS gfy.com



; <<>> DiG 9.10.3-P4-Ubuntu <<>> NS gfy.com

;; global options: +cmd

;; Got answer:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 21089

;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1



;; OPT PSEUDOSECTION:

; EDNS: version: 0, flags:; udp: 512

;; QUESTION SECTION:

;gfy.com.                        IN        NS



;; ANSWER SECTION:

gfy.com.                86399        IN        NS        june.ns.cloudflare.com.

gfy.com.                86399        IN        NS        theo.ns.cloudflare.com.



;; Query time: 56 msec

;; SERVER: 127.0.0.1#53(127.0.0.1)

;; WHEN: Sat Sep 02 12:42:15 EDT 2017

;; MSG SIZE  rcvd: 88

=======================



barry@paragon-DS-7:/media/barry/ADATA3$ curl -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 (Official Build) (64-bit)" "https://gfy.com/"

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" dir="ltr" lang="en">

<head>

        <!-- no cache headers -->

        <meta http-equiv="Pragma" content="no-cache" />

        <meta http-equiv="Expires" content="-1" />

        <meta http-equiv="Cache-Control" content="no-cache" />

        <!-- end no cache headers -->



         <meta name="robots" content="noarchive">

        

        <title>GFY Webmaster Board</title>

freecartoonporn

09-02-2017 10:36 AM

Quote:

Originally Posted by Barry-xlovecam (Post 21982853)

Any other questions :upsidedow

Code:



</html>barry@paragon-DS-7:/media/barry/ADATA3$ dig NS gfy.com



; <<>> DiG 9.10.3-P4-Ubuntu <<>> NS gfy.com

;; global options: +cmd

;; Got answer:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 21089

;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1



;; OPT PSEUDOSECTION:

; EDNS: version: 0, flags:; udp: 512

;; QUESTION SECTION:

;gfy.com.                        IN        NS



;; ANSWER SECTION:

gfy.com.                86399        IN        NS        june.ns.cloudflare.com.

gfy.com.                86399        IN        NS        theo.ns.cloudflare.com.



;; Query time: 56 msec

;; SERVER: 127.0.0.1#53(127.0.0.1)

;; WHEN: Sat Sep 02 12:42:15 EDT 2017

;; MSG SIZE  rcvd: 88

=======================



barry@paragon-DS-7:/media/barry/ADATA3$ curl -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 (Official Build) (64-bit)" "https://gfy.com/"

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" dir="ltr" lang="en">

<head>

        <!-- no cache headers -->

        <meta http-equiv="Pragma" content="no-cache" />

        <meta http-equiv="Expires" content="-1" />

        <meta http-equiv="Cache-Control" content="no-cache" />

        <!-- end no cache headers -->



         <meta name="robots" content="noarchive">

        

        <title>GFY Webmaster Board</title>

what are you trying to show ?

Barry-xlovecam

09-02-2017 10:39 AM

That cloudflare is a submarine with screen doors?

Bladewire

09-02-2017 02:46 PM

They need to tweak their cloudflare setting is all :2 cents:

Not sure if they changed their old IP afternoon cloudflare, as I suggested before, if not they're still wide open.

Edit: nope they didn't change it and they have 64 hostnames pointing to that same IP :/

Step 10 here - change IP after setup

Barry-xlovecam

09-02-2017 03:05 PM

GFY IS using a Cloudflare IP

Luckily botnets are made of stupid computer users. Maybe, Cloudflare can control ddos spikes in traffic.

Cloudflare will not stop lone wolf hackers or scrapers. I just repeated that curl successfully from a Digital Ocean server I have in Amsterdam. So, the AS block is not checked at Cloudflare either.

Bladewire

09-02-2017 03:12 PM

Quote:

Originally Posted by Barry-xlovecam (Post 21983150)

You dope.

The GFY server IP is the same as when it was first attacked.

Then GFY hid behind cloudflare, with the same IP, and continued to go down.

So clearly the only reason the attack stopped was because the attacker stopped hitting the original IP.

I pay for cloudflare business , and use cloudflare. And I looked up GFY's IP.

I just PM'd it to you

freecartoonporn

09-04-2017 04:22 AM

Quote:

Originally Posted by Barry-xlovecam (Post 21982853)

Any other questions :upsidedow

Code:



</html>barry@paragon-DS-7:/media/barry/ADATA3$ dig NS gfy.com



; <<>> DiG 9.10.3-P4-Ubuntu <<>> NS gfy.com

;; global options: +cmd

;; Got answer:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 21089

;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1



;; OPT PSEUDOSECTION:

; EDNS: version: 0, flags:; udp: 512

;; QUESTION SECTION:

;gfy.com.                        IN        NS



;; ANSWER SECTION:

gfy.com.                86399        IN        NS        june.ns.cloudflare.com.

gfy.com.                86399        IN        NS        theo.ns.cloudflare.com.



;; Query time: 56 msec

;; SERVER: 127.0.0.1#53(127.0.0.1)

;; WHEN: Sat Sep 02 12:42:15 EDT 2017

;; MSG SIZE  rcvd: 88

=======================



barry@paragon-DS-7:/media/barry/ADATA3$ curl -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 (Official Build) (64-bit)" "https://gfy.com/"

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" dir="ltr" lang="en">

<head>

        <!-- no cache headers -->

        <meta http-equiv="Pragma" content="no-cache" />

        <meta http-equiv="Expires" content="-1" />

        <meta http-equiv="Cache-Control" content="no-cache" />

        <!-- end no cache headers -->



         <meta name="robots" content="noarchive">

        

        <title>GFY Webmaster Board</title>

i still cant figure out gfy ip from that, what am i missing ?

thanks

Barry-xlovecam

09-04-2017 04:47 AM

BW I don't give a fuck how ...
PM Eric and tell what DNS server that needs to have its records fixed.

All times are GMT -7. The time now is 04:16 AM.