Clam

So, I get this email today from a trade partner saying "you send me a lot of bots".

How do I detect bots? How do I get rid of them so they don't get passed on to my trades?

Any help is appreciated. I'm using TE3 btw...

NickBaer

At the bottom of each of my PHP pages, I have a few lines of code that saves "hits" to a log file.

$fp = fopen('name of the file you want to save.txt', 'a');
$FWwritethis = $date("YmdHis").', '. $_SERVER['PHP_SELF'].', '.$_SERVER["REMOTE_ADDR"].', '.$_SERVER['HTTP_REFERER'].', '.$_SERVER['QUERY_STRING'].'^ '.$_SERVER['HTTP_USER_AGENT'].PHP_EOL;

fwrite($fp, $FWwritethis);
fclose($fp);

I then read that file with a script, that looks for keywords I have picked out, for "Bots". The file is TEXT, so you can also just open it and look it over! Eyeball it!

Most of my sites are getting about 55% BOT traffic!

Here's a few lines from that file: (The first entry shows that bing likes to read gfy forum posts!) You can see some of the different search engine spider/crawler names and IPs.

20160601113239, /index.php, 157.55.39.212, , q=gfy^ Mozilla/5.0 (compatible; bingbot/2.0; +Meet our crawlers) |
20160601113240, /TheBritishBeef.php, 157.55.39.212, , pg=1^ Mozilla/5.0 (compatible; bingbot/2.0; +Meet our crawlers) |
20160601113258, /TheBritishBeefBooks.php, 157.55.39.227, , ^ Mozilla/5.0 (compatible; bingbot/2.0; +Meet our crawlers) |
20160601113258, /MeetThem.php, 157.55.39.227, , pg=2^ Mozilla/5.0 (compatible; bingbot/2.0; +Meet our crawlers) |
20160601113308, /index.php, 157.55.39.122, , q=ygroup^ Mozilla/5.0 (compatible; bingbot/2.0; +Meet our crawlers) |
20160601121733, /TheBritishBeef.php, 66.249.69.162, , pg=4^ Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) |
20160601122856, /TheBritishBeefBooks.php, 66.249.69.156, , ^ Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) |
20160601131022, /TheBritishBeef.php, 66.249.69.150, , pg=2^ Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) |
20160601134745, /TheBritishBeef.php, 66.249.69.162, , pg=3^ Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) |
20160601135957, /TheBritishBeef.php, 66.249.69.162, , ^ Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) |
20160601141145, /TheBritishBeefBooks.php, 92.145.71.253, http://images.google.fr/imgres?imgur...2Fthe-british-
20160614222010, /index.php, 180.76.15.19, , ^ Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html) |
20160614222029, /index.php, 180.76.15.23, , ^ Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html) |
20160603020924, /index.php, 68.180.228.233, , ^ Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) |
20160603015205, /MeetThem.php, 5.9.73.227, , pg=1^ Mozilla/5.0 (compatible; MJ12bot/v1.4.5; http://www.majestic12.co.uk/bot.php?+) |
20160603015208, /index.php, 5.9.73.227, , q=ygroup^ Mozilla/5.0 (compatible; MJ12bot/v1.4.5; http://www.majestic12.co.uk/bot.php?+) |

__________________
Made my first porno in 1986. Sold my first VHS in 1993. Did my first live Cam in 1994. Coach-Karl.com NickBaerGallery.com I host with MojoHost.com and To Buy or Sell Sites Imagine AI Content

NickBaer

The issue of BOT traffic has been raised for years, It is a factor to consider when buying traffic. It is a factor others consider when traffic is sold. Is a Million hits a Million hits, of a half Million hits?

__________________
Made my first porno in 1986. Sold my first VHS in 1993. Did my first live Cam in 1994. Coach-Karl.com NickBaerGallery.com I host with MojoHost.com and To Buy or Sell Sites Imagine AI Content

NickBaer

If you want more info about your hits, just look at the manual page PHP: $_SERVER - Manual

__________________
Made my first porno in 1986. Sold my first VHS in 1993. Did my first live Cam in 1994. Coach-Karl.com NickBaerGallery.com I host with MojoHost.com and To Buy or Sell Sites Imagine AI Content

NickBaer

When someone buys something, like a DVD or Digital Download, I can trace his or her IP and their steps on my site, to see how long they spent, what they looked at that they bought or didn't buy, even if they visited other sites of mine.

__________________
Made my first porno in 1986. Sold my first VHS in 1993. Did my first live Cam in 1994. Coach-Karl.com NickBaerGallery.com I host with MojoHost.com and To Buy or Sell Sites Imagine AI Content