So, in the meantime... I don't know if I made a crap...
But I did this thing...
I searched with grep some bots in the acces.log
Code:
grep ahrefs/var/log/apache2/access.log
Then I copied a few thousand lines and I created a PHP script that creates a file with only the IP, line by line (leaving the duplicates)
Code:
<?
$my_database_txt = 'seznambot.txt';
$array_righi = file($my_database_txt);
foreach($array_righi as $key => $capi){
list($ip, $merda) = explode(" - - ", $capi);
if(strpos($ip_list, $ip) === false){
$ip_list = $ip_list.$ip.'
';
}
}
$nome_file_index = "ip_list.txt";
//chmod($nome_file_index, 0666);
$file = fopen($nome_file_index, "w") ;
fwrite($file, $ip_list);
fclose($file);
?>
From all the games for now came out this file
http://porn-update.com/temp/bad-bot-cidr.txt
Then with the usual while I added the rules to UFW
Code:
while read line; do sudo ufw insert 1 deny from $line to any; done < /var/www/html/bad-bot-cidr.txt
If I did some crap I should be able to go back with this:
Code:
while read line; do sudo ufw delete deny from $line; done < /var/www/html/bad-bot-cidr.txt
Now it seems that something is happening, in the access.log I see a lot of Google and Bing, although for Bing I added crawl-delay: 1 in robots.txt and I also set to 1 the scan frequency on Bing webmaster tools for all My sites.
The CPU graph goes up and down at the moment, but that of load is slowly descending.
I wait a little, and I see what happens...