2

I look after a website that is currently being flooded with HTTP requests from different IP addresses but all using the same user agent string

I've grepped the log file and now have a file with all of the 2696 unique IP addresses per line. It looks every 20 or so IPs belong to the same IP range i.e the first, second and sometimes third octet are the same

Is there a way I can group these IPs together so I can see a list of each IP range? Either some sort of command line wizardry or a tool I can just drop the IPs in and get the IP ranges as output?

Edit:

For example, 24 of the IP addresses start with 104.140 and there are three different third octets. 8 IPs start with 104.140.183, another 8 start with 104.140.211 and the final 8 start with 104.140.4

My firewall manager lets me block IP ranges with network prefix length and allows me to choose from /24 to /32

I don't know enough about subnetting to know what the output should be, but I'd like to A) see how many different ranges there are and B) ideally have the output in a format I can use to block the ranges at the firewall

4
  • An example of some actual lines and the result that you want could help
    – jcbermu
    Mar 4 '15 at 9:52
  • There are multiple options and places where this can be resolved. fail2banis one of those options. Could you tell us a bit more about your setup (eg. which OS, webserver and firewall)? Also: should we provide a working answer for Windows, Linux commandline, ... Which tools do you know and use? What have you tried yourself so far? Try reading into awk. If you find a solution, please answer your oen question.
    – agtoever
    Mar 4 '15 at 11:51
  • @agtoever The OS reports itself as "SLES Expanded Support platform release 6.6" running Apache 2.2.15 which uses nginx as a reverse proxy server. The firewall is a "Cisco ASA 5505 Sec+" device
    – Andy
    Mar 4 '15 at 12:08
  • Did you say 104.140.211? Well, ARIN results for 104.140.211 has info on who the addresses are assigned to, and a comment which happens to include a request that you report abuse (and URLs for doing so). The "Search WhoisRWS" box, in the upper-right corner of that page, is something you can use to check other IP addresses.
    – TOOGAM
    Jan 5 '17 at 6:14
5

Allright, I think this might work for you - assuming you have awk installed and the ip addresses are in ip.txt:

cat ip.txt | awk 'BEGIN { FS = "." } ; { printf("%s.%s.%s\n%s.%s.%s.%s\n", $1, $2, $3, $1, $2, $3, $4) }' | sort | uniq -c | sort -rn

A little explanation:

  • cat prints the file
  • awk splits the ip addresses in 4 variables and prints both the /24 and the /32 ip range
  • sort sorts the ip addresses, so we can count them
  • uniq counts them (thanks to the -c option) and puts the frequency in front of the ip address
  • sort puts them in descending order (-r) of frequency count (-n interprets values instead of string)

Note that the /32 as well as the /24 ranges are shown. If you just want the /32 of the /24 range, adjust the printf statement (so printf("%s.%s.%sn", $1, $2, $3) only prints /24 and printf("%s.%s.%s.%s\n", $1, $2, $3, $4) prints only /32.

1
  • Great stuff. Based on the output from that command I've blocked 24 different ranges and it appears to have completely stopped the HTTP requests coming in from all the IPs :)
    – Andy
    Mar 4 '15 at 16:01

Not the answer you're looking for? Browse other questions tagged or ask your own question.