Fail2ban jail to mitigate DoS attacks against Apache
Requirements
What we would like to accomplish:
- Scanning of all Apache access logs
- Ban the attackers IP if there are more than 300
GETrequests during a time span of 5 mins resulting in HTTP non-200 (OK) status codes: 401, 403, 404, 503 - Ban the attacker for 1 hour
We’re going to scan all customer Apache access.log logfiles which are in a slightly tuned (non-standard) combined log format:
#LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t %I %O \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %V %p" combinedBelow failregex also works for default extended/combined or common Apache LogFormat.
Setup Fail2Ban jail
Let’s call the new Fail2Ban jail apache-get-dos and create the following two files:
[apache-get-dos]
enabled = true
port = http,https
filter = apache-get-dos
logpath = /var/www/*/logs/access.log
datepattern = %%d/%%b/%%Y:%%H:%%M:%%S %%z
maxretry = 300
findtime = 5m
bantime = 1h
In above jail, we didn’t specify a
banaction, as we’re using the global default. You might want to put the following injail.d/custom.conf:jail.d/custom.conf[DEFAULT] bantime = 300 findtime = 300 banaction = iptables-allports
# Fail2Ban filter to scan Apache access.log for DoS attacks
[INCLUDES]
before = common.conf
[Definition]
# Option: failregex
# Notes.: regex to match GET requests in the logfile resulting in one of the
# following status codes: 401, 403, 404, 503.
# The host must be matched by a group named "host". The tag "<HOST>"
# can be used for standard IP/hostname matching and is only an alias for
# (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
# Values: TEXT
failregex = ^<HOST> .*"GET (?!\/robots\.txt).*" (401|403|404|503)\s
# Option: ignoreregex
# Notes.: regex to ignore. If this regex matches, the line is ignored.
# Values: TEXT
#
ignoreregex =
You might want to tune the failregex to your needs. For our use case, I have created a regex101 with some sample log lines which you can play around with. It will ignore any failing GET /robots.txt requests (as most bots request this) and it will not care about “sane” HTTP status codes like 200 (OK), 301 (Moved Permanently), 302 (Found). So, please be aware, that this jail only protects you from some kind of DoS attack that targets a large amount of non-existing URLs/pathes, which is mostly the case for malicious vulnerability scans.
Monitoring jail
After having restarted Fail2Ban, you might want to monitor the new apache-get-dos jail:
# list status of apache-get-dos jail with monitored Apache logs
$ fail2ban-client status apache-get-dos
Status for the jail: apache-get-dos
|- Filter
| |- Currently failed: 80
| |- Total failed: 290517
| `- File list: /var/www/<user>/logs/access.log ...
`- Actions
|- Currently banned: 0
|- Total banned: 5
`- Banned IP list:
# review banning in fail2ban.log
$ grep -Pi 'apache-get-dos\] (un)?ban' /var/log/fail2ban.log
2021-09-14 17:45:47,285 fail2ban.actions [107006]: NOTICE [apache-get-dos] Ban 1.2.3.4
2021-09-14 18:45:45,503 fail2ban.actions [107006]: NOTICE [apache-get-dos] Unban 1.2.3.4
In case you have designed your failregex too aggressively and have banned too many IPs, remember those commands:
# unban an IP from a specific jail
$ fail2ban-client set <JAIL> unbanip <IP>
# unban an IP (or several) from all jails
$ fail2ban-client unban <IP> ... <IP>
# unbans all IP addresses (in all jails and database)
$ fail2ban-client unban --all
Bonus Commands
I’ll provide you with some bonus commands to further investigate such DoS attacks:
# Top 20 list of IPs with a large amount of connections to the server
$ netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -nr | head -n20
You may also want to scan all Apache access logs for the IPs with most requests:
# Top 10 IP list for ALL sites for previous hour
$ grep -h "\[$(date -d -1hour +'%d/%b/%Y:%H:')" /var/www/*/logs/access.log | cut -d' ' -f1 | sort | uniq -c | sort -nr | head -n10
# Top 10 IP list for ALL sites for current hour
$ grep -h "\[$(date +'%d/%b/%Y:%H:')" /var/www/*/logs/access.log | cut -d' ' -f1 | sort | uniq -c | sort -nr | head -n10
# Top 10 IP list for ALL sites with filename
$ grep "\[$(date +'%d/%b/%Y:%H:')" /var/www/*/logs/access.log | cut -d' ' -f1 | sort | uniq -c | sort -nr | head -n10
I recommend you block such IPs directly on your front-end firewall. But in case you can’t, use iptables to quickly block a single IP:
# Block a single IP
$ iptables -A INPUT -s <IP> -j DROP
# Unblock it
$ iptables -D INPUT -s <IP> -j DROP
And now, cross fingers you won’t get hit by a brutal DDoS. I plan to cover DDoS attack mitigation in a future article.
source : https://pipo.blog/articles/20210915-fail2ban-apache-dos
Posted on: March 8, 2025, by : Julian's | 4 views