How to setup Fail2ban to detect Apache 404 attacks?

Now, let’s see how our Dedicated Engineers setup Fail2ban to block 404 scans and invalid request methods.

1) Create filter

Firstly, our Support Engineers create a filter in the location /etc/fail2ban/filter.d. Further, we add a set of rules to ban IPs that cause 404 errors.

For example, to monitor the Apache 404 requests, we create a filter file apache-404.conf in the location /etc/fail2ban/filter.d. The filter looks like this.
failregex = ^<HOST> - .* "(GET|POST|HEAD).*HTTP.*" 404 .*$
ignoreregex =.*(robots.txt|favicon.ico|jpg|png)
We define the regular expression to be matched under the failregex parameter. Here, the above regular expression identifies the IP address that is making too many 404 requests. And, the ignoreregex excludes the valid files such as robots.txt, favicon.io and the images.

2) Create a custom jail

Secondly, we add a new jail in the location /etc/fail2ban/jail.conf. This defines the Apache log path to be checked, maxretry, bantime etc.

For example, to create a custom jail that monitors the Apache 404 requests, we add the following code in the file /etc/fail2ban/jail.conf.
[apache-404]
enabled = true
port = http,https
filter = apache-404
logpath = /var/log/httpd/error_log
logpath = /var/log/httpd/access_log
bantime = 3600
findtime = 600
maxretry = 5
Here, we update the apache log file under logpath parameter. Similarly, the bantime, species how many seconds an offending IP is banned for. We always set this value to an optimum level, so that it’s not short to affect the legitimate users, while not long enough favoring malicious users.

Further, the maxretry parameter specifies the total number of connection attempts. So, if a client makes retry attempts more than maxretry value within the time specified in findtime parameter, they will be banned.