Automating application response to hacking attack without AI or ML!
A few years ago I was working on a project that had me write around 70,000 lines of rails and javascript code. Naturally as a security person, as I worked on the application, I tried to consider how attackers would attempt to find flaws or vulnerabilities and either break my application or compromise it's platform, data or trust.
In the end I found that there were 2 application security controls I could do to make it extremely difficult to break the application and the platform it runs on. This included doing continuous regular upgrades to the libraries and components that my application depended on and automated monitoring and blocking of application events.
I will probably post soon my thoughts on the toil it takes to monitor and keep up to date all of the building blocks and dependencies of any application. That will be a long discussion on upstream and operating system updates that are out of your control and tied to breaking changes that require continuous vigilance and sometimes commercial support.
The second application security control is what I want to focus on in this post, something much more fun! Automated monitoring of potentially hostile actions against my application.
Attacking an applications is very noisy and can take a long time to find potential flaws to exploit. A common method of "scanning" an application uses what is called fuzzing. This is when application variables through urls, cookies, form submissions, or javascript api calls are used by application attackers to submit hundreds if not thousands of different attacks to see if any of the processing applications or APIs can be broken with malicious variable data.
Common best practice in application development to protect against application attacks includes protecting variables using input validation. Input validation is a process to restrict application variables to specifically controlled data types, such as only numbers or matching a regular expression or only 4 characters long.
Rails applications are model based and each model maps to a database table. Each column in the database corresponds to the model variable, each of which can have validations that are "validated" before any new data is written to the database. If any validation fails the entire data posted is rejected and users are generally notified.
An example of model validations can be seen below:
validates :name, :presence => true, :uniqueness => true, :length => {:minimum => 3, :maximum => 23, :message => "can only be 4-23 characters long"},
:format => { :with => /\A[\w\-]+\z/, :message => "can only contain letters, numbers and -" }
validates :homepage, url: true
validates :email, :presence => true, :uniqueness => true, :length => {:minimum => 4, :maximum => 50, :message => "can only be 4-50 characters long"},
:format => { :with => /\A[[:space:]\w\-\@\.]+\z/, :message => "can only contain letters, numbers and - _ @ ." }
validates :api_key, allow_blank: true, :uniqueness => true
validates :api_key, :length => {:minimum => 22, :maximum => 22, :message => "can only be 22 characters long"},
:format => { :with => /\A[\w\_\-]+\z/, :message => "can only contain letters, numbers and _-" },
:if => Proc.new { |m| !m.api_key.nil? && !m.api_key.empty? }
Many developers have for many years developed applications using techniques like input validation. However, one critical user of this validation information is often overlooked. Developers know that they work with the business and platform team to run their applications and often focus on application logs and metrics to ensure up-time and availability of the business application.
However, it is often overlooked that they are also continuously partnered with their security team. Metrics like input validation failures or authentication failures are critical to monitoring the security of the application and critical for the Security Operations Center (SOC) responsible for monitoring business security.
One challenge that security teams have when trying to monitor all of the custom code made by not only enterprise development teams, but also opensource or commercial partners and all of the many applications and components that make up a modern application platform is that every system logs differently!
To get ahead of that problem in the application I was working on, I created a simple library that crafted a standard message format that I could use throughout the application for custom messages put out in a standard format so that any service monitoring the logs could use simple pattern matching and standardize how the application was monitored.
With a standard process for logging across the application I followed a process I found in an old stackoverflow post that took validation errors and send them to the logger(shout out to all other stackoverflow hackers!!).
# https://stackoverflow.com/questions/4678851/rails-log-model-errors-on-failed-save
module ValidationLogger
def self.included(base)
base.send :include, InstanceMethods
base.after_validation :log_errors, if: ->(m) { m.errors.present? }
end
module InstanceMethods
def log_errors
if (defined? remote_ip && !remote_ip.nil?)
Rails.logger.warn " #{remote_ip} :: Failed Validation : #{self.class.name} #{self.id || '(new)'} is invalid: #{self.errors.full_messages.inspect}"
else
Rails.logger.warn " Failed Validation : #{self.class.name} #{self.id || '(new)'} is invalid: #{self.errors.full_messages.inspect}"
end
end
end
end
Since this was all built pre-kubernetes I used a simple tool called fail2ban to monitor the logs for patterns of attacks I wanted to block and then edit a file with the IPs I wanted to block.
Using a simple filter fail2ban is able to run ban and unban scripts. With the input validation error logging I was able to create the following filter definition:
# W, [2023-12-22T20:48:19.304507 #12] WARN -- : [7c9bc9a2-1f35-489d-b8a4-78f84441c537] [testing] [10.200.0.46] [192.168.1.51] [2023-12-22 20:48:19 +0000] 10.200.0.46 :: Failed Validation : Rfq (new) is invalid: ["Title can only contain letters, numbers and -_!."]
failregex = ^W, \[.*\] WARN -- : \[.*\] \[.*\] \[.*\] \[<HOST>] \[.*\].*:: Failed Validation .*$
This would capture the standard validation error and the offending host IP which fail2ban will use to run ban and unban actions on the application.
Many users of fail2ban just use iptables and block the offending IP; but I didn't want to stop them, but instead direct them to a page that said they had a problem and were blocked for a short time with support contact info. I did this because false positives happen and I did not want to block a real customer without a way for them to get support!
To get a full walk through of what was built all those years ago you can watch my BSides Charleston Presentation on it: BSidesCHS 2014: When a Security Architect Writes an Application
Fast forward a few years and now I am working with Kubernetes and other container tools like Pivotal Cloud Foundry (now Tanzu Application Service) and in keeping with the spirit of security I decide to not only make sure my code is all containerized but also the security monitoring I had was fully integrated into the new kubernetes platform.
As things were originally designed, they would not work in kubernetes. Mainly because now there were multiple application processes running in multiple pods and fail2ban isn't able to easily monitor easily nor modify the access control file that was originally used.
The solution I came up with was to leverage kubernetes native logging tools in fluent-bit, nfs for centralized logging and a change from the block file to an API call within the application.
The first step was to update the application with an API to add and remove an IP to it's blocklist. The application already uses memcache to cache some database lookups and page renders, so it was also utilized with the blocklist to allow faster response if an attacker has the system under some load.
Making the application able to quickly identify connections from offending IPs and direct the response to a block-page was a simple before_action cache lookup on the ApplicationController:
class ApplicationController < ActionController::Base
before_action :check_banned_ip
def check_banned_ip
if !request.nil? && !request.remote_ip.nil?
if BlockingIp.lookup_ip(request.remote_ip)
flash[:alert] = "Your IP has logged too many invalid requsts and has been temporarily blocked from #{@current_tenant.name}. If this was a mistake or error please contact support using the chat or one of our contact links below."
respond_to do |type|
type.html {
if params[:controller] == 'home' && params[:action] == 'index'
render "/home/index", :status => 599, layout: tenant_layout
else
redirect_to root_path
end
}
type.all { render :nothing => true, :status => 599 }
end
return false
end
end
end
With the application now able to respond to it's updated blocklist all that was needed was a way to take the application logs and share them with a fail2ban process to process them.
Using fluent-bit I was able to attach an NFS persistent storage to the deployment and configure for all application pods to log to the nfs mounted partition. In kubernetes NFS is only able to be in a single namespace at a time; so whichever namespace fluent-bit is setup in is also where fail2ban needs to be configured.
I am using Tanzu Kubernetes Grid Integrated in my home lab and Linode for the production cluster. TKGi comes with fluent-bit installed in the pks-system namespace where I manually edited the persistent volume and deployment settings. The following examples are for my linode cluster where fluent-bit and fail2ban are installed in the default namespace.
I installed fluent-bit in linode with a simple helm install:
领英推荐
helm repo add fluent https://fluent.github.io/helm-charts
helm upgrade --install fluent-bit fluent/fluent-bit --values fluentbit-values.yaml
The fluentbit-values.yaml contains the NFS volume mount which would need to be created in advance. The PVC I used is a simple 2Gi NFS storage class:
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: fluentbit-pvc
namespace: default
labels:
used-for: application-log-storage
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 2Gi
storageClassName: nfs
The fluentbit-values.yaml file then references that volume and sends all logs for the containers and tags used for my application:
extraVolumes:
- name: fluentbit-volume
persistentVolumeClaim:
claimName: fluentbit-pvc
extraVolumeMounts:
- name: fluentbit-volume
mountPath: /infosecquote-logs
input:
tail:
memBufLimit: 5MB
parser: docker
path: /var/log/containers/*infosecquote*.log
tag: infosecquote.*
ignore_older: ""
dockerMode: false
dockerModeFlush: 4
exclude_path: ""
Now all that remains is to get fail2ban running and monitoring that log file to block and unblock using the new application API.
There is already a fail2ban container available, so all that was needed was to create a configmap with all of my filters, actions and jails and a deployment for fail2ban to monitor application errors and send ban and unban commands to the application API when attack thresholds are met!
A few things I changed in the updated kubernetes fail2ban:
fail2ban-configmap.yaml:
---
apiVersion: v1
kind: ConfigMap
metadata:
name: fail2ban-configmap
namespace: default
data:
TZ: "EST"
F2B_LOG_TARGET: "STDOUT"
F2B_LOG_LEVEL: "INFO"
F2B_DB_PURGE_AGE: "1d"
SSMTP_HOST: "mail.yourserver.com"
SSMTP_PORT: "587"
SSMTP_HOSTNAME: "mail.yourserver.com"
SSMTP_USER: "[email protected]"
SSMTP_PASSWORD: "smtppassword"
SSMTP_TLS: "YES"
APIKEY: keyiusedinmyapi
---
apiVersion: v1
kind: ConfigMap
metadata:
name: infosecquote-webhook-header-conf
namespace: default
data:
infosecquote-webhook-header.conf: |
X-InfoSecQuote-apikey: keyiusedinmyapi
---
apiVersion: v1
kind: ConfigMap
metadata:
name: fail2ban-local
namespace: default
data:
fail2ban.local: |
[DEFAULT]
dbfile = /infosecquote-logs/fail2ban.sqlite3
---
apiVersion: v1
kind: ConfigMap
metadata:
name: infosecquote-auth-conf
namespace: default
data:
infosecquote-auth.conf: |
[INCLUDES]
before = common.conf
# fail2ban filter configuration for infosecquote failed login
[Definition]
# W, [2023-12-22T20:27:29.659141 #12] WARN -- : [bf4eb0fc-6dba-4baf-b0b9-5e2db933848e] [testing] [174.219.15.85] [174.219.15.85] [2023-12-22 20:27:29 +0000] 174.219.15.85 :: Failed login : {"email"=>"", "password_type"=>"[FILTERED]", "password"=>"[FILTERED]", "yubiotp"=>"[FILTERED]"} ::
failregex = ^W, \[.*\] WARN -- : \[.*\] \[.*\] \[.*\] \[<HOST>] \[.*\].*:: Failed login .*$
ignoreregex =
---
apiVersion: v1
kind: ConfigMap
metadata:
name: infosecquote-validation-conf
namespace: default
data:
infosecquote-validation.conf: |
[INCLUDES]
before = common.conf
# fail2ban filter configuration for Input Validation
[Definition]
# W, [2023-12-22T20:48:19.304507 #12] WARN -- : [7c9bc9a2-1f35-489d-b8a4-78f84441c537] [testing] [10.200.0.46] [192.168.1.51] [2023-12-22 20:48:19 +0000] 10.200.0.46 :: Failed Validation : Rfq (new) is invalid: ["Title can only contain letters, numbers and -_!."]
failregex = ^W, \[.*\] WARN -- : \[.*\] \[.*\] \[.*\] \[<HOST>] \[.*\].*:: Failed Validation .*$
ignoreregex =
---
apiVersion: v1
kind: ConfigMap
metadata:
name: infosecquote-block-conf
namespace: default
data:
infosecquote-block.conf: |
# Fail2Ban configuration file
# Author: David M. Zendzian
[Definition]
actionstart =
actionstop =
actioncheck =
actionban = IP=<ip>
curl -d "ipaddr=$IP" -H 'X-InfoSecQuote-Event: block_ip' -H @/data/infosecquote-webhook-header.conf -X POST https://www.infosecquote.com/webhooks
actionunban = IP=<ip>
curl -d "ipaddr=$IP" -H 'X-InfoSecQuote-Event: unblock_ip' -H @/data/infosecquote-webhook-header.conf -X POST https://www.infosecquote.com/webhooks
[Init]
---
apiVersion: v1
kind: ConfigMap
metadata:
name: infosecquote-jail-conf
namespace: default
data:
infosecquote-jail.conf: |
[infosecquote-auth]
enabled = true
filter = infosecquote-auth
logpath = /infosecquote-logs/infosecquote-app.log
backend = polling
action = infosecquote-block
maxretry = 10
findtime = 300
bantime = 300 ; 5 minutes
[infosecquote-validation]
enabled = true
filter = infosecquote-validation
logpath = /infosecquote-logs/infosecquote-app.log
backend = polling
action = infosecquote-block
maxretry = 25
findtime = 60
bantime = 900 ; 15 minutes
The jails are configured to block after 25 failed validation errors in one minute or 10 authentication errors in 5 minutes. Anything scanning will hit that threshold very quick and it should be enough to prevent end-user errors for common validation problems.
Please let me know what number of events in how many minutes and what type of events you would add to this list to monitor!
To get fail2ban running I took the latest image available for fail2ban and created a deployment that loaded the above configmap into configuration files for fail2ban to monitor the fluent-bit log and send ban and unban commands to the application API.
The "external-egress" label in the deployment configuration is tied to my application network policies which block not only ingress but also egress for all pods in the cluster. This label allows the fail2ban access to egress and routing to the application api endpoint for banning and unbanning IP addresses.
apiVersion: apps/v1
kind: Deployment
metadata:
name: fail2ban
namespace: default
labels:
app: fail2ban
spec:
revisionHistoryLimit: 2
replicas: 1
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
selector:
matchLabels:
app: fail2ban
template:
metadata:
labels:
app: fail2ban
external-egress: "true"
annotations:
sidecar.istio.io/inject: "false"
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- fail2ban
topologyKey: "kubernetes.io/hostname"
containers:
- name: fail2ban
image: crazymax/fail2ban:latest
imagePullPolicy: "Always"
resources:
limits:
memory: "450Mi"
envFrom:
- configMapRef:
name: fail2ban-configmap
volumeMounts:
- name: fail2ban-local
mountPath: /etc/fail2ban/fail2ban.local
subPath: fail2ban.local
- name: infosecquote-auth-conf
mountPath: /data/filter.d/infosecquote-auth.conf
subPath: infosecquote-auth.conf
- name: infosecquote-validation-conf
mountPath: /data/filter.d/infosecquote-validation.conf
subPath: infosecquote-validation.conf
- name: infosecquote-block-conf
mountPath: /data/action.d/infosecquote-block.conf
subPath: infosecquote-block.conf
- name: infosecquote-jail-conf
mountPath: /data/jail.d/infosecquote-jail.conf
subPath: infosecquote-jail.conf
- name: infosecquote-webhook-header-conf
mountPath: /data/infosecquote-webhook-header.conf
subPath: infosecquote-webhook-header.conf
- mountPath: /infosecquote-logs
name: fluentbit-volume
volumes:
- name: fail2ban-local
configMap:
name: fail2ban-local
- name: infosecquote-webhook-header-conf
configMap:
name: infosecquote-webhook-header-conf
- name: infosecquote-auth-conf
configMap:
name: infosecquote-auth-conf
- name: infosecquote-validation-conf
configMap:
name: infosecquote-validation-conf
- name: infosecquote-block-conf
configMap:
name: infosecquote-block-conf
- name: infosecquote-jail-conf
configMap:
name: infosecquote-jail-conf
- name: fluentbit-volume
persistentVolumeClaim:
claimName: fluentbit-pvc
kubectl get all -n default
NAME READY STATUS RESTARTS AGE
pod/fail2ban-7df765557d-9rh2n 1/1 Running 0 22h
pod/fluent-bit-5dml7 1/1 Running 0 19d
pod/fluent-bit-5nthx 1/1 Running 0 19d
pod/fluent-bit-mr48g 1/1 Running 0 19d
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/fluent-bit ClusterIP 10.128.192.110 <none> 2020/TCP 328d
service/kubernetes ClusterIP 10.128.0.1 <none> 443/TCP 711d
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/fluent-bit 3 3 3 3 3 <none> 328d
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/fail2ban 1/1 1 1 26d
NAME DESIRED CURRENT READY AGE
replicaset.apps/fail2ban-7df765557d 1 1 1 26d
kubectl logs pod/fail2ban
Setting timezone to EST...
Setting SSMTP configuration...
Initializing files and folders...
Setting Fail2ban configuration...
Checking for custom actions in /data/action.d...
Add custom action infosecquote-block.conf...
Checking for custom filters in /data/filter.d...
Add custom filter infosecquote-auth.conf...
Add custom filter infosecquote-validation.conf...
2024-01-22 00:14:09,584 fail2ban.configreader [1]: INFO Loading configs for fail2ban under /etc/fail2ban
2024-01-22 00:14:09,584 fail2ban.configparserinc[1]: INFO Loading files: ['/etc/fail2ban/fail2ban.conf']
2024-01-22 00:14:09,585 fail2ban.configparserinc[1]: INFO Loading files: ['/etc/fail2ban/fail2ban.local']
2024-01-22 00:14:09,586 fail2ban.configparserinc[1]: INFO Loading files: ['/etc/fail2ban/fail2ban.conf', '/etc/fail2ban/fail2ban.local']
2024-01-22 00:14:09,586 fail2ban [1]: INFO Using socket file /var/run/fail2ban/fail2ban.sock
2024-01-22 00:14:09,586 fail2ban [1]: INFO Using pid file /var/run/fail2ban/fail2ban.pid, [INFO] logging to STDOUT
2024-01-22 00:14:09,588 fail2ban.configreader [1]: INFO Loading configs for jail under /etc/fail2ban
2024-01-22 00:14:09,588 fail2ban.configparserinc[1]: INFO Loading files: ['/etc/fail2ban/jail.conf']
2024-01-22 00:14:09,593 fail2ban.configparserinc[1]: INFO Loading files: ['/etc/fail2ban/paths-debian.conf']
2024-01-22 00:14:09,594 fail2ban.configparserinc[1]: INFO Loading files: ['/etc/fail2ban/paths-common.conf']
2024-01-22 00:14:09,595 fail2ban.configparserinc[1]: INFO Loading files: ['/etc/fail2ban/paths-overrides.local']
2024-01-22 00:14:09,596 fail2ban.configparserinc[1]: INFO Loading files: ['/etc/fail2ban/jail.d/infosecquote-jail.conf']
2024-01-22 00:14:09,596 fail2ban.configparserinc[1]: INFO Loading files: ['/etc/fail2ban/paths-common.conf', '/etc/fail2ban/paths-debian.conf', '/etc/fail2ban/jail.conf', '/etc/fail2ban/jail.d/infosecquote-jail.conf']
2024-01-22 00:14:09,602 fail2ban.configreader [1]: INFO Loading configs for filter.d/infosecquote-auth under /etc/fail2ban
2024-01-22 00:14:09,602 fail2ban.configparserinc[1]: INFO Loading files: ['/etc/fail2ban/filter.d/infosecquote-auth.conf']
2024-01-22 00:14:09,602 fail2ban.configparserinc[1]: INFO Loading files: ['/etc/fail2ban/filter.d/common.conf']
2024-01-22 00:14:09,604 fail2ban.configparserinc[1]: INFO Loading files: ['/etc/fail2ban/filter.d/common.local']
2024-01-22 00:14:09,604 fail2ban.configparserinc[1]: INFO Loading files: ['/etc/fail2ban/filter.d/common.conf', '/etc/fail2ban/filter.d/infosecquote-auth.conf']
2024-01-22 00:14:09,605 fail2ban.configreader [1]: INFO Loading configs for action.d/infosecquote-block under /etc/fail2ban
2024-01-22 00:14:09,605 fail2ban.configparserinc[1]: INFO Loading files: ['/etc/fail2ban/action.d/infosecquote-block.conf']
2024-01-22 00:14:09,606 fail2ban.configparserinc[1]: INFO Loading files: ['/etc/fail2ban/action.d/infosecquote-block.conf']
2024-01-22 00:14:09,606 fail2ban.configreader [1]: INFO Loading configs for filter.d/infosecquote-validation under /etc/fail2ban
2024-01-22 00:14:09,607 fail2ban.configparserinc[1]: INFO Loading files: ['/etc/fail2ban/filter.d/infosecquote-validation.conf']
2024-01-22 00:14:09,607 fail2ban.configparserinc[1]: INFO Loading files: ['/etc/fail2ban/filter.d/common.conf', '/etc/fail2ban/filter.d/infosecquote-validation.conf']
2024-01-22 00:14:09,644 fail2ban.server [1]: INFO --------------------------------------------------
2024-01-22 00:14:09,644 fail2ban.server [1]: INFO Starting Fail2ban v1.0.2
2024-01-22 00:14:09,645 fail2ban.observer [1]: INFO Observer start...
2024-01-22 00:14:09,654 fail2ban.database [1]: INFO Connected to fail2ban persistent database '/infosecquote-logs/fail2ban.sqlite3'
2024-01-22 00:14:09,676 fail2ban.jail [1]: INFO Creating new jail 'infosecquote-auth'
2024-01-22 00:14:09,676 fail2ban.jail [1]: INFO Jail 'infosecquote-auth' uses poller {}
2024-01-22 00:14:09,677 fail2ban.jail [1]: INFO Initiated 'polling' backend
2024-01-22 00:14:09,692 fail2ban.filter [1]: INFO maxRetry: 10
2024-01-22 00:14:09,693 fail2ban.filter [1]: INFO findtime: 300
2024-01-22 00:14:09,693 fail2ban.actions [1]: INFO banTime: 300
2024-01-22 00:14:09,693 fail2ban.filter [1]: INFO encoding: UTF-8
2024-01-22 00:14:09,698 fail2ban.filter [1]: INFO Added logfile: '/infosecquote-logs/infosecquote-app.log' (pos = 143176, hash = 3e89ab46369239d2a105e286cc9bf9ae78fb868c)
2024-01-22 00:14:09,698 fail2ban.jail [1]: INFO Creating new jail 'infosecquote-validation'
2024-01-22 00:14:09,698 fail2ban.jail [1]: INFO Jail 'infosecquote-validation' uses poller {}
2024-01-22 00:14:09,699 fail2ban.jail [1]: INFO Initiated 'polling' backend
2024-01-22 00:14:09,712 fail2ban.filter [1]: INFO maxRetry: 25
2024-01-22 00:14:09,713 fail2ban.filter [1]: INFO findtime: 60
2024-01-22 00:14:09,713 fail2ban.actions [1]: INFO banTime: 900
2024-01-22 00:14:09,713 fail2ban.filter [1]: INFO encoding: UTF-8
2024-01-22 00:14:09,716 fail2ban.filter [1]: INFO Added logfile: '/infosecquote-logs/infosecquote-app.log' (pos = 143176, hash = 3e89ab46369239d2a105e286cc9bf9ae78fb868c)
2024-01-22 00:14:09,720 fail2ban.jail [1]: INFO Jail 'infosecquote-auth' started
2024-01-22 00:14:09,735 fail2ban.jail [1]: INFO Jail 'infosecquote-validation' started
Server ready
Once fail2ban is running and monitoring the application logs, any attempts to attack the application through fuzzing will be identified and blocked limiting the ability of tools to find errors and thereby limiting the ability to find any vulnerabilities to exploit!
2024-01-22 00:32:30,157 fail2ban.filter [1]: INFO [infosecquote-auth] Found IPxxxxx - 2024-01-22 00:32:30
2024-01-22 00:34:03,408 fail2ban.filter [1]: INFO [infosecquote-auth] Found IPxxxxx - 2024-01-22 00:34:03
2024-01-22 00:35:30,664 fail2ban.filter [1]: INFO [infosecquote-validation] Found IPxxxxx - 2024-01-22 00:35:30
2024-01-22 00:36:18,709 fail2ban.filter [1]: INFO [infosecquote-validation] Found IPxxxxx - 2024-01-22 00:36:18
It doesn't prevent any attacks, instead it delays and will most likely cause timeout on most fuzzing attacks preventing many of the tests and limiting the ability of the attacker to identify potential problems or vulnerabilities in the application.
In the real world there are a few other alerts (slack and others) so if there was an active attack those who are monitoring the service will know within minutes.
Could this be improved with AI or ML. Yes 1000% - however is AI or ML needed to identify and block the most common and noisy attacks on your business applications? The answer, as you have seen in this post, that even using old tools and technology it is possible to automatically identify application security events and react to those with possible malicious intent!
Also just for fun and to block scanners who don't pay attention to robots.txt or the warning in the code.
There is a hidden div on the application page template (so it is on every web page on the site):
<!-- Warning: please to NOT follow the next link ("Welcome to the Blackhole") or you will be banned from this site -->
<div style="display:none;">
<a title="Welcome to the Blackhole" rel="nofollow">Attention: Do NOT follow this link, your IP will be blocked!</a>
</div>
And also have robots.txt configured:
User-agent: *
Disallow: */blackhole/*
So normal users won't see the link because the div is hidden, and the robots.txt says to not scan it if you are a robot.
So if anyone does happen to try the /blackhole/ link they will be added to the block list and all of the above notifications will let the team know.