Ip-blocking using MLSecOps

In this article I'm going to explain how we can block a particular IP, using Rhel8, Python script, Jenkins.

Our Problem Statement: Create an automated system which will be useful for a server in terms of the following features:-

1. This system will keep a log of the information about the clients hit or request to the server for example we can get log file of a webserver at location /var/log/httpd/ .

2. This log data of clients will be used for finding the unusual pattern of a client request for example if a client is sending request repeatedly. for this purpose, we can use here clustering to make clusters of different patterns of client request and to identify which cluster of client requests can cause some security and performance issue in the server.

3. If any kind of unusual pattern we got then we can use Jenkins to perform a certain task for example it can run some command to block that IP which is causing this trouble.

Solution: We have to create an architecture in which we will collect all data of IPs which are searching the particular website. For this task , I have created a webpage called index.html. The location of index.html is /etc/www/html . Here all webpages are saved , but I have created only one for testing purpose.

After extracting data , I have saved that log data in a file called alllog.txt which is saved in /logfiles folder. After creating this folder and saving this file I have pushed this file to Github so we can download that data and use it as a dataset.

Because the log data is managed as a record and this looks by only RHEL8 , it has a format which is not easily understood by any programming language . That's why we need to convert into CSV . So python Script can easily take that as a dataset. But, we know this data is having also unwanted features and those features will really create a big problem if we don't remove them , because it can affect its accuracy. So, we have to remove them , It is also called as Feature Selection.

Now , after collecting the data and converting into csv and feature selection , we have to create a small python program by which we will extract only those IPs , which are continuously attacking that particular webpage.

When we get that IP which is continuously attacking that WEBPAGE OR WEBSITE ,we will pass this information into github. After uploading to github we can easily extract that , by cloning this in that RHEL8 OS .

Using Jenkins, we can simply write a following command which is used to block an IP address.

iptables -A INPUT -s IP_ADDRESS_YOU_WANT_TO_BLOCK -j DROP

Step by Step Implementaion,

Accessing all logs from access_log file which is present in /var/log/httpd. I'm saving this file using cat into alllog.txt file because we can't see the access_log file using any text editor like gedit directly.

Now, I will push this allog.txt to Github.

Now I will download the file using git clone command.

Python: In Jupyter notebook, I made a particular to convert log data onto csv, and then to delete the unwanted columns or fields.

1. Importing pandas and NumPy.

2. Creating a code, to convert log to CSV.

3. Removing unwanted space in the CSV.

4. Removing unwanted columns.

5. Renaming the columns

6. Dataset created

7. Counting Ips with their number of attacking.

8. Extracting out that IP which is having attacks more than 12.

9. Saving in the ipoutput.txt file.

Pushing this file into GitHub.

Before Jenkins job building

Jenkins: Created a job to block the IP

After Jenkins job building

Conclusion :

1. we can easily block a particular IP using this concept

2. This will help in securing website from hackers, or even other vulnerable users who just attack to slow down the server.

?