Elastic Lab (part 5) - Logstash

WHEW. That's been a lot of work. At the end of part 4 we're FINALLY seeing some results paying off as we are able to log in to Kibana. Kibana is just the tool to access the Elasticsearch database. How do we get data INTO the database? That component is called "Logstash".

Logstash parses and transforms data so it's ready to get dropped into the big bucket of data. We're going to follow this Install Guide. Again, read through all of it and this page before getting started! Even though we ended part 4 in the browser, we're back to the Terminal (command line) for this part.

1- We are going to skip a few steps that were completed in part 3. We'll run this command:

(DON'T worry about rpm commands- those are RedHat install packages!)

This install only takes a minute or two, Logstash is much smaller than other packages we've installed.

2- To make Logstash startup when we boot our lab VM type "sudo systemctl daemon-reload" then "sudo systemctl enable logstash".

3- Type "sudo systemctl start logstash" to start the service up.

4- I and others have run into Logstash errors. For some crazy reason the install does not set the permissions for its data folder to be editable. So we will enter:

"sudo chmod -R 777 /usr/share/logstash/data" to allow the directory to be written to. (NOTE that we have to be careful with capitalization!!! Make sure the R is capital and the rest is not).

Sidenote: "chmod" is a very important concept in Linux. Just like Windows doesn't let users access the documents of other users (without administrative permission), Linux assigns permissions to every file and folder in the entire system. "chmod" is used to modify who can read, write, and edit files and directories.

5- Now we can follow steps in this guide to test that Logstash is running:

type "cd /usr/share/logstash"

then " ./logstash -e 'input { stdin { } } output { stdout {} }' ". (without the double quotes of course). You can copy the command from the box:

It takes a bit of time but you will get a message "the stdin plugin is now waiting for input:".

Type something (anything!) and hit enter. You'll see your stuff going into the database format shown in that tutorial in part 0. You can type more if you would like.

Press "CTRL-D" to exit the logstash prompt.

CONGRATULATIONS, you have installed the ELK stack! (take a Snapshot!)

So what's next? Well I have some work to do. We need to have data fed to our database to make the database useful. Then we can use Kibana to filter, query, and create graphs and other visualizations of what's happening.

A critical component of Cybersecurity is feeding the "right" information to the tools so we can detect when something is "wrong". This can include logs, sensors, and other information feeds. I'll be working on home lab integration and hopefully get some writeups ready in the next few weeks!

Thanks for the support on this project. Please comment or message me if there are errors, any possible improvements, or other questions!

NEXT installing Winlogbeat to feed information from the host computer to Elastic so we can start seeing some data.