Splunk 'splunks' Cybersecurity Data w.r.t Mitre Att&ck and Cyber Kill Chain
Splunk is a powerful data analytics and monitoring tool that has become a cornerstone in the toolkit of cybersecurity professionals. The necessity to use a tool that specializes in working with log data so that you do not have to read unstructured data from many different sources. That’s where Splunk comes in. Splunk does not detect hazards directly. Nevertheless, it expedites the detection of threats for analysts through the facilitation of data search, analysis, and correlation across multiple security tools. This empowers individuals to detect patterns, irregularities, and possible security risks. Splunk offers a whole suite of various software, apps and APIs, plus a whole lot of flexibility for future analytical needs. Splunk helps you explore things that are not easy to get to otherwise, like log data and messages and machine data. Security analytics tools such as threat detection and security monitoring are deployed to identify and investigate security incidents or potential threats such as external malware, targeted attacks and malicious insiders. Splunk has its own query language called Search Processing Language — SPL. With the ability to detect these threats at early stages, security professionals have the opportunity to stop them before they infiltrate network infrastructure, compromise valuable data and assets or otherwise cause harm to the organization. Though, the cyber kill chain or the MITRE ATT&CK Framework help security professionals get ahead of threats by anticipating their behaviors in a wide variety of contexts. Security analytics solutions aggregate data from numerous sources that include:
Endpoint and user behavior data, Business applications, Operating system event logs, Firewalls, Routers, Virus scanners, External threat intelligence, Contextual data etc.
Splunk technology is used for business and web analytics, application management, compliance, and security.? It correlates, captures, and indexes real-time data, from which it creates alerts, dashboards, graphs, reports, and visualizations. It gives security teams a comprehensive overview of the risks and threats your business faces – so you can protect it on a massive scale. It is advisable to integrate Splunk with a SOC platform and create custom alerts, searches, and workflows tailored to fit the individual needs of your team. Splunk empowers teams with the real-time actionable insight needed for a comprehensive security program. Splunk works through a forwarder collecting data from remote machines and forwarding it on to an index. An indexer then processes that data in real time and stores and indexes it on the disk. End-users then interact with Splunk through the search head, which enables them to search, analyze, and visualize data. Splunk Enterprise Security provides simplified threat management that facilitates quick threat detection and response and minimizes risk. Additionally, the deep integration with MITRE ATT&CK and Cyber Kill chain helps you configure Splunk Enterprise Security by pushing attributions to the Incident Review Dashboard, assess the level of coverage to ATT&CK tactics and techniques, and integrate risk-based events and alerting. Splunk ES can help you achieve continuous monitoring, support your security operations center (SOC), implement incident response, or inform stakeholders about business risks. The solution can be in various environments, such as public and private clouds, on-premises infrastructure, and hybrid deployments. These views include security and performance metrics, trending indicators, and static and dynamic thresholds. Splunk ES also offers a Use Case Library to facilitate quicker detection of new and known threats. It has Notable features include centralized logs, pre-defined reports and correlations, alerts and incidents, correlations for specific views, and incident response workflows. It offers ad hoc search capabilities alongside static, dynamic, and visual correlations to help detect malicious activities. It lets you investigate and pivot on various fields from any data to develop threat context quickly. Splunk ES lets you investigate and analyze breaches to trace activities associated with compromised systems. Library of security posture widgets—you can add widgets to your dashboards or create your own. View security events by categories—you can view events by location, source type, host, geography, and asset groupings. Use KPIs to assess security posture—you can use KPIs to see trends and monitor your security posture.
Splunk ES lets you view incidents as a single event or as a ‘roll-up’ of related events. It also provides an incident management workflow designed for security teams. It applies statistical analysis on proxy data to help understand HTTP-based behavioral outliers. It can help you discover anomalies across various components, including firewalls, DHCP, routers, load balancers, wireless access points, data loss prevention (DLP) devices, and intrusion detection sensors. Splunk Enterprise Security lets you assign a risk score to assets, events, users, and behavior. You can assign scores according to the relative importance of each component or according to its value. Risk scores can help you prioritize security events and investigations. You can also track the security status of components to understand and actively manage your business risk. It Automatically map your data to cybersecurity frameworks such as MITRE ATT&CK and Cyber Kill Chain.
The Components:
Splunk Forwarder – Forwarders ingest data. There are two types of Splunk Forwarders: 1. The Splunk Universal Forwarder, and 2. The Splunk Heavy Forwarder. The Universal Forwarder inputs data, performs preprocessing on the data, and then forwards the data to the indexer.
Splunk Indexer – Used for data parsing and indexing. The indexers receive and store the data being forwarded from the forwarders. They also search the data in response to search head requests. Indexers make it easier to perform search operations.
Search Head – Graphical user interface (GUI) where users can search, analyze, or report data based on key words. Ultimately, this component performs the search management function. The search head consolidates the results from the indexers and serves them to users.
?Splunk is integration friendly. Splunk ES integrates with existing SIEM tools – such as network firewalls and multi-factor authentication tools (MFA) – and leverages machine learning algorithms to identify suspicious activity in your network traffic. Additionally, Splunk apps can be leveraged to further functionality across the platform and with no limitations. With event analysis and correlation rules you can detect suspicious activity by analyzing events from multiple sources in real time. Dashboards and reports provide visibility into your security posture in an easy-to-understand format that allows you to identify potential risks quickly. A correlation search is a type of scheduled search that lets you detect suspicious patterns in your data. You can configure a correlation search to generate a notable event when search results meet specific conditions. Splunk can support any data type. Organizations are able to leverage different apps and add-ons that best suit business needs. Hurricane Labs has identified eight data types that are important for SOC engineers and managers to be aware of for a comprehensive SIEM implementation. Unified security analytics is an approach that incorporates machine learning, anomaly detection and predictive risk-scoring along with data science, to identify behavioral aberrations and suspicious activities that might indicate the presence of security threats. The Common security threats today are Social engineering, Malicious insiders, APTs and advanced malware, Distributed Denial of Service Attacks (DDos), Unpatched vulnerabilities, Compromised and weak credentials and IoT attacks.
?ENTRIES AND HOW TO USE SPLUNK
(All commands here are scenarios-based. They are not one-size fits all. They have to be adapted for the Cybersecurity Dataset and Business Needs.)
what do you have access to?
Copy the code for the following search:??
| eventcount summarize=f index=* | stats count by index
?index=firewall_fortinet srcip="10.99.4.40" dstip="137.99.218.133" dstport=3389 action=blocked | table time srcip, dstp dstport action dvcname vd
?dstip is where the traffic is going to.
?srcip is where the traffic coming from.
?dstport if the destination port of the service. 3389 is RDP/Remote Desktop
action is either blocked or allowed.
?index=_internal
?index=_internal errorfail*?
source="ANDROID.csv" host="DESKTOP-UKGN7IE" index="index_sec" sourcetype="csv" | rex field=_raw "Code=<(?<code>.*)" | dedup Code | table Code
source="ANDROID.csv" host="DESKTOP-UKGN7IE" index="index_sec" sourcetype="csv" | regex _raw!="^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}" | search "code=A" | sort -time
source="ANDROID.csv" host="DESKTOP-UKGN7IE" index="index_sec" sourcetype="csv" srcip="10.99.4.40" dstip="137.99.218.133" dstport=3389 action=blocked | table time srcip, dstp dstport action dvcname vd
source="ANDROID.csv" host="DESKTOP-UKGN7IE" index="index_sec" sourcetype="csv" | where host="localhost "
source="ANDROID.csv" host="DESKTOP-UKGN7IE" index="index_sec" sourcetype="csv" earliest=-1h latest=now
source="ANDROID.csv" host="DESKTOP-UKGN7IE" index="index_sec" sourcetype="csv" | stats count by host, port
source="ANDROID.csv" host="DESKTOP-UKGN7IE" index="index_sec" sourcetype="csv" | top limit=10 host
source="ANDROID.csv" host="DESKTOP-UKGN7IE" index="index_sec" sourcetype="-json"
Looking for executable files caught in suricata IDS logs
index=* sourcetype=suricata .exe
Looking through endpoints to which executable files were downloaded or uploaded. Make sure to change ip values
index=botsv1 sourectype=suricata (dest="ip1" OR dest-ip="ip2") .exe
Looking through endpoints to which executable files were uploaded through http POST request. Make sure to change ip values by specifying the endpoints using IP and domain
index=botsv1 sourcetype=suricata
(dest=domain.com OR dest="ip1")
http.http_method=POST.exe
Let us assume you want to see all IDS[suricata] alerts triggered by the machine infected with APT. Let us assume the infected machine ip is 40.80.140.40, it is going to be index=dataset domain.com src=40.80.140.40
sourcetype=suricata
?Using
?index=dataset sourectype=suricata
alert.signature=*keyword* | stats count by
alert.signature alert.signature_id |sort count
parsing http traffic and using domain as the keyword
index=dataset domain.com sourcetype=stream:http
parsing http traffic for specific source ip
index=botsv1 src=192.168.200.70 sourcetype=stream:http
investigating http protocol with http responses being equal to [200] and count the number of hits per URL
index=* dest=192.168.200.70 sourcetype=stream:http status=200 ! stats count by uri | sort - count
investing IIS logs
index=* sourcetype=iis
parsing http traffic for specific source ip
index=botsv1 src=192.168.250.70
sourcetype=stream:http
looking through events related to n s3 bucket publicly accessible
index="dataset" sourectype="aws:cloudtrail"
eventType=AwsApiCall eventName=PutBucketAcl | reverse
Looking through files that were successfuly uploaded
?
Finding the vendor name of the vendor of a USB inserted into a host
index="dataset" host=hostname usb
?
file sharing events can be found in sysmon
index=dataset
sourcetype=XmlWinEventLog:Microsoft-Windows-Sysmon/Operational host=hostname
?
index=dataset
sourcetype=XmlWinEventlog:Microsoft-Windows-Sysmon/Operational host=hostname
src="filserver.com "
Looking for executable files caught in suricata IDS Logs
index=*? sourcetype=suricata .exe
Looking for process creation events with a named executable
index=dataset name.exe
sourcetype=XmlWingEventlog:Microsoft-Windows-Sysmon/Operational EventCode=1
?
Sorting by MD5 Hashes and
index=dataset name.exe CommandLine=name.exe |stats values(MD5)
?
searching with the hostname of a PC. Replace hostname and its value
index=botsv1 hostname
sourcetype=XmlWinEventLog:Microsoft-Windows-Sysmon/Operational
?
?Looking through endpoints to which executable files were downloaded and uploaded. Chage the ip values
index=botsv1 sourcetype=suricata (dest="ip1" OR dest_ip="ip2") .exe
?
Looking through endpoints to which executable files were uploaded through http POSTrequest. Make sure to change IP Values
by specifying the endpoints using ip and domain
?
index=botsv1 sourcetype=suricata
(dest=domain.com OR dest="ip1") OR dest_ip="ip2")
http.http_method=POST .exe
Looking through the http traffic with domain name as the keyword
index=dataset domain.com sourcetype=stream:http
?
parsing http traffic for specific source ip
index=botsv1 src=192.168.250.70
sourectype=stream:http
?
investigating http protocol with http responses being equal to[200] and count the number of hits per URL
index=* dest=192.168.250.70
sourcetype=stream:http status=200 | stats count by url | sort - count
?index=* sourcetype=iis
?using the stats command to find combination of signature and signature_id column
index=dataset sourcetype=suricata
alert.signature=*keyword* | stats count by
alert.signature alert.signature_id | sort count
Looking through http traffic with domain name a sa keyword
index=dataset domain.com sourcetype=stream:http
parsing http traffic for specific source ip
index=botsv1 src=192.168.250.70
using regularv expressions to display usernames and paswords in http requests and display the count of the password length
index=* sourcetype=stream:http
form_data=*username*passwd*
| rex field=form_data "passwd=(?
<userpassword>\w+)"
| eval lenpword=len(userpassword)
| table userpassword lenpword
?
Using lower case and upper case on the password
index=* sourcetype=stream:http
form_data=*username*passwd*
| rex field=form_data "passwd=(?
<userpassword>\w+)"
| eval lenpword=len(userpassword)
?
parsing general network traffic and cunting the number of requests
index=dataset domain.com sourcetype=stream* |
stats count(src_ip) as Requests by src_ip |
sort - Requests
?
Finding an image transferred between two ips
index=dataset dest=23.22.63.114 "image.jpeg"
src=192.168.250.70
?
Looking through scheduled tasks activity
index="dataset" schtasks.exe
sourcetype="XmlWinEventLog.exe
sourcetype="XmlWinEventLog:Microsoft-Windows-Sysmon/Operational"
| dedup ParentCommandLine
| dedup CommandLine
| table ParentCommandLine CommandLine
?
Looking for hits to malicious websites and providing an ip address
index=dataset sourcetype=fgt_utm
"192.168.250.70" category="Malicious websites"
?
Looking for USB attacks via window registry
index=dataset sourcetype=windowregistry keyword
?
Looking for the attacks sorting by host and object
index=dataset sourcetype=winregistry keyword | table host object data
?
Looking for Sysmon events for the infected system on an external drive and show oldest events first.The external drive letter is[d:]?
index=dataset
sourcetype=XmlWinEventLog:Microsoft-Windows-Sysmon/Operational host=targetpcname "d:\\" | reverse
?
Finding the vendor's name of a USB inserted into a host
index="dataset" host=hostname usb
?
file sharing events that can be found in Sysmon
index=dataset
sourcetype=XmlWinEventLog:Microsoft-Windows-Sysmon/OPerational host=hostname
?
file sharing information about the server address
index=dataset
sourcetype=XmlWinEventLog:Microsoft-Windows-Sysmon/Operational host=hostname
src="filserver.com "
?
status=50*? OR status 404
status!=400
Looking in the cloud
index="dataset" sourcetype="aws:cloudtrail"
useridentity.accesskeyId="AKIAJOGCDXNWSPUPUA"
eventname=CreateAccessKey
?
Looking for unauthorised attempts to describe an account
sourcetype="aws:cloudtrail"
userIdentity.accessKeyID="AKIAJOGCPUPUA5NWXPUA"
eventname="DescribeAccountAttributes"
Investigating coin mining attcks and finding the signature ID
index="dataset" sourcetype="symantec:*" coin
| table time CIDSSignature_ID
?
index="dataset" sourcetype="ms:o365:management"
Workload=OneDrive Operational=FileUpLoaded
| table time srcip user object UserAgent
?
Finding antivirus alerts. For example Symantec antivirus
index="dataset" source="WinEventLog:Application"
sourceName="Symantec Antivirus" "Frothly"
?
looking through what Users created
index="dataset" source="wineventlog:security"
EventCode=4720
?
Finding groups a user is assigned to
index="dataset" source="wineventlog:security"
svcvnc "EventCode=4732"
?
Looking through users added along with their passwords
领英推荐
index="dataset" (adduser OR useradd)
source="/var/log/auth.log"
?
Looking through users added using osquery via linux?
index="botsv3" sourcetype="osquery:results"
useradd
?
finding information about a process listening to [1337]
index="dataset" 1337
sourcetype="osquery:results"
"columns.port"=1337"
??
looking for nightmare vulnerabilty
identifies print spooler
source="WinEventLog:Microsoft-Windows-PrintService/Operational"
EventCode=316 category = "Adding a printer
driver" Message = "*kernelbase.dll,*" Message =
"*UNIDRV.DLL,*" Message = "*.DLL.*"
| stats count min(_time) as firstTime
max(_time) as lastTime by Opcode EventCode
ComputerName Message
?
O365 Events
index="dataset" sourcetype="ms:o365:management"
Workload=Onedrive Operation=FileUploaded
| table time srcip user object UserAgent
?
Looking through full process usage information on aws instance. useful to find coin mining activity
index="dataset" sourcetype= "PerfmonMK:Process"
process_cpu_used_percent=100
| table time host processname
process_cpu_used_percent
?
Finding IAM user across key that generates the most distinct errors when attempting to access IAM resources.
index="dataset" sourcetype="aws:cloudtrail"
user_type="IAMUser" errorcode!="success"
eventsource="iam.amazonaws.com "
| stats dc(errorMessage) by
?
Looking through downlaoded files
index="botsv2" sourcetype="stream:ftp"
method=RETR
| reverse
?
?Listing out the IAM Users that accessed an AWS service (successfully OR unsuccessfully)
index="dataset" sourcetype="aws:cloudtrail"
IAMUser
|dedup user
|table user
?
Looking through events related to n s3 bucket publicly accessible
index="dataset" sourcetype="aws:cloudtrail"
eventType=AwsApiCall eventName=PutBucketAc1 | reverse
?
Looking through files that were successfully uploaded into the s3 bucket
index="dataset"
sourcetype="aws:s3:accesslogs" PUT | reverse |
Looking through events related to n s3 bucket publicly accessible
index="dataset" sourcetype="aws:cloudtrail"
eventType=AwsApicall eventName=Put? BucketAc1 | reverse
index="myIndex" password? Access the index called myIndex and text matching password.
?source="test_data.zip :*"??????? Access the data archive called test_data.zip and parse all its entries (*).
?Does an array contain a specific value?
"array_name{}"=value
?
Events over time
index="my_log"
| bin span=1hr _time
| stats count by _time
OR
index="my_log"
| timechart count span=1hr
?source="/var/log/nginx/access.log" HTTP 500
source="/var/log/nginx/access.log" HTTP (200 or 30*)
source="/var/log/nginx/access.log" status=404 | sort - uri
source="/var/log/nginx/access.log" | head 1000 | top 50 clientip
source="/var/log/nginx/access.log" | head 1000 | top 50 referer
source="/var/log/nginx/access.log" | head 1000 | top 50 uri
source="/var/log/nginx/access.log" | head 1000 | top 50 method
?
?List users and corresponding roles:
? | rest /services/authentication/users splunk_server=?
? | fields title roles realname
?
- List indexes:
? | eventcount summarize=false index=* index=_* | dedup index | fields index
? | rest /services/data/indexes | dedup title | table title
?
- Simple tabling of results:
? | table time srcip src_port dest_ip dest_port proto url method proxy
? | sort _time
?
- Simple count statistics:
index=os sourcetype="wineventlog:security" EventCode=4688
? | stats count, values(Creator_Process_Name) as Creator_Process_Name by New_Process_Name
? | table New_Process_Name, count, Creator_Process_Name
? | sort count
?
- Send e-mail function:
? | sendemail to="[email protected] "
- Create timechart:
? | table _time, <field>, name
? | timechart span=1d sum(<field>) by name
?
- Show rare events:
index=os sourcetype=registry
? | rare process_image
?
- Keep only the results that match a valid email address:
? | regex email="/^([a-z0-9_\.-]+)@([\da-z\.-]+)\.([a-z\.]{2,6})$/"
?
- Use sed syntax to match the regex to a series of numbers and replace them with an anonymized string:
? | rex field=ccnumber mode=sed "s/(\d{4}-){3}/XXXX-XXXX-XXXX-/g"
?
- Expand an event with more than one multivalue field into individual events for each field value:
source="mvexpandData.csv"
? | rex field=_raw "a=(?<a>\d+)" max_match=5
? | rex field=_raw "b=(?<b>\d+)" max_match=5
? | eval fields = mvzip(a,b)?
? | table _time fields
?
?- Match valid IPv4 addresses:
?
? | eval ipv4_valid = if(match(ipv4, "^((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$"), "valid", "invalid")
?
Look for failed attempts to su or sudo to root.
(su OR sudo ) AND (fail* OR error)
Look for errors in sshd logs.
?sshd AND (fail* OR error OR allowed OR identity)
?Look for general authorization failures excluding router messages.
?auth* AND (fail* OR error?) NOT _sourceCategory=routers
?Extract "from" and "to" fields. For example, if a raw event contains "From: Jane To: John", then from=Jane and to=John.
| parse "From: To: *" as (from, top
?Extract the source IP addresses using a regular expression for the four octets of an IP address.
*| parse regex "(?<src_ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"
?
Identify all URL addresses visited, extract them as the url field.
_sourceCategory=apache
| parse "GET * " as url
?
Identify all URL addresses visited, extract them as the url field.
_sourceCategory=apache
| parse "GET * " as url
?
For the Source Category apache, extract src, size, and URL even if the size field is missing from the log message (nodrop).
_sourceCategory=apache
| parse "* " as src_IP
| parse " 200 * " as size nodrop
| parse "GET * " as url
?For the Source Category apache, extract src, size, and URL even if the size field is missing from the log message (nodrop).
_sourceCategory=apache
| parse "* " as src_IP
| parse " 200 * " as size nodrop
| parse "GET * " as url
?
Identify the total number of pages by source IP address.
?_sourceCategory=apache
| parse "* -" as src_ip
| count by src_ip
?Identify the total number of pages by source IP address and re-order them by most frequently loaded pages.
_sourceCategory=apache
| parse "* " as src_ip
| parse "GET * " as url
| count by url
| sort by _count
Identify the top 10 requested pages.
? | parse "GET " as url
| count by url
| top 10 url by _count
?
?Identify the top 10 source IP addresses by bandwidth usage.
?sourceCategory=apache
| parse " 200 * " as size
| parse "* -" as src_ip
| sum(size) as total_bytes by src_ip
| top 10 src_ip by total_bytes
?
Identify the top 100 source IP addresses by number of hits.
?_sourceCategory=apache
| parse "* -" as src_ip
| count by src_ip
| top 100 src_ip by _count
?
?For the Source Category Apache/Access, count messages by status code categories, grouping all 200s, 300s, 400s, and 500s together.
_sourceCategory=Apache/Access
| timeslice 15m
| if (status_code matches "20*",1,0) as resp_200
| if (status_code matches "30*",1,0) as resp_300
| if (status_code matches "40*",1,0) as resp_400
| if (status_code matches "50*",1,0) as resp_500
| if (!(status_code matches "20*" or status_code matches "30*" or status_code matches "40*" or status_code matches "50*"),1,0) as resp_others
| count(*), sum(resp_200) as tot_200, sum(resp_300) as tot_300, sum(resp_400) as tot_400, sum(resp_500) as tot_500, sum(resp_others) as tot_others by _timeslice
??
* host="DESKTOP-UKGN7IE" source="WinEventLog:Security" sourcetype="WinEventLog:Security" EventCode="1102"
?
index=botsv3 earliest=0 sourcetype="stream:ip" (src_max"*?? AND? dest_mac"*)
| table -time src_ip src_mac dest_ip dest_mac
| stats latest(_time) time count by srcip src_mac
| sort - count
| iplocation src_ip
| search Country"*
| head 10
?
Conclusion:
Splunk is a Big Data mining tool. By aggregating, correlating and analyzing the entirety of your data, security analytics gives you a clear and comprehensive window into your threat environment that will let you see — and prevent — emerging attacks well before they compromise your data and harm your organization. By transforming machine data into valuable insights, Splunk enables organizations to detect security incidents, monitor system performance, and ensure compliance with industry standards. Utilize the Splunk community forums, official documentation, user guides and Keep your Splunk software up to date with the latest patches and releases to benefit from new features, performance improvements, and security fixes. Navigating Splunk Enterprise can initially seem daunting, but with a solid understanding of its interface, search functionalities, and alert mechanisms, you can harness its full potential to enhance your cybersecurity operations.
?
References:
?
?
?
?