Weekend learning - SIEM Log Sources

Where is the log files come from?

• Applications, Network, #Firewalls, #Endpoints or #IoT Devices, Servers, #Containers, Hypervisors, Databases, Web Services, etc.

What is the basic anatomy of log files we look for?

• Timestamp
• LogSource: ie AgentName, SourceID, Device Vendor, Device Product
• Event info (such as Activity)
• Related User/ Host/ Machine entity

Typical personas and their focuses

How to identify the required log sources for your SIEM operation?

A note before we go deeper: #SIEM logging focuses on detecting threat from the beginning. Log management provides data but it's up to the security analyst to interpret it and determine whether or not the security incident is real.

There is a very common approach to determine what logs should be sent to your SIEM, is to firstly consider both of the ‘easy-to-understand’ types of log:

1. Technology-specific logs [aka. Logs from your Security Controls]

Technology-specific logs refer to logs generated by specific technologies or components within an IT infrastructure. These logs provide insights into the behavior, performance, and security of those technologies. Here are some examples of technology-specific logs:

• Firewall logs: they provide information about network traffic, including allowed and blocked connections.
• Proxy/Web Filtering Logs: they provide the information about IP, Domain, User-Agent string, URL, etc.
• Network #IDS/IPS logs: capture events related to potential intrusions or malicious activities.
• #ActiveDirectory logs: they contain information about user authentication, authorization, and group policy changes.
• Others: #EndpointDLP logs, #NetFlow data, Sandboxes, etc.

2. Domain-specific logs [aka. Logs from Contextual Insight]

Domain-specific logs refer to different types of logs that are generated by systems or applications within a specific domain or area of interest. By capturing relevant events and activities within that domain, these logs offer valuable context and understanding of the system or application's behavior within its specific context. It helps to enable #SecurityAnalysts to monitor, #investigate, and analyze activities within that specific context. Some common examples of domain-specific logs include authentication logs, access logs, and audit logs. Here's a brief explanation of each:

• Authentication logs: record user login attempts, successful logins, and authentication failures.
• Access logs: track user activities, including file access, system commands executed, and resource interactions.
• Others: Audit Logs, Compliance logs, etc.

Why you don’t need All-the-things to be sent to your SIEM?

• Everything?! In this approach, you store everything, have access to all the possible data that you may need, then do filtering and searching later. What makes you feel safer with this approach is that you might not miss anything but it’s not a 100% assured. In this case, we can look at the them at raw logs with the cost of storage, indexing, transmitting the log to different locations.
• Take only what you need to survive In this scenario, your SIEM/SOC becomes more utilized. It is best to start with what you need and then build upon it to make sure that everything you 'caught' is valuable asset for security-related activities. But still you will think of a risk that you would miss something, we will address it with some of key questions in the following.

Note: You can have a look at this article before moving to the questions Best practices for data collection in Microsoft Sentinel | Microsoft Learn

1st Key question: Why should I look into the second approach?

• In many cases, the answer to what to log is often driven by costs but it is also because of the #SOC operation efficiency.
• It is best to consume logs more aggressively from high-value systems, high-risk systems, and those facing external networks. Then you can save in areas that are of lesser importance from a security perspective.

2nd Key question: If I go with the second approach, how would I start?

• Mapping specific log ingestion to a standards framework will help to focus important log sources and SIEM Use Cases. For organizations with compliance requirements, cybersecurity frameworks such #CIS, #NIST, #ZeroTrust (TIC 3.0) offer a good starting point for example, healthcare organizations complying with #HIPAA, and they may want to prioritize logging access to protected health information.

3rd Key question: What if you need to have all SIEM and LMS primary features? What is the solution you can consider?

Refer to these articles for more details about leveraging #MicrosoftSentinel and #ADX

• Commonly used Microsoft Sentinel workbooks | Microsoft Learn
• Detect threats quickly with near-real-time (NRT) analytics rules in Microsoft Sentinel | Microsoft Learn
• Work with anomaly detection analytics rules in Microsoft Sentinel | Microsoft Learn
• Start an investigation by searching large datasets - Microsoft Sentinel | Microsoft Learn
• Integrate Azure Data Explorer for long-term log retention | Microsoft Learn
• Tutorial: Ingest monitoring data in Azure Data Explorer without code | Microsoft Learn
• Microsoft Sentinel data source schema reference | Microsoft Learn