Optimizing SIEM: Prioritizing Log Collection for Effective Security Monitoring

In today's cybersecurity landscape, managing and analyzing log data is paramount to identifying and mitigating threats. The effectiveness of a Security Information and Event Management (SIEM) system heavily relies on the quality and relevance of the log data it ingests. Below is a prioritized list of recommended event logs that organizations should consider collecting into their SIEM.

This list should be customized to fit the specific needs and threat landscape of your organization, and the priority order may vary accordingly.

1. Security Product Logs (EDR/XDR/IPS)

These logs are crucial as they directly relate to the detection of threats and the protection of endpoints and networks.

2. Windows Domain Controller (DC) Event Logs / LDAP Logs

Essential for tracking authentication and authorization activities, these logs are fundamental in detecting potential compromises.

3. Web Proxy Requests

Capturing web traffic details, these logs help in monitoring and blocking malicious outbound connections.

4. PowerShell Logs

Given the widespread use of PowerShell in both legitimate and malicious activities, these logs are critical for detecting suspicious scripts.

5. DNS Queries (with Endpoint Name/IP)

Monitoring DNS queries can reveal attempts to connect to known malicious domains, making them indispensable for threat detection.

6. VPN Logs

With the rise of remote work, VPN logs have become crucial for monitoring remote access and ensuring it is legitimate.

7. Cloud Service Provider Logs

• Google Cloud Logging
• AWS CloudWatch
• Microsoft M365 Unified Audit Log
• Microsoft Azure AD Sign-In Logs
• Microsoft Cloud App Security Logs
• Microsoft Azure AD Identity Protection
• Microsoft M365 Defender for Identity

These logs are key to monitoring user activities and security events within cloud environments, which are increasingly targeted by attackers.

8. Windows Member Server & Workstation Event Logs

These logs are vital for detecting unauthorized access and suspicious activities on servers and workstations within the network.

9. Linux auditd Logs

Critical for monitoring Linux systems, auditd logs provide detailed records of system events, which are essential for forensic analysis.

10. Email Logs

Given that email is a common attack vector, these logs help in tracking phishing attempts, spam, and other malicious activities.

11. Custom Application Logs

Logs from in-house or custom applications can be a treasure trove of information, particularly for detecting anomalies or unauthorized access.

12. Network Traffic Logs

• Firewall (Ingress and Egress)
• DHCP
• NetFlow
• AWS VPC Flow Logs
• Google VPC Flow Logs
• Microsoft Network Security Group Flow Logs
• Full Packet Capture

These logs are invaluable for monitoring and analyzing network traffic, identifying anomalies, and investigating potential breaches.

Considerations for Log Collection

When establishing a log collection strategy, several key considerations should be taken into account:

1. Data Ingestion and Space Restrictions: Balancing the amount of data collected with the available storage and processing capabilities is crucial. Overloading the SIEM with excessive logs can lead to performance issues.
2. Common Attack Vectors: Prioritize logs that provide visibility into common attack vectors used by adversaries.
3. Detection Sources: Focus on logs that are known to be effective in detecting threats.
4. Expected Noise Levels: Some logs generate a high volume of data with little actionable intelligence. Consider tuning out or filtering these logs to reduce noise.
5. Legal and Regulatory Requirements: Ensure that the logs collected meet the legal and regulatory requirements for your industry.
6. Normalization and Parsing: Normalize field names and content across logs to facilitate correlation and ensure accurate search results. Parse out nested fields to extract valuable data, particularly from complex logs like Windows Event Logs.
7. Time Synchronization: Ensure that all devices sync with Network Time Protocol (NTP) providers and that logs are set to ISO 8601 UTC for consistency.

Event Reduction and Tuning

To optimize SIEM performance and reduce costs, it's advisable to filter out logs that do not provide security value. Consider the following for event reduction:

• Authorized Vulnerability Scanners: Exclude logs generated by legitimate vulnerability scans.
• Debug Logs: Often irrelevant for security purposes, these can usually be excluded.
• Encrypted Data: Unless decrypted, these logs may not provide actionable information.
• Common Application Errors: Filter out non-security-related errors that generate excessive noise.
• File Access by Backup Service Accounts: Typically expected activity that can be safely excluded from analysis.

Detection Layering:

A layered approach to detection means analyzing security events at different levels to get a complete picture. Here’s how it works:

1. Raw Events: Start by checking the raw data for obvious issues. Ask yourself: Are these logs needed? Can some be filtered out? Are the important details being captured and organized correctly?
2. Aggregation of Raw Events: Next, group and display these events on dashboards to spot patterns or trends. Decide which details to focus on and how long to keep this information.
3. Prioritized Alerts: When you identify a real security threat, create alerts based on how important the affected assets are and how reliable the detection method is.
4. Incidents: The most critical alerts need immediate action. These might include known malicious activities, like specific IP addresses or domains that are flagged as dangerous.

This approach ensures you catch important issues at each level of monitoring and respond effectively.

Conclusion

Implementing an effective log collection strategy within your SIEM is essential for robust security monitoring and incident response. By prioritizing the right logs, normalizing and parsing data, and adopting a layered detection approach, organizations can enhance their ability to detect, investigate, and respond to threats in a timely manner.