A Treatise on Detection Engineering for SOC Automation

Threat detection engineering (DE) is a complex process that goes beyond just detecting abnormal activities. This process requires expertise in understanding the tactics, techniques, and procedures employed by threat actors and translating that knowledge into effective detection strategies. DE involves developing an environment inside an organization where multiple teams collaborate to address risks and target potential threats better. It encompasses designing, developing, testing, and maintaining threat detection logic, which can be a rule, a pattern, or even a textual description. The scope of DE is wide and works in multiple dimensions, from risk management to threat intelligence.

Although DE is an increasingly popular term in the information security sector, it is still a relatively new concept in its developing phase. The information available on this topic is limited, and there aren't that many specialists dedicated to this particular area of cybersecurity. However, many companies are testing out their methods to use this theory and enhance their detection and response capabilities. One of the key concepts in detection engineering is Detection-as-Code (DaC). DaC is implemented by using the best software engineering practices and the modern agile CI/CD pipeline. This approach ensures that detection is streamlined and efficient, providing reliable and timely threat detection.

Security engineers play a crucial role in safeguarding organizations from cyber threats. One of the key processes they use is Detection Engineering (DE). By creating, testing, and tuning detections to alert teams about any malicious activities, DE helps minimize false positive alerts and identify risks at an early stage of the attack lifecycle.

What are the benefits of Detection Engineering:

1. Dynamic threat identification: DE employs advanced techniques and tactics to discover dynamic and sophisticated threats in real-time, aiding in spotting possible dangers before they cause serious harm.
2. Improved incident response: Automation and adaptability in detection engineering make activities related to incident response easier. Automated systems can quickly analyze and prioritize security events, resulting in quicker response times. The ability to customize response actions based on the kind and degree of detected threats allows for the optimization of incident mitigation efforts.
3. Reduced dwell time: Detection engineering seeks to shorten the breach discovery period, known as dwell time, by immediately recognizing and reacting to threats. By utilizing comprehensive techniques and routinely modifying detection parameters, organizations can significantly reduce the amount of time that exploits go unnoticed within their networks.

DE is an essential process in the fight against cyber threats, and organizations should consider investing in this proactive approach to enhance their security posture. Improved threat intelligence is crucial for organizations to stay ahead of emerging risks and make informed choices during incident response. By using detection engineering, incorporating threat intelligence feeds and data sources into the detection architecture can enhance detection capabilities by providing the most recent information on new threats, attack patterns, and indicators of compromise.

Why is Detection Engineering Important?

Security analysts work closely with alerts and data provided by security tools to detect suspicious and abnormal activity. However, the main problem with these detection methodologies is that they are not scalable. So, what can be done? Implement, treat detection as well-written code that can be tested, and check source control and review with peers.?

When it comes to a good detection engineering process, it requires both good tooling and excellent detection content to work with. Even simple rules count as long as they can be saved, reused, and allow automatic alerting. A good DE process has many key features for identifying and responding to security alerts.

The trick here is to find a balance between having broad detections that generate a large volume of false positives and having a precise rule with a high level of false negatives. So every query should be tested for alert volume and accuracy.

Overall, detection engineering is a cyclic process that requires a clear picture of the data an organization generates. By knowing the data sources, engineers can understand the detection possibilities and provide better warnings to identify malicious behavior.

EDR (endpoint detection and response) and XDR (Extended Detection and Response) tools are designed to detect threats automatically using internal and external network knowledge and threat intelligence. These tools work as security analysis engines that can detect threats and respond to certain events, such as terminating a process or sending alerts to the security team.

However, relying solely on pre-set rules and standards is not enough. Different networks and organizations have distinct environments, and detection must conform to each particular case. The key to proper detection engineering lies in contextual data based on tactics, techniques, and procedures (TTPs) described in the MITRE ATT&CK? framework or other frameworks.

Threat intelligence platforms provide new updates and feeds about developments in the cyber threat landscape and offer threat context to enrich the detection process. Threat intelligence is the process of collecting data to analyze motives and identify the targets of threat actors. A security analyst with intel on internal and external data can build good detection rules.

While TTPs are the most effective indicators for detection, indicators of compromise (IOCs) are also important in detection engineering. However, IOCs are much less flexible than behavioral-based detection rules. Often, IP addresses, domains, and hashes won’t be reused by the threat actors, and if you’re dealing with a time-sensitive exploit, IOCs might not be published yet. IOCs are effective for retrospective detection to discover if you were targeted in the past.

In conclusion, EDR/XDR platforms play a crucial role in detection engineering. However, organizations must include contextual data based on TTPs and threat intelligence to build good detection rules that conform to each particular case.

Apart from detection engineering, organizations should also leverage threat hunting to strengthen their security posture. Focusing on known threats is essential, yet that alone can’t ensure a complete defense system. Cybersecurity is the industry where 100% protection is a mere myth. But the more comprehensive your approach is, the less vulnerable your systems will end up being. Without threat hunting, DE isn’t complete and might end up being less efficient. It's important to consider this approach to ensure a more robust security posture.