Managing DNS Over HTTPS (DoH) and DNS Over TLS (DoT) Traffic as a SOC Analyst

As a SOC Analyst, managing and detecting malicious activity in networks where DNS over HTTPS (DoH) and DNS over TLS (DoT) are used can be challenging due to the encryption and obfuscation these protocols provide. DNS traditionally sends queries over port 53 in plaintext, which allows monitoring tools to easily inspect DNS traffic for anomalies. However, with the adoption of DoH and DoT, DNS queries are encrypted, making it difficult to inspect them and effectively detect malicious activities or unauthorized data exfiltration.

Here’s how to manage and overcome the visibility challenges related to DoH and DoT traffic in your network, so you can catch attackers attempting to compromise your infrastructure.

Understanding DoH and DoT Traffic

1. DNS over HTTPS (DoH):
2. DNS over TLS (DoT):

Challenges with DoH and DoT Traffic

• Lack of visibility: DoH and DoT traffic is encrypted, and thus, traditional DNS traffic inspection (via DNS logs or IDS/IPS sensors) cannot detect malicious activity within the encrypted traffic.
• Network obfuscation: Attackers may use DoH or DoT to avoid detection from network traffic monitoring systems or DNS-based filtering.
• Bypassing DNS filtering systems: DoH and DoT enable bypassing local DNS servers and firewalls, making it more difficult to enforce security policies or block known malicious domains.
• Encrypted traffic inspection: Without decrypting the traffic, it is impossible to know if a DoH or DoT request is malicious or benign.

Strategies for Managing DoH and DoT Traffic Visibility

To effectively monitor and analyze DoH and DoT traffic in your environment, you'll need to implement strategies that provide visibility into encrypted traffic. Here are several approaches that can help overcome visibility challenges:

1. Enforce DNS Traffic Control Policies

• Block outbound DoH/DoT traffic: If feasible, block outbound traffic to public DoH/DoT servers (e.g., Cloudflare's 1.1.1.1, Google’s 8.8.8.8) at your firewall or perimeter security devices. This forces users to use your internal DNS servers, where you can monitor traffic and enforce security policies.
• Use DNS Filtering: Ensure that your DNS infrastructure uses secure, monitored, and controlled DNS servers that do not support DoH/DoT. You can configure local DNS servers to reject DoH/DoT traffic, ensuring that any DNS query is sent through traditional channels.
• Implement DNS Query Logging: Make sure DNS logs are detailed and capture all relevant information. Even if traffic is encrypted, DNS queries on traditional ports (e.g., 53) will still be observable, and if attackers use DNS tunneling or try to bypass filtering, it can help identify suspicious activity.

2. Inspect Encrypted Traffic (Decryption)

• SSL/TLS Inspection: Use SSL/TLS interception proxies, such as MITM (Man-in-the-Middle) proxies, that decrypt traffic and allow you to inspect HTTPS traffic. If DoH or DoT traffic is going to your environment over port 443 (or 853 in the case of DoT), you can intercept and decrypt this traffic.
• Network Traffic Capture Tools:

3. Detect Anomalous Traffic Patterns

Even though DoH and DoT traffic is encrypted, there are still ways to detect malicious activity using traffic behavior analysis.

• Monitor DNS query frequency: Malicious activity often involves frequent DNS queries to unusual or suspicious domains. Monitor for spikes in DNS queries to new domains or high request frequencies over short time periods.
• Traffic Profiling: Perform traffic profiling by analyzing network traffic patterns and identifying unusual communication patterns or hosts making DNS requests outside of normal business hours. For example, if a device starts making a large number of DNS requests to external servers (not just internal DNS), that could indicate data exfiltration or command and control traffic.
• DNS Query Size: Malicious DNS queries often involve larger-than-normal payloads, particularly with DNS tunneling (e.g., data exfiltration). Track unusually large DNS request sizes and correlate them with suspicious behavior.

4. Leverage Advanced Threat Detection Tools

• DNS Anomaly Detection: Use advanced network monitoring tools such as DNS Security Extensions (DNSSEC) or DNS anomaly detection solutions that analyze patterns in DNS traffic for unusual or malicious behavior. These tools can detect DNS-based attacks like DNS tunneling, DNS amplification attacks, and domain fluxing.
• Next-gen Firewall and IDS/IPS: Implement next-gen firewalls (NGFW) and IDS/IPS that provide DNS filtering capabilities or can identify encrypted DNS traffic patterns. These devices may have functionality to detect DNS over HTTPS or DNS over TLS traffic or identify suspicious traffic even without decrypting it.

5. Utilize DNS Logging and Monitoring Services

• Cloud DNS Security Solutions: If you're using cloud-based DNS solutions like OpenDNS or Cloudflare for Teams, take advantage of their DNS traffic monitoring features. These services often have built-in protections against DoH/DoT traffic and can help you monitor any encrypted DNS traffic attempting to bypass your network security.
• DNS Query Logging: Ensure that your internal DNS servers or proxies log all DNS requests, including metadata (IP, domain name, query type), so that even encrypted DNS traffic can be correlated and analyzed for unusual or malicious activity.

6. Threat Intelligence Feeds

• Threat Intelligence Integration: Incorporate threat intelligence feeds into your SIEM or DNS filtering tools to automatically block or flag DNS queries associated with known malicious IPs or domains. Threat intelligence providers may offer information on the use of DoH/DoT servers by threat actors, which can help you detect and block them proactively.

Practical Example: Implementing a DoH/DoT Detection Strategy

Let's assume you're working in a SOC and need to detect malicious activity involving DoH or DoT. Here's a step-by-step approach:

1. Monitor Outbound DNS Traffic: Set up firewalls or network traffic monitoring tools to detect outbound DNS queries on non-standard ports (443 and 853). Alert on traffic attempting to communicate with known DoH/DoT public servers like Cloudflare or Google’s DoH servers.
2. Enable SSL/TLS Inspection: If feasible, configure a proxy server or SSL/TLS interception mechanism to decrypt and inspect DoH/DoT traffic. If the proxy detects DNS requests that match known attack patterns (e.g., large DNS payloads, unusual domains), an alert is triggered.
3. Use DNS Anomaly Detection: Set up your SIEM to monitor for DNS traffic anomalies, such as unusual query patterns or sudden spikes in traffic to external IP addresses or domains.
4. Block Known Malicious Domains: Leverage threat intelligence feeds to block DNS queries to known malicious domains. You can use DNS blacklists or sinkholes to prevent malicious connections.
5. Threat Hunting and Post-Incident Analysis: Continuously hunt for signs of compromise in encrypted traffic. If an incident occurs, analyze traffic logs to identify signs of DNS tunneling or other evasion techniques. Correlate DNS query data with other logs (e.g., firewall, endpoint security, or authentication logs) to reconstruct the attack path.

Conclusion

While DoH and DoT present significant challenges for visibility and detection in your network, it is still possible to identify malicious activity using a combination of the following techniques:

• Blocking and controlling DoH/DoT traffic through network policies and firewalls.
• Decryption via SSL/TLS inspection or proxying to inspect encrypted DNS traffic.
• Traffic analysis using anomaly detection and pattern recognition to spot suspicious DNS queries.
• Integrating advanced threat detection solutions like DNSSEC and threat intelligence feeds to proactively block malicious domains and IPs.

By using these methods, you can effectively manage encrypted DNS traffic, enhance visibility.