Defending against dastardly cloud data breaches

It's a tough time to be an Australian. It's also hard to find someone not personally affected by the recent data breaches, some of which are the largest to date in Australian history. Ideally I'm sure we'd all like to trust the organisations to whom we provide our personal data, and that they would adequately protect our information.

Unfortunately you can't ever be 100% sure about something until after the fact. Even with a mature and comprehensive cybersecurity programme you don't know your security is inadequate until after an incident has occurred. Despite the significant media attention on a handful of recent incidents, these events have highlighted that breaches frequently occur in organisations of all sizes, and aren't limited to specific industries. But why, and what can organisations do to reduce the likelihood of data being exfiltrated from their environment?

Too often I've observed organisations focusing only on protecting themselves against malicious inbound threats, with little to no consideration given to scenarios where attackers may be inside your network locating and extracting valuable data. One of the most common issues I've seen is organisations lacking outbound network traffic controls, and visibility into what data is going where, and to whom, especially when it's leaving their environment.

If you're not sure if this affects you, look at, or ask your network team to explain your ingress (inbound) and egress (outbound) firewall and network security rules and controls. If you're using a cloud provider it's likely that you will have specific ingress rules for your workloads, with your outbound rules possibly being left as the default:

Outbound rules like the ones shown means any and all outbound traffic originating from within the subnet would be allowed. Cloud environments have high-speed internet access via provisioned internet, NAT, and service gateways and in the absence of other network controls, attackers once inside can quickly and easily send large amounts of data out of your environment.

Before discussing some practical solutions it is worth understanding some common methods attackers use to exfiltrate data from environments.

Common data exfiltration techniques

Attackers are skilled adversaries proficient in data exfiltration techniques. Whilst it's not possible to list them all here are some of the most common. In all of these examples, the attackers make outbound connections to endpoints and services which they control, can control, or can monitor.

• Hypertext Transfer Protocol, commonly called HTTP. HTTPS (HTTP Secure) is just plain-text HTTP being sent over an encrypted TLS connection often called "encryption in transit". Encryption in transit offers no protection against data exfiltration which is a common misconception. Exfiltration via HTTP or HTTPS is arguably the most common technique used given its prevalence in environments and is almost always permitted. Its prevalence also makes it easier for attackers to hide their activity amongst other legitimate traffic.
• File Transfer Protocol (FTP) and its variants such as FTPS or SFTP, or other protocols that support file transfer. Again, file transfer protocols are common in environments and are unlikely to disappear anytime soon. FTPS and SFTP like HTTPS whilst providing encryption in transit, offer no protection against data exfiltration. File transfer protocols also offer an easy way to transfer large files and are commonly installed as part of the underlying operating system meaning attackers don't need to install custom tools.
• Object Storage Buckets. Applications hosted in the cloud commonly use object storage services from the hosting provider to store unstructured data such as user uploaded files. Compute instances are often configured with access to read and write files in buckets, and if a bucket is public, or can be made public via a too-permissive IAM policy then an attacker can simply copy files there and download them. Without access logs enabled it would not be possible to ascertain when the files were download, or by which IP address making this a stealthy option for attackers.
• Simple Mail Transfer Protocol (SMTP), or any of the protocols that support email services. It's not uncommon for applications in environments to send emails via these protocols so are also commonly permitted. Email protocols also support attaching files making it easy for attackers to hide their activities in what may seem like innocuous legitimate traffic.
• Domain Name Service (DNS), a core component of any IP based network. Attackers will often resort to DNS in the absence of other options. Given the limitations of DNS, data must be sent in small chunks and this often makes it slower and more noisy than the other options mentioned.

For fear of boring you with an endless list I won't explain any more, suffice to say without outbound security controls attackers could use any technique or protocol to exfiltrate data from your environment.

Practical solutions to help prevent data exfiltration

As I said you can never be 100% certain about anything, nor is it possible to build and maintain an impenetrable environment. Like it or not, connectivity is required for IT systems to operate and for people to use them. Here's my thoughts on how organisations can better protect their environments from data exfiltration and subsequent disclosure.

Step 1. Constrain your outbound traffic flows

It seems simple but before you can constrain your flows, restricting the inbound and outbound traffic to only that which is required, you first must understand what the rules need to be. It's likely you don't allow all ports and protocols inbound and so it seems only sensible to apply the same scrutiny to outbound traffic. By constraining your outbound traffic flows, you're immediately limiting an attackers ability to arbitrarily connect to other ports and services in your environment or to those on the internet. When reviewing your rules, here are some things to consider:

• Don't allow unfettered egress. For all non internet facing subnets update the outbound 0.0.0.0/0 rules to be the exact CIDR's required.
• Avoid placing resources that need to be public such as load balancers with hosts that don't in the same subnet. Remember the outbound rules for those public subnets will likely allow egress to 0.0.0.0/0 unless you want to have mixed rules applied to the same subnet.
• Scope security rules to the minimum required ports, protocols, sources and/or destinations. It might seem silly but ICMP (ping) outbound should also be disabled unless there is a specific business requirement as like DNS it can be used to exfiltrate data.
• Source and/or destination CIDR ranges should be limited to the absolute minimum required. If a /32 is all that's required (a single IP address), then make it a /32. If you're sending data over the internet, ensure you confirm the 3rd party owns the IP's or ranges and that they are not going to change.

Step 2. Harden your internal environment

By constraining your traffic flows you should now understand what outbound connections are made from your environment and the ports and protocols required. The next step is to limit what data is readily available, implement solutions to improve visibility, and regain control over the data being sent across your network, including ascertaining the exact traffic protocol rather than just assuming based on the port number.

• Implement a network firewall with threat intelligence capabilities to detect and block outbound connections to known bad IP addresses and malicious FQDN's / domains. Ensure threat intelligence databases used are updated in near real-time as attackers frequently change their infrastructure.
• Configure your network firewall to only permit protocol and port combinations you require such as HTTPS traffic on TCP port 443 or SMTP traffic on TCP port 25. This prevents attackers using arbitrary protocols and hopping ports. If you're using a Palo Alto firewall this feature is called App-ID.
• Deploy network canaries such as those by Thinkst Canary. Canaries mimic high-value network devices and application services such as databases. Attackers hunting around in your environment are likely to trigger canaries as they search for data to exfiltrate. Because canaries aren't known to your applications or users, all alerts raised are excellent Indicators of Compromise (IoC's).
• Insert canary data at the beginning and also randomly in your datasets. These are essentially records which aren't used and should never be accessed. Build logic in your applications to raise appropriate alerts when canary data is accessed.
• Implement a HTTP(S) forward proxy in your environment. A forward proxy allows you to inspect outbound web traffic, including that which may be sent encrypted over HTTPS. Creating rules to raise alerts when canary data is detected is another way to identify unauthorised data access, especially if you can't modify exiting applications.
• Compartmentalise your data to reduce the amount that can be easily retrieved. If you need to keep records for historical purposes, consider storing them offline or in a separate enclave with strict access controls such as multi-factor authentication (MFA).
• Use field-level encryption to render exfiltrated data useless. As an example the Payment Card Data Security Standard (PCI-DSS) mandates this for credit card transaction processors.
• Regularly test your egress controls. If you've implemented a network firewall for all outbound traffic and a forward proxy for all web traffic you should regularly confirm that all traffic is inspected, that only allowed traffic flows are permitted, and that your controls cannot be trivially bypassed.

For completeness it's worth noting that this is not an exhaustive list on how to harden your cloud environment. The Australian Cyber Security Center has issued guidelines which are a good read if you're interested.

Step 3. Monitor your traffic flows

By limiting and validating all outbound connections you've restricted the number of paths attackers can use to exfiltrate data. Unfortunately technical controls by themselves cannot completely eliminate the risk of data exfiltration. Correctly configured monitoring and Security Incident Event Management (SIEM) solutions are a common way to detect unauthorised access in your environments.

• Monitor all your traffic flows and compare it against a recent baseline. Unexpected increases in traffic, such as a 20% increase for a specific protocol, or a spike in traffic during an often quiet period, or outbound traffic to an IP address never seen before are all events which should be investigated. Most SIEM's come with out of the box rules. but I would recommend you build and test rules based on your specific requirements.
• Monitor rejected traffic flows using network flow and firewall logs. Rejected traffic may be indicative of a missing rule that's causing an operational issue, or an attacker attempting to connect outbound using a port, protocol, or destination that's not permitted. The stricter your outbound rules, the more likely a rejected log entry will occur as attackers attempt to make outbound connections.
• Monitor Virtual Private Network (VPN) and other remote networking connections. Many remote workers use VPN connections to access internal systems and compromised accounts have been attributed to recent data breaches. Monitor client connecting IP addresses, activity times, session duration, bandwidth, traffic consumption, and MFA challenge responses. Client IP address changes or deviations from expected behaviour should be investigated.
• If you're using field-level encryption, monitor decryption requests. An increase in decryption requests, or requests from an unusual IP address may indicate an attacker is attempting to decrypt values before exfiltrating the data.
• If you have compartmentalised your data, monitor access to data enclaves.
• Monitor DNS queries. It might seem excessive however DNS is a protocol that is almost guaranteed to be permitted, and attackers will default to DNS exfiltration if there are no better options available. Spikes in DNS requests to uncommon domains are a good indicator of exfiltration.

Data breaches are not new. Current trends indicate the number of breaches, the number of records exposed, and the number of individuals impacted will continue to climb. As someone once said "hope is not a strategy" and I would encourage everyone think about how they can better protect their organisation against a data breach.

If you have any other suggestions, please feel free to share them in the comments.