Advanced Threat Detection with AWS Security Hub and Amazon GuardDuty

Introduction

In today’s rapidly and dynamically changing cloud environments, robust security is more critical than ever. Advanced threat detection is a cornerstone of effective cloud security which helps organizations identify, prioritize, and mitigate security risks before they escalate, both proactively. AWS Security Hub and Amazon GuardDuty, two powerful security services from Amazon Web Services (AWS), work together to provide unparalleled visibility into your AWS environment.

In this article we are having a deep dive on how to integrate AWS Security Hub and Amazon GuardDuty for enhanced threat detection. We will explore the features, benefits, and best practices of using these services, providing not only actionable insights but also real world examples for effective implementation. In addition, we will also offer tips on how to scale these solutions for the most complex of environments, and discuss how automation and customization can improve your organization’s security strategy.

AWS Security Hub and GuardDuty: Key Features

Let's meet our two guests today, shall we?

Meet firstly AWS Security Hub and its features

• Centralized Security Management: This tool aggregates security findings from multiple AWS services and third-party tools into a single dashboard, to provide a single pane of glass view for all security operations.
• Compliance Automation: Guarantees compliance with standards like PCI DSS, CIS Benchmarks, HIPAA and GDPR through automated compliance checks for audit readiness and regulatory adherence.
• Customizable Insights: For particular needs, it ensures tailored security insights are created so organizations can focus on the issues that matter the most for them.
• Integration Capabilities: It integrates seamlessly with AWS native and third-party security tools in order to enhance detection, response and reporting capabilities.

And now meet Amazon GuardDuty

• Continuous Threat Detection: Monitors data sources such as VPC Flow Logs, DNS logs, and AWS CloudTrail to identify suspicious activities.
• Threat Intelligence Integration: Uses AWS and third-party feeds to detect known malicious activities, such as compromised IPs and domains.
• Scalable Monitoring: Supports multi-account and multi-region threat detection, ensuring comprehensive coverage across global environments.
• Automated Response: Works with AWS services like Lambda, Systems Manager, and EventBridge to automate incident response and remediation workflows.

When combined, Security Hub and GuardDuty give you a centralized, correlated view of threats across your AWS environment. This integration enhances visibility, reduces manual efforts and accelerates incident response times. Correlating findings from GuardDuty, Config and Macie, organizations can adopt a proactive, layered security approach, so that threats are addressed before they become problems that need fixing.

Setting Up GuardDuty and Security Hub Integration

Any organization wishing to enhance its security posture should consider integrating AWS Security Hub and Amazon GuardDuty. These services not only provide for advanced threat detection but also make sure that the centralized management of findings across accounts and regions is available. Thus, security teams have actionable insights and enhanced visibility with AWS Security Hub aggregating data from GuardDuty and other services like AWS Config and Macie. GuardDuty continuously monitors data sources for proactive threat detection, and Security Hub offers a unified platform on which findings can be correlated and responses automated.

The combination of these tools brings - as we will explore further together - significant benefits:

• Improved Visibility: We can consolidate our findings in a centralized fashion which ensures no threats go unnoticed, even in multi-account, multi-region setups.
• Automation and Efficiency: Processes prone to human errors can be optimized via reduction of manual workloads through automated detection and response workflows.
• Regulatory Compliance: With Security Hub and GuardDuty our ability to maintain adherence to compliance standards like PCI DSS, GDPR, NIS2 and HIPAA (some examples of the vast regulatory jungle out there) with built-in checks and actionable findings skyrockets.

Prerequisites:

Now let's make sure we have everything we need to start "cooking".

• Ensure AWS Security Hub and GuardDuty are enabled in your AWS account.

• Verify permissions using IAM roles with the necessary policies for accessing Security Hub and GuardDuty.
• Confirm both services are available in your desired AWS regions. (This should not pose any problems if you are working with the common regions. Mainland China may face some difficulties)

Step-by-Step Setup:

Now the fun part!

Enable GuardDuty:

In AWS CLI:

aws guardduty create-detector --enable --region us-east-1        

In Terraform:

resource "aws_guardduty_detector" "main" {
  enable = true
}        

In the console:

Enable Security Hub:

In AWS CLI:

aws securityhub enable-security-hub --region us-east-1        

In Terraform:

resource "aws_securityhub_account" "main" {}

resource "aws_securityhub_standards_subscription" "cis" {
  standards_arn = "arn:aws:securityhub:::standards/cis-aws-foundations-benchmark/v/1.2.0"
}        

In the console:

Integrate GuardDuty Findings into Security Hub:

• GuardDuty findings automatically populate Security Hub when both services are active in the same region.
• Configure EventBridge to trigger actions based on GuardDuty findings. This is of utmost importance for automation purposes:

aws events put-rule --name "HighSeverityGuardDutyFinding" --event-pattern '{
  "source": ["aws.guardduty"],
  "detail": {"severity": [{"numeric": [7]}]}
}'        

Multi-Account and Multi-Region Setup:

• Use AWS Organizations to enable GuardDuty and Security Hub across all linked accounts.
• Employ the Delegated Administrator feature for centralized management:

In AWS CLI:

aws organizations register-delegated-administrator \
    --account-id <delegated-admin-account-id> \
    --service-principal securityhub.amazonaws.com        

In Terraform:

resource "aws_securityhub_organization_admin_account" "main" {
  admin_account_id = var.delegated_admin_account_id
}

resource "aws_guardduty_organization_admin_account" "main" {
  admin_account_id = var.delegated_admin_account_id
}        

You are going to need multiple accounts in order to implement this solution.

Aggregate Findings:

Configure Security Hub to aggregate findings from all linked accounts and regions to the management account:

In AWS CLI:

aws securityhub update-finding-aggregation-configuration \
    --region-linking-mode ALL_REGIONS \
    --account-aggregation-source '{"AccountIds":["<linked-account-id>"], "AllRegions":true}'        

In Terraform:

resource "aws_securityhub_finding_aggregation" "example" {
  region_linking_mode = "ALL_REGIONS"
  account_aggregation_source {
    account_ids = var.linked_account_ids
    all_regions  = true
  }
}        

To ensure a scalable and centralized approach to security monitoring and management across your AWS environment, these steps should be followed. This configuration is particularly important for visibility across multiple accounts and regions, and is useful for organizations with a complex topology of distributed operations.

Understanding GuardDuty Findings in Security Hub

To be successful with GuardDuty we have to understand first and foremost how GuardDuty works and how it classifies findings.

GuardDuty generates findings based on suspicious activities, classified by threat types and severity levels:

Types of Threats Detected:

• Reconnaissance: Unusual API calls, IP address scanning, or port scanning activities.
• Data Exfiltration: Abnormal access to S3 buckets or large-scale data transfer volumes.
• Unauthorized Access: Usage of compromised credentials or unauthorized escalation of privileges.
• Malware: Detection of malicious scripts, processes, or unauthorized software installations.

Severity Levels:

• Low: Issues that indicate potential vulnerabilities or misconfigurations, such as failed login attempts.
• Medium: Indicators of compromise that require investigation.
• High: Clear evidence of active threats, such as unauthorized data access or malware activity.

Correlation of Findings: AWS Security Hub consolidates GuardDuty findings with insights from services like Amazon Macie and AWS Config, providing a unified security posture. For example:

• Amazon Macie: Flags anomalies such as sensitive files being downloaded from an S3 bucket by untrusted IPs. These findings are immediately surfaced in Security Hub for review. Also Macie can identify if sensitive data are being stored in public S3 Buckets (such as Credit Card numbers)
• AWS Config: It monitors resource configurations to identify noncompliance or misconfigurations such as publicly accessible S3 buckets or IAM roles with excessive permissions. These findings are also integrated into Security Hub to enable cross service correlation.

By combining findings from these services, Security Hub provides security teams with actionable insights and a comprehensive view of their AWS environment’s security posture, enabling faster remediation and improved compliance tracking.

Creating Custom Insights in Security Hub for GuardDuty Findings

What Are Custom Insights? Custom insights are the tailored views of security findings based on certain criteria such as threat types, affected resources or regions in AWS Security Hub. These insights support security teams to identify and take action on high priority issues without having to dig through unnecessary data.

Some Examples of Custom Insights:

• High-Severity GuardDuty Findings Across All Regions: Create an insight to filter GuardDuty findings with a severity greater than 7 across all AWS regions. This ensures immediate focus on the most critical threats. Example filter:

{
  "Filters": {
    "Severity": [{ "Comparison": "GREATER_THAN_OR_EQUALS", "Value": "7" }],
    "Region": [{ "Comparison": "EXISTS" }]
  }
}        

• Sensitive Data Access in S3: Use Macie findings to create an insight for anomalous access to sensitive data stored in S3 buckets. Combine this with guardrails for unauthorized access prevention:

{
  "Filters": {
    "ResourceType": [{ "Comparison": "EQUALS", "Value": "AWS::S3::Bucket" }],
    "FindingType": [{ "Comparison": "EQUALS", "Value": "SensitiveData:S3" }]
  }
}        

• Compromised IAM Credentials: Focus on detecting and prioritizing IAM-related anomalies, such as unusual API calls or suspected credential exfiltration:

{
  "Filters": {
    "FindingType": [{ "Comparison": "EQUALS", "Value": "UnauthorizedAccess:IAMUser/InstanceCredentialExfiltration" }],
    "Severity": [{ "Comparison": "GREATER_THAN_OR_EQUALS", "Value": "6" }]
  }
}        

Through these tailored views, security teams can focus their efforts on the most critical security issues, thus ensuring faster responses and more efficient resource allocation.

Steps to Create Custom Insights:

• Open the Security Hub console.

• Navigate to the Insights tab.

• Click Create Insight and define filters.

• Save and visualize the insight on the dashboard.

Some more complicated examples of Custom Insights:

• Monitoring Unauthorized Access: To this end, an insight is to be created to track GuardDuty findings that pertain to unauthorized API calls made from untrusted IPs. This should be achieved using EventBridge and Lambda to automatically disable the associated IAM user or notify the security team.

Example EventBridge Rule:

{
  "Name": "HighSeverityGuardDutyEventRule",
  "EventPattern": {
    "source": ["aws.guardduty"],
    "detail-type": ["GuardDuty Finding"],
    "detail": {
      "severity": [{"numeric": [7]}],
      "type": ["UnauthorizedAccess:IAMUser/InstanceCredentialExfiltration"]
    }
  },
  "State": "ENABLED",
  "Targets": [
    {
      "Id": "LambdaFunctionTarget",
      "Arn": "arn:aws:lambda:us-east-1:123456789012:function:DisableCompromisedUser",
      "InputPath": "$.detail"
    }
  ]
}

{
  "source": ["aws.guardduty"],
  "detail-type": ["GuardDuty Finding"],
  "detail": {
    "severity": [{"numeric": [7]}],
    "type": ["UnauthorizedAccess:IAMUser/InstanceCredentialExfiltration"]
  }
}        

Example AWS Lambda Automation:

import boto3

def lambda_handler(event, context):
    # Extract IAM User details from the event
    iam_user = event['detail']['resource']['accessKeyDetails']['userName']

    # Initialize IAM client
    iam = boto3.client('iam')

    # Disable the user and their associated access keys
    access_keys = iam.list_access_keys(UserName=iam_user)
    for key in access_keys['AccessKeyMetadata']:
        iam.update_access_key(
            UserName=iam_user,
            AccessKeyId=key['AccessKeyId'],
            Status='Inactive'
        )

    # Log the action
    print(f"Disabled access for user: {iam_user}")        

• S3 Data Anomalies: To identify irregular data access patterns in S3 buckets, use a combination of GuardDuty and Macie for unexpected geographic locations or anomalous access times. Automate responses like restricting bucket permissions or initiating forensic analysis using Security Hub findings. For example, set a Lambda function to update the bucket policy to restrict access to sensitive files, or integrate with EventBridge to have the security team notified of high risk activities.

Custom Insight Filter Example for S3 Data Anomalies:

{
  "Filters": {
    "ResourceType": [{ "Comparison": "EQUALS", "Value": "AWS::S3::Bucket" }],
    "FindingType": [{ "Comparison": "CONTAINS", "Value": "SensitiveData:S3" }],
    "Severity": [{ "Comparison": "GREATER_THAN_OR_EQUALS", "Value": "6" }]
  }
}        

Automating Incident Response

Incident response automation is an important step for keeping your AWS environment secure and able to withstand threats. Using automation with AWS Security Hub and GuardDuty organizations can cut response times dramatically, reduce the likelihood of manual errors and help to ensure that security incidents are handled consistently.

Why It’s Necessary:

• Fast Threat Mitigation: Automation helps in containing threats as soon as they are detected thus reducing the level of potential damage.
• Consistency and Reliability: This ensures that incidents are always handled in the same performing way repetitive by tasks automated like responses isolating following compromised predefined resources rules.
• Resource Optimization: Reduces the workload on security teams by handling repetitive tasks like isolating compromised resources or revoking access.
• Regulatory and Compliance: Automated workflows can ensure timely responses and accurate logging of security events, helping to meet audit and compliance requirements.

This section describes practical use cases and provides examples of how automation can be effectively used to secure your AWS workloads.

Using AWS Systems Manager Documents (SSM Documents): SSM Documents are JSON or YAML scripts that describe what actions to perform on AWS resources. These are particularly useful for automating the response to security incidents identified by Security Hub and GuardDuty.

Example Use Case: Isolating a Compromised EC2 Instance

There are several times where our resources might become the target of malicious actors. EC2 instances make no exception to that case. We need therefore a plan in order to isolate an instance once we notice that it became compromised. This is where GuardDuty and Security Hub can bring AWS SSM into play.

1. Isolate the Instance: Attach a restrictive security group to cut off all inbound and outbound traffic.
2. Collect Forensic Data: Use Systems Manager to run commands on the instance, such as capturing system logs or memory dumps for further analysis.
3. Take a Snapshot: Create a backup of the root volume to preserve the instance's current state for forensic purposes.
4. Patching or Termination: Depending on the severity of the issue, either apply patches or terminate the instance to prevent further damage.

SSM Document JSON Example:

{
  "schemaVersion": "2.2",
  "description": "Isolate a compromised EC2 instance and collect forensic data.",
  "mainSteps": [
    {
      "action": "aws:runCommand",
      "name": "IsolateInstance",
      "inputs": {
        "DocumentName": "AWS-RunShellScript",
        "Parameters": {
          "commands": [
            "aws ec2 modify-instance-attribute --instance-id {{InstanceId}} --groups sg-xxxxxxxx"
          ]
        }
      }
    },
    {
      "action": "aws:createImage",
      "name": "CreateSnapshot",
      "inputs": {
        "InstanceId": "{{InstanceId}}",
        "NoReboot": true
      }
    }
  ]
}        

Through automation, organizations are able to provide timely and consistent security incident responses with minimal manual effort and reduced risk of human error.

Take this example for instance. Once my instance has been labeled as compromised, I didn't need to do anything. GuardDuty, Security Hub and SSM took care of everything and I got only informed. This demonstrates the power of automation and the promptness of my incident response plan.

Best Practices for Advanced Threat Detection

1. Enable Multi-Region Monitoring:

• Have GuardDuty and Security Hub across all AWS regions activated to prevent blind spots by ensuring that no malicious activity goes undetected in lesser-used regions. I know it sounds costly but without it it can be more costly
• Use AWS Config to continuously monitor and ensure resource configurations comply with security standards in all regions, such as enforcing encryption for RDS instances or ensuring S3 buckets are not publicly accessible. These configurations will not only help you for your security posture but also for audit purposes.
• Leverage AWS Organizations to automate the enablement of these services across all accounts, simplifying the process for large, multi-account setups. Governance is not to be taken lightly. Use strict rules for your cloud hierarchy and policies.
• Use EventBridge to create cross-region event rules for centralized visibility, allowing findings from all regions to initiate automated workflows in case of an incident. Like we said "Automation is very useful".
• Use Terraform or AWS CloudFormation to deploy GuardDuty and Security Hub configurations across multiple accounts and regions, ensuring consistency in security operations. Thus you can have a centralized way of managing your resources

2. Minimize False Positives:

• Use suppression rules in GuardDuty for non-critical activities and regularly update trusted IP lists to focus on genuine threats. Believe me you don't want to be notified about a public bucket that you configured it that way on purpose.
• Feel free to fine-tune your severity filters to ensure findings align with your organization's risk tolerance. Not everything is critical ;)
• Automation, Automation, Automation!! Leverage automation tools like AWS Lambda to suppress repetitive findings dynamically and improve overall efficiency in processing alerts. Not doing so may lead to idle responses due to the "needle in a haystack" effect, where alerts don't have great significance.

3. Leverage Third-Party Tools:

If you have already solutions in place where your infrastructure and landscape is being monitored, well this is your lucky day. AWS GuardDuty and Security Hub offer plenty of integrations with SIEM solutions like Splunk, Datadog, or QRadar for extended analytics and advanced threat correlation.

Conclusion

Using AWS Security Hub in conjunction with Amazon GuardDuty allows organizations to get threat detection and a centralized view for all of their AWS environments. Both services can be used to help businesses defend against threats by identifying security risks so they may be remediated, response actions can be automated, and compliance with industry standards can be maintained.

To enhance your implementation, you could build a multi-region Security Hub analytics pipeline or explore integrations with third party tools to extend your capabilities. Furthermore, focus on automation and customization to ensure these tools meet your organization’s needs. Remain vigilant and incorporate these best practices to enhance your cloud security posture for comprehensive threat detection.

And remember; Security can be expensive but have you tried it without it?