AWS Data Lake Security Controls

Primary Components of a Data Lake:

Architecture Layer of a Data Lake without Security:

[1] Compute:? Server-based - EC2, EMR, Redshift; Serverless??- Lambda, Athena, API Gateway etc, Hybrid Model - Redshift Spectrum

[2] Metadata/Catalog:?Capture data lineage, Easy search with tags/business domain, Curate and assign relevancy score

[3] Storage: S3, EBS, Instance Store

AWS Security Services:

S3 Security and Compliance :

Access Control, Data Protection, Monitor and Audit Security Settings

AWS S3 Common Vulnerabilities:

• Unauthenticated Bucket Access
• Semi-Public Bucket Access
• Improper ACL Permissions

Data loss Prevention (DLP) - Amazon Macie

Data Encryption - Encrypt all data at rest : Server-side or Client-side Encryption

Assign bucket Policies :?Restrict by VPC, HTTPS, IP Filters, KMS Keys

Compliance - Log and Audit all the AWS Activity

Compliance Certifications -HIPAA,PCI-DSS, EfdRAMP, Versioning, Audit Logging

Amazon Redshift Audit Logging:?

• AWS CloudTrail - Redshift API Calls, KMS API Calls, S3 calls
• S3: Connection logs, User logs, User activity logs?

Data Lake Security Best Practices:

• Federate Access
• Setup roles and responsibilities matrix within your organization
• Leverage centralized data catalog
• Use both preemptive and detective controls
• Perform regular audits
• Secure storage, catalog and processing layers
• Incentivize teams to register datasets to catalog
• Streamline process between data producers and data consumers?