How can security data lakes be 10 times more cost effective than traditional SIEM? It's about the architecture they use to store and process the data. Traditional?SIEM?tightly couples compute and storage, even when it runs in the cloud. That matters because typical storage for cloud VMs is way more expensive than blob storage. It also requires the servers to be running 24x7 in order for the storage to be accessed. Compare that to the security data lake model, where the data goes straight to cloud-native storage (e.g. S3 in AWS or blob storage in Azure). When it needs to be queried, compute resources (e.g. a virtual data warehouse in Snowflake) are spun up to search the data and then quickly suspend when the investigation or threat hunt is over. So there are large portions of the week where only minimal compute is active to load the streaming data to storage. This approach of resources being applied elastically "just in time" as needed by the security team is much more aligned to the nature of security operations. The cost shifts from ingest to usage. Not paying for idle SIEM resources drives big savings. Long term, it's inevitable that SIEM solutions that stick with a traditional architecture will lose out to SIEM solutions that run on top of security data lakes. Then the question becomes which solution best operationalizes the security data lake for your team's requirements. #cybersecurity #securitydatalake
Agree! data lakes offer improved scalability, flexibility, and cost-effectiveness compared to traditional SIEM, making them a valuable addition to modern security infrastructures.
This is something similar to the Splunk S2, where the SIEM’s loca storage acts as a cache and the primary storage is a remote object store such as S3. Not really a data lake, however it does decouple compute from storage requirements. I would like to see more SIEM solutions head this route as you are saying.
I will come work for you for free
Great post Omer. Couldn't agree more. Snowflake is the disruptive and most mature model, built from the ground up for Cyber Security Workloads. And further, to your point "Then the question becomes which solution best operationalizes the security data lake for your team's requirements" is critical.
Well said as usual Omer Singer
Echo that, Omer Singer ...we are seeing enterprises wanting to migrate from legacy monolithic stacks to Anvilogic + Snowflake for a modern and cost-effective SIEM strategy. The world is changing.