Busting the Myth of Security Market Consolidation III

The Evolution of Security Operations Technologies: SIEM, XDR, Security Data Lakes and Fabrics

In part I of the series we spoke about why platform consolidation is a myth from a technical perspective, and that the egg-laying milk wool pig is an impossible creature. Part II outlined many of the counter-drivers and inhibitors acting against market consolidation in security. This week, we discuss some of the major shifts occurring in security operations.?

Why It Matters

The cybersecurity landscape is rapidly evolving, with Security Information and Event Management (SIEM) solutions facing competition from emerging technologies like Extended Detection and Response (XDR) and security data lakes. Understanding these trends is crucial for organizations looking to strengthen their security posture in an increasingly complex digital world. As cyber threats become more sophisticated, the tools to combat them are diversifying, creating both opportunities and challenges for security professionals.

By the Numbers

• Market research firms estimate that the SIEM market grew between 5.5% and 17.5% CAGR, with market size estimates ranging between $5.03 billion and $17.97 billion, demonstrating continued strong demand for these solutions.
• XDR adoption is expected to rise from less than 5% today to 30% by 2028, according to industry predictions.
• Security data lakes currently represent about 5% of the market but are projected to grow to 10% by 2028.

The Big Picture

As cyber threats become more sophisticated, the tools to combat them are diversifying. This "Cambrian explosion" in cybersecurity solutions offers more options but also creates challenges in integration and strategy selection.?

What to watch

• Further convergence of SIEM, XDR, CDR and ITDR technologies
• Adoption of Cybersecurity Mesh Architecture (CSMA) and similar frameworks for more integrated security ecosystems
• Greater penetration of security data lake solutions among large enterprises
• Evolution of SOCless/SIEMless and distributed security operations models

The Bottom Line

The future of security operations isn't about finding a single, perfect solution. It's about creating a flexible, integrated ecosystem of security tools tailored to each organization's unique needs and risk profile.

Introduction

Keeping up with cyber threats necessitates ongoing adaptation and innovation. One area that has seen significant changes over the years is Security Information and Event Management (SIEM). As we navigate through 2024 and into 2025 and beyond, it's crucial for security operations leaders and investors to understand the current state of SIEM, its relationship with emerging technologies like Extended Detection and Response (XDR) and security data lakes, and what the future might hold for security operations.?

The Evolution of SIEM

SIEM technology has come a long way since its inception. Originally derived from Security Information Management (SIM) and Security Event Management (SEM), SIEM was coined by Mark Nicolet of Gartner to describe a centralized approach to security operations activities. Over time, SIEM has evolved to accommodate changes in data management, moving from SQL to NoSQL, embracing big data, and adapting to cloud environments and data lakes.

SIEM Generations

1. First Generation—the Intrusion Detection Years: Driven by the need to monitor a growing attack surface, first IDS consoles and systems focused on log aggregation and log search appear, primarily for intrusion detection and network security system monitoring. DragonIDS, ISS Proventia, and Syslog all presage what will become modern SIEM.

2. Second Generation—the Compliance Years: The main driver was compliance. SIEMs expanded the range of supported data sources and introduced real-time correlation capabilities, reporting, and dashboards.? Arcsight is the apex predator in this age, soon to be joined by QRadar, Nitro, and champion of small businesses and MSSPs, Alienvault.

3. Third Generation—the Age of Reckoning: The threat landscape turned dangerous. SIEMs, initially blindsided after years of easy compliance budgets, embraced big data and nosql technologies and began incorporating analytics such as statistical anomaly and outlier detection and machine learning capabilities, including unsupervised clustering. Splunk enters the fray and shifts the goalposts of what a SIEM should look like. Other major players include LogRhythm, IBM, McDonald's, and RSA. Arcsight begins its precipitous decline.

4. Fourth Generation—the advent of AI and automation: Security is a boardroom-level concern. The modern SIEM is cloud-native,? built on a data lake or data warehouse, and comes with integrated capabilities such as UEBA, SOAR, and increasingly AI Copilots. The range of firms is dizzying: Microsoft Sentinel, Exabeam, Securonix, Exabeam, Palo Alto, Crowdstrike, Hunters Security, Panther, and Google are just some of the players active in the market today.

Each generation of SIEM has built upon the strengths of its predecessors while addressing new challenges and leveraging emerging technologies.

Today, a modern SIEM is essentially a bundle of content—including connectors, parsers, correlation rules, dashboards, and reports—developed for specific data stacks to serve tailor-built machine learning and other analytics capabilities. Some SIEM now come “decomposed” or federated, like Anvilogic, and allow you to decouple the data layer. Others are integrating graph databases, like Microsoft or Hunters Security, while others are adding vector embeddings, like Auguria. This evolution has allowed SIEMs to remain relevant in an increasingly complex digital landscape.

It's worth noting that the SIEM market has seen significant changes in recent years. For example, since 2016 , the number of players in the SIEM market has increased from 14 to 22 in 2024 , with 13 net new entrants. Large players that just two years ago were proclaiming the death of SIEM by XDR, now publicly scrap to be seen as SIEM leaders. The space is undergoing a renewed burst of innovation and competition, driven by the need to address evolving security challenges that fundamentally haven’t changed in decades.

The market itself is stable and mature. Market research firms estimate that the SIEM market grew between 5.5% and 17.5% CAGR, with market size estimates ranging between $5.03 billion and $17.97 billion, demonstrating continued strong demand for these solutions (see appendix).

SIEM vs. XDR: Understanding the Differences

New technologies, such as Extended Detection and Response (XDR), have come about as cybersecurity needs change and grow. This has led to some confusion about the relationship between SIEM and XDR.

XDR is often described as an evolution of Endpoint Detection and Response (EDR), optimizing threat detection, investigation, response, and hunting in real-time. It unifies security-relevant endpoint detections with telemetry from various security and business tools. While there is some overlap with SIEM functionality, XDR is typically more focused on native detection capabilities.

The key differences between SIEM and XDR can easily be summarized:

1. Use-case coverage: SIEM tends to offer broader coverage, while XDR provides deeper, more specialized detection capabilities.

2. Detection & Analytics Engineering Effort: XDR generally requires less effort, as it comes with more out-of-the-box content.

3. Customization: SIEM offers more flexibility for organizations that need to develop their own content and use cases.

4. Data Integration: While both handle multiple data sources, XDR is often more tightly integrated with specific vendor ecosystems, whereas SIEM can be more vendor-agnostic.

5. Focus: XDR tends to be more focused on threat detection and response, while SIEM often covers a broader range of use cases, including compliance and general security monitoring.

The Rise of Security Data Lakes and SIEMless Models

While SIEM and XDR dominate the market, some organizations are exploring alternative approaches. Security data lakes are experiencing a burst of popularity. Others are experimenting with automation and analytics, like Netflix's "SOCless Detection Team " concept, introduced in 2018, that did away with a SIEM entirely and instead showcased a stack of different data and micro-solutions built on a security data lake architecture.

SOCless, or better SIEMless

This “SOCless” or “SIEMless” model is particularly appealing to lean-forward organizations, typically found among Fortune 1000 companies, FAANG (Facebook, Amazon, Apple, Netflix, Google), and other tech giants. These companies often have the resources and expertise to build custom solutions tailored to their specific needs and are often engineering-led, with development and infrastructure teams deeply embedded in the security process.

In a SIEMless or SOCless operating model, instead of having analysts sit in front of a console with eyeballs on logs all day, more effort is placed into high-precision detection engineering, more sophisticated analytics for prioritization and enrichment, and aggressive workflow automation for alerting, forensic investigations, and semi-autonomous containment.?

Instead of basing their detection and response program around a SIEM, security responders are alerted using a mechanism like a slack message or a service like Pager Duty, based on high precision detections, with supporting forensic data prefetched and presented for quick review. It’s not a model for everything and everyone. Large, complex heterogeneous environments will struggle to maintain the consistent quality of detections across attack surfaces and use-cases and will be underwhelmed by the lack of generalizability in automation. But for operational teams, for example, protecting a SaaS tech stack, it can be an effective model.

Security data lakes?

Security data lakes are emerging as an alternative to traditional SIEM and newer XDR solutions, offering a way to store and analyze vast amounts of structured and unstructured security data. They are great for advanced analytics and machine learning use-cases but are more complex to set up compared to SIEMs if intended as a full-blown replacement. Traditional SIEMs only retain data for live search for a short period of time, usually days to weeks, and they are designed as data sinks, not as data distributors.? More commonly, large and mature teams are deploying a data lake alongside their SIEM, using the SIEM for correlation, incident management, and reporting, but also utilizing a data lake for long-term historical search, storage, and distribution.?

?Security data lakes offer flexibility, especially for organizations with mature security teams and resources for heavy data engineering. However, for smaller organizations, this complexity can be overwhelming, making traditional solutions like SIEM or XDR more practical. The key trade-off is that data lakes require significant investment in infrastructure and talent to extract actionable insights.?

Currently, only about 5% of large enterprises use security data lakes, but that number could double by 2028. Still, most organizations prefer ready-made solutions like SIEM, much like most people buy cars instead of building their own.

This means that despite the buzz around data lakes, SIEM and XDR will still reign supreme in many environments due to their broader applicability, established ecosystem, and lower expertise and labor requirements.?

The truth, however, is that most modern SIEM and XDR solutions are themselves built on top of a data lake or lake house architecture. What changes is how much of the “bare metal” is exposed to the user. With a pure security data lake, it’s all of it. In the case of XDR, very little. Users can interact with the data via the UI or an API, but not on the data plane.

The SIEM-XDR-Security Data Lake Spectrum

Rather than viewing SIEM, XDR, and Security Data Lakes as entirely separate categories, it can be helpful to think of them as points on a spectrum of security information management solutions. Some vendors are blending SIEM and XDR capabilities or enabling SIEM-like operations on data lakes , creating hybrid solutions that aim to offer the best of several worlds. As the market evolves, we will see further convergent evolution between these technologies.

Emerging Security Operations Frameworks and Operating Models

Security Data Fabric?

In discussing data lakes, it's worth mentioning the concept of a data fabric. According to? Gartner’s Data fabric ? has “emerged as a solution to the common challenge of collecting, connecting, integrating, and delivering data from dispersed data sources to the users who need it." It is basically a design concept for an integrated layer of data and connecting processes. It recommends continuous analytics of discoverable metadata assets to support the design, deployment, and utilization of integrated and reusable data across any environment, including hybrid and multi-cloud platforms.

The idea of a "Security Data Fabric" has been proposed as an extension of this concept specifically for security data. However, it's important to note that this term is in the process of being trademarked by a specific vendor (Avalor, now owned by ZScaler), which very likely will limit its adoption as a general market category by an industry research or advisory firm.

Cybersecurity Mesh Architecture

Another concept that's gaining a small amount of early traction is the Cybersecurity Mesh Architecture (CSMA). As defined by Gartner, CSMA is a “collaborative ecosystem of tools and controls designed to secure a modern, distributed enterprise. It aims to integrate composable, distributed security tools by centralizing the data and control plane, enabling more effective collaboration between different security solutions.”

This approach aligns well with the idea of creating interoperable ecosystems of security tools rather than relying on a single, monolithic solution. As organizations continue to deal with increasingly complex and distributed IT environments, CSMA, or something very similar to it, could play a crucial role in tying together various security technologies, including SIEM, XDR, and security data lakes.?

Google’s Autonomic Security Operations Concept

Google’s Autonomic Security Operations (ASO) is a framework that enhances SOCs by integrating automation and cloud-native technologies to manage modern cybersecurity challenges. ASO focuses on improving four pillars: people, process, technology, and influence, with the goal of automating routine tasks, increasing detection and response speed, and addressing the complexity of cloud and hybrid environments.

ASO emphasizes automation through machine learning and SOAR tools, streamlining incident handling while allowing security teams to focus on higher-priority tasks. By integrating security deeply with DevOps and IT processes, ASO ensures that security operations can scale efficiently, balancing automation with the need for human oversight in addressing complex threats.

What about Security Telemetry Pipelines?

[Disclaimer: I work for a Security Telemetry Pipeline startup called Auguria ]

One emerging technology to enable users to build and deploy federated and decomposed security operations architectures is security telemetry pipelines. Security telemetry pipelines can automate data ingestion, normalization, transformation, and refinement and enrichment,, typically integrating a data lake for raw data storage, replay, and compliance, and can route data optimally prepared to different security data destinations, including SIEMs, data lakes, SOAR, AISecOps or cloud object storage.

Watch out next month for the free special report, Security Telemetry Pipelines Market Overview and Dynamics. Follow the Cyberfuturists if you want to be notified?

Market Trends and Future Outlook: Integrated, not Consolidated

Despite predictions of market consolidation, the cybersecurity landscape continues to expand. According to Gartner's Market Guide for Extended Detection and Response (published August 2023) by the end of 2028, XDR will be deployed in 30% of end-user organizations, up from less than 5% today. Meanwhile, according to another report by the same firm, the SIEM market grew from $5.03 billion in 2022 to $5.7 billion in 2023, with a 13% annual growth rate.

This growth across multiple adjacent markets suggests that we're experiencing what I’ve described as the "Cambrian Explosion " in cybersecurity. Rather than consolidation, what we're actually seeing is horizontalization, due to a diversification of solutions to meet the varied needs of different organizations.

An interesting point to consider is the current state of market consolidation in cybersecurity. Despite expectations of consolidation, the market actually remains quite fragmented. For example, research firm Canalys estimates that in 2024, the top 20 global cybersecurity vendors accounted for only about 64.5% of total spending, but crucially, not a single vendor has yet achieved double digit market share.It may be tempting to assume that consolidation will continue until only a handfullk of large players are left, but if that were treue we would all be using RSA, McAfee and Symantec today.? Instead, it shows that there's still room for innovation and new entrants in the cybersecurity space, with plenty of M&A potential.

Embracing Ecosystem Diversity

The future of security operations isn't about finding a single, perfect solution. Instead, it's about recognizing that different businesses have different needs. A retailer with distributed locations might benefit most from an XDR approach, while a large international bank developing custom applications might prefer a security data lake.

The key is to understand your organization's specific requirements and design a security architecture that leverages the right mix of technologies. Whether you choose SIEM, XDR, a security data lake, or a combination of these and other tools, the focus should be on creating an effective ecosystem of interoperable solutions.

As we move forward, the most successful organizations will be those that can adapt to this diverse landscape, leveraging the strengths of various technologies to create robust, flexible security operations capable of meeting the challenges of an ever-evolving threat landscape.

As I’ve stated before, just like everything eventually becomes a crab, every detection and response solution eventually becomes a SIEM.? Beyond the humorous analogy, I am trying to underscore an important point: regardless of the label—SIEM, XDR, or another acronym—the core capabilities of these systems tend to converge over time. Ultimately, though, the key is not in the name but in how well the solution meets your specific security needs.

In the end, the future of security operations lies not in a single, universal solution, but in the intelligent integration of various tools and approaches to create a robust, adaptable security ecosystem. As threats evolve and new technologies emerge, the organizations that thrive will be those that can effectively leverage this diverse landscape of security solutions.

The key to success in this new paradigm will be the ability to effectively orchestrate these diverse tools and capabilities, creating a security posture that is greater than the sum of its parts. As security professionals, our challenge is to remain adaptable, continuously learning, and always focused on the ultimate goal: protecting our organizations and their stakeholders in an increasingly digital world.

What to Watch

As we move forward, keep an eye on these key areas:

• Further convergence of SIEM, XDR, CDR and ITDR technologies
• Adoption of Cybersecurity Mesh Architecture (CSMA) and similar frameworks for more integrated security ecosystems
• Greater penetration of security data lake solutions among large enterprises
• Evolution of SOCless/SIEMless and distributed security operations models

Appendix

SIEM Market Estimates

https://www.custommarketinsights.com/report/security-information-and-event-management-siem-market/

https://www.grandviewresearch.com/industry-analysis/security-information-event-management-market-report

https://www.marketsandmarkets.com/Market-Reports/security-information-event-management-market-183343191.html

https://www.mordorintelligence.com/industry-reports/global-security-information-and-event-management

https://www.gartner.com/en/documents/5415763 ?

Cybersecurity Market Estimate

https://www.dhirubhai.net/posts/steven-kiernan_cybersecurity-spending-in-q2-was-boosted-activity-7244414459065040896-gk3p