Data Intelligence Transforming Fraud Detection

Financial fraud is an unchecked epidemic, draining billions from the global economy while leaving businesses, taxpayers and consumers to foot the bill.? In 2023 alone, global payment card fraud losses surged to $33.83 billion (Nilson Report), while the U.S. insurance industry bled a staggering $308.6 billion to fraudulent claims (Coalition Against Insurance Fraud).? Government programs face the same relentless threat, Medicare fraud alone is estimated to cost $68.7 billion annually, diverting resources from those in genuine need and eroding public trust in critical institutions.? These losses aren’t just numbers on a balance sheet; they drive up costs for every consumer, policy holder, and taxpayer, making fraud detection one of the most urgent and high-stakes battles of the digital age.

Fraud detection hinges on identifying and preventing unauthorized activities by analyzing massive data streams to spot irregular patterns.? Yet, traditional detection systems are falling behind – unable to keep pace with evolving attack vectors and the sheer scale of data that requires analysis.? Artificial intelligence (AI) is transforming how institutions respond to fraudulent attacks by analyzing large datasets in real-time, identifying subtle anomalies and adapting to new fraud patterns before damage is done

In this article, we will explore the cutting-edge technologies driving sub-millisecond detection capabilities and how the industry’s thought leaders are fortifying their defenses against this growing crisis.? The strategy we map out today is not a luxury or an optional safeguard – it is the lifeblood of institutions that intend to survive and thrive in tomorrow’s financial landscape.

A Data Coherency Problem

Fraud detection is fundamentally a?data coherency problem, much like quantum entanglement - where no single transaction exists in isolation, but rather as part of an interconnected network of financial interactions.? Traditional fraud prevention systems operate like?classical physics, treating each transaction as an independent event, relying on predefined rules, and reacting only after anomalies surface. ?But fraud, much like quantum mechanics,?exploits uncertainty, hidden correlations, and systemic blind spots.? Stopping it isn’t about flagging isolated red flags - it’s about recognizing the?entanglements between data points: location history, device fingerprints, merchant risk, spending behaviors, and transaction velocity.

This is where?AI-driven, high-speed data platforms like DDN Infinia become essential.? Detecting fraud in real-time requires analyzing petabytes of streaming data at?sub-millisecond latencies, identifying?nonlinear patterns, and making?instant, AI-driven decisions?before fraudsters can exploit the lag. ?The old approach - scanning static datasets, applying rules, and chasing fraud after the fact - is already obsolete. ?Without a?coherent, AI-powered, high-speed data infrastructure, institutions are?playing catch-up in a game where milliseconds determine billions in losses.

Traditional Fraud Detection Workflow

Traditional fraud detection systems have long relied on?rigid rule-based models?and?delayed batch processing, leaving financial institutions vulnerable to?evolving fraud tactics. ?These workflows were built for a different era - one where fraud was initiated by humans not sophisticated attacks by machines.? less dynamic, transactions were slower, and real-time analytics weren’t a necessity. ?Today, these legacy systems are?struggling to keep up?with the complexity and speed of modern fraud schemes.

The?old way?of tackling fraud started with?real-time transaction ingestion?from?payment terminals, mobile apps, and online purchases. ?Apache?Spark Streaming?would then process these events, applying?feature engineering?to enrich transaction data with?location history, device fingerprints, merchant risk scores, and behavioral patterns. ?This added context was crucial for identifying anomalies, but at its core, it still relied on?predefined risk models?that couldn't?adapt on the fly.

Once processed, these?structured datasets?were stored in?MapR-DB (a NoSQL database), where transactions were assessed for?risk scoring.? High-risk activity was flagged and written to a separate risk database, triggering downstream reviews. ?At this point,?machine learning models (primarily XGBoost - https://www.nvidia.com/en-us/glossary/xgboost/)?were used to detect fraud patterns and predict risk. ?While?XGBoost?is powerful for structured classification problems, it struggles with?complex fraud rings and relational anomalies, which is why?Graph Neural Networks (GNNs)?have started gaining traction. ?However, training these models required?fast access to stored data, and legacy storage architectures often?became the bottleneck.

After a risk score was generated,?MapR-ES (event streaming service)?would forward flagged transactions to?automated fraud prevention systems and fraud analysts for review. ?At this stage,?speed is everything—the difference between stopping a fraudulent transaction in real-time or allowing it to go through. ?Unfortunately, traditional architectures introduced?latency, preventing fraud detection systems from making instant decisions with?sub-millisecond accuracy.

The?final step?was?post-analysis and retraining, where flagged transactions were fed into?case management systems and used to?update fraud detection models. ?However, retraining required?fast access to historical fraud data, and the persistence layers of traditional systems were?too slow to support real-time model updates. ?This meant fraud detection systems were?always reacting to past fraud rather than proactively stopping emerging threats.

The next generation of fraud prevention must?break free from batch-based bottlenecks, embrace?sub-millisecond decisioning, and harness?AI-powered anomaly detection?to?stay ahead of attackers - not chase them after the damage is done.

Building a Modern Fraud Detection System

Traditional fraud detection systems struggle with the sheer scale and complexity of modern financial crime. ?Fraudsters operate in?networks, not isolated transactions, meaning that?Graph Neural Networks (GNNs)?have become essential for uncovering hidden relationships between transactions, accounts, and behaviors. ?However, running?high-speed, AI-driven fraud detection presents significant challenges - handling massive graph datasets, supporting real-time inference, and ensuring efficient model retraining.? This is where an AI-optimized storage and data intelligence platform like DDN Infinia becomes a critical component.

Fraud Intelligence Without Boundaries

The shift from?legacy fraud detection platforms?like?Vertica, Hadoop, and MapR?to?modern cloud-native architectures?isn’t just about cost savings - it’s about survival. ?Fraud moves fast, and organizations relying on?batch-driven analytics?are playing a losing game. Modern platforms like?Google BigQuery, AWS Redshift, and Snowflake offer?on-demand scalability, real-time query execution, and AI-driven analytics,making them ideal for fraud prevention. But moving to the cloud isn’t just about swapping out one database for another. It requires a?rethink of the entire data pipeline, ensuring that?storage, compute, and analytics?are fully optimized for?real-time fraud detection.

Modern fraud detection is no longer just about?flagging suspicious transactions—it’s about?understanding complex relationships, predicting fraud before it happens, and doing it all in real time.? This level of intelligence demands?high-speed, scalable architectures?that can process petabytes of structured and unstructured data while enabling?AI-driven anomaly detection. ?Cloud data platforms like?BigQuery and AWS Redshift?have become essential for this because they provide?massively parallel processing (MPP), SQL-based querying, and near-instant scaling, making them ideal for handling?billions of transactions, risk scores, and AI-generated fraud patterns.

But?structuring fraud detection data for BigQuery and Redshift?isn’t as simple as dumping logs into a database.?Transactional data, behavioral analytics, device intelligence, and graph-based fraud patterns?all need to be?optimized for rapid querying.?Fraud datasets are often broken down into:

• Raw Event Logs:?High-volume transaction data ingested in near real-time.
• Feature Stores:?Enriched fraud signals - geolocation, merchant risk, spending history.
• Graph-Based Representations:?Entity relationships to identify fraud rings and anomalies.
• Model Training Data:?Historical fraud cases for AI models to learn from.

BigQuery and Redshift?excel at structured fraud analytics—allowing fraud teams to?query billions of transactions, link behaviors, and generate risk scores in seconds. But fraud detection?doesn’t just live in the cloud.?There’s?always a need for?on-prem workloads—whether for?real-time edge processing, regulatory compliance, or historical model retraining.

This is why?hybrid fraud architectures are the future, and why?DDN Infinia is a critical piece of the puzzle.?Cloud platforms alone cannot deliver the ultra-low-latency storage, high-performance AI pipelines, and hybrid data access required for enterprise-scale fraud detection.?Infinia acts as the high-speed backbone, ensuring that?fraud detection models can pull structured data from BigQuery and Redshift while simultaneously accessing raw, high-dimensional fraud patterns from on-prem datasets—without performance bottlenecks.

Even more important is?how organizations evolve legacy fraud detection models. Financial institutions have invested years into?MapR-based analytics and XGBoost-based ML pipelines.? Ripping and replacing these workloads isn’t an option - but?modernizing them with AI is.? With?Infinia’s multi-protocol support (S3, GCS, POSIX), teams can?continue running legacy MapR jobs, while seamlessly transitioning to Graph Neural Networks (GNNs)?for?next-gen fraud pattern detection.? No disruption, no delays - just a seamless evolution toward AI-driven fraud prevention.

At the end of the day,?fraud detection is a real-time problem, and?every millisecond matters.? Platforms like BigQuery and Redshift provide the?scale and analytical power?fraud teams need, but?without an AI-optimized storage and data pipeline like Infinia, organizations will always be one step behind.? Infinia ensures that no matter where fraud is happening - on-prem, in the cloud, or across hybrid environments - data moves at the speed of fraud.? Infinia acts as the mediator between on-prem and cloud and between one format and another

Eliminating Data Bottlenecks & Delivering ROI

A modern?fraud detection system?must eliminate bottlenecks?that slow down traditional AI pipelines while keeping the total cost of ownership (TCO) in check, ensuring business owners are satisfied with the return on investment (ROI).?

To estimate this cost, consider that a typical payment processor like a Visa, Amex or Mastercard processes close to 100,000 transactions per second (TPS).? Real-time inferencing for this workload would require approximately 1000 GPUs, while training and retraining necessitating an additional 4000 GPUs.? The power draw for such a cluster would typically need 10 megawatts (700 Watts per GPU, plus CPU, networking, storage and cooling).? The physical footprint for this setup would require approximately 70 racks.? A typical 5 year TCO for this setup would run approximately $500 million.

The appropriate choice of a data intelligence platform is critical to an organization’s success.? NVIDIA has collaborated with DDN since 2017, leveraging DDN’s data intelligence solutions to enhance performance and efficiency.? Jensen Huang, NVIDIA’s CEO,? stated: “We checkpoint and we restart as often as we can.? Our goal is to continuously drive down the cost and energy associated with the computation.”? The reason is simple, DDN delivers 15X faster checkpoints, 30% less space consumption and 50% more capacity per watt.

Optimizing the Modern Fraud Detection Stack

The modern fraud detection stack must deliver four critical capabilities while maintaining the right price/performance ratio to stay ahead of evolving threats:

• Real-Time Graph Data Ingestion & Feature EngineeringFraud detection begins with?ingesting high-speed transactional data?from payment systems, logs, and account activities.Data must be?enriched in real-time?with location intelligence, device fingerprints, transaction histories, and network linkages.

Challenge:?Storing graph data in traditional databases (Neo4j, TigerGraph) introduces?latency?when retrieving related entities.

Solution:?DDN Infinia’s NVMe-tiered storage accelerates subgraph lookups, ensuring rapid feature extraction for AI models.

• AI-Driven Risk Scoring with Graph Neural Networks (GNNs)Fraud is relational - GNNs analyze how accounts, merchants, and transactions connect over time.Training GNNs requires?fast access to embeddings - the AI representations of nodes and edges in the fraud graph.

Challenge:?Transferring graph embeddings to GPUs is slow in legacy storage architectures, creating a bottleneck in model training.

Solution:?Infinia’s direct-to-GPU data movement?eliminates CPU bottlenecks, enabling faster risk assessment and reducing false positives.

• Real-Time Inference & DecisioningOnce trained, the AI model must evaluate transactions?within milliseconds, flagging high-risk activity before it is approved.

Challenge:?Traditional fraud detection pipelines rely on batch scoring, which?cannot stop fraud as it happens.

Solution:?Infinia’s parallel data streaming?enables real-time fraud inference, ensuring transactions are assessed before they are processed.

• Continuous Model Retraining & AdaptationFraud tactics?evolve constantly, meaning AI models must?retrain on new patterns?to stay effective.

Challenge:?Storing and retrieving past embeddings for retraining is?resource-intensive?and can degrade model performance.

Solution:?Infinia’s AI-assisted retraining pipelines?track graph drift, automatically updating models to maintain fraud detection accuracy.

Conclusion

Fraud detection is a race against time—every millisecond lost to data bottlenecks, slow inference, or outdated models amplifies risk exposure. To stay ahead, fraud detection systems must harness AI-optimized data intelligence and hybrid cloud flexibility, enabling them to operate not just at the speed of fraud, but faster. As attackers increasingly deploy sophisticated AI-driven techniques, defense mechanisms must evolve accordingly, leveraging even more advanced AI to outmaneuver emerging threats.