What Cybersecurity Data Should You Trust?

The Limitations of Security Data
We are constantly being bombarded by cybersecurity data, reports, and marketing collateral—and not all of this information should be treated equally. Security data inherently has limitations and biases, which result in varying value and relevance in how it should be applied. It is important to understand which is significant and how best to allow it to influence your decisions. 

There is a tsunami of security metrics, reports, analyses, blogs, papers, and articles vying for attention. Sources range from reporters, researchers, professional security teams, consultants, dedicated marketing groups, and even security-operations people who are adding data, figures, and opinions to the cauldron. We are flooded with data and all those who have opinions on it. 

It was not always this way. Over a decade ago, it was an information desert, where even speculations were rare. Making decisions driven by data has always been a good practice. Years ago, many advocates were working hard to convince the industry to share information. Even a drop is better than none. Most groups that were capturing metrics were too frightened or embarrassed to share. Data was kept secret by everyone while decision makers were clamoring for security insights based upon industry numbers, which simply were not available. 

What Was the Result?
In the past, fear, uncertainty, and doubt ruled. People began to dread the worst and unscrupulous security marketing advocates took advantage, fanning the flames to sell products and snake oil. They were dark times, promulgated with outlandish claims of easily eradicating cyber threats with their software or appliance products.  The market was riddled with magic boxes, silver-bullet software, and turn-key solutions to easily fix all security woes.  I can remember countless salespeople asserting “we solve security” (which at that point I stopped listening or kicked them out).  The concept of flipping a switch and all the complex problems of compute security forever goes away, was what uninformed organizations wanted to hear, but was simply unrealistic.  Why customers chose to believe such nonsense (when the problem and the effectiveness of potential solutions could not be quantified) is beyond me, but many did.  Trust in the security solutions industry was lost for a period of time.

Slowly, a trickle of informative sources began to produce reports and publish data. Such initiatives gained momentum with others joining in to share in limited amounts. It was a turning point. Armed with data and critical thinking, clarity and common sense began to take root. It was not perfect or quick, but the introduction of data from credible sources empowered security organizations to better understand the challenge and effective ways to maneuver against threats.

As the size of the market and competition grew, additional viewpoints joined the fray. Today, we are bombarded by all manner of cybersecurity information. Some are credible while others are not. There are several types of data being presented, ranging from speculations to hard research. Being well-informed is extremely valuable to decision makers. Now, the problem is figuring out how to filter and organize the data so one is not mislead.

As part of my role as a cybersecurity strategist, I both publish information to the community and consume vast amounts of industry data. To manage the burden and avoid the risks of believing less-than-trustworthy information, I have a quick guide to help structure the process. It is burned into my mind as a set of filters and rules, but I am committing it to paper in order to share. 

I categorize data into four buckets. These are: Speculation, Survey, Actuarial, and Research. Each has its pros and cons. The key to managing security data overload is to understand the limitations of each class, its respective value and its recommended usage.

For example, Survey data is the most unreliable, but does have value in understanding the fears and perceptions of the respondent community.  Research data is normally very accurate but notoriously narrow in scope and may be late to the game.  One of my favorites is Actuarial data.  I am a pragmatic guy.  I want to know what is actually happening so I can make my own conclusions.  But there are limitations to Actuarial data as well.  It tends to be very limited in size and scope, so you can’t look too far into it and it is a reflection of the past, which may not align to the future. 

I hear lots of different complaints and criticisms when it comes to the validity, scope, intent, and usage of data.  I personally have my favorites and those which I refuse to even read.  Security data is notoriously difficult.  There are so many limitations and biases, it is far easier to point out issues than to see the diamond in the rough.  But data can be valuable if filtered, corrected for bias, and the limitations are known.  Don’t go in blind.  Common sense must be applied.  Have a consistent method and structure to avoid pitfalls and maximize the data available to help you manage and maintain an optimal level of security.

Below are a few examples, in my opinion, of credible cybersecurity data across the spectrum of different categories.  Again keep in mind the limitations of each group and don’t make the mistake of using the information improperly!  Look to Speculation for the best opinions, Survey for the pulse of industry perceptions, Actuarial for real events, and Research for deep analysis:

Speculation:

• 2016 Cybersecurity Threat Predictions from McAfee Labs
• $243 billion - $1 trillion.  Potential cost of a single attack against the US Power Grid, per Lloyds Insurance
• ~3 Trillion aggregate economic impact of cybersecurity on technology trends, through 2020.  World Economic Forum 2014 report Risk and Responsibility in a Hyperconnected World
• $90 trillion dollars cyber impact for one scenario (worst case) affecting the global benefits Information and Communications Technologies by 2030. The Atlantic Council’s report estimate.
• 55% CAGR.  Growth of global IoT Security market 2016-2020, per com
• My 2016 Cybersecurity Predictions, in fact most of my blogs are in the Speculation category!

Survey:

• Threat Intelligence Sharing survey.  McAfee Labs Threats Report March 2016
• 20% jump in cybercrime in the UK since 2014 with nearly two-thirds of businesses expressing no confidence in the ability of law enforcement to deal with it, per PwC
• 25% Americans believe they have experienced a data breach or cyber attack.  Travelers survey
• 43% organizations surveyed indicated increases in cybersecurity will drive the most technology spending.  Source 2016 ESG IT spending intentions research report
• 61% of CEO’s believe cyber threats pose a danger to corporate growth per PwC survey

Actuarial:

• 3 out of 5 Californians were victims of data breaches in 2015 according to the CA Attorney General in the 2016 California Data Breach Report
• ~35% of the US population. Top 10 Healthcare breaches of 2015, affected almost 35% of the US population Source: Office of Civil Rights
• Data Breach Investigations Report (DBIR) annual report by Verizon
• 2016 Annual Security Report by Cisco
• 42 million new unique pieces of malware discovered in Q4 2015, bringing the total known samples to almost 500 million, per McAfee Labs Threat Report (March 2016, Malware section)
• Security Intelligence Report (SIR) bi-annual report by Microsoft

Research:

• $325M losses attributed to Cryptowall v3 ransomware, analysis from the Cyber Threat Alliance
• $13.1 billion. U.S. Government spends on cybersecurity in 2015.  Source: FISMA report from OMB
• “Carbanak” advanced attack analysis by Kaspersky

By the way, yes, this very blog would be considered Speculation.  Treat it as such.  

Interested in more?  Follow me on Twitter (@Matt_Rosenquist) and LinkedIn to hear insights and what is going on in cybersecurity.