A CTO asked, “How do I get more signal and less noise from security!”

Here is my very simple, metrics infused answer.

Development teams want meaningful work. That’s what the CTO calls “signal.” Rejected work is “noise.”  

Security noise has two flavors: Irrelevance and Errors.  

Irrelevant security work makes developers fix un-exploitable (and low-impact) vulnerabilities.  They know when this is happening and find it disheartening. 

It’s a distraction. It blocks revenue generating work.  

Errors are “false positives.” They are created by bad security software.  Developers have low patience for false positives.

Here is the problem: Developers hold security responsible for both irrelevance and errors...for noise.

Each are like trust eroding paper cuts. Enough cuts, and development starts to ignore security.  

That’s why security must be merciless in reducing noise!  

Reducing noise helps developers focus on revenue generating work, and it keeps them from ignoring breach causing vulnerabilities.

So, how do you reduce noise? By measuring it! 

Start as simple as humanly possible, sophistication at this point is your enemy!

First, count all accepted and rejected remediation tickets over a time period. 

I like to start with a quarter. For example, January to March. You can use whatever consistent timeframes you like.  

Sum up the count of accepted work items and sum up the rejected work items.  

Create a ratio: Accepted / Accepted + Rejected

Do it again, but move out one month. For example, February to April

Set goals (KPIs) and track the month over month deltas.

Next, decompose this metric into errors and irrelevance categories. Set goals for each.

Next, measure false negatives. These are the work items originally rejected by development only later to come back again as real work. 

You need to measure these and reduce them as well.  Break these down into errors and irrelevance too. 

You now have six metrics and two KPIs!  And we’ve barely scratched the surface!  

These are super simple metrics.  If you aren’t measuring them now, what stops you?

The next level of sophistication includes measuring KPI quality.  That gets a little more advanced.  You can see an example of it below.

This metric embodies much of what I wrote above. The 95% is based on the current three month window.  It also shows the previous 3 months window (-1M) and the window preceding that at (-2M).  It also scores itself...that is the “Meet KPI: 28%”

I will show you how to calculate that last "KPI" metric (and more) in my next article.