Become Part of the Solution

I am frequently asked how someone can become a better computer security professional. This is one solid way to do it. Whenever you find a computer security problem, figure out how to solve it. Anyone and everyone can and does find and bring problems. The best security professionals bring solutions. And then push that solution “upstream” until the ability for the problem to occur is eliminated from its root cause.

Make it a habit to always figure out the best solution to any computer security problem you find as you bring it to the person who is charge of the resource.

Bonus points: Push the solution upstream.

Extra, extra bonus points: Actually, fix the problem.

Do these three things and the people who listen to you will think you are extraordinary.

An Example

Let me use an example of an instance where I did this fairly well. I often use it to help remind myself of the importance of bringing solutions along with problems, although in this case I did not find the initial problem.

I was consulting for a large Fortune 50 company. They had a problem of both internal and external services getting hacked every year. Not a lot of them, but somewhere between a handful to two dozen a year. This may sound like a lot of successful hacking and very poor defending, but they had many millions of servers. Percentage-wise, it was not bad. But even one hack is too many. They knew they had a problem.

They used Qualys (https://www.qualys.com/) to scan their servers for vulnerabilities and security issues. Most scans would find a handful to many dozens of problems that needed to be resolved. The average server report would be 25-50 pages long and contain one to three dozen issues that needed to be fixed. About one-fourth to one-third of the identified issues were ranked as highly critical, needing immediately resolution.

This sounds like a fairly straight forward process and fix until you realize that it means resolving many millions of issues. Where do you start? How do you fix it? And so on. And that is where I came in. I was asked to recommend what issues they needed to fix first.

Every single security decision I think of is driven by the strategy that I have shared in my many articles and my book (https://www.amazon.com/Data-Driven-Computer-Defense-Way-Improve/dp/1092500847/) called a Data-Driven Computer Defense. Its core tenet that underlies the others is “You should defend against the most likely threats first and best.” That seems like common sense, but the vast majority of computer defenders do not do that. It is why hacking and malware are so successful despite decades of knowing what we need to do to be better defenders.

My answer to their question was, “What are the ways most servers were hacked or what are the most likely ways they would be hacked in the near future?” Defend against your most likely risks.

Their answer to my response was a blank stare and silence. I was used to this response whenever I asked it. They did not know. So, I recommended that they find out. They did not have any method already set up to simply query and find out the answer. It took them a month of researching, but when they came back, the answer astounded us all.

The most common reason for the server hacks was hardcoded login credentials. I think most of us thought it would be a buffer overflow vulnerability or unpatched software. But, when the right question was asked and the answer found, the reality was something a lot less sexy – hard coded passwords, where the developers put login credentials in the code of the application running on the server, and either intentionally left it that way or left it in accidentally after the code went live. Many times, developers hard code credentials just to make the frequent testing and debugging of the code easier to do. By hard coding the login credentials in the code, the required login happens very quickly and the rest of the code can be more easily and rapidly tested and examined.

The Qualys scans were finding this problem reliably and was recommending that it be fixed immediately, but that was also alongside dozens of other critical recommendations. How would any report tell stakeholders what the true number one problem was out of all the supposed number ones? (e.g., in this case, hardcoded passwords) was really the number one problem they needed to resolve first and best, out of all the other top priority noise?

Fix the Problem

My solution was to ask if the Qualys finding rankings could be adjusted so that any time hardcoded passwords were found that it was the top finding with the top criticality pushed to report readers. Turns out Qualys did have a separate field in the database that allowed us to adjust scores to get what we wanted. Problem solved. We went into Qualys and adjusted scoring so that hardcoded passwords were the number one finding when it was found in a scan.

Then I thought about how to handle on an ongoing basis. Sure, we could periodically find out the top remaining hacks and modify the Qualys rankings manually, as needed. But it would be better if the process was automated. So, that is what we did. We automated the hacking findings research and created a script which automatically modified the Qualys ranking system dependent on the findings. This part was not easy. It required that we synchronize (i.e., normalize) the hacking findings’ research process to use the same categories and names as what Qualys used (or we could have created a translation table). Within a year, they had an automated process that would capture how their servers were actually being hacked (if the root cause exploit was known) and this process modified the Qualys report to make sure all developers and server owners got reports that pointed out the top real-world problems.

Push the Solution Upstream

But with every problem solved, ask yourself if you have identified the top root cause of the problem. For instance, if unpatched software is your top problem, why is unpatched software such a big problem? What process is broken? What software is not doing its job or what new software has to be bought? Or which department needs to have clearer responsibility over patching? For most security issues, there is usually a root issue that allowed it to become an issue. Is it just one person or department not doing their job? A broken process? What is it?

In this case, we had developers putting in hardcoded credentials. Step one was to educate all developers about the problem and why it was important not to do it. Second, we changed policies to make it clearer that hardcoded credentials were not something that was allowed at the company. At this point, we are starting to address the root of the problem.

The educational process slowed down the number of times that a developer hardcoded credentials in their code. It did not eradicate the problem, but it reduced the occurrence by over 80%. The Qualys report was still catching the remaining instances, but it was still a problem.

I thought, how can we stop this from happening in the first place? Why are developers even being allowed to hardcode login credentials in their code in the first place? The easy answer was that the developer software and platform allowed it. I asked if the software code check-in process (which was automated) could check for hardcoded logins, and if so, could we deny allowing developers checking in code if it contained hardcoded credentials? As it turns out, yes. And with that one additional change, we removed the other remaining 20% of the problem.

Make your own examples. Whenever you find a problem (or someone comes to you with a problem), ask yourself what is the best way to solve it. If you find a problem and take it to someone, always take along the best solution you can think of. Always think about why the problem is a problem in the first place. It is almost always a process or tool issue, or both. Think tactically and strategically.

If you do this…bring root cause solutions to every problem you find…you will find that you will start to be considered the go-to person for solving problems. You will move out of the average pack into the higher echelons of computer security “experts” who get things fixed. And that is not a bad reputation to have.