The Person in the Arena

Preface: Several months ago, I drafted this article, but never finished it. Given recent events, it is time ship it.

~

When I was in Northern Iraq (Mosul), I had the opportunity to talk to locals outside the wire. As a result, I gained insights that I was not able to gather from the news, books, or intelligence briefings. These insights proved invaluable; they altered my perspective. Once I came back from Iraq, I noticed that conversations by people who had studied this topic, but had never been “on the ground,” missed important points. After a time, I understood that studying a topic via books does not result in the same understanding as being on the ground. This does not mean people who learn about a topic via books are not intelligent, it just means they lacked practical experience.

Years later, when I started to work in cybersecurity, I saw a similar trend. There were people that understood the theories behind cybersecurity because they had taken classes, read books, passed certification exams, etc. There were also people who had been through cybersecurity events. In many cases, these people did not see eye to eye when it came to addressing cybersecurity concerns.

I recently read Brian Krebs article on the limits of MFA (you can find it here). The discussion topics in this article resonated because I had interviewed with a technology company for a security position. During one of those interviews, I was asked a series of questions. It was clear the interviewer was looking for check the box answers. For example, one of the questions wanted me to say “2FA,” while another question was looking for the word “hashing.” This got me thinking – if companies are hiring security professionals who answer questions with check the box answers, do they treat real-world problems the same way?

With the recent security-related events impacting multiple organizations and news about how CISO’s may face legal consequences for their actions (or inactions), I found my thoughts turning back to my own experiences. Part of the problem with cybersecurity is the approach that some cybersecurity professionals take. If companies hire cybersecurity professionals who think that solving a security issue is a matter of implementing a specific thing (such as hashing) without thinking through the various scenarios, pros, cons, strengths, and weaknesses of that solution, they may end up with a false sense of security. The risk of implementing these solutions is that organizations may be “worse off” than not having any solution at all. If you have no solution in place, you know you have a vulnerability. If you have a solution in place that you think protects you, but it does not actually protect you, you have a false sense of security.

There are solutions to mitigate this risk. One technique I learned at AWS is to “follow my curiosity.” If I ask a question about how a system works, and someone gives me a check the box answer that I do not understand, I continue to ask questions (sometimes to the point of annoyance) until I understand how the system works. If the person explaining the system cannot answer my questions, that is a flag…it likely means they don’t understand the system either, so I keep asking / find someone else to ask until I understand it.

Another technique is to have a correction of errors / post incident response process that gets to the root cause without assigning blame. Implementing this is far from easy, but allows organizations to drill into the cause of incidents. For many security incidents, the root cause might appear to be that someone did something they should not have…but if an organization has lots of complex policies and procedures, it is accurate to say the root cause is a single person making a mistake? The book Engineering a Safer World provides multiple examples where attributing the root cause of an incident to one person not following the process is often an oversimplification of what happened. Many incidents can be attributed to multiple factors including complex processes that encourage people to work around them, lack of monitoring / alerting / reporting, lack of post incident response processes that root cause issues, lack of risk reporting, lack of prioritization, lack of investment in resolving technical debt, etc.

In addition, blanket statements rarely work. It is easy to say “never deploy on a Friday.” It is more difficult to be in a meeting on Friday morning when your team explains they just found a zero-day exploit that impacts your service. Now you have a decision to make: do you deploy on Friday to resolve the exploit and take the risk that, if something goes wrong because of that deployment, it may impact your customers? Or, do you wait until next week to provide your team time for testing / validation and take the risk that someone leverages that zero-day exploit over the weekend?

These statements are not designed to give organizations a free pass. Every organization needs to take security seriously. This means investing time and money up front, not after an incident has occurred.

Cybersecurity professionals could benefit from a little more empathy. We should avoid rushing to judgement when we see other organizations stumble, thinking ourselves immune from those same incidents. This is especially true for those who have never experienced such incidents themselves. They may be quick to judge and condemn others, without having any real-world experience to draw from.

Cybersecurity professionals would do well to remember that the credit belongs to those in the arena, not those on the sidelines.

~

“It is not the critic who counts; not the man who points out how the strong man stumbles, or where the doer of deeds could have done them better. The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood; who strives valiantly; who errs, who comes short again and again, because there is no effort without error and shortcoming; but who does actually strive to do the deeds; who knows great enthusiasms, the great devotions; who spends himself in a worthy cause; who at the best knows in the end the triumph of high achievement, and who at the worst, if he fails, at least fails while daring greatly, so that his place shall never be with those cold and timid souls who neither know victory nor defeat.”

― Theodore Roosevelt