How to Vulnerability

Originally written 8/2020. Major update 3/2023.

Whether you're an elite security researcher or just starting out in security, odds are that you're terrible at effectively explaining vulnerabilities. Sorry, that's just a fact. It's hard to find security vulnerabilities, but it's really difficult to establish trust, clearly and concisely explain the risk, and compel action all while being great to work with and not bruising any egos. You have to be a great team player and absolutely resist the urge to blame. Otherwise you'll undermine your entire security program and culture -- which is really the whole point of finding vulnerabilities in the first place.

So, in this article, I'm going to give you a recipe to follow for communicating any vulnerability effectively. With some practice, you should be able to create a fantastic, compelling vulnerability writeup in less than five minutes. Of course, you have to really understand the vulnerability first, but that's not what this paper is about. This is about what you do after you find something awesome. Please don't skip steps...I promise you'll be tempted. But it will backfire, so just do them in order.

Background

Before we get to the recipe, I have to spend a minute on why it matters so much. It's useless to discover vulnerabilities unless you can explain them to the folks that need to know. Unless you compel someone to take some action, you might as well not do it at all. As one of my law professors at Georgetown told me, "why don't you go over in the corner and be right."

"An ability to explain what you've done in terms the business can understand.?You have accomplished nothing if you discover a vulnerability that you can't explain.?An enormous part of your job is to make people understand the risk and change their behavior" - How to become a penetration tester

Finding vulnerabilities and reporting them the wrong way might actually be worse than not doing anything. Good security programs are built on mutual trust (See Larry Maccherone's Trust Algorithm). Every time you blast out a raging email about how this application is super vulnerable because it has Reflected Expression Forgery Injection-Jacking and we are doomed, doomed I say, because some APT could use a NOP sled trampoline and take over the world, people roll their eyes and think... really?

Try to keep in mind that hacking stuff and reporting vulnerabilities is just a tiny part of an effective security program. Famed security researcher Michal Zalewski puts it this way

"Prioritization of security efforts is a prudent step, naturally. The problem is that when risk management is done strictly by the numbers, it does deceptively little to actually understand, contain, and manage real-world problems. Instead, it introduces a dangerous fallacy: that structured inadequacy is almost as good as adequacy, and that underfunded security efforts plus risk management are about as good as properly funded security work."

Empathy Is Everything

Nobody likes it when you call their baby ugly. Particularly if almost everything about their baby is absolutely amazing. If you want to be effective at appsec, you're going to want to understand developers' pain and appreciate their skills. And remember, no matter how cool it seems...you are delivering a turd. It's work they have to do that wasn't on their schedule and doesn't delight users (although maybe it should). Newsflash: you may think security moves fast, but development moves much faster. Development teams have to master an amazing amount of constantly changing technology.

So consider your language and approach carefully. Get in the habit of using the word "we" when talking with development teams - even when I've been brought in for an outside review. It's a little thing, but it reminds me that we are on the same team! Be really careful not to blame or shame when describing vulnerabilities. If you call people out, you'll destroy any trust you've built with the team.

You Probably Suck at Vulnerability

It's not that you're wrong. You're probably a really good security researcher. And you probably found a critical security problem. You just didn't build trust and explain the risk right. Trust me -- I learned this lesson the hard way. Over 25 years and many thousands of vulnerabilities, I've probably made every mistake possible explaining vulnerabilities.

• Years ago I used WebScarab to completely bypass a certificate-based digital signature authentication system and gain admin privileges in a major government-wide travel system, I was so excited that I didn't really explain it, I basically just demonstrated it. The program manager didn't get it and said I had "cheated." Had to start over and never really won his trust.
• Another time, at a major newspaper, I found their corporate financial system exposed to the Internet so their consultants could report their time. I also found a really devastating zero day vulnerability in the package. But again, I didn't explain well, and their CIO got mad at me because I couldn't guarantee that there were no additional zero days. Ugh.

I take the blame. It's MY job to explain risks so they are trustworthy, understandable, and compelling. Some people use the word "actionable" but that word doesn't nearly capture a great finding.

Over the years, I've taught scores of consultants how to do this... many of whom went on to become security executives, founders, and leading security researchers. So I know that even though I'm explaining this as a simple recipe that you can just follow... many of you will get excited and hurdle over steps in your mad passion to get to the cool stuff. Sigh.

You're just going to need to practice. There's no other way. Don't practice explaining risks to other security folks. They are not the target audience. I always imagine explaining vulnerabilities to my mom (in her 70's, has trouble using Zoom). Your audience is very smart, very technical, and lives in a completely different world from you. They may not have any experience with security. That's not their fault.

You absolutely cannot be judgmental. It is way more than a full time job to be a great developer. The amount of things that they must learn and keep up with is astounding. If you don't know how to convert anonymous methods to lambda expressions or when to use a streaming XML parser instead of a DOM parser, then find some humility when they don't know what CSRF stands for. In the words of Tuco, "Respect...you got to give it to get it." In other words, it's way past time to get over your security exceptionalism.

The Vulnerability Recipe

These steps are simple. Each one produces just one or two sentences. Focus, knock it out in plain language, and get back to doing what you do best.

Step 1: Explain how the application works

Seems simple, but most security people just blaze right over this step in their excitement to tell you about the wicked cool exploit they came up with. Keep it simple... no need for security jargon here. Reference their application modules and frameworks. You're not describing the problem yet... just how the application actually works. You don't need a ton of detail here, just enough to give the background for the problem you're about to describe. Think of it like you are introducing the characters in the first scene of a movie.

This is incredibly important. You do this to establish trust with your audience. As you describe how the application works, they are deciding if you're an idiot. As you get the details right, they warm up to you. By the end of this, you should have nodding heads. Throw in a compliment about the look and feel -- or the sophisticated microservices. Now people are ready to hear a little tough love.

BACKGROUND: The lunchJammer application takes user input from the jamapp.blt web form in the createSamich module. That input goes directly into the BLT command processor which processes mayoExpressions to generate a new snork report.

Step 2: Explain how an attacker might make the application do something it shouldn't

Now that you've set the stage, you've earned the right to explain how a bad guy can abuse the application. Generally, this should be a factual description of how the exploit actually works -- you don't want to be conjecturing or hypothesizing here. If you're not sure whether or how it can be exploited, you should go back and do some more research. Note that there may be many possible exploit scenarios. Just choose a good one that demonstrates the scope of the problem.

VULNERABILITY: By entering a malicious mayoExpression into the web front end, the attacker can trick the BLT processor into executing arbitrary operating system commands on the host.

Step 3: Describe location and context

You have to specify the "location" of the problem. Your goal is to make it insanely easy for the reader to find the exact right place to make the fix. But remember, contrary to what you might read in the news, real security vulnerabilities very rarely exist in a single line of code. There are multiple different ways to describe the location of problem. The "location" might be at the attack surface (an exposed controller, endpoint, route, message interface, file, trigger, etc...) as specified in code or configuration. Or it might be in intermediate code (filters, actions, plugins, connectors, backend connections, etc...). And it could easily be near the dangerous functions (interpreters, parsers, requests, backend systems, etc...) or security defenses (authentication, access control, encryption, validation, encoding, etc...) invoked during this flow. Don't be afraid to mention multiple locations.

Even more importantly, don't forget to describe the business context of this vulnerable code path. For web apps and web APIs this must include both HTTP request *and* source line of code. Can you depict the full flow of this transaction from the attacker's browser all the way through internal backend systems and back to the browser? Can you reveal the control flow and data flow? Can you talk about why this function exists? Who is it intended for and what is the business purpose? This is the background for understanding the risk.

CONTEXT: The problem can be located at: URL: https://vuln.com/finance/acct/8675309?user=Jenny Code: com.vuln.app.FinanceTransactionModel.java@367 This endpoint provides read-only access to user information stored in several backend systems accessed via APIs. This data is highly sensitive and should only be exposed to account owners and internal associates during a support call.

Step 4: Explain the risk factors

Next, you need to explain why this is a problem. Try to explain the likelihood that an attacker will find and exploit this vulnerability and the impact of a successful attack, both technical and business. It's always best to start by thinking about how you would actually go about finding and exploiting the problem. Then ask yourself "so what" five times until you get to the real issue. Why does this really matter to the business. Note that one organization I worked for really didn't care about a $50m loss... but even the tiniest damage to their reputation was absolutely unacceptable. Ask for their business impact reference table. It'll change your thinking about XSS forever.

Some people more umm... detail oriented... than me would tell you to use a framework like FAIR or the OWASP Risk Rating Methodology to mathematically assess the risk. But my advice is not to go overboard here. Most risks are totally obvious they're either incredibly dangerous or pretty innocuous. Every once in a while, you'll get a risk that's very difficult to rate. To me, your job is to make a logical argument for why something is critical, high, medium, low, or note. You don't have to write a novel. Make the argument as simple as it can be. Generally there are just a few factors that dominate the risk rating -- those are the ones you focus on.

For likelihood, you want to consider who could do this - is it an anonymous internet attacker, authorized user, employee, or admin? Is it an attractive target? Also consider how hard the problem would be to find? Can an attacker use an automated tool to detect the problem? And consider how hard it would be to actually exploit the problem. Are there automated exploits available? Is there some inside knowledge that you would need? Now that you've worked through these factors, stare at the ceiling for a minute and consider whether you really have the risk level right.

For impact, you should start by considering the "technical impact" of a successful exploit. What systems get compromised? What data gets stolen or corrupted? What capabilities would be unavailable? Was the integrity of the business compromised? Then you should work on describing the "business impact." You really can't do this unless you really understand the business that the vulnerable application supports. How long could they survive without it? What kind of revenue does it drive? How sensitive is the data it contains? If you're doing this right, you had this conversation when you started the engagement, but it's always worth running it by some other folks, including your customer, to ensure you get this part right.

RISK: The likelihood of this risk is HIGH, as anyone on the Internet, without having to authenticate can find and exploit it. There are already automated tools for this type of scanning and premade exploits already exist. The impact is also HIGH, because the attacker can completely take over the targeted host, they could steal any code or data, corrupt any resources, or use the server as an internal launching point for other attacks. Therefore, this risk is CRITICAL.

Step 5: Explain the impact with a realistic scenario

Some readers won't understand all the details of the vulnerability or even how you evaluated the risk. So this is your opportunity to make it real for them. Imagine the most realistic scenario for an attacker to exploit this vulnerability and harm the company. Try to keep this big picture so that they can really understand that this could happen. Now tell the story in an interesting way -- how could this really happen?

SCENARIO: A hacker interested in stealing credit card data could automatically search the Internet for applications using the BLT protocol. If they happen to find the lunchJammer application, they can use this vulnerability to take over the server in your Amazon environment with a single malicious HTTP request. From there, they can use your database credentials to steal the entire contents of your database, including names, addresses, CC numbers, and hashed passwords. Most likely they would attempt to sell this information on the black market. They might also launch attacks on the other systems in your Amazon VPC.

Step 6: Explain options and make a recommendation

Remember, the whole purpose of writing this thing up in the first place was to get somebody to do something. Be compelling! That's why we're being so careful about presenting the argument this way. Every application is different and there are many different ways to remediate vulnerabilities. Does their application framework have a defense already that they're not using? You better recommend that instead of writing their own. Do they already have a coding pattern for this? You need to talk with the development team and ask them what would work. What's the future of this application? Should they use parameterized queries? Implement a persistence layer? Transition to APIs? Escape untrusted input? Implement RASP? Don't just parrot some generic advice you got from the OWASP Top Ten.

You'll gain a lot of credibility if you describe several options and then make a recommendation about which one fits their circumstances best. You have to really think about what's best for their business and their development team. If you think architecturally, you may be able to eliminate numerous risks with a single change. For example, maybe you use IAST to generate a Runtime SBOM and find some libraries that have known vulnerabilities that are actually in use in the application. What's your advice? Fix those libraries?

You should consider whether it's possible to eliminate similar risks that already exist or may be introduced in the future. Can you add something to their software pipelines to prevent anything like this from ever being a problem again? Aren't all the vulnerabilities you discover also a problem with the pipeline? Did you point that out?

Be sure to be specific about how they can accomplish this recommendation given their technology stack and capabilities.

RECOMMENDATIONS: The ideal solution would be to change all the database queries to PreparedStatement. However, because this is a legacy application that will be decommissioned in a year, we recommend implementing runtime protection to prevent this vulnerability from being exploited.

Step 7: Create a compelling title

You might be tempted to do this first, but I recommend doing this step last. The title may be the most important piece of each finding. Done right, most readers will fully understand the problem just from the title. You can't write the title first because you won't have thoroughly thought through the specific risks to this company from this vulnerability and the specific threats to their business until you've gone through the steps above. And you want something catchy -- something that will get people to actually fix the problem. What would the newspaper heading in the Wall Street Journal say about this vulnerability? If it's urgent, make an urgent title. If not, make that clear too. If you're so inclined, don't be afraid to add some style -- make it interesting, but not clickbait.

Not so creative? Start with the following simple pattern...

[WHO] can do [WHAT] and [HOW]

You'll want to choose the most likely values for [WHO], [WHAT], and [HOW]. Could be anonymous internet hackers, nation state APT, malicious customers, employees, system administrators, or even inadvertent user error. Sometimes multiple different WHOs might be motivated to find and exploit a vulnerability. Focus on the one that's the most compelling. This is just a title -- you already captured the full risk in Step 3. The WHAT is the harm to the application owner. It might be a technical harm but it's probably better to try to figure out the business harm that would happen if this vulnerability were exploited. There may be a broad range of WHATs, so try to choose an example that captures the seriousness of the problem. The HOW is by exploiting the vulnerability you found. But you don't want to get too technical, stick to the high-level category of problem. Try to tie the how to the exact component of the system with the problem.

For the Struts OGNL Injection problem that caused a large breach a few years ago, what title would you use?

• Option 1: CVE-2017-5638
• Option 2: OGNL Injection in Struts
• Option 3: Struts RCE in Content-Type Header
• Option 4: Anonymous Internet Hacker Can Assume Full Control of Server with Single Unauthenticated HTTP Request
• Option 5: Hacker Can Take Over FooBar System by Exploiting Vulnerable Version of Struts Framework

Really try to put yourself in your audience's shoes. What do they actually care about. Spend some time thinking about exactly what will compel them to take the action that you're recommending. In this case, Option 5 is probably the best choice as it makes the fix obvious, but Option 4 might work for a more technical audience.

Advice on a Few Common Challenges

Not everything is going to go perfectly. Connect with me on LinkedIn and DM me about your challenges communicating vulnerabilities. I'll add my thoughts to this guide! Here are a few common challenges:

Presenting vulnerabilities in meetings

Even in an informal briefing, you still MUST follow all the steps in the recipe. Don't let yourself hurdle steps.

If you're doing this right, as soon as you find a serious vulnerability you really ought to let the application owners know immediately. They may need to take an application offline right away. Be careful in this situation, you can easily create drama or panic. If you do this right, they may want to get started on fixing it right away. And they may ask for a writeup of the vulnerability. That's GREAT! You should have one already.

Briefing executives

Rule number one when briefing executives is no surprises. You want to make sure that everyone has had a chance to review the report and provide feedback. Avoid the "big reveal" like the plague.

The best situation possible is when you report a successful effort that came in on time and on schedule. Share how supportive the development team was. And summarize the three key recommendations from your executive summary. I promise that the right number of key recommendations is three. Don't just choose three vulnerabilities. Stare at the ceiling and ask yourself, what are the three things I would do if I were them. Ideally, you'll be able to explain to the execs that you found some really critical vulnerabilities, but Jean Luc and Beverly were extremely responsive and have remediated all the serious problems already. Developers will love you and management is so happy they'll hire you again! Everybody wins!

Embracing pushback

Even if you follow this guide and do an amazing job of explaining things, your readers may disagree. They may think what you've found is not a problem, or that you're an idiot, or any number of other disagreements. They may have been burned by security in the past and they may be coming in hot. Here's the thing... they may not be wrong. It's really arrogant to think that you know more about their technology and their business than they do.

This isn't you telling them how it is. This is a search for the truth... and you're both on the same team. Don't get defensive. Instead, take the high road and look to understand why they are interpreting the facts differently than you are. Try to diagnose why you're coming to different conclusions. There are three general situations:

1. Maybe you got the technology wrong. You probably only had a few days to understand and break a system that took years to develop and has millions of lines of code. It's impossible. So if they push back, listen to what they're saying. Maybe there's something about the platform, appserver, framework, libraries, or code that you didn't understand. Or maybe it's different in production vs. the system you're actually testing. Be open.
2. Maybe you got the likelihood wrong. Maybe the awesome injection issue you found can only be exploited by an administrator. Or maybe there are compensating controls in place that you didn't consider. Maybe they determined that the insane padding oracle vulnerability you found would require thousands of HTTP requests that their operations team would detect long before an exploit. There's no way you know everything, so you have to discuss it with them.
3. Maybe you got the impact wrong. I had one client that didn't care unless the annual loss expectancy for the risk was $50 million. Many clients are much more concerned about the reputation damage from a vulnerability than the actual breach. You don't know what's important to their business. I can't say this enough... you have to discuss it with them.

Dealing with "duplicates"

Nobody wants 50 separate CSRF findings. Or a separate risk writeup for every JSP that is outside the WEB-INF directory. You are very likely encounter sets of similar or related vulnerabilities in an application. While you might consider writing these up separately, it's often better to combine them into a single finding. You are not being graded on volume. Many organizations consider these duplicates and think of them exactly like false positives. I think the best strategy is is to combine findings based on whether they can be addressed with a single change. Think if an engineer would want a single ticket or if the problems can be fixed separately.

Putting security in "business language"

The most consistently mentioned difficulty with security is communicating with management effectively. You'll hear that they want their security in "business language." Let's face it, they're not going to learn security lingo. So you're just going to have to learn to really communicate at the right level. This guide is a good start, but you should think of every vulnerability as having two explanations - one that's more technical and one that uplevels the risk into business terms. To get good at that business story, you're going to have to understand what the business cares about - is it revenue? innovation? speed to market? growth? social cause? culture? reputation? Be sure you frame your vulnerability in the context of the business.

Achieving 5 minute findings!

It takes practice, but I promise you can create a great finding in 5 minutes. It's actually really important to get fast. If you find 25 vulnerabilities in a penetration test or code review and it takes you 30 minutes to write up each one, that's going to take you over a day and a half just to write them up. And 30 minutes is pretty fast for most security folks -- it could easily take many times that long. Do it my way and you'll be done with all the writing in just over two hours total. And for goodness sake, don't wait to write them up until the end. Do it as you go along. If you don't, you're going to have to redo all the work when it comes time to write up the findings. And it'll end up taking days. There goes your weekend.

Practice, Practice, Practice

Ok, here's a short quiz to see if you were paying attention...

You just found an awesome expression language injection issue. What is the first thing you say to the client?

Your knee-jerk instinct should be to explain how the system works and get confirmation... "Hey, we noticed something. The system takes the X-Screwup header and tries to parse it. When that parsing fails, the error page evaluates the header value as an expression. Is that right?"

Develop this instinct. Fight the urge to blame, call it a critical risk, use security terms, be scary, or jump into vulnerability or attack details. Practice taking a breath and starting with how their system works. Once you get off on the right foot, the rest will probably work out just fine. I highly recommend working with a colleague to improve your ability to vulnerability. You brief them on your findings, they brief you on theirs. Use this guide as a rubric. I guarantee you'll challenge each other and learn in the process.

The ability to communicate risk effectively is critical to your career in security. In my opinion, it's the path towards advancement, leading engagements, running a security team, security management, and communicating with the board effectively. Your ability to explain risk reflects your own depth of understanding and sophistication.

Good luck! If you found this helpful, please let me know. I also greatly appreciate any questions or thoughts on making this guide even better. Thx!