Developer-friendly Security Reporting
Reporting application security vulnerabilities is an awful thing to do to someone. If you want a primer on how to communicate security effectively, check out this article I wrote based on 20 years (of mistakes) in the field.
Communicating Security with Developers via Test Failures
This article is about a specific idea. Rather than generate a massive PDF report, or a zillion JUnit tickets, or a Slack alert volcano, or yet another "single" pane of glass... what if we could simply fail any test cases that cause code with a security problem to run. This is a very natural and non-threatening way to let developers know there's a problem and exactly what to do about it. The goal is to help developers fix vulnerabilities as part of their natural workflow -- without having to run scans and triage reports -- so they can check in clean code.
So, let's try a very simple example. Let's say you want to ban starting native processes in Java. Starting native processes isn't the most dangerous thing in the world, but whatever -- it's *your* coding standard -- this is about how you can effectively automate a broad range of security policies. You could simply grep the code, but the call might be buried deep in a library or other component. It might get loaded at runtime via dependency injection. And anyway, there are quite a few method signatures to search for, so you may not find them all.
Using the Java Observability Toolkit (JOT)
For this article, we're going to use the Java Observability Toolkit (JOT). It's a free and open source library that allows you to create security sensors without writing any code. Sort of a voltmeter that lets you put probes deep into your application so that you can see what's going on way down the stack -- all the way down into libraries, frameworks, appserver, and runtime. It's a very useful tool you can use for a lot of different purposes.
First, we need to create a JOT file with a single sensor (junit.jot). It's just a bit of YAML that defines where to instrument and what the sensors should do. In this case, the only "method" we need to list is the java.lang.ProcessBuilder.<init> constructor -- this is where calls like Runtime.exec() end up. You can add as many banned methods as you want. Then we "scope" this sensor, so it will only fire when the methods are invoked somewhere inside a JUnit test. And finally, we define an "exception" which will cause the JUnit test to error out and tell the developer why.
sensors: - name: "banned-methods" description: "Fails JUnit tests that cause banned methods to be invoked" methods: - "java.lang.ProcessBuilder.<init>" scopes: - "org.junit.platform.commons.util.ReflectionUtils.invokeMethod" exception: "Banned methods are prohibited by Security Directive 27B/6"
Then you'll need an application to test. I have a simple test case here that directly calls Runtime.exec() which in turn calls ProcessBuilder deep in the Java runtime. But the call to Runtime.exec() can happen anywhere inside any code that runs as part of this test and this sensor will find it just fine. For example, if this was an end-to-end test that submits a mock HTTP request to a servlet, JOT will ensure that any calls to create native processes are caught no matter where in that transaction they occur.
@Test void executeRuntime() throws IOException { String cmd = "open -a Calculator"; Runtime.getRuntime().exec( cmd ); }
To run your application with JOT you can download the latest version of the JOT library from github. Probably the easiest way to add JOT to your application is to just use the JAVA_TOOL_OPTIONS environment variable. Then just run your test suite as normal.
$ export JAVA_TOOL_OPTIONS="-javaagent:jot-0.9.1.jar=junit.jot" $ mvn test
That's all it takes. In this case, you can see the test case resulted in an error and lets the developer know exactly what line of code was responsible. Also, the message helpfully references the regulation from the movie Brazil. Probably would be more useful to let them know what they should do instead. Just update your JOT with whatever message you like.
More Sophisticated Sensors
There's no much that you can't make observable with JOT. You could flag transactions that don't make an access control check. Or that use a weak encryption algorithm. You can see any backend connections that are made. Or which APIs consume credit-card information.
You can use "captures" to be very specific about exactly when a sensor will fire. You can filter on the value of the Object, Parameters, and Return of the "methods" listed. Or you can use the full power of Spring Expression Language to invoke methods on those objects (or any static methods) -- and filter on whatever that returns. All this so that your sensors will only fire exactly when you want them to.
You can learn more about JOT from the project homepage. There are more examples of what you can do with JOT in the resource library. You can make any of those JOTs fail test cases by adding the JUnit "scope" and an "exception" as we did above. Let me know what you use it for!
Agile Engineering Executive | Transformational Leader | Influential Change Agent
4 年This is a great suggestion! One of my teams employed this technique to great success especially as it enabled security tests to run in the CI/CD pipeline early and not at the 11th hour before the release was supposed to go live. Security/Style scanners are obviously the most common model but the output can sometimes feel like noise because it often gets muddled in the build log which developers tend to ignore. On the otherhand, a team actively using unit tests to progress a build regularly checks the tests runs to look for that "red" test if something fails. There is a format called a user/abuser story that can be used for this too - you can for say, as a user I will enter special characters into all form fields to see if I can generate a fault (and if the cross site scripting succeeds anywhere, the test goes red to alert the team). Its one example that early feedback is always best, but also less is more sometimes.
Founder | 40 Under 40 | Top Woman in Cybersecurity | Enterprise Software Leader | Market Strategy Expert
4 年I think you've hit the nail on the head, Jeff, with the word “communicate.”?Integrating Developers into the AppSec process isn’t about hitting them over the head with alerts and reports, nor making them responsible for all testing, but rather clearly communicating expectations, and when something doesn’t meet expectations why it does not to enable them to participate in the process. For us, this means we share the feature-specific policies & controls (e.g. security test plan) in their IDE, allow them to attest to following the policy (or not), and then see results. And when there is a security issue to fix, they’re given the context of why it matters to that project.?