An Application Security Testing Primer
Dustin Lehr
Enhancing Culture via Security Champions | Co-founder / CPTO at Katilyst | AppSec Thought Leader and Coach | Community Builder | Software Engineer at Heart
Explore the available security analysis tools you can use in your Software Delivery Pipeline!
As someone who lives and breathes Application Security every day, I am consistently asked by leaders, developers, testers, and others to explain the handful of Application Security Testing (AST) tool types available on the market today, which is a very good thing! I'm glad this topic is gaining momentum. In general, no one tool is perfect for all scenarios, each has its own benefits and drawbacks, and all should be examined properly to determine if they have a place in your organization’s security toolbox.
To help explain the different types of tools, I decided to illustrate the high-level differences between them and where they fit into the Software Delivery Pipeline. Yes, I mean literally “illustrate” - the pun was completely intended. I hope that having a picture handy helps those new to the topic of AST tools to understand the basics. If you're a security expert, feel free to share the details and images within to also assist you in explaining the basic concepts to others, while validating the finer points of my work here for accuracy. If you’re a leader, in security or otherwise, perhaps you can use this to assist in making a case for better software security in your organization. This is only an overview, and more comprehensive details about each of the tools can be found elsewhere, so I’ve added links throughout to help you dig deeper if you wish.
“The best starting point for security analysis is the realization that you need a set of tools - not just a tool - to fully understand the underlying strengths and weaknesses of your software.” — John B. Dickson, CISSP | Principal, Denim Group
So... what is Application Security Testing?
Before getting too deep into the details of the tools, let's start with the basics: What is AST anyway? In simple terms, Application Security Testing is a set of methods, processes, and tools used to examine your in-house developed applications to find security weaknesses or “vulnerabilities”. Fewer vulnerabilities in your code means a lower chance that a threat actor (nowadays commonly called a “hacker", for better or worse) will be able to exploit your application, and thus the security risk to your organization is reduced. If you're new to the security space, you may want to check out the definitions of the terms "threat", "vulnerability", and "risk" on this page: https://www.threatanalysis.com/2010/05/03/threat-vulnerability-risk-commonly-mixed-up-terms/
The types of flaws exposed by AST tools are among the most commonly exploited in Web Applications. Examples include Cross-Site Scripting (XSS), Injection (SQL or otherwise), Broken Authentication, and many others. To start learning about these types of risks, check out the Top Ten list provided by the Open Web Application Security Project (OWASP): https://owasp.org/www-project-top-ten/
Let's dig in to explore the different categories of tools and how they can fit into your Software Delivery Pipeline!
Development Phase - IDE SAST
The first phase of the pipeline where we find Application Security Testing tools is the "development" or "code writing" phase. Here developers can incorporate a Static Application Security Testing (SAST) tool directly into their Integrated Development Environment (IDE) to detect issues right as their code is being written.
Static Application Security Testing (SAST), a form of Static Analysis, is an independent scan tool that examines patterns, and traces variables, in your raw source code to find potential vulnerabilities. This method of analysis does not require a running application or a dedicated environment, only the location of the source code repository or the raw source code files straight from your IDE. More information about Static Analysis can be found here: https://en.wikipedia.org/wiki/Static_program_analysis
An Integrated Development Environment (IDE) is a desktop application that combines several tools which software engineers require to perform their development activities, including editing code, building the application, running, debugging, etc. More information here: https://en.wikipedia.org/wiki/Integrated_development_environment
SAST can be very friendly to developers because an exact line number as well as information about the type of issue found is provided with the scan results. That said, the code examined is limited to what is present in the repository, or what the developer has loaded into their workspace, which may not give the scanner an accurate view of the entire context of the code. Also, this type of scan does not have insight into how the code is actually utilized or the run-time data it works with. These issues can lead to a high number of "false-positives", where the scan may find something that isn’t actually an issue, causing the developers to have to sort through many results to find only a few legitimate ones. Even with these weaknesses, the sooner in the development lifecycle you can find a security defect, the cheaper it is to fix, so SAST is still powerful as the earliest application scanner in the process.
It is also worth mentioning that an even earlier, and thus cheaper, solution is for your developers to avoid writing the flawed code to begin with, which can be helped with adequate and comprehensive secure code training.
Commit Phase - CI SAST
Another opportunity in your pipeline to perform Static Application Security Testing (SAST) is within the full context of your source code repository. More specifically, you can run a SAST scan as part of your Continuous Integration (CI) process. Continuous Integration is the practice of automatically merging individual code changes with the changes of others and back to a central area called a main code "branch". Once this happens you can automatically kick off a SAST code scan which will analyze the entire branch of combined code and report the results, giving the team a more holistic and complete picture of the vulnerabilities present in the app. For more information about Continuous Integration, see here: https://en.wikipedia.org/wiki/Continuous_integration
If you do not have a Continuous Integration pipeline yet, that's okay, you can still set up a “scheduled” scan in many SAST tools, which will periodically connect to your repositories as often as you’d like and review your source code automatically, sending you a report when complete.
Build and Deploy Phase - Prepare for IAST
The next step in the pipeline is the "build" phase. This is when your higher-level source code (Java, C#, C++) is translated or "compiled" into an executable form (an "application") that can be read by a computer. There are some languages, such as Python, that do not need to be compiled before execution, and these are called “interpreted” languages because the code is translated right as it executes.
Once the application is built it can then be deployed (moved/copied) to the pre-production environments used for testing, including Quality Assurance (QA), Integration, User Acceptance Testing (UAT), and Performance. The process of automatically building and preparing an application for deployment, then actually deploying it to an environment, is called “Continuous Deployment” (CD). When this process is paired with Continuous Integration (CI), described above, you now have a CI/CD pipeline.
During the CD process is an opportunity to have your deployment script integrate an Interactive Application Security Testing (IAST) "agent" into your application. This IAST agent is essentially a chunk of code that runs along with your application which analyzes and finds security flaws from the inside, as it executes. This offers a unique solution because the agent has access to not only the actual utilization of the application, including the data used, but also the execution call-stack, which can give a developer a good indication of where the flaw sits within their code. All issues and results found by the agent are reported back to a central location where they can be examined by the team. Unlike SAST, which scans the entire codebase, a strict "passive" IAST agent can only examine the parts of the application that are actually exercised, so good manual and/or automated test coverage is absolutely necessary. There are also “active” IAST solutions and features available that will, to a limited extent, automatically exercise parts of the code for you, which may be worth looking into depending on the maturity of your organization’s testing practices.
Test Phase - IAST and DAST
There is some performance overhead in running the additional IAST agent code as your application executes. For this reason, it is best to utilize IAST in one of your pre-production environments, ideally your QA environment, which is typically where the majority of your application's manual and/or automated feature and regression testing takes place. In this location, you'll have the best chance to exercise enough of your code to maximize your vulnerability findings.
Another method of security analysis found in the testing phase is Dynamic Application Security Testing (DAST). So far, the tools we've covered have been predominantly passive in nature as neither SAST nor IAST tools fully exercise the application by themselves. This is where DAST is different: DAST actively attacks your application like a real threat actor would, executing your application in ways that you may not have accounted for, including sending bogus input (fuzzing), Injection and XSS attacks, and more. The tool then examines the output of the application to determine if the attack was successful and reports the results. In addition, DAST excels at analyzing the environment in which the application runs, which helps find server misconfiguration issues.
Notice here that a DAST tool has no knowledge of the workings of the application. It does not have access to the raw source code like SAST, or an inside look into the way the app is being executed like IAST, so it is difficult for developers to tell exactly where to look for the issue. Also, a dedicated test environment is recommended for this type of testing as it is possible for the persisted data to be modified while running the tool, though many products focus on avoiding this type of situation.
More information about DAST can be found here: https://en.wikipedia.org/wiki/Dynamic_application_security_testing
Note that it is absolutely necessary for developers to run and test their code locally before committing their changes to the common repository, and there are DAST tool products that can be run quickly on the developer's local machine to find issues earlier in the cycle, thus making them cheaper to fix.
Operate Phase - RASP and WAF
Now, imagine if you could take the IAST agent installed into your application— which already sees how potential vulnerabilities are exercised from within— and grant it the power to block the actual attacks taking place? This is exactly what Runtime Application Self-Protection (RASP) does. RASP agents learn the expected behavior of your application and then detect and thwart attacks in real-time as your application executes. Unlike IAST, you would clearly want to run this in production to block actual attacks, so there is a performance hit to consider, but it is typically reasonable compared to the benefits. A word of caution: you do not want to get into the habit of relying too heavily on your RASP solution. Though they are harder to exploit, the underlying vulnerabilities still exist in your code and configuration and should be fixed properly at the source. With that said, RASP can still temporarily fill any gaps in your AST toolbox, and can also provide a good backup for vulnerabilities that are missed by other tools, aligning with the recommended security practice of “defense-in-depth”.
More information about RASP can be found here: https://en.wikipedia.org/wiki/Runtime_application_self-protection
Another similar product which can help protect your applications is a Web Application Firewall (WAF). A WAF monitors and filters the input to your application, as well as the output, according to the exact rules you configure. A WAF acts as a wrapper or proxy; it doesn’t have internal knowledge of the application’s workings like RASP does, and typically requires quite a bit of manual setup and monitoring to determine the type of data that should be disallowed.
More info on WAFs here: https://en.wikipedia.org/wiki/Web_application_firewall
Software Composition Analysis
Most of us don’t write all the code for our applications ourselves. Instead, we rely on a set of open-source libraries and frameworks upon which to build. Several AST tools, particularly on the IAST and SAST side, can help you understand the different components and dependencies present in your codebase, and indicate whether they have vulnerabilities that warrant further research and upgrades. This particular concept is called Software Composition Analysis (SCA), and is worth further investigation for a more comprehensive and secure solution.
This concept is explained further here: https://resources.whitesourcesoftware.com/blog-whitesource/software-composition-security-analysis
A Quick Comparison
As you dig deeper into the details around the different AST tool types, it is also helpful to know their relative strengths and weaknesses. Feel free to use the below chart to begin to understand this.
What’s next?
I hope you've enjoyed this introduction to the various Application Security Testing tool types available today. If you’re new to the space, there’s certainly a lot to learn, and I hope I've helped you get a good start. If you’re an expert or leader, feel free to share the information and images I’ve included here to help those around you better understand how these tools can fit into your organization's Software Delivery Pipeline.
If you’re wondering how to get started using a tool to find and fix security vulnerabilities that may be present in your own or your organization's code, that’s fantastic! I’d encourage you to check out a free tool provided by OWASP called “Zed Attack Proxy” or “ZAP”, which is a DAST tool that you can easily point and run against one of your own applications, either locally or in a non-production environment, with little setup: https://www.zaproxy.org. Warning: do not point this tool at any live public websites or websites that you do not own as this can be seen as a malicious attack on their system.
Another place to start is by researching free or commercial products available per tool type to get further into their details. The below lists are helpful in this endeavor, and there are also some good jump points here to help you begin the process of evaluating them.
- Free Tools: https://owasp.org/www-community/Free_for_Open_Source_Application_Security_Tools
- SAST: https://owasp.org/www-community/Source_Code_Analysis_Tools
- DAST: https://owasp.org/www-community/Vulnerability_Scanning_Tools
- Gartner's list of AST tools: https://www.gartner.com/reviews/market/application-security-testing
- Commercial tools of all types: https://techbeacon.com/app-dev-testing/113-app-sec-vendors-guide-commercial-application-security-products
The tools portion is only one aspect of the Application Security field, where the focus is on understanding and remediating the security risks present in an organization's software products, along with assisting developers in writing secure code. Some of the other important strategies and techniques for accomplishing these goals include Risk Assessments, Threat Modeling, Security Champions, Secure Code Training, and integrating security into the Software Development Lifecycle (SDLC).
Best of luck on your journey deeper into this space - it’s truly a fascinating ride! If you have any comments, questions, or critiques about this article, I'd love to hear them. You can leave a note in the comments section or feel free to contact me directly.
Senior Cyber Security Manager (Head of Security Architecture) at Sopra Steria
2 年Comprehensive and digestible Dustin!
AppSec is a marathon, not a sprint / JPMC Hall of Innovation recipient / OWASP / 2x Above and Beyond Award / 2x President's Club / 1x #1 Global Account Executive / 2x Force Mgmt
2 年When anyone takes the time to efficiently illustrate the AST ecosystem, it's a huge value-add for all. Thank you for taking the time to create this post.
Girl Dad ?? | Helper ?? | Learner ??
4 年Thanks for this insightful write up Dustin. What do you see as the next phase of innovation for software security?
CISO | CTO | Executive Security Advisor | AI Evangelist | Investor | Board Advisor
4 年Dustin: You might want to post this on Peerlyst, too. Limor: Check this out!