Application Testing.
This is a briefing on Automated and Manual Testing methods and how they relate to Pen Testing and Code Review.?This document will explain terminology and options available to you. After reading, you should have more information about the different types of testing available to help you make an informed choice about which options are best for your particular situation.
SAST - Static Application Security Testing.?
How these tools work:
In general, SAST tools perform a full scan of the code base.?SAST tools use signatures or definitions containing a footprint of what coding flaws look like.?Some tools can be customized to an organization’s needs and custom coding methods.?
The tools compare these flaws to what is found in the code. This review can be completed via a manual process or done with a tool, but regardless of which approach you take, all of the libraries and dependencies used to compile the code should be included for thorough and accurate results.?
Performance and Effectiveness:
In general, SAST tools can find a reasonable number of issues and what is found usually aligns with the OWASP (Open Web Application Security Project) Top 10 categories for application vulnerabilities.?SAST tools do, however, create many false positives which will need a manual review or further triaging.?An automated review is faster than a manual review and will often catch things a manual review or peer review won’t.?
Usage best practices: Once processes have matured, automated methods can be used to define risk levels or categories of information or lower risk findings.?Also, SAST tools can find code quality issues which can then feed QA processes in order to build better and more stable code.??
DAST - Dynamic Application Security Testing.?
How these tools work:
DAST tools start with basic static based signatures but contain more precise and through variables and parameters.?It can be as simple as feeding the base SAST signatures with multiple parameters or variances of input parameters or even a pattern based routines for more optimum dynamic testing.?Don’t confuse DAST with Artificial Intelligence (AI), because AI includes even more logic in order to operate effectively. DAST tools simply rely on the ability to rapidly pattern match in order to be successful.?
DAST tools are aimed more toward making testing efforts more efficient through the use of automation.?Often DAST tools are considered Pen Testing tools, however, Pen Testers will use other tools outside the scope of what DAST is intended for.?These other tools are considered to be in the proxy family or single use exploit finders.?As with SAST, DAST uses both automated and manual methods.?A proxy tool allows get and put calls to and from an application to be intercepted, analyzed and manipulated by injecting new parameters, and returning them to the application.?A key point here is that a human is making a logical correlation on the fly and adapting to the data collected.
SAST vs. DAST results can easily be highlighted by using a password field as an example.?A SAST tool would scan the code base for the word password, determine what is associated with it, and trigger a finding if a pattern is matched for something like a clear text password.?DAST, on the other hand, will look for an input field in the web interface and then use a brute force attack and try to exploit the app.?Any successful attempt will result in a finding.
Performance and Effectiveness:
Several distinctions exist which make a DAST tool much different from a SAST tool.?DAST tools are more obtrusive and considered black box testing tools, whereas, SAST is a white box testing tool.?Obtrusive simply means DAST tools are designed to exploit an application and track findings based on successful exploits, whereas a SAST tool is designed to scan the code base for potential exploits and track the findings.?Unlike SAST tools, DAST tools do not scan the code base.?In the case of a web application the URL of the app is provided as an input for testing.?
Usage best practices:
Tools outside of the DAST category often require human interaction to perform the logic decisions needed.?Because applications respond differently with unique error messages and/or responses, human intervention is often needed to interpret the results and provide guidance or decision support on how to react to them.?Once these plans are determined, some of the proxy features may have automation aspects built into them where a found expected result or base signature can be crafted on the fly and variables, parameters or patterns mentioned earlier can be added to them in order to match the uniqueness of the application being tested.?
Beyond proxy and dedicated exploit tools, there are even full distributions of Linux based operating systems (OS) dedicated to application testing with a myriad of Pen Testing, DAST and Proxy-like tools along with scanners dedicated to finding specialized attacks or known exploits in the wild already packaged into the OS.?These can be used by Code Review and Pen Testers.?
IAST - Interactive Application Security Testing.?
How these tools work:
IAST solutions often use agents to interact with the application’s run-time environment which includes all the needed run-time libraries, frameworks and dependencies.?IAST agents can perform the operations or offload the code crunching to dedicated IAST servers where you may also find centralized management interfaces to help manage and review the vulnerabilities.?
IAST is newer technology and less proven than SAST or DAST but it is growing and maturing quickly.?It can add great value to a thorough Application testing program because of the clear differences in the value it provides.?IAST has similar functionality to SAST tools but the integration method is much different.?Integration needs to be done within the run-time environment and IAST has a closer relationship to QA regression testing and findings can be more easily tracked by a bug tracking system.?
Performance and Effectiveness:
Even though IAST is fast, it is not as thorough as SAST or DAST. On the surface it more closely resembles SAST but this depends on how the agents are deployed and the time sequencing that is used to control the workflow.?
领英推荐
Usage Best Practices:
Often IAST is a more Scrum/Agile friendly security integration option whereas SAST and DAST will tend to run in parallel because of the time needed to perform them.?You should think about what CI & CD (Continuous Integration and Continuous Development) methods are being used when determining any solution.?Bottom line: IAST alone will most likely not meet the full scrutiny of audit reviews and most likely will not meet standards and regulatory requirements if used alone.
A combination of one or more of the methods will often be needed and integrated at different steps within SDLC process to be considered thorough enough.?Given the variability present in various development frameworks like Scrum, and the uniqueness of individual business contexts, you will need to think carefully about what processes work for your environment and culture.?
RASP - Runtime Application Self Protection?
How these tools work:
RASP is often believed to be synonymous to Web Application Firewalls (WAF’s), however, they are different. RASPs are more dynamic and reactive to attacks than a WAF.?Like IAST, RASP resides in the run-time environment, deeper in the application stack and environment.?After baselining, trending, and analysis work is completed, rules can be written or out of the box rules turned on to see how applications perform with them, and these rules can be tweaked and tuned in order to generate results as expected.?
Expected results will equate to finally tuned reactive measures that are based on result patterns.?These can be as simple as a log entry or alert a developer to the need for doing a debug capture. They can even go as far as to put in place a new ACL or firewall rule to stop identified behavior.?A scenario where this could be effective is input validation.?Most applications will have logic in place for validating the input from a user, but consider the power in determining if there is a huge influx of inputs flooding the system in hopes to either fool the application or Denial of Service (DOS) it.?The RASP logic could deny input from the source IP attacking it, log it and send an alert to the Security Operations Center (SOC).?
Performance and effectiveness:
A key point about RASP technology is that if it is not implemented properly, these tools can greatly impact the application’s functionality and behavior. In extreme cases, they can cause applications to stop working.?
Usage best practices:
RASP solutions come with costs; the chief impacts are on administration and understanding the interactions of the applications and infrastructure being protected.?To fully understand the interactions, considerable baselining and trending of acceptable behavior will be needed before reactive measures can be put in place.?
Where does Pen Testing fit with Automated or Manual Tools?
In general, Pen Testing is a form of Dynamic Testing that is generally considered manual, however, this type of testing can employ tools, rely on manual processes, or both.?
Concepts familiar to Pen Testing like White and Black are used to describe the testing methods, and these terms apply to automated and manual testing processes as well. In the White Box testing method, testers start off knowing something about what they’re testing and use this to their advantage to expedite the testing plan and procedures.?Black Box testing means the tester knows nothing about what they’re testing and essentially need to start from scratch with planning, procedures and exploit attempts. For this reason, DAST tools are considered black box testing tools, whereas SAST is a white box testing tool.
Pen Testers typically lay out a testing plan based on the white or black box testing methods.?They often include 5 phases:?
White box methods can make many of these steps quicker.?Pen Testers or Code Reviewers can use DAST tools to expedite and/or validate the manual or SAST/IAST test findings.?I will provide more details on Pen Testing in a future article.
Conclusion
So which method is right for you??Understanding each method is half the battle.?Beyond this, clear goals and requirements are needed to determine what your application security program is trying to accomplish.?Best practices for one company are not best practices for the next.?Some solutions may include all of the mentioned methods in a complete security stack within different regions of the SDLC process.?However, this can be very costly.?
My recommendation is to settle for nothing less than Pen Testing and SAST or IAST.?Each discipline will augment each other nicely with some overlap of found vulnerabilities so make sure collaboration of the different disciplines is done since risk can be under- or over-reported.?
Overlap of vulnerabilities and Pen Testing may increase a single finding in SAST or IAST by 10 times or more.?Also, look at the cost differences of SAST and IAST which will include not only licensing costs but the impact to your SDLC processes as a whole.?Carefully consider which method is best for Agile environments, if that is how you are structured.?Some methods will impact them more, or you may need to modify your sprint structure to accommodate bug and security fixes.
Bottom line
If you develop applications, human errors very likely to happen and a good application security program can minimize the human error factor and often lessen the need for 3rd party mitigating controls needed to combat the human errors.?Think about that ROI advantage.?