On the left, on the right and wiggle in the middle
Dispelling appsec urban myths

On the left, on the right and wiggle in the middle

As always you can sign up to our weekly newsletter at www.crashoverride.com

After a decade away, I have found my public speaking voice again. My 2023 topic is to dispel urban myths about application security.?

Here is one such myth and a few inconvenient truths.?

If you look at the recent surveys from HackerOne (you have to register) and BugCrowd about the types of bugs found by bug bounties in 2022, you will see that the overall state of application security has hardly changed in a decade. It's the same old vulnerabilities being found over and over again, and the amount of vulnerabilities are getting worse.?

Financial services companies on Bugcrowd’s platform experienced a 185% increase in the last 12 months for Priority One (P1) submissions, which refer to the most critical vulnerabilities.

The OWASP Top Ten has hardly changed in the last decade and the ‘best practices’ that appsec teams adopt,? seem? to have hardly changed either. Run SAST, DAST and SCA? tools, train developers, do threat modeling and a familiar list of other “best practices” that goes on.?

We have been road testing the people, processes and technology of appsec for over two decades at this point , so how come the industry at large is still doing the same things that are clearly not working?

  • Maybe it's confirmation bias? You teach people what key issues are, and they go off and find them. I think this is likely. The implication being attackers continue to be ahead of defenders.
  • Maybe it’s that people simply don’t correlate effort and results so keep putting effort into things that appear to be sensible but that don’t work? I think this is likely.?
  • Maybe it’s continued unsubstantiated marketing from tools vendors ? Almost definitely.?
  • Maybe it’s peer pressure and wanting to fit in? I think this is likely.?
  • Maybe it’s ineffective or incomplete implementations of best practices? Maybe.?

It is? unlikely that any one thing can be attributed to the impotence of application security as a discipline, but it is likely that a collection of these things, roll up into a ball of appsec? urban myths.?

I think shift-left is one such myth.?

The premise of shift-left is two fold. The first is that the cost to fix a security issue is incrementally cheaper while developing software than it is after it has been built, and therefore as much security should be done upfront by the developers themselves before or as they create code. The second is that a gram of prevention is better than a kilogram of cure.?

Said another way, companies should spend their limited cycles on threat modeling, training developers how to write secure code and running security assessment tools as code is created. There is no doubt in my mind that all of these techniques improve security but shifting your efforts (shift means to move or cause to move from one place to another) to the left, with the goal of better and cheaper security is likely a myth.

We have all seen some variation of the phrase ‘it's 100 x cheaper to fix bugs in development than it is in production’. It’s usually seen as a marketing tagline, often next to ROI calculators. Some examples are Deepsource, Synopsys, IBM , Perforce, Smarttbear (the irony in that name), Whitehatsec and Cigital, Grammatech, Security Boulevard, and there are instances from NIST. The 100 x statistic varies from five x to several hundred x, but the multiplicative mathematics and the promise is the same.?

Some of this stems back to a book, Applied Software measurement by Capers Jones in 1997.?

The ….research by Capers Jones found that the cost to address bugs post-release costs $16,000 to address, but a bug found at the design phase costs $25. That means valuable QA budget is being spent on fixing bugs that could’ve been solved for much less, and earlier on in the release cycle.

This urban myth comes from a so-called IBM Systems Sciences Institute report that, guess what, didn’t even exist. Laurent Bossavit wrote up his analysis Degrees of Dishonesty posted as a Gist on Github that was later covered by the Register in 2021. Yes El Reg. Here is a quote from the Register article.?

Bossavit took the time to investigate the existence of the IBM Systems Science Institute, concluding that it was "an internal training program for employees." No data was available to support the figures in the chart, which shows a neat 100x the cost of fixing a bug once software is in maintenance. "The original project data, if any exist, are not more recent than 1981, and probably older; and could be as old as 1967," said Bossavit, who also described "wanting to crawl into a hole when I encounter bullshit masquerading as empirical support for a claim, such as 'defects cost more to fix the later you fix them'."

That's right, the evidence, that has become the appsec folklore that shift-left makes sense, simply doesn't exist.?

Bossavits excellent ebook, The Leprechauns of Software Engineering, How folklore turns into fact and what to do about it, dispels many other similar myths and is highly recommended.?

I am sure you can find ‘pay to play’ research by firms that specialize in saying what you want them to say in exchange for money. I never cease to be amazed at the bullshit the Ponemon Institute (pokemon?)? chucks out, not unlike the load of nonsense used by shampoo advertisers all the time. They even set their own responsible information practices so you know you can trust them, just like some ‘private’ universities in the US. Fake diploma sites even advertise that they can be trusted while their competitors can’t. Some days I am embarrassed to be part of the security industry.?

If I were to be running an appsec program, which I am not, which I haven’t done for well over a decade, I would apply people, process and tools to the right, to the left and to the middle. I would place my chips where they are needed.

My version of on the left would be something like

  • Make ‘anything and everything security’ available to those that want it
  • Only focus on things that matter. Risky apps, apps that are in production and apps that are important to my business. Scanning all my repos is a waste of time and generates noise and confusion. Do things like threat modeling on these.?
  • Lock down and configure developer tools to use secure defaults. Protected branches, standard tests in PR’s etc.?
  • Train developers about the impact of insecure software to the company, probably a simple lunch and learn, once a year. They read the news and are intelligent for fucks sake.?
  • Train developers JIT (Just In Time) about specific issues when they are found
  • Eradicate classes of vulns by enforcing frameworks
  • Set up a crack team of first responders?
  • Create “office hours” and “phone a friend” for live expert help
  • Tag all the code so you know who owns it, what it does and what it is for?

My version of in the middle would be something like

  • Lock down my CI/CD system including using something like SLSA
  • Run SAST and SCA
  • Use custom SAST to look for “hot spots”, sensitive code changes
  • Use custom SAST to understand the architecture, data flows and any changes to it
  • Build a fast security feedback loop into the development process

My version of on the right would be something like

  • Know what I have in production, know what it does, know how it works and know who is responsible for it. Most people do not know this today.
  • Know how my applications and their associated infrastructure (cloud, online services etc) are configured, what they do and who is responsible for them.
  • Run DAST
  • Invest in production observability tools
  • Invest in protection tools

We should be “on the right, in the middle and on the left” and I think shift-left is a dangerous urban myth.

Kevin Wall

Experienced Application Security developer, OWASP ESAPI project co-lead / committer, secure code reviews

2 年

Always some idiot that comes late to the party, right? I hope all the beers are not gone. Anyhow, while I don't disagree with Mark's main conclusion, I have a different take on how we got here. Let me know what you think. https://off-the-wall-security.blogspot.com/2023/02/why-i-dont-myth-old-days-or-theres-no.html

Steve Schofield

Senior Information Security Specialist

2 年

I jumped into VA role two years ago and handed “DevSecOps”, “Shift-left” and “frictionless” and about every meeting I went to seemed to use them. I like you expression “a gram of prevention is a kg of cure”. I have found much of what you are saying, too many tools, too much noise, people get lost when talk tools vs. capabilities that need to be done at the right time. There is not one right path as much as getting people to “have an attitude of security” and providing actionable results. Show a developer a great report and stuff gets done. Enjoying your articles great to find someone writing ?? about topics I am currently emerished in.

Munya Mufambisi

Security Architecture, Identity and Application Security Specialist

2 年

I get it, the report doesn’t exist, but the fact is almost always supported in our environments. Vulns are harder and definitely more expensive to fix on the right side of the sdlc compared to the left. Prevention is always better than cure. Untangling our intricate mess that is production, to fix a vuln, the downtime caused, the analysis etc is way more effort has this issue been identified in the devs IDE.

Mark Miller

Take responsibility. Give credit. Co-Founder, 2025 Artificial (Un)Intelligence Conference

2 年

Wow! And I always thought Rob England was the original IT Skeptic. The entire industry has been screaming "Shift Left!" for years. Companies have been named after it, thousands of presentation have been heralding it's virtues, and it's an assumed part of a DevOps methodology. Then along comes Mark in 2023... "It is?unlikely that any one thing can be attributed to the impotence of application security as a discipline, but it is likely that a collection of these things, roll up into a ball of appsec?urban myths.?I think shift-left is one such myth." Hold on, hold on, hold on. Is this anti-shift-left attitude evidence based or is Mark just pulling our collective chain? "The evidence, that has become the appsec folklore that shift-left makes sense, simply doesn't exist." In Mark's article he goes over the history of where shift left came from, and the fallacy of the math myth that says "It's 100 x cheaper to fix bugs in development than it is in production". This is all heresy to the DevOps faithful, but take a look, read the case against focusing on shift left, and make your own decision. https://www.dhirubhai.net/pulse/left-right-wiggle-middle-mark-curphey/

John Scott

Satellites, Cybersecurity, Supply Chain, Engineer, Leader and CxO, Baller w/ 2 exits + lots of CrossFit

2 年

I think you are really arguing for a systematic way of sourcing, developing and deploying code - looking at the forest versus just one tree: for example, you can't have meetings to discuss all the findingz on a project every week, if you have a 100 projects, you waste too much time. I'be been thinking about modeling out a CI/CD process ...

要查看或添加评论,请登录

Mark Curphey的更多文章

社区洞察

其他会员也浏览了