When your puzzle has a few broken peices

I read something recently that stated that 99% of new software projects rely on open-source software, and that seems to check out. I haven’t seen an organization yet that hasn’t used OSS in their products. The open-source libraries that are utilized in applications offload many of the mundane tasks that an application must do, or can help solve edge cases that the developer doesn’t need (or want) to write on their own. Think of open-source software as a way to integrate puzzle pieces from multiple different piles of pieces that are pulled together to make a final, beautiful, puzzle.

And the carousel of progress continues with the adoption of open-source growing. For developers and leaders, understanding the significance of OSS is crucial for understanding the future of software development and maintaining competitive advantage.

Because honestly, with the increase in usage comes increased risk. Like bringing the proverbial wooden horse into the walls of Troy.

But how does risk exist in OSS? This software is open for anyone to see. Doesn’t that make it more secure? Well, what if nobody is really watching?

The OSS Process

Creating OSS is a similar process to what many developers would encounter in an enterprise setting. Designs are created, code is written, code is packaged and made available through a repository or some other sharing service. The community that creates OSS could be a team, or one person. One prominent example is that of the Linux kernel which has three primary maintainers. ?Other software could be maintained by a community of developers who may have never even physically met or could be on opposite ends of the globe. If you ever want to peruse a location for OSS take a look at GitHub. There is code for pretty much anything you might be looking for.

So, what goes into maintaining an OSS project? It involves more than just writing code. The individuals that are the shepherds of open-source software are called “maintainers”. Maintainers play a crucial role in steering the project, collaborating with contributors, and ensuring its health and sustainability. They lead the open-source project and play a crucial role in managing the development, ongoing upkeep, and direction of the project. Maintainers make decisions about code inclusion, review contributions, release new versions, handle security fixes, and manage the community.

While functionality and community are a huge part of open-source projects, the pressure to create more secure projects is mounting. Maintainers need to wear the security hat, ensuring that the code that makes it into their projects is free from vulnerabilities.

There are tools and processes to support securing the project. They won’t be a surprise to those of us that work in application security. They range from the simple to the complex but follow the basic secure-by-design approach. Controls, like access controls, strong authentication mechanisms, using two-factor authentication, employing security analysis tools in and around the CI/CD. Likewise, performing regular audits and updating any third-party dependencies can help to prevent vulnerabilities from slipping in.

However, probably the best security tool in the projects tool belt is through good-ole-fashioned code reviews. In most open-source projects, a contributor will make a change in a local branch, commit the changes, and open a pull request. At this point the maintainers or other contributors will review the code before merging it with the main branch. While tooling can help identify some flaws and malicious activity. Some issues require a keen eye. Unfortunately, these are in very short supply.

OSS in the news

In March 2024, a significant security breach was discovered in XZ Utils , a data compression utility integral to Linux-based applications. Microsoft developer Andres Freund found a backdoor consuming excessive processing power, leading to its discovery. This malicious code was introduced by a contributor using the pseudonym Jia Tan (handle: JiaT75), who had gained the trust of the project's community and eventually became a co-maintainer. This elevated role allowed Tan to embed the backdoor without needing further approval, posing a severe risk to any system using the utility.

While getting eyes on the code can be a formidable defense, who’s watching the watchers? The XZ Utils backdoor incident is a stark reminder of the risk in open-source software, and this is even more evident when there is a malicious insider. This situation highlights the importance of enhanced vetting processes for maintainers, reliance on automated security tools to detect anomalies, and community-wide vigilance to safeguard software integrity.

Other attacks are utilizing the OSS ecosystem itself to deliver malware such as the GitCaught campaign against GitHub and FileZilla . This cyber-attack was able to distribute malware, including banking trojans and stealer malware like Atomic, Vidar, Lumma, and Octo. Attackers created counterfeit versions of popular software, deceiving users into downloading malicious files through fake profiles and repositories.

The OSS ecosystem depends on trust, and like most things in cybersecurity, this trust can be exploited. For organizations, understanding that there is no guarantee that the software supply chain is secure will help in developing a program that centers around a defense in depth approach and applying the appropriate level of trust to the software that the organization consumes from third parties. More on that in a bit.

OSS security in the organization

OSS increases an organization’s speed to market by enabling functionality that is already built, but it brings a set of challenges, particularly in the areas of security and maintenance.

• The percentage of codebases with high-risk open-source vulnerabilities increased from 48% in 2022 to 74% in 2023. These risks are classified as those that have been actively exploited, have documented proof-of-concept exploits or are classified as remote code execution vulnerabilities.
• Only 27% of organizations continuously audit code for vulnerabilities.
• 40% of organizations do not use key supply chain security technologies like software composition analysis (SCA) and static application security testing (SAST), which are fundamental in identifying and mitigating vulnerabilities in OSS dependencies and proprietary code.
• An average of 11 malicious NPM packages are reported weekly since the beginning of 2024, according to GitHub's Security Advisory . This high frequency underscores the increasing prevalence of such threats.

While these statistics show how the ecosystem is (or is not) addressing threats in open-source, the frequency of such threats is alarmingly high, and the impact of malicious packages can be severe and multifaceted. They can lead to data breaches, harvesting sensitive personal, financial, and business information. System compromise is another critical risk, as these packages can allow unauthorized access and command execution. Additionally, malicious packages can propagate malware, potentially turning the host system into part of a botnet or crypto miners. Beyond technical damage, the discovery of a malicious package can significantly harm an organization's reputation, eroding trust among users and stakeholders.

To effectively address these concerns, organizations must adopt a comprehensive approach to OSS security. This involves:

• Understanding the difference between malicious packages , which contain harmful code with the intent to infect, and vulnerable packages , which have unintentional security bugs that could be exploited.
• Creating a secure-by-design approach that embeds robust security considerations into products from the outset, companies can reduce exploitable flaws and enhance overall software security.
• Having a strong vulnerability and patch management program that allows your organization to respond rapidly to an identified vulnerability in a package.
• Managing your technical debt by including it in your design planning to ensure you are using up to date software.
• Trusting, but verifying external packages using tools that scan source code and repositories to detect malicious packages, whether added directly or indirectly into a project. These are known as SCA or software composition analysis tools (more on that in a bit).
• Lastly, immediate action is crucial when a malicious package has been detected. This includes removing it from the system, involving security teams to isolate infected hosts, and reporting the package to the relevant OSS community or security organization.

The increasing frequency of malicious OSS packages highlights the critical need for vigilance and robust security measures. Organizations should foster a culture of security, utilizing the right tools to protect their projects and maintain their integrity against evolving threats. By remaining vigilant and proactive, organizations can mitigate the risks associated with malicious open-source software and ensure the security and reliability of their systems.

Where’s the “easy button”?

I’d hate to break it to you, but there is no “easy button”. However, as I mentioned above, there are tools that can help identify vulnerable packages in an organization’s overall application.

“Software composition analysis (SCA) tools will look for known security vulnerabilities and license concerns with third-party and open source software that is used to build the overall application.” – The Application Security Program Handbook        

Some SCA tools offer features like prioritization and auto-remediation to address open-source vulnerabilities, which sounds promising. However, a significant issue with SCA tools is that they often flag a library only after it has been deployed in production. For instance, imagine you built and deployed your application in January, but by July, your SCA tool identifies one of the libraries you used as insecure. When you initially built the software, there was no known issue. The situation becomes even more complex if resolving the flagged issue requires upgrading other libraries or making significant architectural changes. Additionally, the library flagged by the SCA tool might not be insecure itself but could have dependencies on another library that is.

Sounds daunting, right?

One of the biggest pain points for most organizations is that many of these findings are not actually exploitable. And manually determining that is slow for most development teams. There has been a lot of talk (including myself ) about “reachability”. This type of assistance in the scanning tool will attempt to determine whether a package in an application can actually be reached and called by an external source. If a found vulnerability can be reached, it could be exploited.

In the context of OSS security, reachability is a key concept that determines whether a vulnerability poses a real risk. A vulnerable OSS package is considered reachable only if there is an application flow from the application code to the vulnerable code. This is crucial because many transitive dependencies are not actively used by the application.

If a vulnerability is reachable, it means the application code specifically interacts with the vulnerable part of the package, creating a potential pathway for exploitation. This distinction allows developers to focus on addressing vulnerabilities that genuinely impact the application, optimizing their remediation efforts and enhancing overall security. In other words, reducing noise and allowing the developers to focus on what can actually improve the security of the application.

Where does that leave us?

The security hurdles of using OSS can be significant, but the benefits it offers make it an indispensable part of modern software development. OSS fosters innovation, reduces costs, and accelerates time to market by leveraging community-driven resources and pre-built functionalities. And based on the prevalence and increasing adoption of OSS in the enterprise it’s easy to see that OSS is not going anywhere any time soon. And that means that the security challenges aren’t either.

To effectively leverage OSS while maintaining robust security, organizations must focus on reducing false positives and providing actionable feedback from their security tools. Implementing SCA tools that incorporate reachability analysis can help determine if a vulnerability in a given package is actually exploitable, allowing developers to prioritize true risks over false positives. Additionally, encouraging security tools to provide contextual information about vulnerabilities, such as potential impact and suggested remediation steps, makes findings more actionable and relevant for developers.

While the promise of OSS is faster delivery of value to users and builders of software, the security concerns can’t be overlooked. Thankfully, tools like SCA can assist with organizations that depend on OSS to stay competitive.