week 36 - DevOps Metrics and KPIs, Revisit security in the era of DevOps and developers' perception on software testability

Revisit security in the era of DevOps: An evidence-based inquiry into DevSecOps industry

By adopting agile and lean practices, DevOps aims to achieve rapid value delivery by speeding up development and deployment cycles, which however lead to more security concerns that cannot be fully addressed by an isolated security role only in the final stage of development. DevSecOps promotes security as a shared responsibility integrated into the DevOps process that seamlessly intertwines development, operations, and security from the start throughout to the end of cycles. While some companies have already begun to embrace this new strategy, both industry and academia are still seeking a common understanding of the DevSecOps movement. The goal of this study is to report the state-of-the-practice of DevSecOps, including the impact of DevOps on security, practitioners' understanding of DevSecOps, and the practices associated with DevSecOps as well as the challenges of implementing DevSecOps. The authors used a mixed-methods approach for this research. The authors carried out a grey literature review on DevSecOps, and surveyed the practitioners of DevSecOps in industry of China. The status quo of DevSecOps in industry is summarized. Three major software security risks are identified with DevOps, where the establishment of DevOps pipeline provides opportunities for security-related activities. The authors classify the interpretations of DevSecOps into three core aspects of DevSecOps capabilities, cultural enablers, and technological enablers. To materialise the interpretations into daily software production activities, the recommended DevSecOps practices from three perspectives—people, process, and technology. Although a preliminary consensus is that DevSecOps is regarded as an extension of DevOps, there is a debate on whether DevSecOps is a superfluous term. While DevSecOps is attracting an increasing attention by industry, it is still in its infancy and more effort needs to be invested to promote it in both research and industry communities.

See full paper viia Wiley library (last accessed 22 Dec, 2024)

Investigating developers perception on software testability and its effects

The opinions and perspectives of software developers are highly regarded in software engineering research. The experience and knowledge of software practitioners are frequently sought to validate assumptions and evaluate software engineering tools, techniques, and methods. However, experimental evidence may unveil further or different insights, and in some cases even contradict developers’ perspectives. In this work, we investigate the correlation between software developers’ perspectives and experimental evidence about testability smells (i.e., programming practices that may reduce the testability of a software system). Specifically, we first elicit opinions and perspectives of software developers through a questionnaire survey on a catalog of four testability smells, we curated for this work. We also extend our tool DESIGNITEJAVA to automatically detect these smells in order to gather empirical evidence on testability smells. To this end we conduct a large-scale empirical study on Java repositories containing approximately 46 million lines of code to investigate the relationship of testability smells with test quality, number of tests, and reported bugs. Our results show that testability smells do not correlate with test smells at the class granularity or with test suit size. Furthermore, we do not find a causal relationship between testability smells and bugs. Moreover, our results highlight that the empirical evidence does not match developers’ perspective on testability smells. Thus, suggesting that despite developers’ invaluable experience, their opinions and perspectives might need to be complemented with empirical evidence before bringing it into practice. This further confirms the importance of data-driven software engineering, which advocates the need and value of ensuring that all design and development decisions are supported by data.

See full paper via Google Drive (last accessed 22 Dec, 2024)

DevOps Metrics and KPIs: A Multivocal Literature Review

Context: Information Technology (IT) organizations are aiming to implement DevOps capabilities in order to fulfill market, customers and internal needs. While many are successful with DevOps implementation, others still have dificulty to measure DevOps success in their organization. As a result, the effectiveness of assessing DevOps remains erratic. This emphasizes the need to withstand management in measuring the implementation process with suitable DevOps Metrics. But what are these metrics?

Objective: This research seeks to provide relevant DevOps Metrics in order to facilitate the eficiency of DevOps adoption and better analyze DevOps performance in enterprises.

Method: A Multivocal Literature Review (MLR) is conducted, with 139 documents gathered and thoroughly examined from throughout the community, including books, scientific articles, white papers, and conferences, among others.

Results: This article conducts an extensive and rigorous MLR, contributing with a definition of DevOps Metrics, 22 main metrics, their definitions, importance and categorization in sets of KPIs, as well as exposing clear indicators on how to improve them. It is also discussed how metrics could be put into practice and what constitutes a change in the context of DevOps Metrics. The study’s outcomes will assist researchers and practitioners understand DevOps Metrics and how to better implement them.

See full paper via ACM (last accessed 22 Dec, 2024)