week 37 - Generative AI to Generate Test Data Generators, Sentimental Stack Overflow Posts analysis and The readability of test code

Generative AI to Generate Test Data Generators

Firstly shared by Maurício Aniche on X

In this paper the authors present the usage of Large Language Models (LLMs) to master domain expertise, testing fluency, and cultural literacy for generating culturally appropriate test data. The results suggests that LLMs produce executable code synthesizing fake data, ensuring interoperability with existing frameworks. Demonstrated ability to deliver high-quality test data meeting diverse testing constraints and reflecting cultural nuances.

See full paper

Mutation Testing in Practice: Insights from Open-Source Software Developers

Mutation testing drives the creation and improvement of test cases by evaluating their ability to identify synthetic faults. Over the past decades, the technique has gained popularity in academic circles. In practice, however, little is known about its adoption and use. While there are some pilot studies applying mutation testing in industry, the overall usage of mutation testing among developers remains largely unexplored. To fill this gap, this paper presents the results of a qualitative study among open-source developers on the use of mutation testing. Specifically, we report the results of a survey of 104 contributors to open-source projects using a variety of mutation testing tools. The findings of our study provide helpful insights into the use of mutation testing in practice, including its main benefits and limitations. Overall, we observe a high degree of satisfaction with mutation testing across different programming languages and mutation testing tools. Developers find the technique helpful for improving the quality of test suites, detecting bugs, and improving code maintainability. Popularity, usability, and configurability emerge as key factors for the adoption of mutation tools, whereas performance stands overwhelmingly as their main limitation. These results lay the groundwork for new research contributions and tools that meet the needs of developers and boost the widespread adoption of mutation testing.

See full paper

Deconstructing Sentimental Stack Overflow Posts Through Interviews: Exploring the Case of Software Testing

The analysis of sentimental posts about software testing on Stack Overflow reveals that motivation and commitment of developers to use software testing methods is not only influenced by tools and technology. Rather, attitudes are also influenced by socio-technical factors. No prior studies have attempted to talk with Stack Overflow users about the sentimental posts that they write, yet, this is crucial to understand their experiences of which their post is only a fragment. As such, this study explores the precursors that make developers write sentimental posts about software testing on Stack Overflow. Through semi-structured interviews, we reconstruct the individual experiences of Stack Overflow users leading to sentimental posts about testing. We use the post as an anchor point to explore the events that lead to it and how users moved on in the meantime. Using strategies from socio-technical grounded theory (STGT), we derive hypotheses about the socio-technical factors that cause sentiment towards software testing.

See full paper

Software Metrics in Agile Software Development: A Review Report

Modern software systems intensively use the Agile software development processes for their development and maintenance. The Agile development methodology encourages customer satisfaction, early incremental delivery, and overall development simplicity. Agile development methods accept changes in requirements and technology and use a more adaptive or iterative approach to planning. With the adaptation of Agile process models in the development of modern software systems, there is a need for continuous improvement in the Agile processes. Agile development processes are modified and upgraded by utilizing software metrics. There are several proposed software metrics for measuring performance and quality in Agile software development. These include customer satisfaction, story point estimation, velocity, test coverage, defects in production, and other metrics. Each software metric has its own merits and demerits. This research aims to provide a comprehensive review of published research work on software metrics. Specifically, it summarises the software metrics used for measuring the performance of Agile process models. This research paper will help understand the usefulness of various software metrics used in Agile development, and it will serve as a foundation for future research in software metrics for Agile software development.

See full paper

Investigating the readability of test code

Our review of scientific and grey literature showed that test code readability is of interest for academia and industry with a consensus on key influence factors. However, we also found factors only discussed by practitioners. For some of these factors we were able to confirm an impact on readability in a first experiment. Therefore, we see the need to bring together academic and industry viewpoints to achieve a common view on the readability of software test code.

See full paper