week 38 - ChatGPT Usage in Software Engineering Practice and The State of the Practice in Software Testing
A Cost and Resource Aware Model for E2e Testing in the Cloud
ContextContinuous integration (CI) practices have significantly reduced software development cycles from months or years to weeks or days. In CI, new software versions are frequently released and tested, which can require substantial testing resources, especially for end-to-end (E2E) testing. Migrating E2E testing to the Cloud is recognized as a solution to address time/resource constraints by leveraging the Cloud unlimited and elastic infrastructure. However, it also introduces new challenges for testers in face of the broad range of configuration options available in the Cloud.
Objective: The aim of RETORCH* is to support testers in comparing multiple Cloud alternatives for executing their test suite, going beyond the mere cost billed by the provider. It enables testers to select the Cloud infrastructure that best aligns with their testing objectives.
Method: RETORCH* analyzes the cost billed by the Cloud Provider and breaks it down into the actual testing cost, which is the cost incurred to execute the test suite, and the cost of possible unused infrastructure (over provisioning). Based on these distinct costs, testers can compare different Cloud alternatives, taking into account various Billing Options and test suite execution plan.
Results: RETORCH* has been applied to three case studies using a real-world application's E2E test suite. We compare various deployment alternatives and test suite execution plan options, demonstrating the impact of different decisions made during the migration of the E2E test suite to the Cloud on the cost of executing the test suite.
Conclusion: RETORCH* provides a means to compare different Cloud infrastructures for the purpose of E2E testing not solely based on the cost billed by Cloud Providers. The case studies highlight how the different decisions made during the migration of E2E test suites to the Cloud may have a significant impact on all the costs related to executing the test suite.
领英推荐
Beyond Code Generation: An Observational Study of ChatGPT Usage in Software Engineering Practice
Large Language Models (LLMs) are frequently discussed in academia and the general public as support tools for virtually any use case that relies on the production of text, including software engineering. Currently there is much debate, but little empirical evidence, regarding the practical usefulness of LLM-based tools such as ChatGPT for engineers in industry. We conduct an observational study of 24 professional software engineers who have been using ChatGPT over a period of one week in their jobs, and qualitatively analyse their dialogues with the chatbot as well as their overall experience (as captured by an exit survey). We find that, rather than expecting ChatGPT to generate ready-to-use software artifacts (e.g., code), practitioners more often use ChatGPT to receive guidance on how to solve their tasks or learn about a topic in more abstract terms. We also propose a theoretical framework for how (i) purpose of the interaction, (ii) internal factors (e.g., the user’s personality), and (iii) external factors (e.g., company policy) together shape the experience (in terms of perceived usefulness and trust). We envision that our framework can be used by future research to further the academic discussion on LLM usage by software engineering practitioners, and to serve as a reference point for the design of future empirical LLM research in this domain.
State of the Practice in Software Testing Teaching in Four European Countries
Software testing is an indispensable component of software development, yet it often receives insufficient attention. The lack of a robust testing culture within computer science and informatics curricula contributes to a shortage of testing expertise in the software industry. Addressing this problem at its root —education— is paramount. In this paper, we conduct a comprehensive mapping review of software testing courses, elucidating their core attributes and shedding light on prevalent subjects and instructional methodologies. We mapped 117 courses offered by Computer Science (and related) degrees in 49 academic institutions from four Western European countries, namely Belgium, Italy, Portugal and Spain. The testing subjects were mapped against the conceptual framework provided by the ISO/IEC/IEEE 29119 standard on software testing. Among the results, the study showed that dedicated software testing courses are offered by only 39% of the analysed universities, whereas the basics of software testing are taught in at least one course at every university. The analysis of the software testing topics highlights the gaps that need to be filled in order to better align the current academic offerings with the real industry needs.