The impact of AI on software testing

By Martin Sutcliffe, QA practice lead at Iridium

AI-powered testing can mean accelerated delivery cycles and more robust quality control. However, the impact will vary based on the organisation’s readiness for AI adoption and its approach to integrating AI within existing testing processes. While the technology is promising, it’s essential to understand its scope and set realistic expectations, as the transition to AI-driven automation requires planning and strategic investment.

Understanding AI automation

AI automation in testing involves using machine learning (ML) and data-driven techniques to enhance traditional test automation. Instead of rigid, rule-based scripting, AI testing frameworks can dynamically identify and validate user interfaces, simulate user behavior, and predict potential defects in new builds by leveraging past testing data. Tools such as “self-healing” scripts, which can adapt to UI changes without human intervention, are popular applications of AI in this field. This flexibility helps QA teams tackle the complexity of modern software systems.

For businesses new to AI automation, education and assessment are critical first steps. Understanding what AI can and cannot do, as well as how it differs from traditional automation, helps set clear goals. Businesses should explore use cases, consult industry experts, and even run pilot projects to gain practical insight into how AI could fit within their QA strategy.

AI automation isn’t a magic wand

AI automation may simplify repetitive testing, but it isn’t a complete substitute for human oversight. Like any other technology, AI requires effective implementation and continued monitoring. Businesses must configure machine learning algorithms to accurately interpret testing scenarios, validate results, and adjust to new data. The process is akin to learning to drive a car—AI is the vehicle, but it requires a competent driver who knows where to steer it.

It requires ongoing supervision to ensure the AI is making accurate interpretations and adjustments. Implementing AI without a well-defined plan can lead to negative outcomes, such as missed bugs or false positives and QA teams should be prepared to adjust the AI parameters and intervene if test outcomes are inconsistent.

Preparing a business for AI test automation

To prepare for AI automation, strong foundations are essential:?

1. Define objectives and KPIs: Establish what the business aims to achieve with AI test automation, whether it’s faster test cycles, reduced costs, or improved defect detection rates.
2. Data availability and quality: AI thrives on data, so businesses must ensure access to relevant historical test data to train the AI models effectively. Quality data is crucial for AI to learn accurately and improve its decision-making capabilities.
3. Infrastructure readiness: For AI testing to run effectively, underlying systems should support AI applications, which may require modernising infrastructure and embracing cloud-based or DevOps-compatible solutions.
4. Team training: Investing in training for the QA team on AI tools, scripting languages, and data analysis ensures the team can handle AI-assisted testing tasks and make necessary adjustments.”

AI testing for established firms with legacy systems

Start-ups and newer companies are often more agile and can adopt AI testing more seamlessly due to fewer dependencies on legacy systems. However, the adaptability of AI testing isn’t exclusive to start-ups - well-established firms, too, can benefit if they address specific challenges unique to their larger, more complex systems.

For established firms, implementing AI-based testing may seem daunting, especially if they rely on legacy systems or older mainframes. AI testing frameworks are typically designed for modern architectures, so retrofitting them to work with outdated systems can be challenging. This doesn’t mean AI is out of reach, but it does require a more strategic approach:

1. Layered approach: Businesses can adopt a “layered” approach to AI testing, where AI tools are applied to specific, non-legacy parts of the system first. This helps minimise disruption and allows teams to gradually test AI’s effectiveness before expanding to other areas.
2. Partial modernisation: For firms with significant legacy systems, it may be beneficial to modernise critical portions of the infrastructure or adopt cloud services where feasible. This doesn’t require a complete overhaul but enables AI-driven testing on select modules.
3. Third-party tools and integrations: Some AI testing tools are built to integrate with older environments. Evaluating third-party solutions that support legacy system integration may provide a feasible workaround.
4. Phased transformation programme: Undertaking a digital transformation programme could prepare an organisation for AI-driven testing. This ensures that critical dependencies are addressed, and modern testing frameworks can be implemented gradually across systems without jeopardising existing operations.

Transformational change before AI test automation

For most established businesses, particularly those reliant on mainframe systems, a transformation programme would be ideal before implementing AI-driven testing. This transformation would modernise core systems, align QA practices with DevOps principles, and improve data availability for AI models. Transformation doesn’t necessarily require a complete overhaul, it may involve selectively upgrading certain systems or migrating specific applications to cloud-based platforms that better support AI tools.

Brett Hargreaves , Iridium’s Cloud and Architecture practice lead, adds: “Modernising core systems—whether through selective upgrades or migrating key applications to cloud-based platforms—lays the groundwork for smoother AI integration.?

“Cloud-based environments not only improve data accessibility and scalability but also enable real-time collaboration aligned with DevOps principles. With this foundation, AI can be applied more effectively to testing processes, enhancing accuracy and speed while avoiding the bottlenecks and compatibility issues that often come with legacy technology.”

Reaping the benefits of AI-powered testing

Once infrastructure and QA workflows are modernised, AI can be introduced more effectively, providing tangible benefits without the complexity of navigating outdated technology constraints.

Satya Nadella, CEO of Microsoft, recently said: "Artificial Intelligence is fundamentally changing how we develop and test software, enabling us to achieve higher quality and faster delivery through automation and intelligent insights." I couldn’t agree more.?

Speak to Martin

Please don’t hesitate to reach out to [email protected] to discuss your software testing needs.