AI in software testing

Probably if you google the heading, you will get all the information which is going to be there below. I am just trying to structure my thoughts on how I am seeing AI helping/can help in software testing eventually improving software quality.

Software testing essentially is a process by which you get sufficient confidence and assurance that incremental release of the product will not break existing functionality and the new features/capabilities released incrementally will work as desired. Easier said than done ..given the pace at which new releases/changes/experiments are required to goto production in market and given the complexity involved in architecture, usage patterns and other variables involved.

In these times, AI can add considerable value and transform software testing by automating several tasks which takes time..say testcases generation, smart execution of test suites by selecting relevant testcases, Test data generation, Improving defect detection rate etc. Lets dive in...

Testcases: In the space of testcases generation, AI can be used to observe/learn from application usage patterns and observability tools. This should essentially give most used flows (and hence critical paths) of the application which needs assurance and deeper and wider testcases coverage in them should make the regression suite bulletproof. There are some tools like testim, functionalize etc which are doing some parts of this already.

Automation: Biggest pain of test automation is maintainability. AI can help here effectively by learning from execution history and predicting/flagging problem areas which can give more failures in automation execution. AI can also help with self healing scripts which essentially update themselves when UI elements/layout changes. This is very pertinent requirement given the rate at which businesses want to experiment with UI screens/flows to hit that customer sweet spot. AI can also help in clearly defining what needs to be run in test automation, prioritizing testcases. Some plugins are available for Jenkins which does this.

Defect Data Analysis: Looking at historical data of defects can be a good training data for AI models which can predict where bugs can be in future. They can flag product areas and any other patterns which can greatly help development team focus and put measures to improve on incrementally. Same can also be extended to support data. Various metrics and data captured in support processes can be used to analyze "problem" areas of the product and its usage. This can set the priority for product/design team to work on...essentially bringing down support costs and creating customer delight. "Smart" analysis of support data can also give several improvement points to quality engineering team.

Perf and Sec: Moving outside functional testing to 2 other important aspects of quality namely Performance and Security. Crux of performance testing is "how you test" and "how/what you infer" from the results leading to "actions taken". Here AI can certainly help in finding patterns and flagging anomalies. AI can also help generate real world user scenarios for stress testing and load testing. Similar usecase in security testing too where AI can help mimic real world usage and run pen tests which are more effective than targeted pen tests run by humans. AI could also help in coding phase itself, analyzing patterns and flagging risks and showing mitigation suggestions for the developer.

Overall I see multiple opportunities for AI to better testing and improve efficiency. Talking about shift left, AI can make huge impact in code review phase by suggesting improvements and catching bugs early in the cycle. Several tools/plugins are available already for this. In terms of maintaining uptime and service quality, several applications are there in observability space. Devops can greatly improve in terms of predicting events, taking preventive actions and finding optimizations in resources usage by application of AI.

What are your thoughts on this? Happy to engage...please comment!