What CTOs need to know about GenAI Testing and More

What do CTOs need to know about GenAI in Testing?

Have you tried Browser Use yet?

Want to know a new tool to help you analyze and debug tests.

Find out in this edition of the Test Guild New Shows Newsletter for the week of March 16. So, grab your favorite cup of coffee or tea, and let's do this.

Before we get into it if you haven’t already, please make sure to subscribe to our Test Guild LinkedIn New show newsletter that I’ll have a link for down below never miss another episode.

Show Sponsor ??

Shout out to our sponsor ZAPTESTAI—a platform you'll want to check out if you're serious about automation. Their AI-powered tools, like Plan Studio, turn your ALM data into reusable, automation-ready modules without the usual headaches. At the same time, Copilot eliminates tedious tasks like script generation and object management. The best part? They're offering a 6-month No-Risk Proof of Concept, so you can test-drive the platform and see actual results before committing.

If you want to level up your automation, ?? ZAPTEST.AI is worth a serious look!

Browser Use, one of the tools powering Manus?

A little-known AI tool, Browser Use, is experiencing a surge in popularity, driven by its integration into the viral AI agent platform Manus. Developed by Gregor Zunic and Magnus Müller, Browser Use facilitates easier interaction between AI agents and websites. Its daily downloads increased more than fivefold in a week, reaching 28,000 on March 10th, from approximately 5,000 on March 3rd.

This growth is directly attributed to a social media post highlighting Manus' use of Browser Use, which garnered over 2.4 million views. Browser Use extracts website elements like buttons and widgets, allowing AI models to interact with them. The tool also manages multiple browser tabs and handles various input methods. While a paid managed version exists, the freely available, self-hosted version is the one driving this recent increase in usage.

Software testers can leverage Browser Use to create automated tests that simulate user interactions on websites.?You can also sue Manus to help with things like:

Here are some possible use cases of Manus for automation testing and software testing:

1.??? Test Script Generation - Manus can analyze application requirements and automatically generate test scripts in various languages like Python, JavaScript, or Java.

2.??? Test Case Development - Manus can help create comprehensive test cases based on user stories, requirements documents, or existing code.

3.??? API Testing Automation - Manus can set up automated API testing frameworks, write test scripts, and validate responses against expected results.

4.??? UI Testing - Manus can create and maintain UI test automation using tools like Selenium, Cypress, or Playwright.

5.??? Performance Testing - Manus can develop performance test scripts, execute them, and analyze results to identify bottlenecks.

6.??? Load Testing - Manus can create scripts to simulate high user loads and monitor system behavior under stress.

7.??? Security Testing - Manus can implement automated security scans and vulnerability assessments.

8.??? Regression Testing - Manus can maintain and update regression test suites as applications evolve.

9.??? Test Data Generation - Manus can create realistic test data sets for various testing scenarios.

10. Test Environment Setup - Manus can automate the configuration of test environments with necessary dependencies.

11. Continuous Integration/Continuous Deployment (CI/CD) Pipeline Integration - Manus can help integrate testing into CI/CD workflows.

12. Test Result Analysis - Manus can analyze test results, identify patterns in failures, and suggest potential fixes.

13. Cross-browser and Cross-platform Testing - Manus can set up frameworks to test applications across different browsers and platforms.

14. Mobile Application Testing - Manus can create automated tests for mobile applications using frameworks like Appium.

15. Database Testing - Manus can write scripts to validate database operations, integrity, and performance.

16. Documentation Generation - Manus can automatically generate test documentation, reports, and coverage metrics.

17. Mock Service Creation - Manus can develop mock services to simulate dependencies for isolated testing.

18. Exploratory Testing Assistance - Manus can suggest areas to explore based on code changes or user behavior patterns.

19. Test Maintenance - Manus can update test scripts when application changes break existing tests.

20. End-to-End Testing - Manus can create comprehensive end-to-end test scenarios that validate complete user workflows.

Pretty cool - not a silver bullet but I think its worth playing around with to see for yourself.

And currently Manus is free to use (even thought I hit a daily use limit all the time :))

Webinar of the week

Traditional software testing methods are struggling to keep pace with accelerated development cycles, leading to increased costs and delayed releases. So in an?upcoming TestGuild?free training Seamless, Scalable, Swift:? what every CTO Should Know about The GenAI Testing BlingIO will join use to share the challenges of current testing practices and showcased GenAI's potential to address these issues.

The event will feature speakers from BlinqIO, including CEO Tal Barmeir, a veteran of the testing industry, and Sapnesh Naik, Head of QA, who discussed real-world applications of GenAI in testing. Rebecca Villalba, BlinqIO's Operations Manager, also contributed to the discussion.

The webinar will include a live demonstration of GenAI-powered testing tools, aiming to provide attendees with practical insights into implementation and best practices. A key focus was on quantifiable benefits and return on investment (ROI) through real-world case studies.

I built auto-healing tool for Playwright tests

On LinkedIn Felip demonstrated A new tool he designed?to analyze and debug tests.?The tool, analyzes data from test traces to identify and suggest fixes for common errors. Unlike standard error messages, which often lack crucial context, this system delves deeper into timing issues, dead clicks, network problems, and other potential causes of test failure.

The MCP server integrates with a Cursor IDE, providing developers with specific remediation advice, whether the issue lies within the test code itself or the application being tested. This proactive approach aims to shorten debugging cycles and streamline the testing process. The system's ability to diagnose the root cause of test failures, rather than merely indicating failure, represents a considerable advancement in automated testing capabilities.

Chrome DevTools Integrates AI Assistance for Web Developers

Google Chrome's developer tools now include an AI assistance panel powered by Gemini, Google's large language model. This new feature allows developers to interact directly with the AI within DevTools, leveraging context-specific information from the inspected webpage. The AI can answer questions about styling, network requests, source code, and performance, offering explanations and potential solutions to debugging challenges.

However, users are cautioned against sharing sensitive information, and data is retained for up to 18 months in an anonymized format.

I think this is a quick way for you to utilize the AI assistance to see if it can help.

Boyka Framework is now officially a part of Selenium Ecosystem.

Wasiq Bhamla the announced a few days ago that his Boyka Framework is now officially a part of Selenium Ecosystem. We covered the Boyka framework awhile agao on this show but if you missed it its helps you create highly maintainable Automated tests for Web, API, Android and iOS applications.

For example you can use it to

• Automate Rest API, perform schema validations, validate response headers, status and body.
• Automate Web apps on all the major browsers, run on local browsers, run on Selenium Grid, run on cloud providers like BrowserStack and LambdaTest.
• Automate Android apps on local Android Emulator, real devices, cloud provided devices by BrowserStack and LambdaTest.

Create Your Own AI Agents?

OpenAI has released new tools aimed at simplifying the development of AI agents for businesses. These tools, part of the new Responses API, are designed to replace the soon-to-be-retired Assistants API. The API allows businesses to create AI agents capable of performing tasks like web searches, file scanning, and website navigation, similar to OpenAI's Operator product.

The new tools utilize GPT-4o search and GPT-4o mini search models, boasting high factual accuracy rates on OpenAI's SimpleQA benchmark. A key component is the Computer-Using Agent (CUA) model, which enables automation of computer tasks. Importantly, OpenAI also released an open-source Agents SDK to facilitate integration and monitoring of these AI agents. While acknowledging ongoing challenges with AI agent reliability and potential for errors, OpenAI emphasizes these are early iterations of the technology.

To me this is a great way to familiarize yourself with AI agents and the potential towards testing the interactions between AI agents and existing systems, and verifying the reliability of the AI agents' actions and outputs. Which I think is a skill that is going to grow more and more in demand

AI autonomous debugging?

Lightrun, a developer observability platform, has introduced an AI-powered autonomous debugger designed to significantly reduce the time and effort required for software troubleshooting. This new tool automates the root cause analysis process, moving from identifying a problem to implementing a code fix.

The system analyzes logs, traces, and metrics in real-time to detect anomalies and predict potential failures. It correlates events across distributed systems and suggests fixes based on identified patterns and historical data. The debugger operates within the developer's Integrated Development Environment (IDE), streamlining the debugging workflow.

The process starts with the developer providing the debugger with the issue description. The AI then identifies the relevant code files, suggests troubleshooting steps and root cause hypotheses, and guides developers in validating these hypotheses using Lightrun's debugging actions. The entire process, from ticket to fix, is substantially automated. Lightrun claims this approach dramatically decreases Mean Time to Resolution (MTTR) and enhances software quality.

Testers could proactively incorporate AI-driven debugging suggestions into their testing process. By using the AI's hypotheses to guide testing efforts, you can focus your validation on the most likely problem.

Detectify

Detectify, a cybersecurity firm, announced the release of Alfred, an AI-driven system designed to automate vulnerability assessments.

Alfred leverages large language models (LLMs) to process vulnerability data from sources like the CVE database, enriching it with information gathered from various online resources. This automated process prioritizes vulnerabilities based on exploit likelihood using the Exploit Prediction Scoring System (EPSS) framework and generates payload-based security tests. These tests are then integrated into Detectify’s platform after quality assurance checks by human researchers.

The system is designed to address the challenge of the ever-increasing number of vulnerabilities and the limitations of traditional automated scanners, which often produce high rates of false positives and miss vulnerabilities not documented in the CVE system. Alfred is currently being rolled out to all Surface Monitoring and Application Scanning customers.

I think Software testers should be aware of tool like Alfred and their implications for automated vulnerability scanning.?

That's a Wrap

So that's it for this Test Guild News Show Newsletter edition.

Make sure to ?? subscribe to never miss another episode.

I'm Joe Colantonio, and my mission is to help you succeed in creating end-to-end full-stack DevSecOps automation awesomeness.

As always, test everything and keep the good.

Cheers!

Next Steps

Join our ?????? private Automation Testing Community and get access to other like minded experts 24x7.