GenAI Learning Journey, Contract Testing, AI Safety Testing and more

What to learn about a real-world tester's GenAI / LLM learning experience journey?

How can Contract Testing Bridges the Gap in Integration Testing

Have you seen the new open-source platform that was released for AI safety testing?

Find out in this edition of the Test Guild New Shows Newsletter recap for the week of Aug 4th.

So, grab your favorite cup of coffee or tea, and let's do this.

GenAI / LLM learning journey

In a blog post titled ?? How's your GenAI / LLM learning journey going?, renowned Agile tester Lisa Crispin discusses the growing impact of Generative AI (GenAI) and Large Language Models (LLM) on software testing.

Lisa Crispin emphasizes how these technologies are helping tester by automating mundane tasks and generating valuable insights. She also highlights the importance of continuous learning and adaptability for testers to leverage these advancements effectively.

They also provide examples of how GenAI can assist in areas like test case generation, defect prediction, and even in simulating real-world usage to uncover hidden issues. Lisa encourages testers to embrace these technologies and integrate them into their workflows to enhance efficiency and effectiveness. However, the post also cautions about over-reliance on these tools and stresses the need for human oversight to ensure quality and reliability.

Software testers should know that Generative AI and Large Language Models can significantly augment their testing processes by automating repetitive tasks and providing deeper insights. The key takeaway is to embrace these technologies while maintaining a critical eye to ensure the quality and reliability of their outcomes.

How Contract Testing Bridges the Gap in Integration Testing

Bas Dijkstra latest post is about ?? Contract testing and how it has emerged as a critical approach in integration testing, providing an efficient and robust method to validate the interactions between different services. As microservices architecture becomes increasingly prevalent, traditional integration testing often falls short due to its complexity and time consumption. He goes over how Contract testing addresses these issues by defining and enforcing clear contracts between services, ensuring they communicate correctly without requiring complete end-to-end tests.

Contract testing works by establishing agreements between the consumer and provider services, which specify the expected requests and responses. This method reduces the dependency on shared environments and makes it easier to identify where issues may arise when services interact. As a result, it streamlines the testing process, making it faster and more reliable.

The article outlines the mechanics of contract testing, emphasizing its critical role in the development lifecycle. By validating interactions at a micro-level, developers can catch errors early, reducing costly fixes later in the pipeline. Furthermore, contract testing supports parallel development streams by decoupling service dependencies, thus enhancing productivity.

Check My Links

Sanjay Kumar announced the release of a new browser extension, ?? Check My Links, is available for developers, SEO professionals, and content creators. This free tool quickly scans web pages to identify broken, active, and valid redirect links, providing valuable insights in just a few seconds. Available for Chrome, Edge, Opera, and Brave browsers, it helps maintain website integrity by detecting link issues.

Features:

• Highlights valid links in green, invalid links in red.
• Allows users to customize link highlight colors.
• Exports link data for further analysis
• Includes settings to exclude certain pages from scans.

The Dependency Inversion Principle

?In her latest blog post Kristin Jackvony goes over the ?? Dependency Inversion Principle (DIP) - one of the SOLID principles - applies to software testing.

The Dependency Inversion Principle advocates that high-level modules should not depend on low-level modules. Instead, both should depend on abstractions, which means that the core codebase should rely on interfaces or abstracts classes rather than concrete implementations. This approach encourages a more flexible and decoupled architecture.

When applied to software testing, this principle can drastically improve the maintainability and scalability of tests. Test code can become cumbersome and brittle if it tightly couples with specific implementations. By utilizing interfaces and dependency injection, testers can easily swap out components with mocks or stubs, increasing the robustness of the tests and making the entire test suite more adaptable to changes.

For software testers, embracing the Dependency Inversion Principle can lead to more resilient and adaptive test suites. The principle allows testers to focus on testing behaviors rather than implementation details, thereby enhancing the ability to maintain and extend the test code efficiently.

This is the last post in a series on solid principles so make sure to check out all her other post on this topic.

GameDriver

GameDriver announced the release of its latest version, ?? GameDriver 2024.07, designed for Unreal Engine and Unity. This update brings a variety of new features aimed at enhancing automated testing for game developers and QA professionals. The release focuses on improving workflow efficiency, expanding platform support, and integrating more seamlessly with CI/CD pipelines.

Key additions include enhanced interaction with game objects, updated support for various versions of Unreal Engine and Unity, and advanced debugging tools. These improvements are set to streamline testing processes, allowing developers to identify and resolve issues more quickly.

In line with its mission to drive innovation in game QA, GameDriver 2024.07 aims to empower testers with tools that not only simplify but also accelerate the testing phase, ensuring more reliable game releases.

?Sounds cool - right?

Try GameDriver out for yourself by checking out our FREE course ?????? Getting Started with Video Game Test Automation using GameDriver

Follow the Money

Checkly has announced it raised ?? $20 million in Series B funding. The financial boost is expected to aid Checkly in enhancing its platform, which enables developers to detect and resolve issues at unprecedented speeds—up to ten times faster than traditional methods.

With the new investment, Checkly aims to expand its product's capabilities and reach, thereby facilitating more seamless integration and real-time debugging processes for DevOps teams globally.

?

Build Automated Full Stack Tests Quickly from Manual Test Runs (Don't Promote)

Nuwan Samarasekera posted on LinkedIn about a tool I've never heard of (TestChimp) and mentioned that he had worked in organizations ranging from million to trillion-dollar scales, consistently finding testing to be a critical yet suboptimal area.

As development work increasingly relies on fast, inexpensive, but often inaccurate large language models (LLMs), testing is becoming the main bottleneck in the development process.

Because of this ?? he sees the role of humans in testing is changing. While some advocate for fully AI-driven testing, a more practical approach involves humans guiding the process through exploration and insights, while AI handles repetitive tasks like automation and scaling.

He then mentioned TestChimp which is a new tool he developed, which aims to support this balanced approach, combining human intuition and creativity with AI efficiency. The platform's goal is to create a seamless integration of human and AI efforts in the testing process.

Synthesizing Observability

Over the recent years observability has become a critical component to ensure robust and efficient systems. A recent pivotal development in this field is the integration of synthetic monitoring techniques with OpenTelemetry, an open-source project providing a unified set of APIs, libraries, agents, and instrumentation to collect distributed traces and metrics.

This post goes over why ?? Maintaining optimal performance and reliability of software systems is critical. One innovative approach gaining traction is the use of synthetic monitoring combined with the powerful capabilities of OpenTelemetry.

Synthetic monitoring, which involves simulating user interactions and transactions to test application performance, is being revolutionized through OpenTelemetry's flexible, open-source tools.

This synergy enables organizations to preemptively detect performance issues and outages in their software, ensuring a superior user experience. By capturing and analyzing comprehensive telemetry data, developers can pinpoint inefficiencies and rectify issues proactively, thus safeguarding the reliability of their applications.

This approach provides a proactive layer of observability, allowing teams to understand system behavior under predefined conditions without waiting for real users to encounter problems. It’s an invaluable asset for diagnosing and resolving potential bottlenecks

Open Source AI Safety Testing

The National Institute of Standards and Technology (NIST) has announced the release of an open-source platform designed to ?? enhance the safety testing of artificial intelligence (AI) systems. This new initiative aims to provide a standardized method for evaluating AI technologies, ensuring that they function reliably and are free from biases. By creating this platform, NIST seeks to offer developers, researchers, and policymakers tools to assess the safety and effectiveness of AI applications comprehensively.

Called Dioptra (after the classical astronomical and surveying instrument), the modular, open source web-based tool, first released in 2022, seeks to help companies training AI models — and the people using these models — assess, analyze and track AI risks.

NIST's open-source platform includes a suite of testing tools and frameworks to rigorously test AI systems. These tools are geared toward identifying potential vulnerabilities and ensuring that AI technologies comply with established safety standards. The initiative aligns with NIST's broader mission to promote innovation and industrial competitiveness by advancing measurement science, standards, and technology.

Project Highlights:

• Framework for Testing: Dioptra offers tools and methodologies for consistent ML model evaluation.
• Focus Areas: The project emphasizes accuracy, robustness, and security in ML models.
• Accessibility: Resources and documentation are available to support users in implementing best practices for ML testing.

Software testers should note that the introduction of NIST’s open-source platform provides a valuable resource for assessing AI systems' safety and reliability. Testers can now leverage these tools to identify and mitigate risks, ensuring that AI applications are both secure and unbiased.

That's a Wrap

So that's it for this Test Guild News Show Newsletter edition.

?? Make sure to subscribe to never miss another episode.

I'm Joe Colantonio, and my mission is to help you succeed in creating end-to-end full-stack DevSecOps automation awesomeness.

As always, test everything and keep the good.

Cheers!

Next Steps

Join our ?????? private Automation Testing Community and get access to other like minded experts 24x7.