Sideways Test Pyramid, WebDriverIO Visual Testing and More

What is a Sideways Test Pyramid in testing?

Have you seen the latest WebDriverIO feature that allows developers and teste to easily verify web page content and layout.

What is an autonomous AI SRE?

Find out in this edition of the Test Guild New Shows Newsletter for the week of March 24.

So, grab your favorite cup of coffee or tea, and let's do this.

TestResultsIO Announces Partnership with the Test Guild

First up is an experiment. We're trying out at the Test Guild. What is it?

We've partnered with TestResultsio, and we want to announce this collaboration that offers many benefits to users who opt-in for TestResultsio plans. Now, purchasing any plan from TestResultsio grants access to their awesome end-to-end test automation tool and includes a bonus. And that is the free 5-day passes to our annual Automation Guild conference. Three copies of our Automation Awesomeness e-book, and this partnership aims to provide users with the tools and knowledge necessary to scale your end-to-end test automation efforts more effectively. Test Results offers multiple packages that you should check out.

They are designed to cater to various needs and feature benefits such as a few sets, unlimited user journeys, and integration capabilities. With the 5-day Automation Guild pass, you get 40 over-the-shoulder video training sessions with Q&A sessions and two 90-minute workshops. Plus, you get access to our community-only test automation accelerated with one new interactive and premium live training session hosted by community experts monthly for nine months from March to November. And you'll also get an exclusive discount for Automation Guild 2025. Please take advantage of it now, and I hope to see you at the Guild!

The Sideways Test Pyramid

"The Sideways Test Pyramid" introduces a new perspective on integrating tests into the continuous testing pipeline, Noemi Ferrera wrote this in response to an inquiry about test distribution.

Traditionally visualized as an upright pyramid with unit tests at the base, integration tests in the middle, and end-to-end tests at the apex, the blog proposes a "sideways" orientation to better align with the flow of Continuous Integration and Delivery (CI/CD) pipelines. This reorientation positions unit tests at the pipeline's commencement and end-to-end tests closer to deployment, reflecting their role in the development process.

The post emphasizes that while the ideal placement of tests may vary due to specific application and team needs, a common theme is that most testing should occur early in the pipeline to identify issues swiftly. However, it also acknowledges the necessity of some test repetition across different deployment environments to ensure quality. Additionally, the discussion extends to other types of tests, such as performance and security, suggesting their placement based on the application's readiness at various stages.

This insightful approach encourages a strategic distribution of testing efforts to enhance efficiency and effectiveness in software development.

Hands-on Experience Project for Aspiring Automation Professionals

Bas Dijkstra has recently shared updates on his ongoing open source test automation project, aimed at those looking to enhance their skills after completing video courses or tutorials. This project is designed to equip testers and developers with the essential skills required in the current market for test automation roles.

Upon completion, it can be utilized as a portfolio project to showcase to potential or current employers. Dijkstra has also included a reference project link in the README, which, for now, features two initial dummy tests in C# using NUnit—one that passes and another that fails—alongside a CI definition for GitHub Actions. This project is expected to expand further with detailed instructions, providing valuable hands-on experience for aspiring automation professionals.

Storybook 8

The UI tool Storybook, which is an industry staple for developing, testing, and documenting UI components, has launched its 8th edition, Storybook 8, enhancing its capabilities in testing, documentation, framework compatibility, and user experience. This release, driven by contributions from 218 individuals across 8,865 commits and 773 PRs, integrates major JavaScript frameworks and works in harmony with leading design and development tools. Storybook 8 debuts built-in visual testing through a new addon, offers experimental support for React Server Components, upgrades autogeneration of controls for Vue and React, and introduces a rearchitected support for Vite, among other improvements. It notably boosts test builds speed by 2-4 times and refreshes both desktop and mobile user interfaces. React's dependency has been removed for non-React projects, addressing a common concern among developers using different frameworks. This update promises a more streamlined, efficient, and versatile tool for UI developers and designers.

New Snapshot Testing Teature

WebdriverIO, the popular open source testing framework, released a new snapshot testing feature that allows developers to easily verify web page content and layout. Wim Selles, a WebdriverIO contributor, demonstrated the new capabilities in a tutorial video.? Using simple commands, testers can now capture and compare snapshots of text content, DOM elements, and full page screenshots. This streamlines test creation and maintenance.

The snapshot feature works across desktop web, mobile web, and even native mobile apps on Android and iOS. It integrates with WebdriverIO's new visual regression testing service for storing and comparing screenshots. Wim highlighted some best practices when using the snapshot feature, such as storing baseline images for the specific OS and browser where tests run. He also showed how to hide dynamic page elements to avoid flaky tests. The WebdriverIO team believes these enhancements will make testing more efficient for the many developers who rely on the framework. Full documentation is available on the WebdriverIO site.

Simplify and Automate Front End Testing Webinar

If you want to move to streamline front-end testing, I highy recommend you check out my upcoming webinar "Simplify and Automate Front End Testing with Synthetic Monitoring" taking place April 4th, 2024, at 1:00 PM EDT.

This free training aims to address the growing complexity and resource demands of end-to-end testing in agile development workflows. Presenters Mike Peralta, a Product Manager, and Bridgitte Kwong, a Product Marketing Manager at Datadog, will demonstrate how their Synthetic Monitoring tool can expedite test creation without code, simulate real user journeys, and integrate synthetic tests into CI/CD pipelines for earlier issue detection.

This webinar promises to equip teams with the tools to conduct comprehensive testing across various locations, browsers, devices, and network protocols, aiming for more efficient, smarter testing workflows. Registration is open for those seeking to enhance their testing processes with this innovative technique.

2024 State of DevOps Report

Puppet's 2024 State of DevOps Report, focused on platform engineering, is now freely available. Starting in 2012, Puppet has annually surveyed the DevOps community, drawing from over 40,000 respondents to explore emerging topics within the field. This year's edition delves deeper into platform engineering, examining its definitions, benefits, and practical applications within organizations. The report reveals key insights, such as the majority of platforms being at least three years old, with 70% of respondents incorporating security from the outset.

However, there appears to be a divide between senior management and developers regarding platform adoption. Additionally, the report highlights a significant underinvestment in product management skills within platform teams, despite 52% of respondents citing a product manager's role as crucial. For those interested in platform engineering's impact on DevOps, the report offers statistics, strategies, and expert recommendations.

AI SRE

Cleric recently unveiled an autonomous AI designed to function as a site reliability engineer (SRE). This AI tool, introduced on March 21, 2024, aims to assist engineering teams by taking over the continuous monitoring and issue resolution tasks, traditionally a significant part of their workload. The AI can analyze thousands of system alerts simultaneously and is capable of identifying and resolving complex cloud infrastructure problems within an average time of five minutes.

The creation of this autonomous AI SRE was driven by Cleric's co-founders, Shahram Anver and Willem Pienaar, who have secured $4.3 million in seed funding for their project. Their experience in managing large-scale infrastructures at companies like Gojek and Tecton highlighted the intense resource and time demands of traditional on-call support. Their solution is designed to alleviate these burdens, allowing engineering teams to devote more time to core development tasks rather than troubleshooting. Cleric's AI operates 24/7, integrating data from various sources to diagnose issues quickly and provide actionable resolution suggestions.

With this development, Cleric aims to streamline engineering operations, offering a model that promises enhanced efficiency and productivity for tech companies. The company is also focusing on expanding its R&D efforts and seeking strategic partnerships to further develop and integrate this technology.

Next Steps

Looking for more tips, tools, techniques and stragies to help you with e2e automation?

Chekc out my new book, Automation Awesomeness: 260 Actionable Affirmations to Improve Your QA and Automation Testing Skills, is now in PDF format.

I hope to inspire you through the insights of some of the smartest testers I've had the privilege to interview over nine years on my Test Guild podcasts.

I want you to know that I aim for you to glean an actionable tip, tool, best practice, or mindset every business day that you can apply to your daily software testing and career. Drawing from over 600 interviews conducted across nine years, I've distilled bite-sized, actionable daily advice for your benefit.

That's a Wrap

So that's it for this Test Guild News Show Newsletter edition.

I'm Joe Colantonio, and my mission is to help you succeed in creating end-to-end full-stack DevSecops automation awesomeness.

As always, test everything and keep the good. Cheers!