The Pros and Cons of different UI automation test tools - Selenium

In this next part of my blog series on testing tools, we are looking at Selenium. Perhaps one of the most well-known and widely used automation tools in the world.

Unlike the first two testing tools we looked at - UFT and Tosca - Selenium is primarily web-based and not as adaptable across different types of applications as the commercial tools. However, because it's open source, it can also be adopted and used for free which is great for many companies and especially those whose applications are primarily web-driven. It has also benefitted from a wide number of corporate collaborations over the years and grown significantly as a tool thanks to these contributions to its development. Selenium can also integrate with other open-source drivers to be able to test APIs and certain databases as well and so is growing in its usability over the years.

The architecture of Selenium is also highly componentized which makes it easy for its different components to scale up and run independently. The different modules then make use of the Selenium API to communicate with each other asynchronously. This enables its execution runners like WebDriver (which executes the tests against a web browser) to run parallel and against different browsers at the same time. It also supports different programming languages, so that teams can choose a language they are familiar with, and this will hopefully speed up development times.

This architecture has also allowed a variety of different drivers to be developed over-tine that allow it to better integrate with other application components like APIs and databases, though some of this support is still limited and it doesn’t handle all the different OS functions particularly well and so is best kept to web-based testing, though its partner application – which I will discuss at another time – Appium, has a driver which is best suited for mobile devices and supports operation on desktop OS a little better.

As the first open-source testing tool that was popularized in the industry, Selenium is widely used, well supported and has been able to build up a sizable set of features that have been established from the cross-collaboration of the many companies that can have developed its code base over time. However, as with all the older tools, it means that its initial architecture is not in keeping with modern design trends and this is why the likes of Cypress and Playwright have grown in popularity on the open-source market.

For many companies, it might seem a little weird as to why teams would even need commercial tools when Selenium is so good at what it does. However, there are still big drawbacks to the tool where it simply cannot compete with the bigger commercial tools, and it also depends on how important certain features are for companies.

Below is a summary of the history of Selenium and the different versions:

Selenium 1 (2004-2008): The first version of Selenium was released by Jason Huggins in 2004 as a JavaScript library to automate browser actions. The library was called "JavaScriptTestRunner," and it was used to test internal web applications at ThoughtWorks. The library evolved and was renamed to "Selenium Core" in 2005. Selenium Core was integrated with other testing frameworks and became a popular tool among developers.

Selenium 2 (2008-2011): Selenium 2 was a major upgrade to Selenium 1, and it was released in 2008. The new version of Selenium introduced the WebDriver API, which enabled developers to write tests in multiple programming languages. This made Selenium more accessible to a wider range of developers.

Selenium 3 (2011-2018): Selenium 3 was released in 2011, and it was the first major release under the Selenium WebDriver project. The new version of Selenium introduced a new architecture that made it more stable, faster, and reliable. Selenium 3 also introduced a new interface called the "WebDriver API" that provided a consistent API across different browsers.

Selenium 4 (2018-Present): Selenium 4 is the latest version of the Selenium testing tool. It was released in 2021 after several years of development. Selenium 4 includes many new features, including the ability to automate mobile applications, improved support for browser extensions, and improved support for browser vendors.

Some of the pros of using Selenium include:

• Open-source: Selenium is free to use and can be easily integrated with other open-source tools and frameworks. It has also been heavily modified and improved over the years with good investment from many organizations to make it better with the latest Selenium 4 update adding a lot of significant features that have brought it up-to-date with modern applications.
• Language independent: Selenium works with most programming languages and allows companies to build a framework in a language of their choice and simply wrap Selenium as an automation tool, allowing for greater tool independence. It also integrates with a wide number of existing frameworks like Maven, ANT, and TestNG allowing companies who use those frameworks to easily adopt Selenium into their development processes.
• Cross-browser compatibility: Selenium supports a wide range of web browsers, including Chrome, Firefox, Internet Explorer, Edge, and Safari, allowing users to test their applications on multiple browsers with minimal setup.
• Robust API: The Selenium API doesn’t just allow for great interoperability between its different components, but also allows for easy integration with other tools which can use of the powerful API to automate a lot of commands.?
• Can run locally: Selenium does not require us to start any server before testing and provides a direct interpretation of code onto the web services.
• Parallel Testing: Through its modular architecture it supports parallel testing against different browsers and thanks to the small size of the WebDriver runner, it's also easy to run these tests within a CI pipeline.
• Large community: Selenium has a large and active community of users and developers, which means that users can easily find help and support when needed.
• Extensibility: Selenium can be easily extended with additional functionality through the use of plugins and extensions.
• Integrations: Because selenium has been around for a long-time, it has the benefit of being widely supported and therefore easily integrated into a wide number of other tools.

Some of the cons of using Selenium include:

• Depends on 3rd-party support for reporting: If you want to get detailed test results out of your Selenium test execution you need to make use of 3rd-party frameworks like TestNG or Cucumber, as it does not have its own built-in reporting tool.
• Limited support for other application types: Selenium is primarily designed for testing web applications and may not be as well-suited for testing other types of applications, such as mobile or desktop applications. There are drivers that can assist with this, but its object identification model is simply not able to automate many desktop and mobile applications.
• Ineffective execution on complex web elements: Selenium is not effective at interacting with advanced web elements like frames, pop-ups, video, audio, and other dynamic web elements.
• Steep learning curve for non-coders: While it has improved over the years, Selenium requires a certain level of programming knowledge, and requires a strong understanding of testing frameworks and solid test automation design to work effectively. While this leads to better-automated testing frameworks, it does mean that testers who aren’t very strong at coding may struggle more than the previously mentioned commercial tools that provide a better suite of libraries that solve many of these challenges.
• No easy integration to Test Management suites: While Selenium is supported by many test management suites, it doesn’t have any supporting test management tool that allows for seamless integration like many of the paid-for testing suites that offer everything all-in-one.

?When to select Selenium:

• If your application-under-test is primarily web-based
• When you want to limit your tool spend and are comfortable working with open-source tooling
• Your testing automation has a strong understanding of programming fundamentals

In summary, Selenium is a powerful and versatile web application testing framework with a large community of users. However, its limited support for other application types, steep learning curve, limited reporting capabilities, and flaky test can be a downside for some users.