Accessibility Testing with Robot Framework using the AXE?Library
Vanderlan Alves, MBA,CTAL-TM, ASF
QA Engineer | QA Tech | Lead Expert in Test Automation & DevOps | Cypress, Robot Framework, API Testing, Azure, Postman
About a one year ago, I had the opportunity to participate in an event here at Riachuelo organized by Julio Cesar Nietto, which also featured Marcelo Sales! A true expert on accessibility. His lecture on the topic: “everything you do in your daily life is accessibility!” made me reflect on how my work in quality could contribute to such an important subject!
Why is accessibility important??
According to data from the World Health Organization, about 10% of the population has some form of disability. In Brazil, around 45,606,048 million people have some type of disability, which corresponds to 23.9% of the general population, according to data released by the Brazilian Institute of Geography and Statistics (IBGE). This disability can be visual, auditory, motor, mental, or intellectual. Furthermore, according to the 2010 IBGE census, the most recurrent disability in Brazil is visual (18.6%), followed by motor (7%), auditory (5.10%), and finally, mental disability (1.40%).
We must consider that the quality of a given system is directly influenced by the customer’s expectations, and for at least 23.9% of the Brazilian population, if a system’s accessibility has not been considered, it lacks quality! And as we well know, quality today is not only a competitive advantage but a prerequisite for a company to place its product in the global market. Ignoring accessibility is akin to losing money!
How to implement automated tests??
Within the DEVOPS flow, automated tests play an increasingly important role in our daily lives. Therefore, improving validations by adding accessibility tests helps us perform more comprehensive work. However, it is essential to remember that automated accessibility tests do not validate the completeness of WCAG rules, so manual validations are still necessary in the quality process.
Robot Framework?
Robot Framework is an open-source automation and RPA framework based on Python. Its main strengths are its ease of learning and flexibility, allowing it to operate across various platforms such as MOBILE, WEB, DESKTOP, API, and many others!
AXE Library?
AXE is an open-source project developed by Deque Labs. It is an accessibility testing engine for websites and other HTML interfaces. It is based on WCAG rules and can cover up to 57% of these rules during validations.?
WCAG?
WCAG (Web Content Accessibility Guidelines) are accessibility recommendations for web content, meaning they are guidelines explaining how to make web content accessible to everyone. I recommend this link to learn more about this topic.?
Installing the tools?
First, we need to install the dependencies for Robot Framework. If you’re new to this, here’s a great article: Simplifying test automation with Robot Framework.
pip install robotframework-axelibrary==0.1.5
For more information, check out the library’s?
Creating the test project?
This project can be fully reviewed on my GitHub, which I’ll make available for reference.?
First, let’s choose the site to be evaluated in the accessibility tests. For this example, I thought it would be cool to use the homepage of Riachuelo’s marketplace.
First, let’s structure Robot’s directories, remembering that we will use the Page Object Model to structure it. If you want more information on this model, read the article below:?
We will create three folders:
In the test-case folder, we will create the file “evaluate_home_screen.robot.”
领英推荐
In the Settings section, we will add the resources from the files used to define the keywords. In Suite Setup and Suite Teardown, we will include the opening and closing of the browser.
In the Test-case section, we will write the test scenarios that will be executed. We will use BDD to standardize the scenarios. For more information on using this methodology, read the article below:?
In the Resources folder, we will store the keywords separated by page. We will start by creating the file “resources.robot,” where we will store the keywords used regardless of the page.
In the Settings section, we will specify the libraries used in the tests.
In the Keywords section, the keywords we will use in the tests will be declared. Accessibility tests will be declared in: “evaluate page accessibility,” where we execute the AXE Library keyword: “Run Accessibility Tests.” Here, the site’s accessibility will be evaluated. The test result will be stored in the dictionary &{results}, which we can later process to evaluate the test, as we will see in the next steps. In this keyword, we also specify the JSON file where the AXE report will be stored.
In the keyword: “Log Readable Accessibility Result,” we will log the test results in Robot’s standard report.
Finally, in the keyword: “the result should not show accessibility violations,” we will create an assert to fail the test if the violations exceed a certain number.
Finally, in the “home.robot” file, we will include access to the page to be validated by the tests.
Here is the GitHub with the code used in these tests: https://github.com/Rommelfoxx/axe_library.
Evaluating the accessibility test?results?
To run the tests, execute the command below:
robot -d results test-case
Our test failed, so let’s analyze Robot’s report to understand what happened.
Opening the log of the “Log Readable Accessibility Results” keyword, we can see more details of the errors found. Seven accessibility violations were found.
In the report, we can see a description of the violation, as well as a link providing more details about the error and its impact level.
Clicking the link takes you to a URL with more detailed information about this violation.
I also recommend checking the guide at https://guia-wcag.com/ to see which standard was violated.
Conclusion?
Accessibility is a fascinating topic, filled with a wealth of information and technologies to assist us in testing. Therefore, there is no justification for overlooking this subject. Accessibility is something that cannot and should not be ignored. The world will become an increasingly accessible place, and those who do not follow this path risk being left behind.