Leveraging ChatGPT to Build an Automation Framework using the Page Object Model

1. Introduction

In a previous article, we explored using Playwright's Codegen tool to generate test cases for the NUnit framework using C#. The generated test cases were functional but lacked abstractions and presented maintenance challenges due hard-coded values throughout the generated code. In this article, we leverage Codegen again, but this time, we use ChatGPT to build a well-structured, readable, and maintainable testing framework using the Page Object Model (POM). While our example focuses on C# and NUnit, the principles discussed are adaptable to other technologies like Java and JUnit, ensuring flexibility and scalability in automation efforts

2. Manual Test Case

Before diving into automation, it is recommended to start with a manual test case. Let us consider a scenario where a user performs a quick smoke test to verify links on the home page of an e-commerce website. In this example, we will use “https://demo.nopcommerce.com”

3. Recording the test case with codegen

By analyzing the steps outlined in the manual test case, we can identify some of the pages for the application. Each user action described in the manual test case navigates the user to a different page, namely, ,“Home” page, the “Registration” page, the ?“Computers” page, and the "Electronics" page.

?

Our first task towards building the framework is to identify the locators for elements on this page, leveraging Playwright's Codegen tool.

Codegen allows us to record user interactions and generates code snippets based on those interactions. To run Codegen from the command line, with the URL of the application under test, we can start the Playwright inspector tool with the following command.

npx playwright codegen https://demo.nopcommerce.com/        

This command opens ?the “Playwright inspector” and the browser side by side, as shown in Figure 1. and Figure 2.

Our primary objective of using codegen is to identify the locators for the pages under construction. By convention, we will use “Page_<Name> to name our pages, so our home page is called “Page_Home”.

4. Identify home page locators

Here are the complete steps for identifying the locators for the home page using the functionality of Codegen. In this phase, we make use of the “Record Button” and the “Assert Visibility” button.

1.?? Click the record button on the browser window

2.?? Click on the “Assert Visibility” button

3.?? Click on the URL (https://demo.nopcommerce.com/) in the address bar.

Notice how the state of the “Assert Visibility” button changes momentarily when we click on the element we are asserting, like the address bar in this case. If you have a fast computer, you may not see the change!

4.?? Once the “Assert Visibility” button is back to its original state, click on it again, then click on “Registration”, then wait until the icon goes back to its original state.

5.?? Next, click on “Login in”, then wait until the icon goes back to its original state.

6.?? Next, click on “Computers”, then wait until the icon goes back to its original state.

7.?? Finally, click on “Electronics”, then wait until the icon goes back to its original state.

Here are the results of our interaction with the page using “Codegen” for the NUnit target settings.

5. Building our first page object

One of the challenges we face when creating an automation framework using POM is, ironically, the creation of the objects themselves. The subsequent hurdles involve building the initial set of objects, refining them iteratively, and continually expanding the collection of objects as more tests are integrated into the framework. To streamline this process, we have opted to leverage ChatGPT to accelerate our progress.

Notice that the generated code, from “Codegen”, has a couple of issues, which we need to fix before moving forward. (Author note: I am not sure if these are “Codegen” bugs or just a display issue on my end.)

1.?? Line 11 in Figure 5 is missing the statement to create a “page” instance. The line should have been:

???????????var page = await context.NewPageAsync();        

2.?? The lines starting with “await Expect(page…” are missing the "Assertions". For example, "Electronics" should have been.

await Assertions.Expect (page.GetByRole(AriaRole.Link, new() { Name = "Electronics" }).First).ToBeVisibleAsync();        

?

Anyway, I thought this is a good opportunity to challenge ChatGPT.

To get started with “ChatGPT”, let us issue the following prompt. It is a bit long, but I wanted to investigate how far I can push ChatGPT.

After a few seconds, we get the following response.

The test case generated by ChatGPT for this test case was correct and complete. I made some minor formatting modifications. Other than that, ?I copied it into Visual Studio and executed it.

Our next task is to generate a page object for this test case. Here is the ChatGPT prompt that I used.

1.?? Create a page object for the test case "VerifyPageElementsAreVisible".

2.?? Use “Page_Home_Tests” for the class name

3.?? Use Pages_generated_chatgpt for the namespace

Overall, we can use ChatGPT as a senior consultant looking over our shoulders as we iterate through the lifecycle of creating the framework.

6. Create an NUnit/C# project

At this point, we are ready to put our efforts into action.

• Start Visual Studio
• Create an NUnit project,
• Install the following NuGet packages. Microsoft.Playwright Microsoft.Playwright.NUnit Microsoft.Playwright.TestAdapter

?

Please note that I am using VS 2022, but any version should work just as well.

?

Next, create 3 folders to organize the automation framework

1.?? WebElements_recorded_codegen

Create a class under this folder, then paste the generated code by “Codegen” (from Figure 5) into it. After pasting it, comment out the code, since we only need this code to copy and paste into the ChatGPT prompt.

2.?? Tests_generated_chatgpt

Create a class under this folder, then paste the code generated by ChatGPT ?(Figure 7) into it. This is a complete test case. Compile it and run it. Congratulations!

3.?? Pages_generated_chatgpt

Create a class under this folder, then paste the code generated by ChatGPT ?(Figure 8) into it. This is a complete page object.

?

Here is the structure of the solution after completing the steps above.

This completes our minimal framework with the help of ChatGPT. In a future article, we will continue to leverage ChatGPT? by creating additional test cases, updating the framework with additional page objects to support the new test cases, and refactor the code as needed.

7. Points of Interest

ChatGPT is a great tool to use, and it is here to stay. Here are some of my thoughts on using it for this project.

1.?? The shorter and the more specific the prompts, the better results I got.

For example, in my initial prompts I was curious to see what the response would be when a prompt with multiple objectives. Here is an example where I provided ChatGPT with the code snippet in Figure 5.

I tried this prompt with different AI models, specifically ChatGPT, Gemini, and Copilot, with varying degrees of success.

For this specific prompt, ChatGPT came on top, and it tried to address all the points from the prompt. So, to summarize, the shorter and specific the prompts, the better.

2.?? In my quest to iterate between creating test cases and page objects, ChatGPT seemed, at times, to get confused, even when my prompt was specific. For example, it would mix code from page objects and tests.

3.?? ChatGPT seems to have a shot term memory issue. For example, after it generated an NUnit test case using the [Setup] and [Teardown], I requested that it uses the AAA style (Arrange, Act, Assert). Initially, it updated the test case with the AAA style. But I tried to refine the same example, giving ?ChatGPT short and specific requests, responses were not consistent. Sometimes, it used NUnit style while other times, it used the AAA style.

8. GitHub Repo

Here is the code for this initial release of the framework on GitHub.

9. Conclusion

Leveraging ChatGPT alongside the Playwright's “Codegen” tool has proved to be an effective approach for rapidly developing automated testing frameworks. By generating well-structured code snippets and page objects, we were able to streamline and accelerate the process of building an initial version of the framework.

Through the combination of manual test case analysis, automated code generation, and iterative refinement, ChatGPT served as a valuable assistant throughout the development process, providing helpful suggestions and even complete generation of pages objects and tests cases.

Moving forward, we will continue to use ChatGPT to iterate on our framework, incorporating additional test cases, refining existing code, and addressing any maintenance challenges that arise. With ChatGPT's assistance, we can ensure that our testing framework remains flexible, scalable, and aligned with best practices in software testing.