Let the power of UI Tests help your Sitecore Solution

Hello dear Sitecorians, I hope you guys are ok out there. Corona has struck us all hard, keep your head up and keep going.

I was thinking of showing you guys how powerful UI Tests can be in a solution.

UI Tests have been around for awhíle, it’s nothing new but it’s something you rarely see in a Sitecore solution and that is too bad really! UI tests are quite easy to set up and they will be your friend when you deploy to DEV/QA/STAGE/PRODUCTION. *UI tests can also be used when you are developing (similar to unit tests), but BDD – Behavior-Driven Development.

Let’s see what the Helix docs tell us about UI tests and where they should reside in a Helix solution:

When you are doing automated testing – and especially for automated browser testing – you are often running the tests on an assembled solution and therefore the tests might be dependent on the combined pages of the website. Even though your tests might just focus on the features or functionality in a single module, they depend on the functionality of integration mapping in other modules or layers. Therefore, the tests belong in the Project layer as opposed to together with the module in the Feature or Foundation layer.

*3.3.3. Integration, Acceptance or other automated testing methods

Yes, the message is quite clear. Put the UI tests in the PROJECT layer ??

Next will be to decide what UI test framework to use. There are a bunch of them out there, here is a great site(on GitHub) that can help you to decide what UI test framework you want to work with:

Awesome Test Automation

Microsoft recently released a quite interesting UI test library, open-source Node.js library – Playwright

But the pick will be… SpecFlow. So what is SpecFlow?

*SpecFlow has been around for a while and you will find a lot of help on the net ??

SpecFlow is a test automation solution for .NET built upon the BDD paradigm, and part of the Cucumber family. Use SpecFlow to define, manage and automatically execute human-readable acceptance tests in .NET projects (Full Framework and .NET Core).

What I really like about SpecFlow is that it’s using the Gherkin syntax:

SpecFlow tests are written using Gherkin, which allows you to write test cases using natural languages. SpecFlow uses the official Gherkin parser, which supports over 70 languages. These tests are then tied to your application code using so-called “bindings”, allowing you to execute the tests using the testing framework of your choice.

Example of a Gherkin syntax:

Scenario: Promo cards visible

Given I am a website visitor

When I navigate to the Home page

Then I expect to see promo cards

Gherkin syntax is also common when creating user stories – Acceptance Criteria for User Stories.

This means you can use the Acceptance Criteria (Gherkin syntax) from the user stories when you create your SpecFlow UI tests – Kill two birds with one stone

*I prefer the Swedish saying here, it’s less cruel. Nobody wants to kill birds – Sl? tv? flugor i en sm?ll ??

Before we begin we should install SpecFlow for Visual Studio 2019 from the VisualStudio MarketPlace. It is optional but it will help you a lot ??

The Visual Studio integration includes a number of features that make it easier to edit Gherkin files and navigate to and from bindings in Visual Studio. You can also generate skeleton code including step definition methods from feature files. The Visual Studio integration also allows you to execute tests from Visual Studio’s Test Explorer

*You can read more about it at – https://specflow.org/documentation/Visual-Studio-Integration/

Time to set up the UI tests in our Helix solution, as always we will use the Sitecore Helix Examples.

We will be working in my fork – helix-basic-tds-with-docker-vs2019-project-type

How about if we would set up/add a simple UI test for the promo cards? The promo cards are located in Feature.BasicContent

According to the Helix docs, we should put the UI tests in our Project layer. Let’s create a tests folder in Project/BasicCompany.

We should preserve the “naming”, meaning the new UI test project will be BasicCompany.Feature.BasicContent.UITests. A good thing is to postfix the UI test projects – UITests. It will be easier to find/filter them in CI/CD builds.

We will also set up a Foundation project for the UI tests, here we will have “shared code”. Extension methods, base classes, configurations, etc. Let’s call it BasicCompany.Foundation.Common.UITests

The project types will be MsTest and yes it’s in .Net Core:

Why .Net Core then? This is something I will share with you in my next post. Let me give you a hint, it’s all about cross-platform ??

*Guess what, .Net Core projects and .Net Framework projects play quite well together(in a solution).

Time to add the SpecFlow NuGet packages, we need the following three:

We also need some Selenium NuGet packages, but why? Well, Selenium is the fundament. It allows us to connect and automate browsers.

Selenium is a set of different software tools each with a different approach

to supporting browser automation. These tools are highly flexible, allowing

many options for locating and manipulating elements within a browser, and one

of its key features is the support for automating multiple browser platforms

Here are the Selenium packages:

Note the last one, Selenium.WebDriver.ChromeDriver. We need to match the correct Chrome version with the one we have on our local machine.

This means we will run our UI tests using Chrome (for now ??)

In our newly created MSTest Project, we will add the following folders: Features and Steps

The Features folder will contain “SpecFlow Feature” files(Gherkin files with code behind). A Feature can contain one or many Scenarios, each Scenario has one or several Steps.

So what are Features, Scenarios, and Steps? Let me give you a quick explanation ??

Features

The feature element provides a header for the feature file. The feature element includes the name and a high-level description of the corresponding feature in your application. SpecFlow generates a unit test class for the feature element, with the class name derived from the name of the feature.

Scenarios

A feature file may contain multiple scenarios used to describe the feature’s acceptance tests. Scenarios have a name and can consist of multiple scenario steps. SpecFlow generates a unit test method for each scenario, with the method name derived from the name of the scenario.

Scenario Steps

Scenarios can contain multiple scenario steps. There are three types of steps that define either the preconditions, actions or verification steps that make up the acceptance test (these three types are also referred to as arrange, act and assert). The different types of steps begin with either the Given, When or Then keywords respectively (in English feature files), and subsequent steps of the same type can be linked using the And and But keywords. There may be other alternative keywords for specifying steps.

The Gherkin syntax allows any combination of these three types of steps, but a scenario usually has distinct blocks of Given, When and Then statements.

Scenario steps are defined using text and can have an additional table (called DataTable) or multi-line text (called DocString) arguments.

SpecFlow generates a call inside the unit test method for each scenario step. The call is performed by the SpecFlow runtime that will execute the step definition matching the scenario step. The matching is done at runtime, so the generated tests can be compiled and executed even if the binding is not yet implemented.

The scenario steps are the primary way to execute any custom code to automate the application.

Here is an example:

Feature: Footer
    As a website visitor 
    I want to see a footer containing various information

Scenario: Footer exists
    Given I am a website visitor
    And I navigate to the Home page
    When I scroll down to the bottom of the page
    Then I expect to see a footer

Scenario: Footer has a copyright
    Given I am a website visitor
    And I navigate to the Home page
    When I scroll down to the footer
    Then I expect to see the copyright text 'COPYRIGHT ? 2018-2020'

The Steps folder will contain the logic for each step.

Something like this:

[When(@"I scroll down to the bottom of the page")]
public void WhenIScrollDownToTheBottomOfThePage()
{
    var footerElement = _driver.FindElement(By.Id("footer"));
    Actions actions = new Actions(_driver);
    actions.MoveToElement(footerElement );
    actions.Perform();
}

Let’s create a SpecFlow Feature file and call it PromoCard.feature. We will do this by right-clicking the Features folders and select Add New Item. In the open dialogue, we will have SpecFlow as an option under “Visual C# Items”. Click on it and you will see several suggestions, select SpecFlow Feature File and name it PromoCard.feature.

You should now have an example Feature with a Scenario and some Steps

Next will be to add a Feature, a Scenario, and Steps to the PromoCard.feature file. Let’s start with something simple, we will create a UI test that checks if the home page has promo cards.

*The purple Steps, means that there are NO logic/code connected/bound to the steps.

Hey Prince, I said purple steps, not Purple Rain!

Behave, Prince! Sorry for the interruption, we will now continue.

Time to generate the logic/code for the Steps, we will do this by right-clicking the purple Steps and select “Generate Step Definitions”:

Here you choose what Step Definition Style to use:

Regular expressions in attributes:

[Given(@"I have entered (.*) into the calculator")]
public void GivenIHaveEnteredNumberIntoTheCalculator(int number)
{
  ...
}

Method name – underscores:

[Given]
public void Given_I_have_entered_NUMBER_into_the_calculator(int number)
{
  ...
}

Method name – Pascal-case:

[Given]
public void GivenIHaveEnteredNUMBERIntoTheCalculator(int number)
{
  ...
}

We will go for “Regular Expression in attributes” ??

Next is to select where the code should reside, we will put it in the newly created Steps folder:

Now your Steps should be white and shiny ?? This means that the Steps are now connected/bound.

And here is the generated code in PromoCardSteps.cs:

using TechTalk.SpecFlow;
 
namespace BasicCompany.Feature.BasicContent.UITests.Steps
{
    [Binding]
    public class PromoCardSteps
    {
        [Given(@"I am a website visitor")]
        public void GivenIAmAWebsiteVisitor()
        {
            ScenarioContext.Current.Pending();
        }
 
        [When(@"I navigate to the Home page")]
        public void WhenINavigateToTheHomePage()
        {
            ScenarioContext.Current.Pending();
        }
 
        [Then(@"I expect to see promo cards")]
        public void ThenIExpectToSeePromoCards()
        {
            ScenarioContext.Current.Pending();
        }
    }
}

Next is to change/update the “default generated” code but wait a minute we have a warning:

Ok, we need to fix that. Let’s do as they suggest. Get the ScenarioContext via Context Injection and while we are at it, we also need a connector(driver) for the browser to run the step. If you guys remember we will use Selenium for that. There will probably be more “Steps classes” in the future, so I was thinking… Instead of doing this in every “Steps class”, let’s put it in a base class.

Here is the base class(which will be inherited by the Steps classes) and it will reside in the BasicCompany.Foundation.Common.UITests project:

using OpenQA.Selenium;
using OpenQA.Selenium.Support.UI;
using System;
using TechTalk.SpecFlow;
 
namespace BasicCompany.Foundation.Common.UITests.Steps
{
 
  public class BaseSteps: TechTalk.SpecFlow.Steps
  {
    protected readonly ScenarioContext _scenarioContext;
    protected readonly FeatureContext _featureContext;
    protected static IWebDriver _driver;
    protected readonly IConfiguration _configuration;
 
 
    protected BaseSteps(ScenarioContext scenarioContext, FeatureContext featureContext)
    {
      _scenarioContext = scenarioContext ?? throw new ArgumentNullException(nameof(scenarioContext));
      _featureContext = featureContext ?? throw new ArgumentNullException(nameof(featureContext));
      _configuration = scenarioContext.ScenarioContainer.Resolve<IConfiguration>();
      _driver = _featureContext.SeleniumDriver(_configuration[Constants.EnvironmentVariableKeys.Browser]);
    }
 
  }
} 

We will inject ScenarioContext and FeatureContext(as they suggested).

Notice the IConfiguration, this is how we get application settings(or environment variables) into our UI test application. Let me explain how it works. In SpecFlow you use a specflow.json file for configuration. Here is the specflow.json file in our solution/project(The JSON file is shared between the UI test projects):

{
  "stepAssemblies": [
    { "assembly": "BasicCompany.Foundation.Common.UITests" }
  ],
  "BaseUrl": "https://localhost:44001/",
  "Browser": "Chrome"
}

The stepAssemblies allows us to share “Bindings” between projects. Hey, stop! What are Bindings?

The Gherkin feature files are closer to free-text than to code – they cannot be executed as they are. The automation that connects the specification to the application interface has to be developed first. The automation that connects the Gherkin specifications to source code is called a binding. The binding classes and methods can be defined in the SpecFlow project or in external binding assemblies.

The best place for the shared bindings will be in the BasicCompany.Foundation.Common.UITests project.

The cool thing is that we can also add/set application settings variables too – BaseUrl and Browser. But how do we set/register the application settings to IConfiguration? Well, thanks to the stepAssemblies we can now create a shared “Binding” class, we will call it ConfigurationBinding.cs.

using Microsoft.Extensions.Configuration;
using System.IO;
using TechTalk.SpecFlow;
 
namespace BasicCompany.Foundation.Common.UITests.Infrastructure
{
 
  [Binding]
  public  class ConfigurationBinding
  {
    private readonly ScenarioContext _scenarioContext;
 
    private static IConfiguration config;
 
    public ConfigurationBinding(ScenarioContext scenarioContext)
    {
      _scenarioContext = scenarioContext;
    }
 
 
    [BeforeScenario()]
    public void RegisterConfiguration()
    {
 
      if (config == null)
      {
 
        config = new ConfigurationBuilder()
          .SetBasePath(Directory.GetCurrentDirectory())
          .AddEnvironmentVariables()
          .AddJsonFile("specflow.json", optional: false, reloadOnChange: true)
          .Build();
      }
 
      _scenarioContext.ScenarioContainer.RegisterInstanceAs<IConfiguration>(config);
 
    }
 
  }
}

We inject the ScenarioContext and we also hook up/listen to the event binding – [BeforeScenario()]. Hey, stop! What is an event binding?

Hooks (event bindings) can be used to perform additional automation logic at specific times, such as any setup required prior to executing a scenario. In order to use hooks, you need to add the Binding attribute to your class

The supported hooks(event bindings) are:

[BeforeTestRun] [AfterTestRun]

Automation logic that has to run before/after the entire test run

Note: As most of the unit test runners do not provide a hook for executing logic once the tests have been executed, the [AfterTestRun] event is triggered by the test assembly unload event. The exact timing and thread of this execution may therefore differ for each test runner.

The method it is applied to must be static.

[BeforeFeature] [AfterFeature]

Automation logic that has to run before/after executing each feature

The method it is applied to must be static.

[BeforeScenario] or [Before] [AfterScenario] or [After]

Automation logic that has to run before/after executing each scenario or scenario outline example

Short attribute names are available from v1.8.

[BeforeScenarioBlock] [AfterScenarioBlock]

Automation logic that has to run before/after executing each scenario block (e.g. between the “givens” and the “whens”)

[BeforeStep] [AfterStep]

Automation logic that has to run before/after executing each scenario step

This means we will register the application settings variables(from specflow.json) to IConfiguration before the Scenario loads.

[BeforeScenario()]
public void RegisterConfiguration()
{
 
  if (config == null)
  {
 
    config = new ConfigurationBuilder()
      .SetBasePath(Directory.GetCurrentDirectory())
      .AddEnvironmentVariables()
      .AddJsonFile("specflow.json", optional: false, reloadOnChange: true)
      .Build();
  }
 
  _scenarioContext.ScenarioContainer.RegisterInstanceAs<IConfiguration>(config);
 
}

*The .AddEnvironmentVariables() will override the application settings variables from specflow.json. This will come in handy when working with environment variables which we will do in the next post ??

So IConfiguration is now injected into the ScenarioContext.ScenarioContainer.

Let’s go back to the BaseSteps.cs class and have a look at the constructor.

protected BaseSteps(ScenarioContext scenarioContext, FeatureContext featureContext)
{
  _scenarioContext = scenarioContext ?? throw new ArgumentNullException(nameof(scenarioContext));
  _featureContext = featureContext ?? throw new ArgumentNullException(nameof(featureContext));
  _configuration = scenarioContext.ScenarioContainer.Resolve<IConfiguration>();
  _driver = _featureContext.SeleniumDriver(_configuration[Constants.EnvironmentVariableKeys.Browser]);

}

The IWebDriver(_driver) will be set from our extension method – SeleniumDriver.

The IWebDriver interface is the main interface to use for testing, which represents an idealized web browser

Here is the extension method – SeleniumDriver:

using BasicCompany.Foundation.Common.UITests.Infrastructure;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Edge;
using OpenQA.Selenium.Firefox;
using OpenQA.Selenium.IE;
using OpenQA.Selenium.Safari;
using System;
using TechTalk.SpecFlow;
 
namespace BasicCompany.Foundation.Common.UITests.Extensions
{
  public static class SpecFlowExtensions
  {
    public static IWebDriver SeleniumDriver(this FeatureContext featureContext, string browser)
    {
      if (featureContext == null)
        throw new ArgumentNullException(nameof(featureContext));
 
      if (featureContext.ContainsKey(typeof(IWebDriver).FullName ?? throw new InvalidOperationException()))
        return featureContext.Get<IWebDriver>();
 
      System.Enum.TryParse(browser, true, out BrowserTypes browserType);
 
      return CreateSeleniumDriver(featureContext, browserType);
    }
 
    private static IWebDriver CreateSeleniumDriver(this SpecFlowContext specFlowContext, BrowserTypes browser)
    {
      switch (browser)
      {
 
        case BrowserTypes.Edge:
          EdgeOptions options = new EdgeOptions
          {
            PageLoadStrategy = PageLoadStrategy.Normal
          };
 
          specFlowContext.Set((IWebDriver)new EdgeDriver(options));
          break;
 
        case BrowserTypes.Firefox:
          specFlowContext.Set((IWebDriver)new FirefoxDriver());
          break;
 
        case BrowserTypes.InternetExplorer:
          specFlowContext.Set((IWebDriver)new InternetExplorerDriver());
          break;
 
        case BrowserTypes.Safari:
          SafariDriverService safariDriverService = SafariDriverService.CreateDefaultService();
          SafariOptions safariOptions = new SafariOptions
          {
            PageLoadStrategy = PageLoadStrategy.Normal
          };
          specFlowContext.Set((IWebDriver)new SafariDriver(safariDriverService));
          break;
 
 
        case BrowserTypes.None:
        case BrowserTypes.Chrome:
        default:
        {
          ChromeDriverService chromeDriverService = ChromeDriverService.CreateDefaultService();
          chromeDriverService.SuppressInitialDiagnosticInformation = true;
          specFlowContext.Set((IWebDriver)new ChromeDriver(chromeDriverService));
        }
 
          break;
      }
 
      return specFlowContext.Get<IWebDriver>();
    }
  }
}

Depending on what “browser type”, it will set the expected “browser driver”. Chrome for ChromeDriver, Edge for EdgeDriver and so on.

Finally, we can now continue with PromoCardSteps.cs:

using BasicCompany.Foundation.Common.UITests;
using BasicCompany.Foundation.Common.UITests.Steps;
using FluentAssertions;
using OpenQA.Selenium;
using TechTalk.SpecFlow;
 
namespace BasicCompany.Feature.BasicContent.UITests.Steps
{
  [Binding]
  public class PromoCardSteps : BaseSteps
  {
 
    protected PromoCardSteps(ScenarioContext scenarioContext, FeatureContext featureContext) : base(scenarioContext, featureContext)
    {
    }
 
 
    [Given(@"I am a website visitor")]
    public void GivenIAmAWebsiteVisitor()
    {
      _driver.Url = _configuration[Constants.EnvironmentVariableKeys.BaseUrl];
      _driver.Manage().Window.Maximize();
    }
 
 
    [When(@"I navigate to the Home page")]
    public void WhenINavigateToTheHomePage()
    {
      _driver.Navigate().GoToUrl($"{_configuration[Constants.EnvironmentVariableKeys.BaseUrl]}/en");
      var element = _driver.FindElement(By.CssSelector("a.navbar-item.is-tab.is-active"));
      element.Text.Trim().Should().Be("Home");
    }
 
    [Then(@"I expect to see promo cards")]
    public void ThenIExpectToSeePromoCards()
    {
      var promoCards = _driver.FindElements(By.CssSelector("div.column.promo-column.graphQlPromoCard"));
      promoCards.Count.Should().Be(5);
    }
 
 
  }
}

Let me give you guys a quick explanation of the code:

In GivenIAmAWebsiteVisitor we decide what URL to use, this is set in specflow.json – BaseUrl. Then in WhenINavigateToTheHomePage we navigate to the homepage. To verify it is the homepage we check in primary navigation that the Home tab is active:

var element = _driver.FindElement(By.CssSelector("a.navbar-item.is-tab.is-active"));
element.Text.Trim().Should().Be("Home");

In ThenIExpectToSeePromoCards we will check for the promo cards and it should be five promo cards.

var promoCards = _driver.FindElements(By.CssSelector("div.column.promo-column.graphQlPromoCard"));

promoCards.Count.Should().Be(5); 

The moment of truth! Open the Test Explorer and locate the UI test “PromoCardsVisible”, right-click and select run:

And success! The Chrome browser is now running the UI test ??

That was one big blog-post, sorry for that guys.

As always you will find code in my fork from Helix Examples – helix-basic-tds-with-docker-vs2019-project-type

Stay tuned for a second blog-post about Automated UI tests!

That’s all for now folks ??