Lessons Learned during Automation Testing of “Share Point Online” Application using Selenium

Recently I got an oppotunity to perform functional automation testing of a web app based on Share Point Online (SPO 2015). I used Selenium Webdriver3 with Java (TestNG, Page Object Model, Extent Report v2, Maven) and I learned some interesting lessons that I think will be useful to the testing community. Share Point on premise and Share Point Online have many distinct differences and I am going to talk about SPO only (w.r.t. scope of my AUT).

Let me first share the crux of web application under test. It was a document management system that was supposed to be used by above 20,000 users, distributed geographically in 13 countries. While the Basic feature set was same for all the users, but each country team used their “own version” of the product containing the distinct features according to their own requirements. The application’s version was differentiated by the access URL, while all the users got authenticated via Azure Active Directory (AD) as Office365 is bundled with SPO. In order to view the documents, there were LIBRARIES, VIEWS, WORKFLOWS (these are some SPO jargons but self-explanatory) and there were 5 user role categories with different rights. Roles and rights were managed by AD user groups.

My goal was to automate the ‘Basic Feature Set’ so that our manual regression effort could be minimized. Here are the challenges that I faced, issues which I resolved and lessons that I learned.

1.      Authentication:

In simple web applications, we login with valid credentials and then use the application. Same goes when we automate it with Selenium. But, with SPO life is not that simple. Login page of Web App, right after entering the user name and clicking on password input field, redirected to Microsoft AD authentication page. On that page password is entered and after authentication from AD user was redirected to application. This was still a bit simple, yet another level of authentication was added on client’s demand which is “PingID”. It’s a desktop utility which generates a 6 digit code to authenticate the user. AD password page redirected to PingID authentication page where the user had to enter those 6 digits generated by his desktop app recently (and that expires in 1 minutes I guess and user(s) need to get a new 6 digit code).

So, my automation logic for login process was like this:

Open the application login page --> Enter user name in the relevant input field --> Click password input field --> wait for AD authentication page to load --> enter the password and click Login --> wait for PingID page to load --> select your device from list of PingID devices and click login again --> wait for PingID code verification page to load --> get latest 6 digit code From PingID desktop app --> enter that code on the verification page and click login again --> successful authentication message is displayed and user is redirected to homepage of the SPO application in the end.

The very first and main challenge for me was PingID desktop utility part. None of my peers had worked on such scenario in their automation experience so far and so I had to explore it and resolve it on my own. I found ‘AutoIT’ at first and tried to build script for my required actions, but failed. I watched online demos, tutorials, videos etc. but could not find the solution of my problem using AutoIT. The main issue I encountered was the inability of ‘AutoIT’ to recognize the buttons on ‘PingID’ program’s user interface. After spending a day and a half and trying my best, I decided to find some alternate solution. Then I came across ‘Sikuli’. I watched couple of videos about using Sikuli and got the idea that how I can devise the solution of my problem with it. I added dependencies in my project’s POM.xml and took the snaps of PingID icon, UI, and command buttons etc. and implemented the logic. From the get go, in the first try, my script was successful and I literally jumped with joy. I tried the script again and again and again and identified that happy path was covered but some alternates were not covered by the logic that I coded. So, I covered the alternate scenarios too and finally this challenge was completed.

Here are the lessons learned: 1- ‘AutoIT’ and ‘Sikuli’ both can be used to handle non-web tasks during automation with selenium. 2- If your approach with one of these tools is not succeeding then instead of wasting more time, try the alternate (in my case Sikuli came to rescue, in your case AutoIT may do the job for you). 3- Sikuli is OCR (optical character recognition) tool and in order to get your job done by it smoothly you must provide the uniquely identifiable snapshots to it. 4- While matching the expected and actual screens, Sikuli is quite rigid (I think ‘Pixel Sensitive’ is the technical term for this behavior) so don’t assume that your snapshot taken on one machine (with specific screen resolution) will work fine on some other machine (with different resolution). 5- The Sikuli tutorials available on internet lack information about both these issues mentioned in point 3 and 4, but I am thankful to guys who at least shared the basics to help first time users of these tools (like I was :) ).

2.      Web Element Locators:

This sounds so simple, right? Selenium webdriver, by default, provides so many options to locate web elements like ‘id’, ‘name’, ‘css’, ‘xpath’, ‘linktext’ etc. Most of the trainings/videos/blogs tell that prefer to locate a web element using ‘id’ then ‘name’ then ‘css’ and then ‘xpath’ (if the former is not available than use the latter one) and ‘id’ locator is assumed to be the best in general. This is exactly what I used to believe too but, the application I am talking about really changed my thoughts.

In SPO, ids were available (wherever possible) for all the web elements on every web page. But the challenging part was, same element had different id for different user of same version (of the web application e.g. for user ‘AdminRole’ of country XZY the id of a button is ‘abc123asdf_btn_abc123’; the id of same button for user ‘nonAdminRole’ of same country XYZ was ‘abcd121sdf_btn_1a2b3c’). Shocked? Trust me it is true, moreover, same element had different id for different user of different version of the application. I had one option to design custom locator using “starts-with”, “ends-with” or “contains” approach based on id. But, after thoroughly viewing the behavior of ids for several elements with different users on different versions (of the web application), I figured out that this approach was risky as the changing pattern of ids was varying too. To solve this problem I wrote custom CSS locator as it was same for the elements regardless of user or application’s version (let me remind you that by application version, here I mean the web application access URL differentiated that which feature set is supposed to be available to the logged in user according to his/her country). For some web elements, I still had to use “starts-with” and “ends-with”, “contains” and even “nth-child” options while making CSS locators, however, it was not risky but definite. Another interesting behavior that I observed and tackled was, same web-element (e.g. a table or a menu ribbon button or menu tab) had different ‘id’ or ‘css’ in different mode (i.e. edit mode, view only mode). One more tricky part was the thing that a web elements which apparently looked like ‘DropDownList’ was not actually the dropdown; it was a div which contained all the options in an overlay table behind the scene while on UI its behavior and appearance was like an ordinary dropdownlist for which ‘select’ locator is used simply. This element existed in each row of every table in all the library view pages and so it was required some serious attention. The same thing I observed in another application on which one of my colleagues is working and he needed some assistance from me.

Here are the lessons learned: 1- Locating a web element by ‘id’ is surely best approach, but in case of SPO based application (with different user groups, roles and rights) it will not work. So design the custom CSS for web elements identification in order to ensure your tests execute smoothly. 2- Simple CSS or Xpath locators again may not work in SPO based web apps, give proper attention to this part at first time or you will end up tweaking your tests and web elements again and again. 3- Pay proper attention to actual structure of web elements in the source code, its appearance might be a deception. 4- In different modes, the locator of same web element will differ and you must handle it or you will be surprised during the test execution. 5- Several properties of web element’s CSS remain same in different modes (and for different users) so you should be wise enough to utilize this in your favor while scripting your tests.

3.      Access Token:

In order to use the SPO application, successfully authenticated user is given an ‘Authorization Key’ and ‘SharePoint Access’ token. This key and token pair was specially required for testing the APIs (I will soon share all the lessons learned w.r.t. APIs testing in a separate post). So, another big challenge was to get these two (key and token). My first approach was to get this token from Azure Active Directory via my automation script, and after a lot of search on internet I found a way that used some custom java library created by a person. This was not acceptable and I knew that in the code/architecture review of my project I won’t be able to defend how and why I trusted that library and used it in my code. Then I was really worried as this hurdle was sort of a blocker for me and was wasting my time. Suddenly, the idea came into my mind that why not I get these two keys from page source of the landing page after the successful login. I checked the page source and both keys were present there, which I extracted and stored in global variables in my base class. This was acceptable by the architecture reviewer too as my purpose was to test the functionality aspects of the application, not to verify whether Azure Active Directory is providing the authorization key and SharePoint access token accurately or not. In another application (on which I am working these days), the authorization key is stored in the ‘SessionStorage’ and extracting it from there is just like peanuts.

Here are the lessons learned: 1- ‘PageSource’ of webpage is really useful object that can be utilized during the test automation. 2- When you are writing automated tests for an application which is a blend of multiple tools/technologies, focus primarily on the functionality coverage rather than trying to verify/simulate the interoperability areas of those tools/technologies. 3- Automated test usually simulates the flow of already existing manual black-box test, if your manual black-box test ignores the technical implementation details of a flow, then during automation scripting you should also ignore that. 4- I will say this again, if one approach of doing a task fails, don’t give up but try to find some other.

Landing page of the application was a blend of C#, ASP.Net, AngularJS2 and several REST APIs were used in the application which I automated using rest-assured libraries. Soon, I will also share a detailed post about the lessons learned during that activity.

Regards,

Enjoy Knowledge Sharing.