EA and RSAT review by Axtension

It is common knowledge that after every adjustment in D365FO countless processes must be checked and played through. Questions arise like:

·        Can the individual processes be executed as usual by the users?

·        Is the expected value displayed / calculated?

Nobody likes to release a release without knowing if the system will work afterwards.

This topic has been up-to-date for years and the fight against the windmills takes place in different places:

·        Quality controls at the manufacturer (Microsoft)

·        Tests by the partner before handing over an adjustment to the customer

·        The customer tests the processes with the KeyUsers and removes them

In addition, the complexity of today's systems makes predicting what kind of side effects an update will entail becoming increasingly difficult and costly.

Wouldn’t it be nice if a "fairy godmother" checked the most important processes every night?

Before such a review, however, the first step must be to define what the most important process is: each company has different core processes that must not be affected by an adjustment. In the first step, it should therefore be precisely defined what should be tested and how these tests can be carried out reproducibly. Only when this has been clarified can a technical implementation be started.

In this entry I discuss the possibilities of automated testing in D365FO and show you the possibilities of the following 2 variants, and compare them afterwards. The advantages and disadvantages of the 2 variants are explained from my own personal point of view.

·        RSAT from Microsoft

·        Executive Automats

RSAT (Regression Suite Automation Tool)

The official test tool from Microsoft. The idea is simple:

·        The user records a process with the task recorder, which can then be automatically played back.

·        The task created in this way is subsequently stored in the LCS.

·        DevOps (code management of D365) synchronizes with the LCS and different tasks can be merged into one process.

·        On a server / computer, a program must be installed, which carries out the individual tests in the D365FO system.

·        The test program runs the individual processes in D365FO and stores the results in DevOps. DevOps has the entire history. From faulty tests, new work packages for development can be created directly.

·        If a test is successful, an adaptation to another system could also be automatically rolled out. The possibilities and actions are huge.

Blog entry about automation in DevOps (https://blog.axperience.ch/2019/07/09/azure-devops-tasks-for-microsoft-dynamics-365-part-1/)

Further links: https://blog.armaninollp.com/dynamics/2018/11/27/test-automation-using-the-regression-suite-automation-tool/

Executive Automats / https://executiveautomats.com

The mission of the Polish company XPLUS for its tool Executive Automats is clear: the customer should be able to record and play their processes as simply as possible.

·        For the customer, no installation is necessary and the setup is completed in 15 minutes. SaaS at it's best.

·        The idea is simple, too. The D365FO instance is accessed through a proxy from Executive Automats and can thus register and record every click.

·        The individual steps can be subsequently adapted in a simple interface and enriched with logic.

As an example, I recorded the process of creating a new customer with Executive Automats.

Creating a single customer in most test programs is not a challenge. It is more difficult if the customer numbers / information should come from an Excel spreadsheet or a number range. If, depending on the customer group, another process is to be carried out, it will be complicated.

But not with Executive Automats. The administration interface is organized and offers many possibilities. Individual processes can be batched and executed in parallel to simulate load tests.

All results are stored in one place and presented clearly. So I became a fan of this tool. I recommend the following videos for a quick overview:

https://executiveautomats.com/introduction-to-executive-automats/

For AX2012 a solution is also offered by Executive Automats, which I have not looked at more closely.

RSAT by Microsoft vs. Executive Automats

Price:

RSTA can be operated +/- with the existing licenses. Executive Automats incur additional charges.

However, the installation, operation, creation of test cases and evaluation are much more efficient with Executive Automats. Thus, a balance has to be made between internal and external costs.

Set up:

For RSAT, the test tool must be installed on a server and granted access to the D365FO / DevOps / LCS. With each update it must be checked, if everything still works. In my test I had the problem that an automatic installation of the latest version of Chrome (browser) new drivers were needed for the tests. These are things that you really do not want to worry about.

With Executive Automats I have a web portal and after 15 minutes I was able to do my first tests in my D365FO.

Customize and create test cases:

RSTA works with the task logs from D365FO. For the individual versions, an Excel must be created in which the individual test is defined. Inserting new steps or even loops is very difficult.

Executive Automats has its own editor here, which can be operated very intuitively.

Conclusion:

With RSAT, Microsoft is certainly heading in the right direction to provide the customer with a test tool, but it is still in its infancy. Executive Automats offers in my view already a mature solution.

I would be delighted if our customers increasingly started paying more attention to the subject of "Testing" and thus came to more stable processes in the D365FO.

Original article in German on Axperience : https://blog.axperience.ch/2019/11/09/automatisiertes-testen-in-d365fo/