Test automation with pure python

This is the fifth part in my series on test automation with Test Complete and open source.

You can find the first episode here.

In my recent post I have written about problems of TestComplete. Now, it is time to write about automating OpenApp, which was done with python supported by pywinauto open source library.

These are the features and advantages of OpenApp automation:

1. It is free, so everybody can install the framework on his/her machine and run the automation
2. It has a classical, line oriented logging, which can be tailed and grepped.
3. It has a tracing of progress, so if you are restarting a particular suite, it runs only tests, which have not passed before
4. It generates HTML documents with contents of the important tables
5. It compares database rows with expectations and prints a color-coded, clear illustration of what is as expected and what is not (in HTML document from the previous point)
6. It creates screenshots after clicks and keyboard events
7. It creates reports, which can be copied and pasted to a final test report document

I am proud of my tool. However, let’s be honest – none of features given above required sophisticated solution. Log files, html generation, making screenshots – if you have some development experience these are all no-brainers. Should I need something else, I will implement it, with all python libraries for my disposal.

As I have written before, TestComplete had some problems. Did pure python solution have problems? Of course it did! Some features did not work as expected, as promised by a documentation, so I was forced to use some workarounds. It is important that you understand what point I am proving here. And what point I am not trying to prove. I am not trying to prove that TestComplete is bad and open source is perfect. It would be a lie. I have written it couple of times and I will do it again – TestComplete is a decent piece of software, which provides support in test automation. What I am trying to prove is this – whichever route you take (TestComplete or open source) you will have to invest time in understanding what is given to you out of the box, how to adjust stuff to your needs and how to use the tool in the right way.

One may think that it is a little bit unfair, that I have devoted the whole post to talk about TestComplete problems. In the same time for open source I have just said “there were problems”. There is a couple of reasons for it. First of all, going too deep into some technical aspects of an open source framework would result in revealing confidential information on the solution implemented for my client. Secondly, when talking about TestComplete problems, I strived to focus on problems, which one does not expect to find in commercial “we do all the dirty work for you” platform. If I had some problems with pure python solution, they were mostly general python problems so the community has already worked out some reasonable workarounds. Or it was a problem with pywinauto gui automation library, which, by the way, also has some community around it. The biggest problem I had, was an error during using some style of gui controls addressing in pywinauto. It worked in most of places, but there was one place, where it did not. The solution was to use a different style of control addressing. In general, controls addressing is a little bit tricky in pywinauto, and sometimes requires some debugging. It is a consequence of smartness in locating the controls by the pywinauto library. In most cases this smartness does the job. But there are situations, when this smartness has to be supported by some manual hints.

Apart, from the aforementionned problems, I do not consider any challenge I have faced as something really susprising, unexpected. If during the development and recurring test cycles I realized that something can be improved, I just did so. I was not limited by some closed source features deeply embeded in the framework. I was also not limited to libraries installed with the python distribution used by TestComplete. Plethora of packages was there, waiting to be used, "pip" away from me.