Automation with commercial or open source - summary

This is the sixth part in my series on test automation with Test Complete and open source. It is time for a wrap-up.

You can find the first episode here.

Recommendation

I recommend using widely known open source tools. They require some time to make them work for you. But so are commercial tools. So why pay for them, if delivering a complete solution requires the same amount of time? The only reason for purchasing a commercial tool is when it supports some GUI controls, which no other tool does.

Before I finish this series, let me talk about few more factors worth considering.

IT job market

Whenever one talks about an investment, one talks about a payoff. If you are about to invest some time (which is the most precious, unrecoverable resource for IT people) in learning something, you should ask yourself a question – what is the use of this knowledge I am about to acquire. If you develop something with pure python, using well known python libraries, you learn universal stuff. Your knowledge is likely to fit in other python based projects. If you invest your time in learning specialized commercial tool, like TestComplete, you acquire a specialized knowledge, which will be useful mainly for companies, which use TestComplete. Which is better? Well, it depends. IT job market rewards for expertise in commonly used tools and languages, especially the hot ones. And python is both – commonly used and hot. However, the market also rewards for expertise with some specialized niche technologies. In other words – if you automate with pure python, you are more likely to find a nice job, since python is respected, widely used and there is a demand for it. However, since it so popular, the provision is also higher. If you specialize in TestComplete and you happen to find a customer, which has invested heavily in this tool – you might be on a position to expect higher salary, since the provision of TestComplete experts is much lower. It might work this way, it does not have to work this way. In IT job market luck is also an important factor. Is TestComplete this kind of a specialized tool, giving huge premium for knowing it – I don’t know. And I don’t believe there is an oracle, which would give you an ultimate answer to this question. You have to do your homework – google, talk to people and then decide.

You are going to see it differently, if you are the owner/sponsor of a test automation effort. If you buy an expensive commercial tool, you will suffer from a low provision of people knowing it. You might be tempted by a promise that with this commercial tool no expertise is needed. However, I have never seen such a promise kept.

Support

Commercial tools' vendors promise support. However, my experience is that such a support is capable of helping you only in simple basic matters. That is my recurring experience with such a support. And it is perfectly logical, it has to be that way. Why? Because if you are a vendor, you do not put your experts in a call center and have them answering mostly basic questions. You use them to develop new features, you use them as sales support, you use them as senior consultants in projects. The ultimate source of real help for real problems is google. However, google does not produce knowledge. It searches it. The more people use the tool, the more knowledge is accumulated for google to search in. If the population of users is small, so is the searchable knowledge base on the internet.

Success stories

One final note, a kind of a hint and a warning. Commercial tools vendors are really good in speaking the language managers and decision makers understand. They have several diagrams and tables showing, that despite a breathtaking price of one license, you are saving a lot of money buying it. The more you buy the more you save. Moreover, they have success stories, in which their customers testify that indeed, using the software was a huge success and has brought significant savings. When reading such stories, remember this – when you are a decision maker and decide to spend big bucks on a commercial tool, it will be in your best interest to prove your supervisors (be it CIO, CEO, the board) that you have made the right decision. Should the tool fail to deliver the efficiency it has promised – you are to blame. You have paid for the wrong tool. Therefore, if a commercial tool vendor will ask you to testify, that purchasing their tool was a very good decision – denying it would mean admitting that you have made an expensive mistake. This is not to say, that all success stories are fakes. However, I have seen the phenomenon described above too many times to simply ignore it.