Saturday, June 8, 2013

Which automation framework should I choose?

Lately, I've been hearing a lot of folks ask questions like "which one should I choose QTP or Selenium?" or "which one has a better future, QTP or Selenium?". In this post I will attempt to clear up the misconceptions regarding the comparisons of QTP to Selenium.

By far the most common argument I hear agasint QTP is that if you want to use anything other than just record and playback, you must learn VBScript (this is an easy enough task to do, however, especially if you are a novice programmer). As well you must learn some type of language in order to realize and / or get the most juice out of any framework you choose for your automation projects. Which leads me to one of the main advantages of QTP, it is a framework! What that means is that you not only get to drive your Application Under Test (AUT), you also get reporting of test results, you get support for data driven tests, you get an object repository. Basically is a framework youc an use right out of the box.

Selenium on the other hand, is just an API, a good one at that! The reason it is so good is because all of the development efforts go into having a robust API for automation architects and programmers to use and incorporate into their own frameworks, rather than having to support all of the features a ready-to-use framework provides. And there lies the difference! With Selenium you don't get reporting, you don't get data driven capabilities, you don't get an object repository so it is not a framework. However, it IS flexible enough that you can incorporate into your own framework that already has all these features. The caveat is that you must design and build these features yourself. A lot of folks prefer to have this type of control. I am one of them :)

A more realistic comparison would be SmartBear's TestComplete -> Froglogic Squish -> HP QTP; all three of these are frameworks that give you, right out of the box, reporting, object repositories, data driven capabilities, keyword (another type of data driven) capabilities, etc. Again, in order to realize the full potential of any of the just mentioned frameworks (not including Selenium which is not a framework) you have to learn some language (e.g. Perl, PHP, Python, Ruby, VBScript, TCL, Java).

That being said, lets not forget about many general purpose frameworks that already exist and that are ready for you, the automation architect, to "just add water" (or Selenium if you wish ;), some examples:

STAF: http://staf.sourceforge.net/
Robot Framework: http://robotframework.org/

So there you have it, I hope this helps some folks clear up (at least in their minds) what to choose or not to choose when designing a framework to be used in any type of software verification. Contact me if you are interested in learning more or wish to secure my services to design your Automation framework.
Creative Commons License
VGP-Miami Web and Mobile Automation Blog by Alfred Vega is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.