Saturday, June 8, 2013

Which automation framework should I choose?

Lately, I've been hearing a lot of folks ask questions like "which one should I choose QTP or Selenium?" or "which one has a better future, QTP or Selenium?". In this post I will attempt to clear up the misconceptions regarding the comparisons of QTP to Selenium.

By far the most common argument I hear agasint QTP is that if you want to use anything other than just record and playback, you must learn VBScript (this is an easy enough task to do, however, especially if you are a novice programmer). As well you must learn some type of language in order to realize and / or get the most juice out of any framework you choose for your automation projects. Which leads me to one of the main advantages of QTP, it is a framework! What that means is that you not only get to drive your Application Under Test (AUT), you also get reporting of test results, you get support for data driven tests, you get an object repository. Basically is a framework youc an use right out of the box.

Selenium on the other hand, is just an API, a good one at that! The reason it is so good is because all of the development efforts go into having a robust API for automation architects and programmers to use and incorporate into their own frameworks, rather than having to support all of the features a ready-to-use framework provides. And there lies the difference! With Selenium you don't get reporting, you don't get data driven capabilities, you don't get an object repository so it is not a framework. However, it IS flexible enough that you can incorporate into your own framework that already has all these features. The caveat is that you must design and build these features yourself. A lot of folks prefer to have this type of control. I am one of them :)

A more realistic comparison would be SmartBear's TestComplete -> Froglogic Squish -> HP QTP; all three of these are frameworks that give you, right out of the box, reporting, object repositories, data driven capabilities, keyword (another type of data driven) capabilities, etc. Again, in order to realize the full potential of any of the just mentioned frameworks (not including Selenium which is not a framework) you have to learn some language (e.g. Perl, PHP, Python, Ruby, VBScript, TCL, Java).

That being said, lets not forget about many general purpose frameworks that already exist and that are ready for you, the automation architect, to "just add water" (or Selenium if you wish ;), some examples:

STAF: http://staf.sourceforge.net/
Robot Framework: http://robotframework.org/

So there you have it, I hope this helps some folks clear up (at least in their minds) what to choose or not to choose when designing a framework to be used in any type of software verification. Contact me if you are interested in learning more or wish to secure my services to design your Automation framework.

2 comments:

  1. Great post. Also wanted to mention a few things:

    RobotFramework already offers third party Selenium libraries for web testing, so you get Selenium + test framework. But if you don't like the existing Selenium library interfaces, you are free to modify it or build your own Selenium library for RobotFramework. There currently is Selenium library in Python for WebDriver and Selenium RC, and one in Java for WebDriver.

    For those already experienced with QTP & VBScript moving to Selenium, you can take your VBScript experience with you, don't necessarily have to learn Java or another language (though still good to). While not popular compared to other languages, there is VBScript bindings to WebDriver (http://htejera.users.sourceforge.net/vbswebdriver/) and Selenium RC (http://blueducksda.sourceforge.net/). Using VBScript with Selenium shortens the migration path for those already versed in QTP/VBScript than to have to learn new language. You can then later move to an official Selenium language like Java when ready.

    ReplyDelete
  2. Good info on the VBScript bindings for WebDriver, I was not aware of this. thanks for sharing!

    ReplyDelete

Creative Commons License
VGP-Miami Web and Mobile Automation Blog by Alfred Vega is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.