Thursday, July 21, 2011

Automation System, Explained - Part 2 "Components"

This is Part 2 of, well, Part 1 :). I will outline the system components that make up this functional and web test automation tool implemented using all open source software (Linux, Selenium RC, MySQL, Perl). The challenges our client faced and we aimed to address were mainly:
  • multiple dynamic sites, structure changed often
Which in turn led to:
  • too much maintenance time spent on test scripts.

Since there were so many sites we decided to split the testing by site, although there would be a group of commonly used functions housed in a Perl package for reference by any of the classes or scripts. This conclusion was drawn after numerous discussions where the following was identified:
  • Some areas / functions of the sites where similar
  • Most sites implemented unique html layouts
  • Conditional access functional testing was required
  • Gender specific functional testing was required
  • Back-end validation was required
  • A central point of reference for automation system status was necessary
For this we created a structure that required a class per web site, or more. I say "or more" because, although this client has not had the need to yet, it is possible that your class can get so extensive that it becomes harder to manage and update. At this point, it may make sense to split the class into smaller, more manageable modules. I should mention that, although we implemented this using Perl, you can apply the same principles to any language and come up with a similar system.

So now have a class called WebSiteOne and another called WebSiteTwo. Both of these correspond to property one website and property two website. This website objects include methods that encapsulate users actions. Lets use a login page example. Both websiteone and websitetwo have a login page that they chose to implement differently, but even if they hadn't, I still would have two login methods (one per property). Just because in time, most likely these two websites will end up with slightly different  login pages. Login into the website is a function that we can test with Selenium. In order to achieve this all we have to do, and it can be this simple, is use IDE to record the function. Then replace the hard coded field data with your own test data from a database or wherever, add any other logic that is required to test the function such as loops, conditions, etc., and you're done.

When you think about it; all the method is doing is performing a function that a human normally would on the site. These functions vary depending on the context of the web site. Here are some examples:
  • Signup to a newsletter
  • Upgrade your account
  • Check your transactions
  • Log off
  • Log on
As a test developer you may also have access to methods that things like:
  • Verify images are displayed on the screen
  • Verify certain text and links are displayed to the user
  • Capture a screen shot
  • Writes specific test debug info to a specific location, etc
This modular approach made sense for this client mainly because there were so many properties to develop and later maintain tests for.

The next component is made up of test scripts that make use of the exposed methods of WebSiteOne and WebSiteTwo. These scripts are nothing more than a list of functions that have been grouped together based on what made sense. For example using the same login example; you may have a test script that tests the login function of the web site under positive conditions (correct user name, password, etc) and another test script that tests the login function of the web site under negative conditions (wrong user name, password, etc). The idea is that for every function that  a human can do on a site, there is a corresponding method that you can call in your scripts to perform the action.

Most functional tests need some type of data input so that they can perform their functions. We chose a data driven test approach because of the massive amount of test data that would be consumed by these tests. Here are some of the factors that went into our decision to implement a database into our design:
  • As mentioned, lots of permutations, meant lots of data
  • Ease of maintenance
  • Faster execution times (vs using flat text files or csv's)
  • Test data integrity
The next logical step was to implement a dashboard that will present the test results to all interested parties via the web and without the need to install any other plugins. For this, Perl came to the rescue again and we implemented a web based dashboard that is constantly mining the database's test results tables and making that information available as soon as the tests are done. The dashboard is extendable and can be customized to add any data set we choose.

As a matter of fact, the entire system is extendable in that if you add functional areas, you add more scripts to your test arsenal. If a new property (web site) is added then we add a new class. Lets say for example you want to be able to track how many times a specific credit card biller is used in your signup form. All you would have to do add this functionality to the test framework is add a table to track the biller, a method to the class that will do all the work, and then call this method from your scripts and display this info on the dashboard.

So far, this system is running without any major hiccups and have even survived a partial migration from Selenium RC to WebDriver that was done by the customers staff with almost no support from us.

We welcome any feedback, questions or comments and look forward to interacting with you.

No comments:

Post a Comment

Creative Commons License
VGP-Miami Web and Mobile Automation Blog by Alfred Vega is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.