Wednesday, October 22, 2014

#stop29119. Campaign? Or a classic example of the "We Have to Do Something" fallacy?

 There is no denying that there has been a lot of activity regarding ISO29119 since August of this year and it doesn't seem like its going to be dying down anytime soon. The standard has certainly created a rift in the training and consulting space that has aggravated a long time rivalry between two schools of thought.

Now, in order to run a successful and, more importantly, profitable business we need to be able to compete and use any tool at our disposal to reach our vision, this includes public debates and functions. One of the things we must keep in mind when talking about standards and certifications is that its a business, most folks know this already but if you don't now you do. And its a business whether you issue a certificate at the end of the training or not, by the way. Its a business with a bottom line just like Sears. Its a business that needs to fight for its existence or fall prey to its competition.

Aside from a response from Stuard Reid the WG convenor, the ISO camp has been fairly quiet throughout this debacle. This hasn't been the case for the stop campaign side, however. From them we see statements used like: "where is your skin in the game" or "if the standard is approved all testers will be forced to succumb to and abide by it". You also read some folks say "you'll be forced to produce tons of wasteful documentation" or "before your every move you'll need to get a sign off" when talking or interacting (via social media) with folks that either don't know of the petitions' existence or have decided to abstain from signing it for a variety of different reasons.

In following this debate on Twitter, LinkedIn and the web (via individual's blogs). I have noticed a pattern in the rhetoric used by the stop campaign folks which I believe its an almost perfect implementation of the "Scare Tactic"[*1] argument which inevitably leads to a "We Have to Do Something"[*2] fallacy. In other words the standard is going to be so bad that we should all unite and do something, no matter what that something is. Even if the something is just to stop the darn thing. Sounds counter productive doesn't it? Why not offer a real solution rather than a call to arms? This is the part that has me, and a lot of others, baffled a bit.

Finally, and for the record once again, I want to be clear that I am not saying that the arguments raised by James Christie based on his own experiences and knowledge is in any way shape or form invalid. I am saying, however, that the ensuing madness does appear to fall within the model of a "Scare Tactic" and "We Have to Do Something" fallacies.

I'm not advocating for just silently accepting the standard either, I'm advocating for doing your own research and coming to your own conclusions based on your own independent investigation.

Even if one believes the allegations expressed by the supporters of the stop campaign it makes you wonder why the scare tactic? As humans we are thinking creatures. We like to be presented with information and be able to analyze that information and come up with our own conclusions. But when things are framed in a way that it is meant to scare or force people into signing, it makes one wonder if there is anything more to this debate. Anything more than business profit, business market share, and of course the all important human mind share.

What say you?

[*1] Scare Tactic (Also Paranoia): A variety of Playing on Emotions, a raw appeal to fear. A corrupted argument from Pathos.(E.g., "If you don't do what I say we're all gonna die! In this moment of crisis you can't afford the luxury of thinking or trying to second-guess my decisions. Our very lives are in peril!  We need united action, now!")

[*2] We Have to Do Something: The dangerous contemporary fallacy that in moments of crisis one must do something, anything, at once, even if it is an overreaction, is totally ineffective or makes the situation worse, rather than "just sit there doing nothing." (E.g., "Banning air passengers from carrying ham sandwiches onto the plane probably does nothing to deter potential hijackers, but we have to do something to respond to this crisis!") This is a corrupted argument from pathos.

Thursday, October 16, 2014

Purpose, Mission and Vision keep testers self focused on things that matter

Purpose, Mission and Vision may sound like pointy hair  mumbo jumbo to you but what if it isn't?. In fact I believe it applies to testing and, specifically to Context Aware Testing.

To be context aware means to adapt according to the location, the collection of nearby people, and accessible resources as well as to changes to such things over time. To be a context aware tester means that you have the capabilities to examine the computing and human environments and react to changes to the environments that may affect the product under test.

To help guide us in our journey the Context Aware Tester always lays out his "Test Pact" from the onset. The pact includes the Purpose (the why), Mission (the how) and Vision (the what) for her testing. Lets review an example:

As a contractor she bids for and wins an assignment to test Widget A (an address book application for Windows). During a meeting with the stakeholders you find out this application is for internal use only, their main concern is stability and don't want the app to crash and cause loss of contact information.

From this bit of information we can begin to define our Test Pact:
  • You start with your purpose. To make sure the contacts application is stable by testing it using real world scenarios.
  • We then lay out what it is that we anticipate when we're done, our vision. This is our definition of done. In this case we anticipate application stability.
  • Next we state the mission. How are we going to accomplish our goal of verifying the state of the application stability. Load testing, stress testing, volume testing.
By defining your purpose, mission and vision before starting your testing project (no matter how small) you'd have given yourself a road map as well as a set of constraints to wrap around your testing effort to help keep you focused on the things that matter most (i.e. what's important). Once you start working, this is also a great way to gauge if what you are being asked to do now (an interruption) interferes with or contradicts any of the Test Pacts you are currently working on.

In nutshell Test Pacts encapsulate the definition of testing for its specific context in the form of purpose, vision and mission. This implies that for a context-aware tester, the definition of testing is not only depending on context, but also possibly different each time.

To a context aware tester, purpose (why) is her guide while the mission (how) is what drives her towards the vision (what). This keeps us closely and tightly aligned with, not only the technical aspects, but also the vision, of the stakeholders as captured in the Test Pacts.

Wednesday, October 15, 2014

Run a test suite using Perl

So you have a bunch of functionality you've automated and you'd like to execute the scripts unattended both on a set schedule as well as on certain triggers.
In this quick post (which I also did a while back here) I show you one way of running a test suite executing tests sequentially (i.e. one after the other) from the command line by using Perl. The script is appropriately named
The code
1:  #!C:/Perl64/bin/perl  
2:  use strict;  
3:  use warnings;  
5:   my $result_dir = "C:\\Automation\\Tests\\$ARGV[0]\\";  
6:   opendir (my ($dh), $result_dir) or die "can't open dir: $!";  
8:   while (readdir $dh){  
9:    if ($_ =~ /pl$/){  
10:     system ( $result_dir . $_ );  
11:    }  
12:   }  
This script takes one parameter [line 5], the name of directory in the C:\Automation\Tests folder which is where the test scripts should reside. We use this value to read all of the files in the directory [line 6] and, using regular expressions, only action on the ones that end in pl [line 9] (since we're looking for Perl files). Finally we use the system function to execute the script [line 10]
Once you have the above script, assuming you have perl.exe in your path and Perl mapped to open your pl files, to run manually type
1:  c:\> functional_tests   
Alternatively you can add the Perl script, using your services manager or crontab, to the list of services to be run periodically. Or add it to your CI flow to be executed when certain conditions are met.

Thursday, October 9, 2014

The Context Aware Tester

What is Context?
"The word "context" stems from a study of human "text"; and the idea of "situated cognition," that context changes the interpretation of text" ***

What does it means to be a Context Aware Tester?
A Context Aware tester knows (see above) that context changes the interpretation of what "testing" and "tests" means. And, as well:
  1. A context-aware tester knows that each situation will most likely require a custom approach.
  2. Likewise, A context-aware tester rejects the notion that a specific approach is the only approach to all problems.
  3. A context-aware tester does not reject any practice, technique, or method (not even another approach) when it comes to the who, what, when, where, and why of testing.

In his 1994 paper at the Workshop on Mobile Computing Systems and Applications (WMCSA), Bill Schilit introduces the concept of context-aware computing and describes it as follows:
Such context-aware software adapts according to the location of use, the collection of nearby people, hosts, and accessible devices, as well as to changes to such things over time. A system with these capabilities can examine the computing environment and react to changes to the environment.
-- Schilit et al 1994

Just like Schilit describes "context-aware software" as adapting "according to the location of use, the collection of nearby people, and accessible devices as well as to changes to such things over time.", so is a Context Aware tester and in doing so has the capabilities to examine the computing and human environment and react to changes to the environment that may affect the product under test.
Oh, and no, being a Context Aware tester does not mean you are now a member of a school. Context Aware is not a school. Is an approach to help solve hard, and easy, testing problems.


Monday, October 6, 2014

A method to Data Drive your Selenium automated tests

I was just approached by a colleague who wanted to know how I data drive my tests when automating. Since this is probably a common enough question I figured I share in my blog.

One of my favorite approaches to data drive my tests is to use database tables to store the data. By using a database, such as MySql, you save yourself a lot of time in the maintenance and mining of the test data. You also get to, based on the query, select only certain type of data to be included during the current test run.

Lets say for example that you want to test the login function of your web application / site. You already have a user table set up in your database that has a large amount of user names, passwords in different combinations (e.g. valid user, invalid user, male, female, etc). You can leverage MySql by constructing a query that covers the specific users you would like run the test for.

Below is a quick example to try to illustrate the above.

DB test_data_tbl:
user_id        password        gender        active
user1          user1pw           male            1
user2          user2pw           male            0
user3          user3pw           female         1

Pseudo code:
1) Query the database for all valid users that are male
2) For each of the users returned
2a) Instantiate Selenium
2b) Open the home / start page
2c) Login to the application
2d) Close the session

Sample code in Perl:
1:  my $query = "SELECT user_id, password  
2:             FROM test_data_tbl  
3:             WHERE account_active = true  
4:             AND gender = 'male'  
5:             LIMIT 3;";  
7:  while ( my( $user_id, $password ) = $sth->fetchrow_array() ) {  
8:    # SsApp is a custom module where the stuff to set up selenium has been abstracted.  
9:    my $driver = Custom::SsApp::setup_selenium( \%desired_capabilities );  
10:    $driver->get( APPHOME );  
11:    $driver->find_element( $user_name_target, $user_name_locator )->send_keys( $user_id );  
12:    $driver->find_element( $password_target, $password_locator )->send_keys( $password );  
13:    $driver->quit();  
14:  }  

With this approach you can have easy access to your test data by just querying a database table. You can also replace the while loop with a foreach loop or whatever looping construct your programming language supports. The basic idea is to iterate through your data and execute each test using each data set.
Creative Commons License
VGP-Miami Web and Mobile Automation Blog by Alfred Vega is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.