One of the first things I identified as being needed for this implementation of my framework (other than the basics: reporting, logging, etc.) was the need to have a group of virtual users (little robots if you will) that will mimic not only real users of the system, but their behaviors as well.
User State System
Enter the User State System. While the name may sound fancy this is nothing more than a series of database tables who's sole responsibility is to keep stored the state of each user as well as their interactions with the SaaS system and any artifacts they create along the way. When running tests in parallel (or even sequentially) keeping track of user state has the following advantages:
- Restore the entire user list to a known default state
- Restore a particular user to a known or default state
- Test scripts can share users and their current states (e.g. logged in, not logged in)
As an example, lets say you have a system that supports a SaaS for an accounting firm. You have SLA's in place that guarantee certain response times and load times as well as unscheduled down time. In a user state table for this type of system all users will share a common starting state, however this state is only guaranteed when the user record is first added or reset to a default state; from then on each user’s state will be represented in this table. In other words this is a dynamic table constantly being updated / changed by the running test scripts or it may be updated by resetting it to a default state either globally (for all users) or for a particular group of users (i.e. 1 or more users).
With a user state system implemented when test scripts execute they request, as part of the test setup, a specific user with a specific set of characteristics from the user state system. The specific set of characteristics would be determined by the features / functions being tested. Test scripts then execute using the requested user. For example if you wanted to test invoices being generated properly you would request a user that has the appropriate rights (i.e. is able to add invoices) and that has an account configured with a budget with enough money to satisfy the invoice. Or conversely a user that has an account configured with a budget that does not have enough money to satisfy the invoice.
As you can see in the above example you can effectively use a state system to pinpoint specific functionality as required by the test you are designing without the need of building this test data into the test. This level of abstraction gives the test designer, and the automated test system, the flexibility and speed that is gained from using shared virtual users with a known current state.
As well, although there is no one size fit all solution when it comes to saving user state, there are some common features that can be shared in any user state system that will be utilized for functional verification in QA or even for TiP (testing in production). I will share some of these in a future post.