Model-Based Testing For Web Applications With TestOptimal - (Part 3) Generating Sequences

So far we've prepared our TestOptimal project with the model we created in Part 1 (Model-Based Testing For Web Applications With TestOptimal - (Part 1) Preparing The Model) and the mScript we crafted in Part 2 (Model-Based Testing For Web Applications With TestOptimal - (Part 2) mScript). Now it's time to see how this all comes together by generating our test sequences.

TestOptimal offers 5 sequence modes. They are:

• Optimal - uses the proven optimization algorithms to generate the optimal test sequence to achieve 100% coverage of your application. Because only the minimum number of test steps have to be repeated, the test execution completes in least amount of time.
  
• Random - generates the test sequence by randomly selecting a transition from the current state. You can set conditions when to stop the test execution
  
• Greedy - works the same way as Random except that it prefers un-traversed (un-tested) transitions over already traversed transitions. This helps to achieve the desired coverage with less number of test steps.
  
• mCase Optimal - generates the optimal test sequences to cover all states and transitions contained in mCases. Use mCaseSequencer if you just want to test certain scenarios or parts of your application. You may also use mCaseOptimal with virtual users to simulate concurrent users running certain test scenarios.
• mCase Serial - is a variation from mCaseOptimal. It differs from mCaseOptimal by preserving the order of transitions/states in mCase(s) in the test sequences it generates.

Each of these sequence types describe how TestOptimal will traverse our graph, and consequently how much coverage we will achieve. I tried out the first three sequence types (mCase Serial and Optimal won't be covered in this article) and took them for a "dry run" to see what was produced.

For each sequence type I selected, I received a report on Testing Coverage from the Monitor tab:

Greedy

Random

Optimal

As you can see, the Monitor tab shows how many traversals were done for each state and transition, as well as how many states/transitions were covered or not.  Since I set up the sequencer to seek 100% coverage, there were no un-covered states/transitions.  I was surprised to see the Random sequence achieve a greater efficiency over the Greedy sequence.  And I was very impressed to see how efficient the Optimal sequencer was.

In addition to the information above provided in the Monitor tab, TestOptimal provides several other views for each run.  There's a Sequence Log, which shows the last 1000 traversals and lists each state and transition traversed:

And a Stats tab, which provides state and transition coverage information with performance times:

After trying out these sequences in "dry-run" mode, I decided to run the test sequences against the actual target - maps.google.com.  It worked flawlessly.  After pressing the Run button, TestOptimal launched a browser window and began running each of the several hundred test sequences.  It was fun to watch and, more importantly, demonstrated the power of model-based testing.  Next week we'll take a look at some of the details within TestOptimal reports and analyze our results.