Sahi Tutorial

PushToTest Examples - Tutorial on Ajax RIA Testing with Sahi

Build A Test of a Rich Internet Application
In 30 Minutes Or Less.

This article shows how to build a test of a Rich Internet Application (RIA) using Ajax in PushToTest TestMaker with Sahi in minutes. This article then shows the tremendous power to repurpose the test as a load and performance test and a business service monitor with no extra coding.

Note: Find all of the code and data described in this article in TestMaker_home/example_agents/WebTest_Sahi_Tutorial

Enterprises are challenged to surface performance and load issues, regression and functional issues, and to accomplish business service production monitoring in Rich Internet Application (RIA) environments that use Ajax technology. Successful applications on the public Internet, such as Google Docs, Flikr, Facebook, and Google Maps, drive users and managers to expect rich user interfaces and advanced functions in an organization's Web applications. For instance, the following Ajax application has the look-and-feel of a desktop application, and it is running entirely in a Web browser.

This tutorial builds a test of this Ajax application. The tutorial will show how to use the test for regression and smoke testing. Then the tutorial will repurpose the test as a load and performance test and business service monitor.

The tutorial will make use of Sahi integrated support in PushToTest TestMaker. Sahi is an open-source Web test suite development tool that works well with Ajax applications. PushToTest built the TestMaker Object Designer using Sahi technology. The Designer provides an easy-to-use record/playback technology to visually build tests. The Designer records test scripts in Internet Explorer (IE), Firefox, Chrome, Opera, and Safari browsers. Playback tests in these browsers and the Htmlunit headless browser. TestMaker repurposes the same test as a functional test, load and performance test, and business service monitor.

Tasks and Scenario

Our challenge in this tutorial is to build a test of the Ajax application. The tutorial will show how to use the test for regression and smoke testing. Then the tutorial will repurpose the test as a load and performance test and business service monitor.

Build and run a functional test of the Ajax application
Run the test in a TestNode using the Firefox browser
Run the test in a TestNode using the HtmlUnit browser
Reuse the test as a data-driven test with a Data Production Library that reads data from a Comma-Separated-Value (CSV) file
Reuse the test as a load and performance test
Reuse the test as a business service monitor
Transform the test into a Jython script and Java test class
Debugging Ajax Applications: Tips and Techniques

Test Use Case

A software quality tester needs to test an Ajax system. The tester follows these steps:

Open the Ajax test application and view the Desktop
Add A Company to the directory
Change an Employee's information
Exit from the application and test

Implementation

We built the test and provide the test in a set of files available in the TestMaker_home/examples/WebTest_Sahi_Tutorial directory.

Preparation To Use This Tutorial

This tutorial uses an example Ajax application that ships with TestMaker 6 and later. Run the TestMaker Installer program and choose to install the Tutorial Services. TestMaker Installer installs the Ajax tutorial service into the localhost TestNode. When TestMaker runs it makes the tutorial service available at http://localhost:8080/AjaxApp/examples/desktop/desktop.html

This tutorial uses the following file name conventions:

Sahi test script files end in .sah
TestMaker Object Designer files end in .ds
Test Scenario files end in .scenario

Task 1: Record and Playback A Web Application Functional Test

In this task we will create a test of the Ajax application using the Sahi record/playback utility.

Open TestMaker
Choose the Windows menu -> Designer command or click the Designer icon in the toolbar.
TestMaker Object Designer appears. Click the Record button.
Choose Firefox and click Record Test.
Enter the starting URL to the Ajax tutorial application of http://localhost:8080/AjaxApp/examples/desktop/desktop.html. Then click Ok.
Click back to the main Firefox window. Point Firefox to http://localhost:8080/Ajax. The Designer opens Firefox and shows the Ajax application window.

Designer is now recording your use of the Ajax application.

Double-click the Company List icon in the Ajax application. Click Add Company. Name the company PushToTest, set the price to 101, and set the Change value to 0.50. Click the Save button.
Scroll down to the bottom of the Company list. Verify the PushToTest entry in the Company list.
Add a Checkpoint to the test to verify the PushToTest entry is in the Company list. In the Designer window, click the Checkpoints icon. This is the second icon to the left in the icons above the End Recording button.

Roll the mouse pointer over the PushToTest entry in the Company list. Hold the Control key down and click on the PushToTest row in the Company list. This updates the Assertion Type to AssertExists.

Click the Add button.
Click End Recording in the Designer window.
Save the test script to a file. Choose File menu -> Save As command. Name the file AddCompany.ds. Designer will add a .ds file extension automatically.
Next we will see the test operate in play-back mode. Click the Play Test icon.
Choose a browser to play the test.
Playing a test is useful for debugging the test script.

Ajax applications are asynchronous by their nature. For example, the example Ajax application displays the Add Company dialog box by running a Javascript function within the browser. At times you may need to manually add steps to the test script to wait for a Javascript function to finish processing before the test proceeds. We added Line 4 of the finished AddCompany script manually: assertVisible textbox("company)

Designer supports saving recorded tests in Sahi, Selenium, and Flex formats. Each script type has its advantages and disadvantages. For example, Sahi element accessors are more powerful and less complex to keep. Change the type by operating the Script Type drop-down menu.

In the above steps you operated the test of an Ajax application using a Web browser. Designer operates the recorded test script step-by-step. This works well when building and debugging tests.

In the next part of this tutorial you will operate the recorded test in the PushToTest TestMaker test automation platform. TestMaker automates operation of the test, data-enables the Sahi test, produces reports for the operation of the test, enables the test to be a step within a larger use case, and repurposes the test to be a load and performance test and business service monitor from the same single test.

Task 2: Run The Test In A TestNode

PushToTest TestMaker makes it easy to repurpose a Sahi test as a functional test, load and performance test, and production monitor. TestMaker operates tests in a distributed test environment. PushToTest provides an easy-to-use central console that sends the test to one or more TestNodes. Each TestNode is equipped with script runners to operate the test. This environment scales up to meet your testing needs from hundreds to millions of virtual users. TestNodes optionally data-enable tests by injecting operational test data into Sahi tests from csv data files, relational database queries, and data generating objects. The TestNodes record the time it takes to operate a use case and each of the steps in the Sahi test for results analysis reporting.

The TestMaker console sends the test to one or more TestNodes. Each TestNode operates the test using a Script Runner. This tutorial uses the Sahi Script Runner. The Sahi Script Runner operates the test in Firefox, Chrome and other browsers through the Sahi Proxy. The Script Runner optionally uses the built-in HTMLUnit browser to interact with the Ajax application. The TestNode logs the time it takes for the application to respond and returns the transaction logs to the PushToTest console.

With TestMaker still open from Task 1, save the test script to a file. Choose File menu -> Export Text Object As command. Choose Sahi format.

Designer shows the Sahi script and asks for a file name. Click Save As. Name the file AjaxTest. Designer automatically adds a .sah file extension.

The PushToTest console identifies the operating parameters of a functional test, load and performance test, and business service monitor in a TestScenario. Use a Controller window to display the controls to run, pause, and stop a TestScenario. Use the TestMaker Editor to define a TestScenario. Find a TestScenario document already created for you in Ajax-Functional-Test.scenario
From the TestMaker window choose File -> Open and select the file TestMaker_home/tutorials/WebTest_Sahi_Tutorial/Ajax-Functional-Test.scenario
TestMaker opens a new controller panel for the TestScenario. The controller panel includes icons to run, pause, and stop the test.
Click the Edit icon in the controller panel to view the source of the TestScenario document. This action opens the Editor and displays the TestScenario.
Editor displays the TestScenario. Use the tabs to view the TestScenario settings. For example, the TestScenario defines use of the Sahi test in the Use Cases tab.

The TestNodes tab tells TestMaker to send the Sahi recorded test to the TestNodes that will operate the test.

The UseCases tab tells TestMaker to run a use case that runs the Sahi test in AddCompany.sah. The Browser field chooses the Web browser to operate the test. Enter 'firefox' in the Browser field. If Firefox is not installed on your machine then use an alternative: iexplore for Microsoft Internet Explorer, chrome for Google Chrome, safari for Apple Safari.

The General tab tells TestMaker that this is a functional test and to run the use case once.
Click the Run button and watch the results.

As the test operates you will see the Firefox browser open and Sahi operates the application. At the end of the test TestMaker closes the Firefox browser session for the test.

Functional tests run a test as a single user. TestMaker displays check-marks next to each step of the Sahi test that operate successfully. TestMaker shows the amount of time to operate the use case and each step in the Sahi test.

TestMaker displays an X mark for a step that fails. Use the Windows menu -> Output command (or click the Output button in the Controller window) to view the logs to learn the cause of the failure. In the Output window click the Local TestNode Error and Local TestNode Output tabs.

See tips and techniques to solve failures.

By the way, TestMaker supports Sahi with many browsers. For example the browser attribute may be any of the following installed browsers: chrome, konqueror, piiexplore, iehta, mock, opera, pifirefox, safari, and iexplore.

Task 3: Run A Test In A TestNode Using HtmlUnit

Task 2 shows how to operate a Sahi test in a TestNode using Firefox. TestMaker provides the SahiHtmlUnit library to operate Sahi tests in the HtmlUnit browser. HtmlUnit is a headless browser.

HtmlUnit is very efficient at using memory and CPU resources compared to browsers like Internet Explorer (IE) and Firefox. While we could get 3 or 4 copies of IE to run on a server, TestMaker runs hundreds of HtmlUnit browsers concurrently in the same machine. HtmlUnit's efficience is the best choice for repurposing Sahi test scripts to be load and performance tests and production monitors.

In the TestMaker window use the File -> Open TestScenario command. Use the file selector to choose WebTest_Sahi_Tutorial/Ajax-Functional-Test-HtmlUnit.scenario
TestMaker opens a new controller panel for the TestScenario. The controller panel includes icons to run, pause, and stop the test.
Click the Edit icon in the controller panel. We only made 1 change from the TestScenario that operates the Sahi test with the internal HTMLUnit browser:

The change tells TestMaker to use the HtmlUnit browser.
That's all! With Firefox installed, click the PushToTest Run button and watch the test operate.

HtmlUnit is a headless browser. You will see no browser window appear as we did in Task 1. Click the Output button in the Controller window to see the logs for HtmlUnit.

TestMaker logs its activies while Sahi runs the test in the HtmlUnit headless Web browser.

Task 3: Make A Data-Driven Functional Test

Task 3 enhances the functional test of the Ajax application from Task 1 and 2 to use a Data Production Library (DPL.) The DPL reads data from a comma separated value (CSV) flat file. The values provide input to the test and validation values, including CompanyName, PriceValue, and ChangeValue.

Create a Comma-Separated-Value file. Use your favorite text editor or spreadsheet program. Name the file data.csv. The contents must be in the following form.

A quick explanation: The first row of the data file contains data mapping names. These will be used to map values into the Sahi test.
Open the Sahi test script in the Designer. Click the Designer icon in the tool bar of the TestMaker Console window.
In the Designer, use the File -> Open Test Object menu command to open AddCompany-Data-Driven.ds
Change the Sahi test to refer to the data mapping names. For example:

setValue, textbox("company"), "PushToTest"

to

setValue, textbox("company"), ${CompanyName}

An easy way to accomplish this task is to use the Data tab in the Designer. Click the Data tab. Click the Browse button. Choose data.csv from the Tutotirla directory.

Click-and-drag the PushToTest value in the Data tab, CompanyName column and drop the value in the test script on the Value field. Do the same for PriceValue and ChangeValue.
Run the test in the Designer to confirm your changes to the Sahi test script. Click the Run icon in the Designer window.

A quick explanation: Designer maps the data from the named column in the CSV data file to the Sahi test data using the name you just entered.
Steps 1-5 ran the Sahi test in the Designer on your local machine. Next, we will run the Sahi test in a TestNode using the CSV Data Production Library (DPL.)
Export to a Sahi test script. Use the File menu -> Export As Test Object -> Sahi command. Click Save As... and name the file AddCompany-Data-Driven.sah.
Connect the Data Production Library (DPL) to the Sahi test in the Editor. Open Ajax-Functional-Test-Data-Driven.scenario in the Editor. Click the Use Cases tab.
Click the Add DPL link.
Set the DPL type to HashDPL. HashDPL reads data from a comma separated value (CSV) file and provides it at test run time to the ScriptRunner.
Set the Action pop-up menu to Get Next Row of Data.
Click Browse and select the data.csv file.
By default TestMaker runs the use case in a Functional Test once. Click the General tab and set the Repeat value to a higher value to repeat the use case for the additional rows of data in the data.csv file.
That's all! Click the PushToTest Run button and watch the results.

A quick explanation: The TestScenario operates a functional test by running the Sahi test once. The data file we created contains 5 rows of data. Change the repeat value to 5 to have TestMaker repeat the test for each row of data.

Task 4: Repurpose A Functional Test As A Load And Performance Tests

The test we built in Task 3 operated a functional test by a single user. Load and performance testing identifies the scalability index of a target host application by operating a test with multiple simultaneous concurrently running virtual users. PushToTest repurposes tests to be load and performance tests without requiring any change to the test.

In the TestMaker window use the File -> Open TestScenario command. Use the file selector to choose WebTest_Sahi_Tutorial/Ajax-Load-Test.scenario
Click the Edit icon in the TestScenario Controller panel.
Set the test to be a load test in the General tab.

The settings tells TestMaker to operate a load and performance test at three levels of concurrently running simulated users. The test operates 1 user, then 2 users, and then 4 users.
In the Use Cases tab change the browser to sahihtmlunit.
Click the PushToTest Run button and watch the results.

PushToTest operates the test in 1 concurrently running simulated users, then 2 users, then 4 users. The Real Time Scalability Index contrasts the Transaction Per Second (TPS) for each of the three user levels. A perfectly scalable system will increase TPS in linear proportion to the increase in users. For example, at 2 users doing 0.03 TPS a 4 user level should be twice the TPS, or 0.06 or higher. The above chart shows the actual application throughput on top of what ideal application throughput with linear scalability.

See the Results Analysis chapter of the TestMaker User Guide for explanation of the thousands of additional charts to identify the root cause of a performance bottleneck.

Task 5: Repurpose A Functional Test As A Business Service Monitor

Business Service Monitor (BSM) testing enforces and proves a Service Level Agreement (SLA) by operating a test periodically. For example, a monitor runs a test every 30 minutes. TestMaker reuses functional tests as a monitor with no changes to the Sahi test and one change .

In the TestMaker window use the File -> Open TestScenario command. Use the file selector to choose Ajax_Sahi_Tutorial/Ajax-Monitor-Test.scenario
Click the Edit icon in the TestScenario Controller panel.

This TestScenario runs the test use case as a business service monitor. The monitor operates the test use case every 10 minutes. The monitor keeps running until it encounters an exception.
TestMaker optionally sends notification email messages. Click the Notifications tab to set the message.
Click the Run button and watch the results.

The monitor controller panel is a dashboard to the status of the service being tested.

Task 6: Transform A Recorded Test To A Script-Driven Test (Java, Jython)

PushToTest provides Web application record/playback test capability with Sahi. There may be times when you need to accomplish a test flow that is more than can be modeled with the Designer. For these occassions PushToTest provides an Export utility to convert a recorded Sahi test into Javascript, Java, Jython, and several other languages. The resulting class or script is a fully functional test with complete access any of Java or Jython's capabilities.

In the Designer use the File menu -> Export command to save the test to a Javascript file.

For more details, read the Script-Driven Testing chapter of the TestMaker User Guide.

8. Debugging Ajax Applications: Tips and Techniques

Ajax applications provide rich user experiences. In an Ajax application components may update themselves asynchronously. Ajax applications communicate to information sources external to the browser - like the backend server - or even with other components running on the same page. This flexibility challenges the tests we write. Our Ajax tests must work within the asynchronous nature of the Ajax application. And, we often need to make manual changes to the test because of the lack of a standard for the Designer to follow when recording an Ajax application.

You may find the following tips and techniques useful:

a) When you are not certain of the state of your test and the application, use the Designer's breakpoints and step functions to view the application as the Designer operates the test.

b) Set HtmlUnit into "debug" mode. In the Editor use the Options tab to set "Save received responses to temp directory" and SahiHtmlUnit log level.

Additional documentation, product downloads and updates are at www.PushToTest.com. While the PushToTest TestMaker software is distributed under an open-source license, the documentation remains (c) 2011 PushToTest. All rights reserved. PushToTest is a trademark of the PushToTest company.