Archived PushToTest site

Web Testing With Selenium and PushToTest

Build A Selenium Test of a
Web Application
In 31 Minutes Or Less.

The Selenium TestMaker Tutorial

This article shows how to build a test of a Web application in Selenium in minutes. This article then shows TestMaker repurposing the test as a load and performance test and a business service monitor with no extra coding.

Note: Find all of the code and data described in this article in tutorials/Selenium_WebTest_Tutorial

Download TestMaker and Run Selenium Tests Watch the Tutorial Screencast

Enterprises are challenged to surface performance and load issues, regression and functional issues, and to accomplish business service production monitoring in Web environments. For example, TestMaker comes with an example Web application named BrewBizWeb.

BrewBizWeb log in screen

This tutorial builds a test of the BrewBiz Web application. The tutorial will show how to use the test for regression and smoke testing. Then the tutorial will repurpose the test as a load and performance test and business service monitor.

The tutorial will make use of Selenium support in TestMaker. Selenium is a popular and widely used open-source Web test suite development tool. Selenium enables testers to build functional tests of Web applications in a Firefox-based graphical test development environment. Playback these tests in Firefox, Internet Explorer, Opera, Safari and other browsers. Then repurpose the same tests as functional tests, load and performance tests, and business service monitors in the TestMaker distributed test environment.

Tasks and Scenario

Download TestMaker and Run Selenium Tests Watch the Tutorial Screencast

Our challenge in this tutorial is to build a test of the Web application. The tutorial will show how to use the test for regression and smoke testing. Then the tutorial will repurpose the test as a load and performance test and business service monitor.
  1. Build and run a functional test of the Web application
  2. Run the test in a TestNode using the HTMLUnit browser
  3. Run the test in a TestNode using Selenium RC and a browser
  4. Reuse the test as a data-driven test with a Data Production Library that reads data from a Comma-Separated-Value (CSV) file
  5. Reuse the test as a load and performance test
  6. Reuse the test as a business service monitor
  7. Transform the test into a Jython script and Java test class
  8. Debugging Web Applications: Tips and Techniques

Preparation To Use This Tutorial

This tutorial uses the Web application that ships with TestMaker 6 and later. TestMaker Installer automatically installs the BrewBizWeb tutorial service.

This tutorial uses the following file name conventions:
  • Designer Script test object file ends in .ds
  • Selenium table format test script files end in .selenium
  • Test Scenario files end in .scenario

Task 1: Record and Playback A Web Application Functional Test

In this task we will create a test of the Web application using the TestMaker Designer record/playback utility.
  1. From the TestMaker console click the Designer icon or choose the Tools menu, Designer command.

  2. Select Selenium in the Script Type drop-down menu.

    Designer upon startup

  3. Click the Record button. Choose a browser. We wrote this test using the Firefox browser.

  4. Enter http://localhost:8080/BrewBizWeb/login.html for the Starting URL. Designer opens a browser window.

  5. In the browser window, type bert in the ID field, type biz in the password field. Click the Log-in button.

  6. Type 1200 in the Search field. Click Search.

    Searching the product catalog

  7. Add a Checkpoint to validate the product is in the catalog. Click the Checkpoint icon in the Designer window.

    Checkpoint icon

  8. Use Designer's locator assistance to define the Checkpoint. Select the browser window. Hold the Control key down while rolling the mouse over the product 1200 listing. Notice Designer identifies the locator for you automatically.

    Checkpoint window

  9. Click Add.

  10. Click End Recording. Designer closes the browser window.

    A Quick Tip: Double-click in the Action, Accessor, and Value fields in any row of the test to see a helpful pop-menu of options. For example, the Action pop-up shows you a sorted list of available action commands.

  11. Choose the Save command in the File menu. Name the file BrewBizTest.ds

    Export test script to file

  12. Next we will see the test operate in play-back mode. Click the Play icon.

    Play test icon

  13. Choose the browser. Click Ok.

  14. Designer opens a browser window and operates the test.

  15. As an option, set a breakpoint at one of the test commands. Right-click on a test command and choose Set Breakpoint. Click the Play icon. Use the Stop and Step icons.

    In the above steps you operated the test of a Web application using a Web browser. The Designer reads the Selenium test instructions and operates the test in the Firefox Web browser. This works well when building and debugging tests.

    Download TestMaker and Run Selenium Tests Watch the Tutorial Screencast
    In the next part of this tutorial you will operate the recorded test in the PushToTest TestMaker test automation platform. TestMaker automates operation of the Selenium test, data-enables the Selenium test, produces reports for the operation of the Selenium test, enables the Selenium test to be a step within a larger use case, and repurposes the Selenium to be a load and performance test and business service monitor from the same single Selenium test.

Task 2: Data-Enable The Test

Task 3 enhances the functional test of the Ajax application from Task 1 and 2 to use a Data Production Library (DPL.) The DPL reads data from a comma separated value (CSV) flat file. The values provide input to the test and validation values, including UserID, Password, and Product.
  1. Create a Comma-Separated-Value file. Use your favorite text editor or spreadsheet program. Name the file data.csv. The contents of the file must be have double-quotes surrounding the data, fields separated by a comma, and rows separated by a return character. In a text editor this looks like:

    Comma separate value (CSV) file

    A quick explanation: The first row of the data file contains data mapping names. These will be used to map values into the Sahi test.

  2. In the Designer, reuse the Designer Script test we created in Task 1. Save the file as BrewBizTestData.ds using the File menu, Save As... command.

  3. Click on the Data tab.

  4. In the Data tab, click Browse. Select the data file.

    Data tab showing comma separate value (CSV) file

  5. Drag-and-drop the UserID value to the Value field in the 2nd line of the test. Do the same for the Password field. Drag the Product field to line 6 and line 8.

    Mapping fields in place

  6. Click the Data Production Library (DPL) icon.

    DPL Icon in the Designer tool bar

    Alert showing DPL Execution Type

    These control the action Designer takes when it runs the test. For example, with "Get random row" selected Designer will choose a random row from the CSV file to provide test data to the test.

  7. Run the test in the Designer to confirm your changes to the Sahi test script. Click the Run icon in the Designer window. Click the Logs button to view the DPL provided data used during the test.

    Results of running a data enabled test.

    A quick explanation
    : Designer maps the data from the named column in the CSV data file to the test data using the $name notation you just entered.

  8. Steps 1-5 ran the test in the Designer on your local machine. Next, we will run the test in a TestNode using the CSV Data Production Library (DPL)

Task 3: Run The Test In A TestNode

PushToTest TestMaker makes it easy to repurpose a Selenium test in its test automation platform. PushToTest TestMaker is a distributed test environment. PushToTest provides an easy-to-use central console that sends the test to one or more TestNodes. Each TestNode is equipped with a script runner to operate the test. This makes it easy to operate tests in a rack of servers to generate hundreds-to-thousands of simulated users and to run the test from several locations around a network or the Internet. The TestNodes also inject operational test data into Selenium tests from data files, relational database queries, and data generating objects. The TestNodes record the time it takes to operate a use case and each of the steps in the Selenium test for results analysis reporting.

PushToTest distributed test environment

The TestMaker console sends the test to one or more TestNodes. Each TestNode operates the test using a Script Runner. This tutorial uses the Selenium Script Runner. The Selenium Script Runner uses integrated Selenium RC technology to run the test in Internet Explorer, Firefox, Chrome, Safari, and Opera. The Script Runner optionally runs the test in HtmlUnit, a headless highly-efficient browser. The TestNode logs the time it takes for the application to respond and returns the transaction logs to the PushToTest console.

Operating a test using the internal HTMLUnit browser

The PushToTest console identifies the operating parameters of a functional test, load and performance test, and business service monitor in a TestScenario. Use the TestMaker Editor to define a TestScenario.
  1. In TestMaker, choose the File menu, Open command. Select tutorials/Selenium_WebTest_Tutorial/Functional_Test.scenario

  2. TestMaker opens a new controller panel for the TestScenario. The controller panel includes icons to run, pause, and stop the test.

    Controller panel shows the TestScenario

  3. Click the Edit icon in the controller panel to view the source of the TestScenario document. This action opens the Editor.

    Editor for building TestScenarios

  4. Editor displays the TestScenario. Use the tabs to view the TestScenario settings. For example, the TestScenario defines use of the Selenium test in the Use Cases tab.

    Editor use case tab

    The UseCases tab tells TestMaker to operate a test use case that runs the Selenium test in BrewBizTestData.ds. The Command Language field tells TestMaker to run the Designer Script (.ds) test object using the Selenium command language. Browser identifies the browser to operate the test. Browser choices include Internet Explorer (iexplore), Firefox, Chrome, Safari, and Opera. Choose Htmlunit to use the headless highly-efficient Web browser.

    The General tab tells TestMaker that this is a functional test and to run the use case once.

  5. Click the Run button and watch the results.

    Results of running the functional test

    Functional tests run a test as a single user. TestMaker displays check-marks next to each step of the Selenium test that operate successfully. Designer automatically adds waitForElementPresent commands to synchronize the test with the application. TestMaker shows the amount of time to operate the use case and each step in the Selenium test.

    TestMaker displays an X mark for a step that fails. Use the Output command in the Windows menu to view the logs to learn the cause of the failure. Click the Local TestNode Error and Local TestNode Output tabs.

    Please note: Designer adds waitForElementPresent commands automatically. This makes sure the test waits until the browser finishes updating the page to show the page.

Task 4: Run Data-Driven Functional Tests In A TestNode

Task 2 created a data-enabled test object and run it in Designer with a Data Production Library (DPL) on the local machine. This task shows how to run data-enabled tests in a TestNode. TestMaker sends the comma separated value (CSV) file to the TestNode automatically. The DPL reads data from the CSV file and injects data into the test and validation data value.
  1. Use the Editor to connect the DPL to the Selenium test. Click the Use Cases tab.

  2. Click the Add DPL link. Set the DPL type to HashDPL. HashDPL reads data from a comma separated value (CSV) file and provides it at test run time to the ScriptRunner. Set the Action to Get Next Row of Data.

    Setting a DPL in the use case

  3. By default TestMaker runs the use case in a Functional Test once. Click the General tab and select the Repeat For All Data control to repeat the use case for the additional rows of data in the data.csv file.

  4. That's all! Click the PushToTest Run button and watch the results.

    Results of running the functional test

Task 5: Repurpose A Functional Test As A Load And Performance Tests

Load and performance testing identifies how close to linar scalability the application under test achieves. An application with linear scalability services 100 users concurrently at the same speed as 1000 users. TestMaker's Scalability Index is the quickest read of an application's ability to scale linearly.

TestMaker operates load tests by running multiple concurrently running users for a set amont of time.

PushToTest repurposes the functional tests as load and performance tests without requiring any change to the test!
  1. With the TestScenario from the previous task open in the Editor, click the General tab.

  2. Choose the Save As command in the File menu. Save the TestScenario to a file named Selenium_LoadTest.scenario

    Click the Load and Performance Test test type.

  3. Click the Add Virtual User Level 3 times. Enter the values 1, 2, 4.

    Editing a load test

    The settings tells TestMaker to operate a load and performance test at three levels of concurrently running simulated users (crlevel.) The test operates 1 user, then 2 users, and then 4.

  4. That's all! Click the PushToTest Run button and watch the results.

    Test results from a load test

    PushToTest operates the test in 1 simulated virtual user, then 2 concurrently running virtual users, then 4 users. The Real Time Scalability Index contrasts the Transaction Per Second (TPS) for each of the three user levels.

    A perfectly scalable system will increase TPS in linear proportion to the increase in users. For example, at 2 users doing 0.03 TPS a 4 user level should be twice the TPS, or 0.06 or higher. The above chart shows how close the application under test got to linear scalability. The Transaction Distribution chart shows the amount of time each step took to operate as the test proceeded.

    Test results from a load test

    Click the right triangle icon to view the Transaction Distribution at 2 and 4 virtual users.

Task 6: Repurpose A Functional Test As A Business Service Monitor

Business Service Monitor (BSM) testing enforces and proves a Service Level Agreement (SLA) by operating a test periodically. For example, a monitor runs a test every 30 minutes. TestMaker repurposes functional tests as a service monitor with no changes to the Selenium test.
  1. In the TestMaker window use the File -> Open TestScenario command. Use the file selector to choose WebRIASeleniumTest/Web_Monitor.scenario.

  2. Click the Edit icon in the TestScenario Controller panel.

    Editing a business service monitor

    This TestScenario runs the test use case as a business service monitor. The monitor operates the test use case every 10 minutes. The monitor keeps running until it encounters an exception.

  3. That's all! Click the PushToTest Run button and watch the results.

    Business Service Monitor (BSM)

    The monitor controller panel is a dashboard to the status of the service being tested.

Download TestMaker and Run Selenium Tests Watch the Tutorial Screencast


Additional documentation, product downloads and updates are at While the PushToTest TestMaker software is distributed under an open-source license, the documentation remains (c) 2011 PushToTest. All rights reserved. PushToTest is a trademark of the PushToTest company.