<basics>

Defines the TestScenario name and default directory.

<basics>

<name>GeospatialTest</>

<defaultdirectory>./geospatialtest</defaultdirectory/>

<description>Test description if desired</description>

</basics>

The <name> element defines the test scenario name. TestMaker uses the name in the Controller Panel, in the status Output Panel and in the results files. The <defaultdirectory> element defines the default path to find scripts, JAR files and other resources. The <desciption> element stands for a brief description of the test.

About Paths And The Default Directory

PushToTest TestMaker works well in a distributed environment where developers and QA engineers run tests within any
directory structure (i.e. wherever they've checked out the project files). TestMaker TestScenario test orchestration documents determine the default directory path for test resources. The following options overriding the default directory and other elements in the TestScenario. For example, consider the following directory structure:

ScenarioDirectory
- Scenarios
- - MyScenario.xml
- Resources
- - resource.csv

TestMaker provides a {scenario} operator that resolves to the path of the TestScenario document. This operator may be used in the <defaultdirectory> element, in <resource> elements, and in individual resource paths. For example. use the above resource.csv with

path="{scenario}/../Resources/resource.csv"

The following is also valid:

Defaultdirectory ="{scenario}/../" and path="Resources/resource.csv"

<testnodes>

Defines a TestNode name and URL for running the test.

<testnodes>

<node name="node1" location="http://localhost:8080/TestNetwork/TestNode"/>

</testnodes>

TestMaker provides an special form of the <node> tag to identify a TestNode and a PTTMonitor.

<testnodes>
<node
name="node1"
location="http://localhost:8080/TestNetwork/ws/TestNode"
monitor="http:///localhost:51111"/>
</testnodes>

<arguments>

Defines a set of name / value pairs passed to running tests; allows for parameterized test scenarios

<arguments>

<argument name="targethost" value="10.0.0.1/gis"/>

<argument name="id" value="admin"/>

<argument name="password" value="admin"/>

</arguments>

A running test has access to the name / value pairs through a dictionary (Jython) or HashMap (JAVA).

<resources>

Identifies files to be dynamically loaded when the test is run on a TestNode

<resources>

<jar path="./lib/test.jar"/>

<testgen4web path="./PTT_Examples_UnitTest.testgen4web"/>

<soapui path="example_agents/soapui_examples/PTT_Examples-soapui-project.xml"/>

<module name="myJythonModule" path="./example_agents/scriptTutorial/example1.py"/>

<module name="myRubyModule" path="./example_agents/rubyExample/calc.rb"/>

<selenese path="CalendarTest.selenium"/>

<data path="./example_agents/dplCSVExample/csv.txt"/>

</resources>

TestScenarios recoganizes several types of resource files described in the following:

• • <jar> - JAVA Archive Resource containing compiled JAVA classes and data.
• • <testgen4web> - TestGen4Web recorded unit tests of a Web application.
• • <soapui> - soapUI recorded test suite of a soap-based Web service.
• • <module> - a unit test script.
• • <data> - any data file, including a comma separated value file, an image file, a file the unit test will use for validation. They are accessed using DPL resources.
• • <selenese> - Selenium recorded test suite of a Web application.

<DataSources> and Data Production Library (DPL)

Identifies Data Production Library (DPL) providing dynamic operational test data when the test is run.

<DataSources>

<dplDefType name="SimpleDPL"

library="./example_agents/dplExample/dplSimple.jar" lang="java"

classname="com.dpl.SimpleDPL"/>

<dpl name="lingo" type="LingoDPL"/>

<dpl name="simple" type="SimpleDPL">

<argument name="argrun2" value="Data"/>

</dpl>

<dpl name="simple" type="SimpleDPL">

<argument name="argrun2" dpl="lingo" value="getNextData"/>

</dpl>

</DataSources>

See the Data Production Library page for examples of the DataSources function, including HashDPL, dplCSVExample, dplExample, and counterDPLExample. DPL can use standard arguments and DPL arguments from previous defined DPLs.

<logs>

Defines one or more log handlers handling logged entries during a running test

<logs>

<log type="file" path="logs/mylog.txt" level="2"/>

</logs>

The <logs> option includes optional attributes to contract the summary reports for a load test

<logs summaryreport="true" showsummary="true"/>

In the above example summaryreport tells TestMaker to save all charts and create the summary report in HTML form for easier sharing of test results. The showsummary attribute opens the summary report in a browser at the end of a load TestScenario.

<logs logerror="true"/>

The logerror option tells the TestNodes to capture console exception messages to a log file. This is useful while debugging a test.

Step Logging

Step logging enables root cause analysis of slowly performing steps within a transaction. For instance, the following TestScenario defines a transaction with three steps: geo_insert, geo_query, geo_update, geo_delete

<test>

<run name="geo_insert" testclass="com.pushtotest.example.test"

method="insert" langtype="java"/>

<run name="geo_query" testclass="com.pushtotest.example.test"

method="query" langtype="java"/>

<run name="geo_update" testclass="com.pushtotest.example.test"

method="update" langtype="java"/>

<run name="geo_delete" testclass="com.pushtotest.example.test"

method="delete" langtype="java"/>

</test>

With step logging enabled, each <run> is a step within the test transaction. PushToTest logs the step values to a transaction log. For instance, the following is part of a transaction log for the above testusecase

<testmakerlog version="200" usecase="Selenese usecase" target="AppHost"
node="localhost" dataindex="1" description="Selenese usecase" users="1" duration="60">
<results>
<tx id="0" seq="0" dur="4594" res="true" ts="1224604845575">
<steps times="1469,16,15,1766"/>
</tx>
<tx id="0" seq="0" dur="1719" res="true" ts="1224604851169">
<steps times="891,15,16,453"/>
</tx>
…
</results>
<definitions>
<sequence name="Selenese sequence" id="0">
<step id="0" name="geo_insert"/>
<step id="1" name="geo_query"/>
<step id="2" name="geo_update"/>
<step id="3" name="geo_delete"/>
</sequence>
</definitions>
<duration seconds="67"/>
</testmakerlog>

The <step> entries show the time it took to perform each step.To enable step logging add the following to the <logging> element in a TestScenario.

<logs summaryreport="true" showsummary="false" step="true"/>

The <step> entries show the time it took to perform each step.To enable step logging add the following to the <logging> element in a TestScenario

<logs summaryreport="true" showsummary="false" step="true"/>

User Defined Log Step Entries in Test Scripts

Tests implemented in Java, Jython, and other script languages support user defined log entries. TestMaker implements a special log listener at com.pushtotest.tool.PTTStepListener. Use the following method call to log step information within a test script:

import com.pushtotest.tool.PTTStepListener;

public class tester {

public void runTest()
{
PTTStepListener.startStep( "my custom step log entry" );

/* Perform my test step here /*

PTTStepListener.endStep();
}

TestMaker uses the string value given in startStep as the name of the step. This value appears in the transaction log files as a sequence step name. The PTTStepListener lets you create an arbitrary number of steps. However, each operation of the test use case must contain the same number of steps.

Logging To A Relational Database Management Server (RDBMS)

Using a database logger, PushToTest TestMaker stores data in a Relational Database Management Server (RDBMS) database including results, performance snapshots and all other statistics. This is a description of that database schema:

To enable RDBMS logging, create the database and tables. See testmaker_home/docs/resources/Standard_DB_for_logging.sql for commands to create the database and tables in MySQL. Then, add the following lines to a TestScenario document:

<logs>

<log type="console" level="info"/>

<log type="file" path="mylog.txt" level="info"/>

<log type="database"

connector="com.mysql.jdbc.Driver"

url="jdbc:mysql://localhost/testmaker"

loggin="llara"

password="12345"/>

</logs>

Use multiple RDBMS log handlers to log data to multiple databases. The database default name is testmaker , but it can be easily changed.

<functionaltest>

Operate a testusecase in a single concurrent virtual user (CVU). The repeat attribute specifies the number of times to run the testusecase

<functionaltest repeat="1">

TestMaker displays a Functional Test in the following styled controller panel.

<loadtest>

Operate a testusecase in multiple concurrent virtual users (CVUs)

<loadtest>

The testusecase requires <dimensions> and <crlevels> elements.

TestMaker displays a Load Test in the following styled controller panel.

<monitor>

Periodically operate a testusecase in a single concurrent virtual user (CVU). The timeBetweenTest element identifies the number of milliseconds (1000 milliseconds = 1 second) to wait between operations. This <monitor> element requires the <until> element to define the exceptions that stop the monitor

<monitor timeBetweenTest="10000">

<until>

<exception type="java.lang.Exception"></exception>

</until>

</monitor>

Service monitors show an up-or-down icon to indicate the last known condition of the service under test.

<shocktest>

<shocktest> makes concurrent requests to an application host. A ShockTest stages a test by communicating a test usecase - defined in a TestScenario - to the defined TestNode(s). Each TestNode instantiates the needed number of threads. The threads execute the test usecase immediately when all the Threads on all of the TestNodes exist. The threads run the test use case once and then the test ends. The results directory for the ShockTest updates with the time it took to operate all of the threads on all the TestNodes.

<ShockTest>
<testusecase>
<dimensions>
<usecases>
<usecase name="Untitled_usecase">
<sequence name="Untitled_sequence" proportion="100">
<test>

<run
name="Appcelerator"
instance="I1"
testclass="org.appcelerator.client.ServiceBrokerClient" method="run"
langtype="java">
<argument name="theUrl" value="http://localhost:8080/AjaxService/"/>
<argument name="message" value="Hello World"/>
</run>

</test>
</sequence>
</usecase>
</usecases>
</dimensions>
</testusecase>

</ShockTest>

<options>

Defines default and optional values for the test. The following child tags may be put within the <options> command.

<sleeptime>0</sleeptime>

Sleeptime sets the duration TestMaker waits between running the test use case. The value is milliseconds (1000 = 1 second.) Set the value to 0 for no delay.

<delayBetweenStartingUseCases time="10"/>

delayBetweenStartingUseCases sets the delay time between starting concurrently running user. TestMaker creates one Thread for each simulated user. This option facilitates pacing the amount of time it takes to stage a load and performance test. The value is milliseconds (1000 = 1 second.) Set the value to 0 for no delay.

<delayBetweenTestCases time="60"/>

delayBetweenTestCases sets the duration TestMaker waits between running a concurrently running simulated user level (CRLEVEL) in a load and performance test. The value is milliseconds (1000 = 1 second.) Set the value to 0 for no delay.

<testTime time="60000"/>

testTime sets the amount of time TestMaker operates a concurrently running simulated user level (CRLEVEL) in a load and performance test. The value is milliseconds (1000 = 1 second.) Set the value to 0 for no delay. The <testTime> value does not include the time setup the test - when the TestNodes instantiate the Threads - or the time it takes to teardown the threads.

<target name="PushToTest_WebServer"/>

Target sets the name of the target application host under test. TestMaker uses the target name of AppHost if none is defined in the TestScenario.

<htmlunit
DebuggingWebConnection="true"
HtmlUnitContextFactory="true"
XmlTestResultImpl="true"
Csswarning="true"
Jswarning="true"/>

TestMaker uses the HTMLUnit Web browser to operate Selenium (not Selenium RC) and TestGen4Web tests. The <htmlunit> tag controls many features that come with HTMLUnit. The options contains the following tag options:

• DebuggingWebConnection a wrapper around a "real" WebConnection that will use the wrapped web connection to do the real job and save all received responses in the your local operating environment's temp directory with an overview page. This may be useful at test development/debug time to understand what is "browsed". For more information check HTMLUnit documentation.

• HtmlUnitContextFactory enables or disables the HTMLUnit debugger that logs stack entries and exceptions. Enabling the debugger may be useful if HtmlUnit is having trouble with JavaScript, especially if you are using some of the more advanced libraries like Dojo, Prototype or jQuery. Currently only implemented for TestGen4Web.

• XmlTestResultImpl logs results using xml output. Currently only implemented for TestGen4Web.

• Jswarning supresses JavaScript warnings to avoid excessive logging that may artificially increase transaction duration. Useful in load testing.

• Csswarning supresses CSS warnings to avoid excessive logging that may artificially increase transaction duration. Useful in load testing.

<selenium>

TestMaker uses the <Selenium>option to the system start the internal Selenium remote control (Selenium RC) server and control logging and behavior. Control logging and exception handling for Selenium tests (langtype="selenium"):

<selenium
loglevel="off"
throwExceptionOnFailingStatusCode="true"
throwExceptionOnScriptError="true"/>

• loglevel = "off", "info", "debug" controls the Selenium HTMLUnit logging

• throwExceptionOnFailingStatusCode, when set to true HTMLUnit will ignore errors when the source of an external script cannot be loaded

• throwExceptionOnScriptError, when set to true HTMLUnit will ignore JavaScript execution errors

<notifications>

Sets the notification parameters for the email notification triggered when a test changes state

<notifications>

<email on="start" email=" This e-mail address is being protected from spambots. You need JavaScript enabled to view it "/>

<email on="end" email=" This e-mail address is being protected from spambots. You need JavaScript enabled to view it "/>

<email on="interrupted" email=" This e-mail address is being protected from spambots. You need JavaScript enabled to view it "/>

<email on="testcasestart" email=" This e-mail address is being protected from spambots. You need JavaScript enabled to view it "/>

<email on="hourly" email=" This e-mail address is being protected from spambots. You need JavaScript enabled to view it "/>

</notifications>

<mail>

<username value="yourUsename"/>

<password value="yourPassword"/>

<hostname value="YourMailHost"/>

</mail>

<notifications>

<email on="start" email="EmailDirection" subject="test scenario started"

message="The test started. - From Pushtotes"/>

<email on="testcasestart" email="EmailDirection"

subject="test case started"/>

<email on="end" email="EmailDirection"

message="The test scenario had ended. - From push to test"/>

</notifications>

<testusecase>

Defines the functional test made-up of one or more unit test operations. <testusecase> operates a single unit test, a sequence of unit tests, or a mix of sequences for the duration of the test.

A <testusecase> for a functional test and a service monitor operates one or more unit tests in sequence. The functional test operates the <testusecase> once in one simulated user. The service monitor operates the <testusecase> in one simulated user repeatidly with a delay timer between operations. TestMaker offers several options for <testusecase> when run in a load and performance test.

The following sections show how to configure a <testusecase> in a TestScenario XML file.

Mustrun Option

A <sequence> has a proportions attribute, which defines the number of threads that will execute the actual sequence. This number of threads is calculated from the concurrent running virtual user <crlevel> . In the case the number of threads ends in decimals (3.7 threads in a sequence for instance) this number is rounded to the nearest integer number (4 threads). It can happen that the number of threads in some sequence falls below 0.5 and then TestMaker does not run the sequence at all. To allow the user to have the capability of choosing which sequences to must run at least in one thread, TestMaker introduced a new attribute: <sequence mustrun='true'>. When mustrun is true, the sequence will be run in AT LEAST one thread, even if the number of threads for this sequence is zero.

Number of Threads per sequence using the mustrun option:

Sequence\Users	2	4	10
inserts	0	1	2
queries	2	3	8

Instance Variables

PushToTest TestScenarios feature the <setup>, <run>, <teardown> elements instantiate and run unit tests implemented in JAVA and many other supported dynamic scripting languages. Each call instantiates a new class. Instance variables enable PushToTest to instantiate a class once and reuse it. The following example illustrates instance variables and reuse:

<resources>

<jar path="./example_agents/instancestest/SampleClass.jar"/>

</resources>

<loadtest>

<testusecase>

<dimensions>

<crlevels>

<crlevel value="10"/>

<crlevel value="20"/>

</crlevels>

<usecases>

<usecase name="INT VALUES">

<sequence name="InstancesTest" proportion="100">

<test>

<run instance="I1" testclass="SampleClass" method="open" langtype="java">

<argument name="id" value="1"/>

</run>

<run instance="I1" testclass="SampleClass" method="close" langtype="java"/>

<run instance="I2" testclass="SampleClass" method="open" langtype="java">

<argument name="id" value="1"/>

</run>

<run instance="I2" testclass="SampleClass" method="close" langtype="java"/>

</test>

</sequence>

</usecase>

</usecases>

</dimensions>

</testusecase>

</loadtest>

The above example uses the class defined below.

public class SampleClass

{

int v1;

static int s1= 0;

public void open(int id){

v1 = id;

}

public void close(){

System.out.println(v1+"-"+s1);

s1++;

}

}

<while> - Variable Test Use Case Periods

PushToTest TestScenarios operate a set of <testusecases> for a period of time defined by <testTime> in the <options> section. Certain load TestScenario situations operate a test until it reaches a certain condition and then moves to the next crlevel or data level. PushToTest provides the <while> operator for these circumstances.

For instance, a data driven test finishes operating a <testusecase> when the test exhausts the data from a set of data files or a database. Consider the following example:

<loadtest>

<testusecase>

<dimensions>

<crlevels>

<crlevel value="10"/>

<crlevel value="20"/>

</crlevels>

<usecases>

<usecase name="WHILE Test">

<sequence name="while test" proportion="100">

<setup testclass="whiletest.While" method="setup" langtype="java">

<argument name="quantity" value="100"/>

</setup>

<test>

<while testclass="whiletest.While" method="mustContinue" langtype="java"/>

<run testclass="whiletest.While" method="run" langtype="java"/>

</test>

</sequence>

</usecase>

</usecases>

</dimensions>

</testusecase>

</loadtest>

<options>

<sleeptime>1</sleeptime>

<delayBetweenStartingUseCases time="1000"/>

<delayBetweenTestCases time="100"/>

<testTime time="20000"/>

</options>

In the above example, PushToTest operates 10 Threads for 20 seconds ( time="20000" .) Each time through the <test> if the <while> value returns false, then the ScriptRunner ends the usecase - it takes down the Threads and moves onto the 20 CR level.

<monitoring>

PushToTest TestMaker provides PTTMonitor, an agent-based monitoring service, to observe machine and server performance as TestMaker operates a test. PTTMonitor by default observes CPU, network, and memory use. TestMaker polls the PTTMonitor periodically and correlates performance hotspots to test activity.

PTTMonitor installs on TestNodes and on one or more target systems that run the target application. TestMaker TestScenarios use <monitoring> to identify the target host and turn monitoring on or off when presenting results:

<monitoring enablednodes="true" enabledtarget="false" targethost="http://localhost" />

TestMaker allows monitoring of multiple target hosts. Consider the following PushToTest TestMaker provides a Monitor to correlate performance hotspots to testusecase activity. The Monitor installs on TestNodes and the target system. TestMaker TestScenarios use <monitoring> to identify the target host and turn monitoring on or off when presenting results:

<monitoring enablednodes="true">
<host name="host1" location="http://192.168.1.102:8011/targethost1" enabled="true"/>
<host name="host2" location="http://192.168.1.102/targethost1" enabled="true"/>
<host name="host3" location="http://192.168.1.102/targethost1" enabled="false"/>
</monitoring>

TestMaker provides an special form of the <testnodes> <node> tag to identify a TestNode and a PTTMonitor.

<testnodes>
<node
name="node1"
location="http://localhost:8080/TestNetwork/ws/TestNode"
monitor="http:///localhost:51111"/>
</testnodes>

<run> and Timeout

The <run>, <setup> , and <teardown> elements of a TestScenario runs a unit test that may return long after the testuecase finishes. The timeout attribute in <run>, <setup> , and <teardown> causes these to throw a TimeoutException if the unit test does not complete in the defined number of milliseconds

<usecases>

<usecase name="Examples_usecase">

<sequence name="Examples_sequence" proportion="100">

<test>

<run name="MyTest?" testclass="PTT_Examples_UnitTest.testgen4web"

method="testGen4Web" langtype="testgen4web" timeout="1500"/>

</test>

</sequence>

</usecase>

</usecases>

If the <run> element call takes more than 1500 milliseconds then a TimeoutException is triggered.

<Run>, <Setup>, <TearDown>

<run> indicates the call to a function or a method to operate the step of a test. <run> has various behaviors depending on the langtype. For example, Selenium and TestGen4Web run a testcase or a transaction file respectively. <setup> is used to initialize the test and envnironment. <teardown> is used to finalize the environment after the test operates. Both tags can be used several times in the scenario. Their meaning differs according to the location and they are optional.

<usecases>
<startup name="NodeStartup"/>
<usecase name="Untitled_usecase">
<startup name="UsecaseStartup"/>
<sequence name="Untitled_sequence" >
<startup name="AgentStartup"/>
<test>
<startup name="TransactionStartup"/>
<run module="myJythonScript"/>
<teardown name="TransactionTeardown"/>
</test>
<teardown name="AgentTeardown"/>
</sequence>
<teardown name="UsecaseTeardown"/>
</usecase>
<teardown name="NodeTeardown"/>
</usecases>

Examples of the above <startup>:

• NodeStartup: Initialize the node. There can be only one NodeStartup and it is executed right after the scenario is loaded. It is useful to set up global information.

• UsecaseStartup: Initialization at the beginning of the usecase. There can be only one startup per usecase.

• AgentStartup: Initialize the agent (thread). A testcase is composed by a usecase with certain data index and certain number of agents. Every time a test case is run, new agents are created. AgentStartup is executed to initialize these agents, and it is executed right after the agents are created. Useful to open connections to servers.

• TransactionStartup: Indicates an initialization in a transaction. Its time is taken into transaction time, then even when it syntactically differs from a run, its behavior is the same of a run.

Examples of the above <teardown>:

• TransactionTeardown: Same as the TransactionStartup, It is a run with a different mane. Useful to indicate the finalization of a transaction.

• AgentTeardown: Indicates the finalization of the agent. It is run by every agent after the test time and the cold down time is done. Useful to close connections to server.

• UsecaseTeardown: Indicates the finalization of the usecase. It is run after all the threads had exhausted their test time and cold down time, and done their AgentTeardown (if any).

• NodeTeardown: Indicates the finalization of a scenario. It is run at the end of the scenario, before sending the xml files to the console.

Types of <Run>

As mentioned before the demeanor of each transaction depends on its language type. The following sections show <run> for each language type.

Java

A standard call to Java:

<run name="javacall" testclass="com.examples.PrintOne" method="runTest" langtype="java">
<argument name="testval" value="b"/>
</run>

TestMaker maps the above <run> method to to call the Java sequence show in the following code:

com.examples.PrintOne obj = new com.examples.PrintOne();
obj.runTest(“b”);

TestMaker handles data type conversion automatically. For example, when an argumentn value expected by the Java method's runTest is a number, TestMaker will try to change the value to a number.

<run name="javacall" testclass="com.examples.PrintOne" method="runTest" langtype="java">
<argument name="testval" value="5"/>
</run>

com.examples.PrintOne obj = new com.examples.PrintOne();
obj.runTest(5);

Data Production Library (DPL) Argument For Data Driven Tests

DPL's provide operational test data to a <run> method in a TestScenario. The following gives an example DPL use:

<dpl name="mydpl" type="HashDPL">
...
</dpl>
…
<run name="javacall" testclass="com.examples.PrintOne" method="runTest" langtype="java">
<argument dpl="mydpl" name="DPL_Properties" value="getData"/>
</run>

The run can be mapped to the next java sequence:

HashDPL mydpl = new HashDPL();
…
com.examples.PrintOne obj = new com.examples.PrintOne();
obj.runTest(mydpl.getData());

TestMaker supports DPL use with langtypes, including Selenium, Java, soapUI, and the script languages (Jython, Groovy, and others.)

Instance System

Many times a test needs to run methods within the same instance of a test object. For example, a test implements methods to connect to a server, make a transaction, check that the transaction is finished, and close the connection. Using the instance option, the first <run> instantiates the test object and calls the connect-to-a-server method and then uses the same object instance to call the other methods. The following shows an example:

<run name="javacall" testclass="com.examples.PrintOne" method="runTest" langtype="java">
<argument name="testval" value="5"/>
</run>
<run name="javacall" testclass="com.examples.PrintTwo" method="runTest" langtype="java">
<argument name="testval" value="7"/>
</run>

TestMaker transforms these to:

com.examples.PrintOne Object@1abc7b9 = new com.examples.PrintOne();
Object@1abc7b9.runTest(5);
com.examples.PrintOne Object@c55e36 = new com.examples.PrintOne();
Object@c55e36.runTest(7);

The following uses an instance attribute:

<run name="javacall" testclass="com.examples.PrintOne" method="runTest" langtype="java" instance="counter">
<argument name="testval" value="5"/>
</run>
<run name="javacall" testclass="com.examples.PrintTwo" method="runTest" langtype="java" instance="counter">
<argument name="testval" value="7"/>
</run>

TestMaker transforms the above into:

com.examples.PrintOne counter = new com.examples.PrintOne();
counter.runTest(5);
counter.runTest(7);

For two calls to use the same method, it is required to both calls contains the same langtype, testclasss and instance. Instance system could be used with langtypes selenium, seleniumrc, java, script languajes (jython, groovy, etc..)

Array Argument System

TestMaker supports a system to pass an array of data values to a <run>.

<run name="javacall" testclass="com.examples.PrintOne" method="runTest" langtype="java">
<argument name="testval" value="a"/>
<argument name="testval" value="b"/>
<argument name="testval" value="c"/>
</run>

com.examples.PrintOne obj = new com.examples.PrintOne();
testval = new String[3];
testval[0] = “a”;
testval[1] = “b”;
testval[2] = “c”;
obj.runTest(testval);

TestMaker handles data type conversion automatically. For example, when an argumentn value expected by the Java method's runTest is a number, TestMaker will try to change the value to a number in the array.


<run name="javacall" testclass="com.examples.PrintOne" method="runTest" langtype="java">
<argument name="testval" value="1"/>
<argument name="testval" value="2"/>
<argument name="testval" value="2>"/>
</run>

com.examples.PrintOne obj = new com.examples.PrintOne();
testval = new int[3];
testval[0] = “1”;
testval[1] = “2”;
testval[2] = “2”;
obj.runTest(testval);

Currently Array argument system could be used with langtype java only.

Script Languages Support: Java, Jython, Groovy, JRuby, Perl, PHP

TestMaker supports the Java 1.6 ScriptEngine API (JSR 223) to run test cases written in Java, Jython, Groovy, JRuby, and several other scripting languages. To add a resource tag you first need to add the file where the script is in the resources, as a module.

<resources>
<module name="myJythonScript" path="./example_agents/Miscellaneous/SleepTest/printone.py"/>
</resources>
…
<run module="myJythonScript" instance="i1" name="printone" testclass="PrintOne" method="runTest" langtype="jython">
<argument name="testval" value="a"/>
</run>

In the run target you need to indicate the module from were you are running the script. Given the fact that you can not create directly script objects with the java script engines, you need to define a function to create the object. The function should have the name “get” plus to testclass method. Then in our example at the end of script file, there should be a function called getPrintOne, similar to:

def getPrintOne():
return PrintOne();

Selenium

TestMaker runs a Selenese file using the headless HTMLUnit browser. This is useful for load test because it is less memory intensive because there is no graphical interface. In order to use Selenium, declare the file in the resource as a Selenese resource.

<resources path="./test/data">
<selenese path="calendar.selenium"/>
</resources>
…
<run langtype="selenium" name="testweb" testclass="ptttest.selenium"/>

TestMaker passes operational test data using the HashDPL to a Selenium test operation.

<dpl name="mydpl" type="HashDPL">
</dpl>
…
<run name="CalendarTest" testclass="calendar.selenium" langtype="selenium">
<argument name="hashdata" dpl="mydpl" value="getNextData"/>
</run>

SeleniumRC

TestMaker runs a Selenese test file in a real browser using SeleniumRC and TestNode technology. TestMaker requires the browser identifier.

<resources path="./test/data">
<selenese path=" ptttest.selenium"/>
</resources>
…
<run langtype="seleniumrc" method="*firefox" name="testweb" testclass="ptttest.selenium"/>
…
<options>
<selenium rc="true"/>
</options>

Additionally, SeleniumRC instructs TestMaker to start the Selenium server. Also, TestMaker supports the DPL and argument system for Selenese tests.

TestGen4Web

Run a TestGen4Web file in TestMaker. The file should be added in the resource section as a TestGen4Web element. This Script Runner uses the headless HTMLUnit browser. Also, TestMaker supports the DPL and argument system for TestGen4Web tests.

<resources>
<testgen4web path="BrewBizWeb.testgen4web"/>
</resources>
….
<run name="testweb" testclass="BrewBizWeb.testgen4web" langtype="testgen4web">
<argument dpl="mydpl" name="DPL_Properties" value="getData"/>
</run>

soapUI

TestMaker operates soapUI tests.

<run name="BrewBizService" testclass="TestSuite" method="TestCase" langtype="soapui"/>

The test class is the name of the test suite. The method should be the test case of the suit, which will be run.
The parameters system in soapUI is a bit different. First, you need to have a properties step:

Inside the soapUI test, use Properties as usual. Then, in the scenario, you need to add a DPL argument. The argument name must be the same name of the properties step and the DPL should return a map object.

<run name="BrewBizService" testclass="TestSuite" method="TestCase" langtype="soapui">
<argument dpl="mydpl" name="DPL_Properties" value="getData"/>
</run>

The name that appears in the properties step will be search in the map, if there is and object different from null in the DPL map, the DPL map value will be set in the properties.

Goals

Modern software test automation tools need to meet these criteria:

1. 1. Test suites need to be intelligent. Testing Web application and soap-based Web Services software requires a test to intelligently drive the application as a typical user would.
2. 2. Test suites need to test systems for scalability and performance under near-production loads.
3. 3. Test tools need to be used. Many engineers do not have access to test tools due to cost or corporate standards.

TOOL is a free open-source design for a library of test objects. The TOOL design enables developers to build and run intelligent test agents. The test agents drive Web applications and soap-based Web Services as a typical user would. Multiple concurrent agents test an application for scalability and performance closely simulating production settings.

The design of TOOL should be language and implementation independent. TOOL does not need to be implemented in a language that explicitly supports OO structures (e.g., a pure C implementation - using OO techniques in C should be possible) independtly of using good Object Oriented design and architecture.

Design

TOOL is a library, which on it’s own wont accomplish anything, and can’t be used without a user interface. It is intended to be used in other projects that offer a user interface, so the audience for TOOL is software developers. TOOL offers software developers a flexible framework for building test suites to test Web systems.

Class Heirarchy

(or namespace or modules depending on what language is used)

Object Relationships

(Hopefully the pseudo-UML in the following section makes a sense to you.)

datasource:

- DataSource - An interface that needs to be implemented by DataSource

implementations to allow access to the data.

- Csv - Access data in a Comma Separated Values flat file.

- Tsv - Access data in a Tab Separated Values flat file.

- Db - Access information that is stored in a Database. If needed

by the language, make Database specific implementations of the

Db to access a specific database type.

Datasource <-- Csv

Tsv

Db <-- PostgreDb (if necessary)

OracleDb

protocolhandler:

- Protocol - Interface to implement to create a new Protocol type.

- Header - An interface to setting Header values for a protocol. Protocol

specific implementations will need to be provided.

- Body - An interface to setting the Body of a Protocol request. Protocol

specific implementations will need to be provided.

- ProtocolHandler - A facade in front of all of the Protocols that allows an

easy way to instantiate new objects (factory methods) and call methods on a

Protocol regardless of which specific type it is.

- *Protocol - The specific implementations of the different Protocol handlers.

Protocol <-- HTTPProtocol

soapProtocol

?

<>-- Header <-- HTTPHeader

IMAPHeader

?

Body <-- POP3Body

soapBody

?

ProtocolHandler <>-- Protocol

response:

- Response - Interface to implement to define a response that will be returned

by a specific protocol.
- ResponseLink - An interface that defines the access to Response items that

can be chained together with the ResponseChain to define how to deal with a

Response object. This is how searching responses and things of that sort

will be implemented.

- ResponseLinkConfig - An Object that is created to hold values used to

initialize a ResponseLink.

- *Response - The implementations of Protocol specific response types.

- ResponseChain - A Chain of Responsibility pattern class that allows a

response to be passed among many ResponseLinks.

Response <-- HTTPResponse

soapResponse

?

ResponseLink <-- HTTPSearchLink

?

<>-- ResponseConfig

ResponseChain <>-- ResponseLink

util:

- Codec - Interface to encoding/decoding methods.

- Crypt - Interface to cryptographic routines.

- Lingo - Utility that creates pseudo-text to use as subjects or bodies of

messages, form posts, etc.

- Queue - For languages where there is no Queue class, this provides a Queue

implementation.

- PropertyFinder - A class that hides the details of loading property files

that contain strings and other configuration directives and handles

dynamically changing the values of these.

Codec <-- Base64Codec

URLCodec

HexCodec

Crypt <-- ?

PropertyFinder

Lingo

Queue

JavaDoc API Documentation for TOOL

TOOL is written in JAVA and comes with JAVAdoc code documentation . JAVAdoc is the standard way JAVA engineers annotate their code. The JAVAdoc code documentation for TOOL is also found on-line at docs.pushtotest.com .

Additional documentation, product downloads and updates are at www.PushToTest.com . While the PushToTest testMaker software is distributed under an open-source license, the documenation remains (c) 2008 PushToTest. All rights reserved. PushToTest is a trademark of the PushToTest company.