Frequently Asked Questions

Frequently Asked Questions (FAQ) for PushToTest, TestMaker, TestNetwork

Product Information

Product Pricing, Costs, Support

Open Source (GPL) License

Why did PushToTest choose to license TestMaker under GPL version 2 since this has been called one of the most restrictive and viral open source licenses?
Does shipping TestMaker with our software product/platform mean that we need to adopt the GPL license too?
Does building a program against the open API in TestMaker mean that we will need to adopt the GPL license?
Does PushToTest support of the GPL eliminate our ability to negotiate an agreeable license to use, bundle, and OEM TestMaker?
Why does PushToTest use a dual license strategy?

Installation and Running

TestMaker Capabilities

Creating Tests

Technology Questions

Company Information

Everything Else

What is PushToTest?

PushToTest is a solutions provider to enable enterprises and institutions to monitor in real-time information services and provide immediate root-cause analysis and remediation for performance bottlenecks and downtime at development time, QA time, and runtime. PushToTest offers free open-source test software, free performance kits, and global services solutions that test, monitor and automate Web service systems for reliability, functionality, scalability and performance. The result is the right size datacenter, qualification of software patches and updates, and happy users.

What are the details behind the PushToTest company?

PushToTest was founded in 2001 by Frank Cohen. PushToTest is a privately held small business. PushToTest offices are in Campbell, California, Washington DC, San Jose, Costa Rica, and London UK. PushToTest is a small business with 135,000 users, 22 engineers, and 5 managers. PushToTest was founded by Frank Cohen.

As an open-source company what is the PushToTest business model?

PushToTest provides solutions that enable enterprises and instiutions to monitor in real-time information services and provide immediate analysis and remediation for performance bottlenecks and downtime. Our customers require solutions, that is different from requiring software code they can compile into a test tool. PushToTest provides solutions based on the amont of customer support needed, the size of the tests (concurrent virtual users, days of testing,) the number of service monitors, training classes needed, and customization to TestMaker. Contact sales@pushtotest.com for details.

What products does PushToTest offer?

PushToTest offers the TestMaker service governance and test automation platform. TestMaker is a platform for software developers, QA groups, and IT management to test, monitor, and govern information systems. At any given time new software needs to be installed, existing software modules and database software need to be patched, application software and databases need to be tuned and optimized, and the root-causes of crashes, downtime, and performance bottlenecks needs to be analyzed and remediatedquickly.

Software developers use PushToTest Version 5 to turn their unit tests into functional tests in a test automation platform that runs on their development machine. PushToTest Version 5 includes Wizards and Recorders to automatically build tests and supports a variety of languages, including Java, .NET, Jython, Groovy, PHP, Ruby, and many others. Plus PushToTest Version 5 supports SOA, Web Service, AJAX, and REST services using HTTP, HTTPS, SOAP, XML-RPC, and the email protocols.

The PushToTest Version 5 runtime environment automatically turns these same functional tests into load tests, scalability and performance tests, regression tests, and service monitors for QA technicians, IT operations managers, and CIOs. PushToTest Version 5 runtime load tests and service monitors integrate into Service Registry/Repository products and database and application performance optimization and root-cause analysis tools.

What product support is offered?

PushToTest offers a variety of free and for-pay support options to assist you with TestMaker and TestNetwork. These are described on the Support Page. We pride ourselves on building exceptional relationships with customers and partners. PushToTest offers professional services to help solve difficult enterprise problems.

How Much Technical Expertise Will I Need?

PushToTest TestMaker is an open-source end-to-end service governance and test automation platform. PushToTest provides many options to build test:

Using the included Wizards and Recorders to automatically build tests that requires no programming experience at all.
Using the Network Proxy Recorder to watch you use a browser or your application and writes a Jython script for you. This requires no programming effort but does require a familiarty with the Jython language.
Start from scratch and write a script or class that implements a JUnit TestCase in any of the supported languages. Plus, TestMaker delivers a library of protocol handlers called the Test Object Oriented Library (TOOL) of native SOA and Web service protocols, includingHTTP, HTTPS, SOAP, XML-RPC, and the email protocols.

What is the product pricing?

PushToTest distributes TestMaker under dual licenses:

The PushToTest TestMaker source code is licensed under the GPL version 2 license.
PushToTest users may download pre-build, tested, and ready-to-run TestMaker software that is licensed under a commercial license where the first 200 concurrent virtual users (CVUs) and 10 service monitors (CSMs) are free. PushToTest sells a license for additional CVUs and CSMs.

Please contact us by telephone at (408) 871-0122 (USA, California Pacific time) or by email at sales@pushtotest.com.

We are using another testing product. How does TestMaker compare?

PushToTest TestMaker was designed for Web, Service Oriented Architecture (SOA), and Web Service architectures. TestMaker has these architectural strengths:

Designed For IT Integration and Continuous Monitoring

Root cause analysis and remediation through integration with application and database analysis tools

Governance Platform Integration to manage tests as corporate assets using Registry/Repository

Automate test operation using Continuous Integration Environments

Reduces downtime by qualifying patches, updates, hardware changes
Designed for multiple forms of testing

Regression, Functional, Load, Scalability, Performance, Service Monitoring

Designed for SOA and Web services

Distributed test environment: vertical deployments for load and scalability tests, horizontal deployments to analyze users experiences on multiple networks/ISPs

Multiple protocols (HTTP, HTTPS, SOAP, REST, AJAX, email) and extensible to add more
Build tests in a variety of languages, including Java, .NET, Jython, Groovy, PHP, Ruby, and many other

Designed for multiple user roles

Software developers - runs on their development machines

QA technicians - Recorders and Wizards avoid coding and do not require deep technical skills

IT managers - operates tests as service monitors to prove SLAs with no extra effort

Proven deployments

Commercial licenses at GM, BEA, AMP, European Union, TIBCO, Raining Data
Consistent and regular product updates/upgrades since 2002
Large open-source user community - 130,000 users

Cost effective solutions

PushToTest Test and Remediation Services
PushToTest Global Services
Less expensive and more powerful than many commercial test tools

What companies provide references to PushToTest products and capabilities?

General Motors, European Union, Lockheed Martin, US Army, Sun, IBM, BEA, Microsoft, CapeClear, Software AG

Does TestMaker support complex data types in SOAP-based Web Services?

PushToTest TestMaker and TestNetwork provide full support for complex SOAP data types. See http://www-106.ibm.com/developerworks/webservices/library/ws-complex.html

Does TestMaker support HTTP, HTTPS and JMS protocols for Web Services?

PushToTest TestMaker provides direct support for making SOAP calls over HTTP, HTTPS and JMS.

What platforms does PushToTest TestMaker support?

PushToTest solutions are designed for Java, .NET, J2EE, Spring, Plain Old Java Object (POJO) and legacy systems that use open-standards protocols. PushToTest TestMaker is a 100% Java application and runs on any platform that supports Java.

Does TestMaker provide support for SOAP RPC, Document Literal and other Web Service encoding styles?

PushToTest TestMaker provides direct support for SOAP RPC, Document Literal, SOAP RPC Literal and many other Web Service encoding formats.The Test Object Oriented Library (TOOL) in TestMaker is an extensible library of protocol handlers. Support for other data types is easily added using TOOL APIs.

Is PushToTest technology extensible?

PushToTest TestMaker is implemented as an extensible SWING-based application written 100% in Java. The Test Object Oriented Library (TOOL) in TestMaker is an extensible library of protocol handlers. Support for other data types is easily added using TOOL APIs. Full source code and support are offered by PushToTest.

What is TestMaker?

PushToTest is a platform for software developers, QA groups, and IT management to test, monitor, and govern information systems. At any given time new software needs to be installed, existing software modules and database software need to be patched, application software and databases need to be tuned and optimized, and the root-cuases of crashes, downtime, and performance bottlenecks needs to be analyzed and remediated quickly.

Software developers use PushToTest Version 5 to turn their unit tests into functional tests in a test automation platform that runs on their development machine. PushToTest Version 5 includes Wizards and Recorders to automatically build tests and supports a variety of languages, including Java, .NET, Jython, Groovy, PHP, Ruby, and many others. Plus PushToTest Version 5 supports SOA, Web Service, AJAX, and REST services using HTTP, HTTPS, SOAP, XML-RPC, and the email protocols.

Click here to download TestMaker.

From Freshmeat.net: PushToTest TestMaker is a platform for real-time monitoring and governance of information systems. Software developers use TestMaker turn their unit tests into functional tests that run on their development machine. TestMaker includes Wizards and Recorders to automatically build tests and supports a variety of languages to build tests, including Java, .NET, Jython, Groovy, PHP, Ruby, and many others. Plus TestMaker supports SOA, Web Service, AJAX, and REST services using HTTP, HTTPS, SOAP, XML-RPC, and the email protocols. The TestMaker test runtime environment automatically turns these same functional tests into load tests, scalability and performance tests, regression tests, and service monitors for QA technicians, IT operations managers, and CIOs.

What is TestNetwork?

TestNetwork enables businesses to automate system tests to check for correctly operating functions, to handle increasing user loads, and continuously monitor systems to proof service levels. TestNetwork builds on TestMaker by delivering these features:

Run agents in greater scale than on a single machine.

For example, on a single 1 GHz Pentium system TestMaker runs up to 150 concurrent test agents. With TestNetwork running 10,000 or more concurrent test agents is possible.

Run test agents on multiple remote machines.

TestNetwork turns remote systems into TestNodes that remotely run test agents and report the results back to a central console.
TestNetwork's console/TestNode architecture turns test agents into mini-servers that are able to handle operations autonomously.

TestNetwork's console/TestNode architecture turns test agents into mini-servers that are able to handle operations autonomously.

What is PushToTest TestMaker Monitor?

PushToTest TestMaker Monitor (PTTMonitor) is an agent-based utility to monitor CPU, network, memory, and application server statistics (threads, pools, objects, call lists.) Statistics gathered from the monitor provide a root-cause analysis function in the charts and dashboards PushToTest provides by correlating performance statistics with resource usage.

What is required to use and run TestMaker?

TestMaker comes with everything needed to build and run intelligent test agents. TestMaker requires a Java 1.6 or higher virtual machine.

Does PushToTest test Windows desktop applications?

PushToTest is an open-source test automation platform for testing Web applications, Ajax, Web service, and Service Oriented Architecture (SOA) services. We use PushToTest to test Java desktop applications for functionality. We are investigating technology to test Windows desktop applications. Information on one promising technology is at:
http://www.pushtotest.com/thecohenblog/functional-testing-on-windows

How do I install TestMaker?

Instructions on installation are found in the Installation Instructions page.

How do I get TestMaker?

TestMaker is available for free download from the PushToTest Web site and through anonymous FTP download. See the downloads page on the PushToTest web site to download TestMaker using a browser.

For FTP download:

ftp.pushtotest.com

user name: anonymous

user password: enter-your-email-address

The file TestMaker.zip (capitalization is important) is always the latest release of TestMaker.

How do I run TestMaker?

TestMaker comes with start-up scripts for Windows and Unix systems, including Linux, Mac OS X and Solaris. To start TestMaker on Windows, run the testmaker_home/TestMaker.bat script; for Unix, run the testmaker_home/TestMaker.sh script. These scripts are simple shell scripts that build a Java classpath containing the libraries TestMaker comes with, and executes Java telling it the TestMaker object to run. Additional instructions and a tutorial on using TestMaker are found in the Tutorial.

What are the most common installation problems?

TestMaker normally requires very little configuration to install and run. The TestMaker distribution comes with an intstaller application. The installer requires Java 1.6 or higher runtime environment. Sun provides the Java runtime environment at java.net. You may install either the Java Runtime Environment (JRE) or the Java Software Development Kit (JSDK) depending on your intended use of TestMaker. To install TestMaker run or double-click one of these installers:

PushToTest_TestMaker_Install.jar - this installs the PushToTest TestMaker environment. TestMaker is the development environment to build tests and the central console to operate tests. By default this also installs a local TestNode to operate tests from your local machine.
TestNode_Install.jar - this installs a TestNode, the remote environment that receives commands from TestMaker and runs a test.
Monitor_Install.jar - this installs the PushToTest Monitor to observe CPU, network, and memory utilization as tests operate.

These installers create a directory with the TestMaker files. In this directory you will find the TestMaker.bat (for Windows) and TestMaker.sh (for Unix machines, including Linux, Mac OS X, and Solaris) launcher scripts. The script builds a Java classpath, executes Java, and tells Java to run the TestMaker application.

The launcher script and many other Java utilities depend on a system environment variable named JAVA_HOME to points to the Java installation. For UNIX machines set the JAVA_HOME variable in the.cshrc, .bashrc, or other shell configuration file. For Windows systems set the JAVA_HOME by following these instructions:

Click Start->Control Panel->System. Then click Advanced, then click "Environment Variables".
Under System variables, click "New". For Variable Name, use JAVA_HOME (exactly like that, case and all). For Variable Value, use the path to your Java installation. For example, c:\jsdk1.6. Do not set the path to the bin directory within the Java installation. Click OK until the Control Panel is closed.
Verify the setting. Click Start->Run, and in the command box type "cmd" and click OK. At the command prompt, type echo %JAVA_HOME% and verify that the value returned matches the Variable Value from step 2.

For Mac OS X 10.4 or greater set the JAVA_HOME by following these instructions:

Use the Terminal applicat to open a new shell window (/Applications/Utilities/terminal).
Type vi .profile and return
Enter the following:
JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home
export JAVA_HOME
Type :w and return
Type :q and return

What issues, bugs, and problems exist in TestMaker?

While earlier versions of Java introduced problems with video drivers and memory management, Java 1.5 and later seems to have solved these and PushToTest knows of no Java related problems. Check http://bugs.pushtotest.com for latest problem and issue reports.

Can TestMaker stress test a software product with "real world" usage, including ftp transfers, telnet sesions, web surfing, windows NETBIOS file transfers, everything that a user would ever do?

As far as we know there is no single product that will conduct stress tests in all of these methods. Even the expensive commercial products would have a hard time doing it all. PushToTest TestMaker is a framework and utility for building such a test. TestMaker's integrated Test Object Oriented Library (TOOL) is an extensible API that provides protocol handlers with which to write intelligent test agent scripts. So, TOOL can certainly be extended to provide support for ftp, telnet, and NETBIOS. Under TestMaker's open-source license you can do the work yourself, or you can contract to PushToTest Global Services to extend TestMaker for you.

Will TestMaker work for Java Client/Server applications as well as Web applications?

Yes, TestMaker is appropriate for testing Java Client/Server applications. For example, Sun Microsystems engaged PushToTest to use TestMaker to test its bug tracking system. Sun deploys a Java application to provide a Swing-based user interface. The application makes SOAP calls to a middle-tier that then makes calls to a Siebel host, Oracle database, and LDAP server. TestMaker drives the bug tracking system by making calls directly to the command objects in the Java application. By varying the number of concurrent requests and implementing the behavior of archetypal users, the TestMaker system can report on the functionality, scalability, and performance of the bug tracking system.

Can I run TestMaker agents from the command-line or shell?

TestMaker function tests, load tests, and service monitors may be run from the command-shell. See the TestMaker_home/TestMaker.sh and TestMaker_home/TestMaker.bat scripts for details.

Does TestMaker allow me to test applications that use WS-Security Specifications (specifically the Username Token Profile)?

TestMaker 4 offers no support for WS-Security directly at this time. However, because TestMaker supports Java and the Jython scripting language you can easily import and use an Java library that implements the security library and profile you need.

Can TestMaker analyze a WSDL file and generate the necessary test objects?

TestMaker comes with the Eviware soapUI utility to analyze a WSDL file and visually create a TestSuite. Use the Tools menu, Start soapUI command to start soapUI from within TestMaker.

Does TestMaker do full load testing and analysis of results?

TestMaker operates unit tests as load tests. TestMaker provides a scalability index chart showing transactions per second (TPS), a distribution chart showing response times during the test, and a monitor of CPU, network, and memory utilization as the test operates. TestMaker logs all transactional data to a set of XML-formatted documents for import into other results anlalysis products.

Does TestMaker support the testing of simultaneous sessions from one and multiple IPs?

TestMaker is architected to operate tests under multiple IP addresses using the distributed TestNetwork environment.

Does TestMaker support peer-to-peer communications protocols such as JXTA?

TestMaker does not offer any specific P2P protocol handlers (such as JXTA.) However, TestMaker's Test Object Oriented Library (TOOL) was build around an extensible architecture. That makes it easy to extend TOOL to provide a JXTA protocol handler. PushToTest can provide you with this or support your own effort.

Will any of the work here become part of a commerical product?

TestMaker is distributed under a dual license. The PushToTest TestMaker source code is licensed under the GPL version 2 license. PushToTest users may download pre-build, tested, and ready-to-run TestMaker software that is licensed under a commercial license where the first 200 concurrent virtual users (CVUs) and 10 service monitors (CSMs) are free. PushToTest sells a license for additional CVUs and CSMs.

What is the latest update to TestMaker?

TestMaker's home is at the PushToTest Web site. The download page contains the most up-to-date version of TestMaker. TestMaker is usually updated few months to incorporate bug fixes, documentation updates and new features. Between releases the modules, object libraries and working-in-progress code is available at the PushToTest CVS server.

How do I find bug fixes and work-in-progress code between releases of TestMaker?

TestMaker packages bug fixes, documentation updates and new features into binary distributions that are ready-to-run. Between releases the modules, object libraries and working-in-progress code is available at the PushToTest CVS server. CVS is a version control system that enables developers to work on TestMaker as a team.

The PushToTest CVS server allows anonymous download of the TestMaker code. Updates and patches should be sent to Frank Cohen, principal maintainer for TestMaker, for consideration. With enough support Frank may choose to give you CVS commit privileges. Current there are 4 committers.

From Unix systems:

$export CVSROOT=:pserver:anon@cvs.pushtotest.com:/var/cvsroot

From Windows we recommend using a CVS client:

C:> set CVSROOT=:pserver:anon@cvs.pushtotest.com:/var/cvsroot

In either case, the checkouts are:

$ cvs login
(Logging in to anonymous@cvs.pushtotest.com)
CVS password: <ENTER>
$  cvs co tm5
cvs server: Updating tm5
...

The tm5 module includes:

TestMaker - The stand-alone Java application code for TestMaker

tool - the Test Object Oriented Library (TOOL) provides protocol handler objects

j-0-1.21-tmbranch - The TestMaker branch of the J text editor

maxq-tmbranch - The TestMaker brand of the MaxQ Proxy Recorder

TestNetwork - The distributed test environment of TestNodes

To receive email announcements when new TestMaker code is changed in cvs subscribe to the cvscommits email list. Discussion among the TestMaker developers is on dev@lists.pushtotest.com. This is a good place to post questions and suggest new features.

What is SOAP? What are Web Services?

Modern systems today use technology that enables software applications to communicate over private intranets and the Internet. The protocols (HTTP, HTTPS, SOAP, XML-RPC) are accepted standards and supported by the entire computer industry. We call these systems Web Services. Enterprise adoption of Web services has sparked an explosion of new software projects, all of which have interoperability and connectivity requirements. A Web browser may retrieve a Web page, a pager may receive an email, a portal may collect data from many services to present a single Web page, and many systems located in various places may be integrated into a productive system.

Server software needs to exchange data with other server software. Past attempts for server data exchange include Corba, DCOM, RMI, EDI and and host of other protocols. SOAP is an emerging industry standard that has a better chance than previous attempts. SOAP is to servers what HTTP and HTML are to the Web, a lightweight means to communicate.

Here is the SOAP definition from the draft W3C specification: SOAP is a lightweight protocol for exchange of information in a decentralized, distributed environment. It is an XML based protocol that consists of three parts: an envelope that defines a framework for describing what is in a message and how to process it, a set of encoding rules for expressing instances of application-defined datatypes, and a convention for representing remote procedure calls and responses.

The SOAP specification is at: http://www.w3.org/TR/SOAP/

How may I extend TestMaker to work with custom SOAP data types?

SOAP uses serializer and deserializer objects to convert data in XML-encoded request and response documents into Java objects that your objects may use. TestMaker uses the Apache SOAP library to make SOAP calls. Detailed instructions are found in the IBM developerWorks article titled Complex Data Types and Axis, Apache Soap and TestMaker by Frank Cohen. Additionally, we recomment checking the Apache SOAP Web site for details on adding custom SOAP data types to TestMaker. Also, the docs.pushtotest.com site has documentation on implementing custom serializers.

What is the PushToTest web site privacy policy?

Click here for the PushToTest privacy policy.

Where can I learn more about Web Services and TestMaker?

docs.pushtotest.com is your single source for additional information, articles, books and news on TestMaker and Web Services.

Frank Cohen, principal maintainer for TestMaker, is author of FastSOA (Morgan Kaufmann 2006,) Java Testing and Design: From Unit Tests to Automated Web Tests (Prentice Hall 2004.) The book covers many of the experiences Frank has had in designing and testing Web Services in many varied environments using TestMaker.

Many articles on PushToTest technology and products have appeared on Web sites, in magazines and journals. These links take you to articles that teach more about Web Services and TestMaker:

Performance testing SOAP-based applications

Myths and Misunderstandings surrounding SOAP

Using Web services for e-Commerce single sign-in

IBM's article on PushToTest and Load.

IBM jStart Web Services Program

Additionally, IBM, Sun, BEA and CapeClear have partnered with PushToTest because we each have a common goal to establish the Web Services standard. Each company offers tutorials, documentation, software and utilities at little or no cost.

I teach Java, Testing and Web Services, may I use TestMaker as a teaching tool?

TestMaker makes an ideal environment for students learning new technologies, including Java, software test methodologies and Web Services. TestMaker includes example test agents which may be used to form a course featuring practical examples of new technologies in action. The docs.pushtotest.com site offers many free downloads which may help you construct a course. Frank Cohen is available to speak at conferences, classes and lectures.

What documentation exists for TestMaker?

TestMaker comes with a a User Guide, Tutorial, Sample Agents and Javadoc documentation of the test objects. You will find all of these in the docs directory that is installed with TestMaker. The docs.pushtotest.com Web site is home to these same documents and more.

What is JDOM and how can I use it to work with XML data?

SOAP-based Web Services use XML-encoded request and response data. TestMaker comes with JDOM, a Java framework for working with XML data. The JDOM objects and methods are accessed through the TestMaker scripting language. Look at the Book_price.a test agent for examples of how to use JDOM methods to parse and validate XML-encoded request and response data.

Chapter 6 of Java Testing and Design: From Unit Tests to Automated Web Tests covers SOAP and Web Services in detail, with lots of example TestMaker scripts.

Can TestMaker automatically build test agents for Web pages that use JavaScript?

PushToTest TestMaker includes the SpikeSource TestGen4Web recorder as a FireFox add-on that watches you operate a Web browser-based application and creates a unit test automatically. These unit tests support dynamic Web pages, JavaScript, and Ajax.

Can TestMaker record my interaction with a Web site and build an agent automatically?

Can TestMaker record my interaction with a secure Web site using HTTPS and SSL?

How do I record a test agent script?

PushToTest TestMaker provides two ways to watch you operate a browser-based Web application and "record" a unit test script.

FireFox and TestGen4Web - A FireFox addon that adds a TestGen4Web control bar to the top of Firefox. Use the control bar to start, stop, and edit functional tests of a Web application. Save the recorded test to a TestGen4Web file in XML format that may be payed-back in the TestMaker environment. TestGen4Web supports HTTP, HTTPS, and Ajax applications. See the tutorial for step-by-step examples of using TestGen4Web and TestMaker.
MaxQ Proxy Recorder - A network proxy recorder that watches the network communication between your use of a Web browser and the target Web application server and creates a unit test script automatically. The Recorder outputs a Jython script to implement a JUnit TestCase test class. The Recorder supports HTTP and Applets, but not HTTPS and Ajax applications. See the tutorial for a step-by-step example of using the Recorder.

Does TestMaker support testing secure Web Services?

TestMaker and Load use the JSSE library from Sun to implement HTTPS connections over SSL protocols to the server. JSSE has certificate handlers for Verisign and Thawte certificates by default. The keytool utility that comes with JSSE is used to install additional certificates. Details on using keytool are on the TestMaker Security wiki page. Additional information on keytool is available on the java.sun.com site.

How do I use the Network Monitor to watch network traffic?

Follow these steps to use the Network Monitor to watch network traffic:

Open TestMaker
Choose Tools -> Network Monitor -> Admin tab
Enter 8091 as the port number and make certain Proxy is selected.
Click Add button
Click the Port 8091 tab
The TOOL ProtocolHandlers for HTTP, HTTPS and SOAP have proxy setting methods. Additionally, the setProxy() method in agentbase.py for TestMaker 4.4 sets the proxy values.
When you run a script configured in this way the Network Monitor, Port 8091 tab shows the network traffic the script creates.

Does TestMaker support MIME encoded email file attachments?

PushToTest TestMaker supports MIME encoding for email tests. Email messages use MIME encoding to attach files, icons, and binary data to a message body. The Mail protocol handler supports MIME encoding too. The agent script below illustrates how to send an email message with a file attachment: Click here for details on the Mail protocol handler.

How can I use my Java objects from within a test agent script?

TestMaker comes with the Jython scripting language. Jython is a 100% Java implementation of the popular Python language. Jython's unique design gives you access to all of Python's objects and any Java object that is on the TestMaker classpath. This means you may use your own custom Java objects from within a TestMaker test agent. To use your Java objects in TestMaker 4.x, follow these steps:

1) Create the following Java class file.

package com.pushtotest.myapplication;

public class myApp
{
public String getName()
{
return "Frank";
}
}

2) Compile the class into myApp.class and put it into a myApp.jar file.

3) With TestMaker 4.0, modify testmaker_home/TestMaker.bat (for Windows) or testmaker_home/TestMaker.sh (for Unix/Linux) to add myApp.jar to the classpath.

4) Create a new TestMaker script as follows:

from com.pushtotest.myapplication import myApp
a = myApp()
print "myApp says: Thank you", a.getName(), \
", I will buy a support agreement."

The output window will display:

myApp says: Thank you, Frank, I will buy a support agreement.

Additional information on using Java objects from within a test agent script is found at http://www.jython.org/docs/usejava.html.

How do I use TestMaker from behind a proxy server?

Proxy servers provide an IT team with a way to manage network traffic.A proxy server is a router that forwards network traffic to a host. If your network uses a proxy server chances are TestMaker will give you errors when you try to run a test agent script or use the New Agent Wizard to record a script. You may see errors like these:

Encountered an error: java.net.NoRouteToHostException: No route to host: connect
Encountered an error: java.net.ConnectException: Connection timed out: connect

To use a proxy server from a TestMaker test agent script use setProxyHost, setProxyPort, setProxyUser, setProxyPass methods of the protocol handler. Below is an example script showing an HTTPProtocol handler using a proxy server:

protocol = ProtocolHandler.getProtocol("https")
protocol.setProxyHost( "localhost" )
protocol.setProxyPort( 9001 )
protocol.setProxyUser( "frank" ) #optional
protocol.setProxyPass( "password" ) #optional

Many times the proxy server does not require a user or password. In this case, setProxyUser and setProxyPass are optional.

The TestMaker New Agent Wizard supports proxy servers too. When using the New Agent Wizard to record a test agent script, set the proxy information for the recorder by clicking the Help drop-down menu and choosing the Preferences command. Click on the Recorder tab. Enter the values for your proxy server.

How do I use the network monitor to sniff network traffic?

Software developers, QA technicians, and IT managers often need to see the HTTP, SOAP, and XML-RPC messages being used to invoke Web applications and web services along with the results of those messages. The network monitor provides a way to monitor the messages being used.To start the network monitor use the Tools -> Network Monitor command in TestMaker. The network monitor window will appear and the administrative tab will appear by default. Click here for directions on using the network monitor.

How can I use TestMaker on a Macintosh computer running Mac OS X?

To launch PushToTest TestMaker on a Macintosh computer running Mac OS X version 10.4 or greater follow these instructions:

1) Make certain your computer has Java version 1.6 or greater. To do so open a Terminal shell window from the Applications directory. Type java -version and press enter. The Java version number will appear. If you have an older version of Java use the Software Update feature found in the System Preferences icon on the Dock to download the new Java runtime environment. Details on the Java virtual machine provided by Apple Computer are found at http://developer.apple.com/java/

2) From a Terminal shell window change the current directory to the location of the TestMaker directory. Type ./TestMaker.sh and press the enter key.

We are unaware of Java runtime environments for Macintosh operating systems prior to Mac OS version 10.4. If you figure out how to run on Mac OS 9.2, for example, then please let us know by sending email to info@pushtotest.com.

What is J and how do I edit TestMaker test agent scripts?

J is an open-source text editor. TestMaker embeds J as a test agent script editor. J offers many features and functions that are described in the J User's Guide.

Does TestMaker support testing applets?

Typically applets are used for presentation and communication with hosts in some fashion that is not easily possible using HTML pages in a browser. TestMaker does not provide special functions to test an applets ability to present data. However, TestMaker does fully support all of the major Web protocols (HTTP, SOAP, XML-RPC, SMTP, etc.) an applet would use to request something from a host. Additionally, TestMaker's Recorder feature automatically detects an applet's HTTP communication with a host and writes a TestMaker intelligent test agent script that emulates the request from the applet to a host.

Does PushToTest TestMaker support multiple Web (HTTP) user sessions through cookies?

PushToTest TestMaker 5 includes the TestGen4Web plug-in to Firefox to record a functional unit test of a Web application. The TestMaker TestScenario documents define the orchestration of a functional test, load and performance test, and service monitor. For a load test you identify the number of concurrent running users (CRs) and PushToTest operates the TestGen4Web functional test concurrently at that level of users. Each user runs in its own thread concurrently and each user has its own set of cookies. To the application under test, it appears that a number of individual users are operating the application independently.

What books do you recommend to teach me about building Web-enabled applications?

PushToTest recommends the following books:

Java Testing and Design: From Unit Tests to Automated Web Tests by Frank Cohen (Prentice Hall)
Lessons Learned in Software Testing, Cem Kaner, James Bach, Bret Pettichord, (Wiley Computer Publishing)

If you know of additional books, please let us know by sending email to info@pushtotest.com and we will list them here.

Does TestMaker support batch execution of scripts using the console or otherwise?

PushToTest tests may be run from the command-line. This enables you to schedule scripts using external tools and "cron" jobs.

We do not currently have a scheduler built into TestMaker and have one on the product road map.

PushToTest comes with the PushToTest As A Service (PAAS) interface to SOA registry/repository products. For instance, SoftwareAG X-Registry may start a PushToTest test scenario as the result of a service policy being triggered and then receive the results and transactional statistics in a Governance Statistics Result Set (GSRS) XML response that is stored as metadata in the repository.

Does TestMaker support outputting test results as HTML Web pages?

PushToTest test scenario output is in a set of charts (Scalability index, transaction distribution, and resource utilization for CPU, Network, and memory use during the test) that are output in PNG format, the statistical data is output in XML format, and the transactional data is output in XML format.

The logging capability outputs to HTML format as well and is customizable by you or PushToTest's global services consultants.

Are TestMaker scripts human readable using a browser, the notepad, or text editor?

PushToTest TestMaker test scripts implement a JUnit TestCase and then are orchestrated as functional tests, load tests, and service monitors using a TestScenario XML document. TestMaker supports JUnit TestCases implemented in Java, Jython, JRuby, Rhino (JavaScript,) PHP, and several other dynamic scripting languages. All TestMaker TestScenario scripts are human readable using a browser, the notepad, or the text editor.

Are TestMaker tests composed of individual files that may be checked-into a configuration management system?

PushToTest TestMaker tests are composed of individual files that may be checked into a configuration management system as code.

Can TestMaker tests check SOAP messages for mandatory custom headers?

PushToTest TestMaker provides the Test Object Oriented Library (TOOL) that delivers SOAP and other protocol handlers. TOOL provides APIs to check SOAP messages for mandatory custom headers.

Does TestMaker support testing services available over HTTP, HTTPS, and JMS?

PushToTest TestMaker provides the Test Object Oriented Library (TOOL) that delivers HTTP, HTTPS, XML-RPC, SOAP, IMAP, SMTP, POP, SIP and other protocol handlers. A JMS driver for TOOL is planned.

Does TestMaker support Digital Security Certificates, including x509 and 2-way SSL authentication?

PushToTest TestMaker is build on the Java platform and supports HTTP Bound 1-way and 2-way digital certificates through the Jave Security (JSSE) library, including x509 XML encrypting support and 2-way SSL authentication.

Can TestMaker use data from Excel, databases, and CSV files as input to tests?

PushToTest TestMaker provides Data Production Libraries (DPL) for CSV, RDBMS, and custom DPLs implemented by any developer. See the TestMaker_home/examples/dpl directory for examples.

What support does TestMaker have for building tests of SOAP-based Web services?

PushToTest TestMaker integrates the Eviware soapUI utility to visually create tests. You provide a WSDL definition, soapUI visually displays a skeleton of the SOAP request document, you fill it in with data, and the utility shows you the SOAP response from the service. This requires no programming experience.

What build tools like ANT and MAVEN are supported?

PushToTest supports ANT and MAVEN. An ANT build script TestMaker operates a TestMaker test from a custom ANT task. PushToTest TestMaker supports MAVEN through a command-line call.

Can TestMaker be configured to produce HTML run reports automatically during test execution without requiring manual intervention?

PushToTest TestMaker has an extensible Log Handler to perform results reporting automation. The typical configuration effort requires the user to define the TestScenario parameters in an XML document, build a unit test in the language of your choice, and then call PushToTest from the graphical console, the command-line, or as a PushToTest As A Service call.

What format are the test scripts - XML, C, Java?

Does TestMaker offer a Web interface to allow people to review tests?

PushToTest has plans to provide a Web interface to allow people to review, orchestrate, and operate tests. This will appear in an upcoming version of TestMaker. To enable this, PushToTest TestMaker is architected to support deployment as a service. For example, we now offer integration with Software AG's X-Registry. PushToTest runs as a Web service, receiving a PushToTest As A Service (PAAS) request and providing a Governance Statistics Result Set (GSRS) response. This will be the basis for offering a Web interface to operate tests.

How may I use TestMaker to validate a response?

PushToTest TestMaker provides a protocol handler library called the Test Object Oriented Library (TOOL.) TOOL provides a SOAP protocol handler, including APIs to set custom headers and methods to validate response headers. PushToTest TestMaker provides multiple XML parsers, including an XPath and an XQuery engine, to validate results. (And also to compare XML parser performance.) PushToTest TestMaker provides Data Production Libraries (DPL) to work with data stored in CSV, RDBMS, and custom DPLs implemented by any developer to validate response data.

Where may I find TestMaker screen shots, sample test reports, demonstrations and screencasts?

Find screen shots, sample test reports, demonstrations, and screencasts on TestMaker in the Tutorial

Why did PushToTest choose to license TestMaker under GPL version 2 since this has been called one of the most restrictive and viral open source licenses?

PushToTest began offering TestMaker under a free open-source license in 2001. We find there are generally two types of users: those who want to benefit from TestMaker immediately and do not have the technical skills or just don't want to build TestMaker from its sources, and those who have improvements to solve problems and generally make TestMaker a better product. To satisfy both types of users PushToTest licenses TestMaker under two licenses.

a) For those that want to use TestMaker immediately, PushToTest distributes the pre-built and ready-to-run TestMaker under a commercial license. TestMaker comes with an installer program and takes a few minutes to get up and running. The commercial license lets TestMaker users operate tests up to 200 concurrent virtual users (CVUs) and 10 concurrent service monitors (CSMs.) Additional CVUs and CSMs are available for license from PushToTest.

b) For those that want to learn the inner-workings and offer improvements to TestMaker, PushToTest distributes TestMaker under the GPL version 2 license. The GPL license provides our community of users and partners with a license that lets them receive the source code, fix bugs, and contribute improvements. PushToTest's goal is to make certain that any beneficial changes to TestMaker are made available back to the users.

Does shipping TestMaker with our software product/platform mean that we need to adopt the GPL license too?

No. Shipping the pre-built and ready-to-run TestMaker with your software product/platform under the commercial license costs your company nothing and does not require your company to adopt the GPL license too.

Does building a program against the open API in TestMaker mean that we will need to adopt the GPL license?

No. The GPL version 2 license does not require you to adopt the GPL license for your programs that call the TestMaker API. If your software project is a derivative of TestMaker source code then the GPL rules say that you must publish your source code for the benefit of the user community. Writing your software to interact with the TestMaker API (service interface) means there is no requirement for your company to adopt the GPL license.

Does PushToTest support of the GPL eliminate our ability to negotiate an agreeable license to use, bundle, and OEM TestMaker?

PushToTest provides dual licenses (Commercial license for the pre-built and ready-to-run TestMaker and GPL for the TestMaker source code) to enable you to negotiate an agreeable license to use, bundle, and OEM TestMaker with your product. Please contact info@pushtotest.com for details.

Why does PushToTest use a dual license strategy?

PushToTest uses a dual license strategy (Commercial license for the pre-built and ready-to-run TestMaker and GPL for the TestMaker source code) to allow technical users to have access to the source code for the reasons of improving TestMaker through bug fixes and enhancements and business and organizations to immediately benefit from PushToTest services around TestMaker, including commercial licenses to run larger tests and more service monitors, receive technical support, and training.

My question wasn't answered here. How do I get it answered?

There are many ways to get your question answered. Send us an email message at info@pushtotest.com, call us at (408) 871-0122 (California USA time,) or use the Ask A Question form on the PushToTest Web site.

Sections

Personal tools