Archived PushToTest site

Frequently Asked Questions About PushToTest

Frequently Asked Questions

What is PushToTest?

PushToTest is a solution provider to enable enterprises and institutions to monitor in real-time information services and provide immediate root-cause analysis and remediation for performance bottlenecks and downtime at development, QA and run time. PushToTest offers free open-source test software, free performance kits and global services solutions that test, monitor and automate Web service systems for reliability, functionality, scalability and performance. The result is the right size datacenter, qualification of software patches and updates and efficient system users.

As an open-source company what is the PushToTest business model?

PushToTest delivers TestMaker in three ways:

  • TestMaker Community is for the individual tester needing a no-cost test tool to test Web and Rich Internet Applications (RIA) for functionality, load and performance, and production monitoring. It  comes with free use up to 50 virtual users in a performance test and 10 business service monitors.
  • TestMaker Enterprise is for organizations running large scale business-critical Web and Rich Internet Applications (RIA, using Ajax, Flash, Flex,) Service Oriented Architecture (SOA,) and Business Process Management (BPM) and need world-class tools, methodology, training, services and support. PushToTest sells licenses to use TestMaker Enterprise.
  • Download the source code, build it yourself, support it yourself, and participate in the community for support. The source is licensed under a GPL version 2 open-source license. Building TestMaker from source requires expert knowledge in Java, Ant, NetBeans, XML, Web services, and testing. Find the source code at http://www.pushtotest.com/community.html

PushToTest provides Global Services to get you started with PushToTest quickly, efficiently, and inexpensively.

What products does PushToTest offer?

PushToTest offers the TestMaker service governance and test automation platform. TestMaker is a platform for software developers, QA groups and IT management to test, monitor and govern information systems.

What companies provide references to PushToTest products and capabilities?

PepsiCo, Best Buy, Deloitte, Measured Progress, and Bell Aliant.

Does TestMaker support complex data types in soap-based Web Services?

PushToTest TestMaker provides full support for complex soap data types. TestMaker comes with a ScriptRunner for soapUI and soapUI Pro test suites.

See http://www-106.ibm.com/developerworks/webservices/library/ws-complex.html

Does TestMaker support HTTP, HTTPS and JMS protocols for Web Services?

PushToTest TestMaker provides direct support for making soap calls over HTTP, HTTPS and JMS. See our SOA Knowledge Kit which includes a fully implement test of a system using HTTP, HTTPS, and JMS protocols for Web services.

What platforms does PushToTest TestMaker support?

PushToTest solutions are designed for Java, .NET, J2EE, Spring, Plain Old Java Object (POJO) and legacy systems that use open-standard protocols. PushToTest TestMaker is a 100% Java application and runs on any platform that supports Java.

Does TestMaker provide support for soap RPC, Document Literal and other Web Service encoding styles?

PushToTest TestMaker provides direct support for soap RPC, Document Literal, soap RPC Literal and many other Web Service encoding formats.The Test Object Oriented Library (TOOL) in TestMaker is an extensible library of protocol handlers. Support for other data types is easily added using TOOL APIs.

Is PushToTest TestMaker technology extensible?

PushToTest TestMaker is implemented as an extensible SWING-based application written 100% in Java. The Test Object Oriented Library (TOOL) in TestMaker is an extensible library of protocol handlers. Support for other data types is easily added using TOOL APIs. Full source code and support are offered by PushToTest.

What is TestMaker?

PushToTest is a platform for software developers, QA groups and IT management to test, monitor and govern information systems. At any given time new software needs to be installed, existing software modules and database software need to be patched, application software and databases need to be tuned and optimized and the root-cuases of crashes, downtime and performance bottlenecks needs to be analyzed and remediated quickly.

Software developers use PushToTest to turn their unit tests into functional tests in a test automation platform that runs on their development machine. PushToTest includes Wizards and Recorders to automatically build tests and supports a variety of languages, including Java, .NET, Jython, Groovy, PHP, Ruby and many others. Plus PushToTest supports SOA, Web Service, Ajax and REST services using HTTP, HTTPS, soap, XML-RPC and email protocols.

The PushToTest runtime environment automatically turns these same functional tests into load tests, scalability and performance tests, regression tests and service monitors for QA technicians, IT operations managers and CIOs. PushToTest runtime load tests and service monitors integrate into Service Registry / Repository products and database and application performance optimization and root-cause analysis tools.

What is PushToTest TestMaker Monitor?

PushToTest TestMaker Monitor (PTTMonitor) is an agent-based utility to monitor CPU, network, memory and application server statistics (threads, pools, objects, call lists). Statistics gathered from the monitor provide a root-cause analysis function in the charts and dashboards PushToTest provides by correlating performance statistics with resource usage.

What is required to use and run TestMaker?

TestMaker comes with everything needed to build and run functional tests, load and performance tests, and business service monitors out of the box.

How do I run TestMaker?

TestMaker comes with start-up scripts for Windows and Mac OS X environments.

What are the most common installation problems?

Read the Installation Guide in the TestMaker User Guide for installation problems and mitigation.

What issues, bugs and problems exist in TestMaker?

Check http://bugs.pushtotest.com for the latest problem and issue reports.

Can TestMaker stress test a software product with "real world" usage, including ftp transfers, telnet sessions, web surfing, windows NETBIOS file transfers, everything that a user would ever do?

PushToTest TestMaker is a framework and utility for building such a test. TestMaker's integrated Test Object Oriented Library (TOOL) is an extensible API that provides protocol handlers with which to write intelligent test agent scripts. So, TOOL can certainly be extended to provide support for ftp, telnet and NETBIOS. Under TestMaker's open-source license you can do the work yourself, or you can contract with PushToTest Global Services to extend TestMaker for you.

Can I run TestMaker agents from the command-line or shell?

TestMaker function tests, load tests and service monitors may be run from the command-shell. Continuous Integration (CI) environments like Hudson, Cruise Control, Jenkins, and Bamboo integrate TestMaker through a service call.

Can TestMaker analyze a WSDL file and generate the necessary test objects?

TestMaker comes with the soapUI utility to analyze a WSDL file and visually create a TestSuite.

Does TestMaker do full load testing and analysis of results?

TestMaker operates unit tests as load tests. TestMaker provides a scalability index chart showing transactions per second (TPS), a distribution chart showing response times during the test and a monitor of CPU, network and memory utilization as the test operates. TestMaker logs all transactional data to a set of XML-formatted documents for import into other results analysis products.

Does TestMaker support the testing of simultaneous sessions from one and multiple IPs?

TestMaker is architected to operate tests under single and multiple IP addresses using the distributed TestNode environment.

Will any of the work here become part of a commerical product?

TestMaker is distributed under a dual license. PushToTest TestMaker source code is licensed under the GNU General Public License. PushToTest users may download pre-built, tested and ready-to-run TestMaker software that is licensed under a Commercial License where the first 50 concurrent virtual users (CVUs) and 10 service monitors (CSMs) are free. PushToTest sells a license for additional CVUs and CSMs.

How do I find bug fixes and work-in-progress code between releases of TestMaker?

TestMaker packages bug fixes, documentation updates and new features into binary distributions that are ready-to-run. Between releases the modules, object libraries and work-in-progress code is available at a Subversion (svn) repository. svn is a version control system that enables developers to work on TestMaker as a team. Learn the location of the source code at http://www.pushtotest.com/community.html. Nightly builds from svn are found at http://ci.pushtotest.com

What is SOAP? What are Web Services?

Modern systems today use technology that enables software applications to communicate over private Intranets and the Internet. The protocols (HTTP, HTTPS, soap and XML-RPC) are accepted standards and supported by the entire computer industry. These systems are Web services. Enterprise adoption of Web services has sparked an explosion of new software projects, all of which have inter-operability and connectivity requirements. A Web browser may retrieve a Web page, a pager may receive an email, a portal may collect data from many services to present a single Web page and many systems located in various places may be integrated into a productive system.

Server software needs to exchange data with other server software. Past attempts for server data exchange include Corba, DCOM, RMI, EDI and a host of other protocols. soap is an emerging industry standard that has a better chance than previous attempts. soap is to servers what HTTP and HTML are to the Web, a lightweight means to communicate.

Here is the soap definition from the draft W3C specification: soap is a lightweight protocol for exchange of information in a decentralized, distributed environment. It is an XML based protocol that consists of three parts: an envelope that defines a framework for describing what is in a message and how to process it, a set of encoding rules for expressing instances of application-defined datatypes and a convention for representing remote procedure calls and responses.

The soap specification is at: http://www.w3.org/TR/soap/

Where can I learn more about Web Services and TestMaker?

Read the soapUI Tutorial.

I teach Java, Testing and Web Services, may I use TestMaker as a teaching tool?

TestMaker makes an ideal environment for students learning new technologies, including Java, software test methodologies and Web Services. TestMaker includes example test agents which may be used to form a course featuring practical examples of new technologies in action. The PushToTest site offers many free downloads which may help you construct a course. Frank Cohen is available to speak at conferences, classes and lectures.

What documentation exists for TestMaker?

TestMaker comes with a a User Guide, Tutorial, Sample Agents and Javadoc documentation of the test objects. You will find all of these in the docs directory that is installed with TestMaker. View the Support page on the PushToTest Web site for details.

Can TestMaker automatically build test agents for Web pages that use JavaScript?

Yes. Read the Web Test Tutorial

Can TestMaker record my interaction with a Web site and build an prescribed user agent automatically?

Yes. Read the Web Test Tutorial

Can TestMaker record my interaction with a secure Web site using HTTPS and SSL?

Yes. Read the Web Test Tutorial

Does TestMaker support testing secure Web Services?

Yes. Read the Installation and Integration Guide for details.

How do I use the Network Monitor to watch network traffic?

Follow these steps to use the Network Monitor to watch network traffic:

  1. Open TestMaker
  2. Choose Tools -> Network Monitor -> Admin tab
  3. Enter 8091 as the port number and make certain Proxy is selected.
  4. Click Add button
  5. Click the Port 8091 tab

When you run a script configured in this way the Network Monitor, Port 8091 tab shows the network traffic the script creates.

How can I use my Java objects from within a test agent script?

TestMaker comes with ScriptRunners for Java, Jython, and Groovy languageg. See the Editor User Guide for details.

How do I use TestMaker from behind a proxy server?

Proxy servers provide an IT team with a way to manage network traffic. A proxy server is a router that forwards network traffic to a host. If your network uses a proxy server chances are TestMaker will give you errors when you try to run a test agent script or use the New Agent Wizard to record a script. You may see errors like these:

Encountered an error: java.net.NoRouteToHostException: No route to host: connect

Encountered an error: java.net.ConnectException: Connection timed out: connect

To use a proxy server from a TestMaker script configure the proxy settings in the TestScenario using the TestMaker Editor.

How can I use TestMaker on a Macintosh computer running Mac OS X?

TestMaker requires Java 6 or later. Apple ships its own Java 6 runtime environment with Mac OS X 10.5 (Leopard). From a Terminal shell window change the current directory to the location of the TestMaker directory. Open the TestMaker icon.

Does TestMaker support testing applets?

Yes, watch the Java Applet Testing (JAT) screencast.

What books do you recommend to teach me about building Web-enabled applications?

PushToTest recommends the following books:

If you know of additional books, please let us know by sending email to This e-mail address is being protected from spambots. You need JavaScript enabled to view it and we will list them here.

Does TestMaker support batch execution of scripts using the console or otherwise?

PushToTest tests may be run from the command-line. See the Continuous Integration Guide for details.

Does TestMaker support outputting test results as HTML Web pages?

Yes, see the Results Analysis User Guide for details.

Are TestMaker tests composed of individual files that may be checked-into a configuration management system?

PushToTest TestMaker tests are composed of individual files that may be checked into a configuration management system as code.

Can TestMaker tests check SOAP messages for mandatory custom headers?

Yes, soapUI test suites support assertions for headers.

Does TestMaker support Digital Security Certificates, including x509 and 2-way SSL authentication?

Yes, see the Installation and Integration User Guide for details.

Can TestMaker use data from Excel, databases and CSV files as input to tests?

PushToTest TestMaker provides Data Production Library (DPL) for CSV, RDBMS and custom DPLs implemented by any developer.

What support does TestMaker have for building tests of SOAP and REST-based Web services?

PushToTest TestMaker integrates the Eviware soapUI utility to visually create tests. You provide a WSDL or WADL definition, soapUI visually displays a skeleton of the soap request document, you fill it in with data and the utility shows you the soap response from the service. This requires no programming experience.

What build tools like ANT and MAVEN are supported?

PushToTest supports ANT and MAVEN. An ANT build script TestMaker operates a TestMaker test from a custom ANT task. PushToTest TestMaker supports MAVEN through a command-line call.

What format are the test scripts - XML, C, Java, VB?

PushToTest TestMaker test scripts implement in TestMaker Object Designer are XML format. TestMaker uniquely repurposes tests as functional tests, load tests and service monitors using a TestScenario XML document. TestMaker supports JUnit TestCases implemented in Java, Jython, JRuby, Rhino (JavaScript,) PHP and several other dynamic scripting languages.

Does TestMaker offer a Web interface to allow people to review tests?

PushToTest has plans to provide a Web interface to allow people to review, orchestrate and operate tests. This will appear in an upcoming version of TestMaker Enterprise.

How may I use TestMaker to validate a response?

PushToTest TestMaker provides a protocol handler library called the Test Object Oriented Library (TOOL). TOOL provides a soap protocol handler, including APIs to set custom headers and methods to validate response headers. PushToTest TestMaker provides multiple XML parsers, including an XPath and an XQuery engine, to validate results (and also to compare XML parser performance). PushToTest TestMaker provides Data Production Library (DPL) to work with data stored in CSV, RDBMS and custom DPLs implemented by any developer to validate response data.

Where may I find TestMaker screen shots, sample test reports, demonstrations and screencasts?

Find screen shots, sample test reports, demonstrations and screencasts on TestMaker here.

Why did PushToTest choose to license TestMaker under GPL version 2?

PushToTest began offering TestMaker under a free open-source license in 2001. We find there are generally two types of users: those who want to benefit from TestMaker immediately and do not have the technical skills or just don't want to build TestMaker from its sources and those who have improvements to solve problems and generally make TestMaker a better product. To satisfy both types of users PushToTest licenses TestMaker under two licenses.

  1. For those that want to use TestMaker immediately, PushToTest distributes the pre-built and ready-to-run TestMaker under a commercial license. TestMaker comes with an installer program and takes a few minutes to get up and running. The commercial license lets TestMaker users operate tests up to 50 concurrent virtual users (CVUs) and 10 concurrent service monitors (CSMs). Additional CVUs and CSMs are available for license from PushToTest.
  2. For those that want to learn the inner-workings and offer improvements to TestMaker, PushToTest distributes TestMaker under the GPL version 2 license. The GPL license provides our community of users and partners with a license that lets them receive the source code, fix bugs and contribute improvements. PushToTest's goal is to make certain that any beneficial changes to TestMaker are made available back to the users.

Does shipping TestMaker with our software product / platform mean that we need to adopt the GPL license too?

No. Shipping the pre-built and ready-to-run TestMaker with your software product / platformunder the commercial license costs your company nothing and does not require your company to adopt the GPL license too.

Does building a program against the open API in TestMaker mean that we will need to adopt the GPL license?

No. The GPL version 2 license does not require you to adopt the GPL license for your programs that call the TestMaker API. If your software project is a derivative of TestMaker source code then the GPL rules say that you must publish your source code for the benefit of the user community. Writing your software to interact with the TestMaker API (service interface) means there is no requirement for your company to adopt the GPL license.

Does PushToTest support of the GPL eliminate our ability to negotiate an agreeable license to use, bundle and OEM TestMaker?

PushToTest provides dual licenses (Commercial license for the pre-built and ready-to-run TestMaker and GPL for the TestMaker source code) to enable you to negotiate an agreeable license to use, bundle and OEM TestMaker with your product. Please contact This e-mail address is being protected from spambots. You need JavaScript enabled to view it for details.

Why does PushToTest use a dual license strategy?

PushToTest uses a dual license strategy (Commercial license for the pre-built and ready-to-run TestMaker and GPL for the TestMaker source code) to allow technical users to have access to the source code for the reasons of improving TestMaker through bug fixes and enhancements and business and organizations to immediately benefit from PushToTest services around TestMaker, including commercial licenses to run larger tests and more service monitors, receive technical support and training.


Additional documentation, product downloads and updates are at www.PushToTest.com. While the PushToTest TestMaker software is distributed under an open-source license, the documenation remains (c) 2012 PushToTest. All rights reserved. PushToTest is a trademark of the PushToTest company.