TestMaker 5.5TestMaker 5.5 is a major feature enhancement and bug fix release. The new software includes major feature improvements in the following areas:
- Grid Testing - TestMaker runs your tests on a Grid of your test equipment, in a Cloud Computing environment, or both.
- New Script Runners to operate tests written in .NET, Visual Basic (VB) and from the command line
- Scalability Index Plus - TestMaker shows graphically where the application under test should be performing with linear scalability
- Failure and Warning Report Detail - 16 new reports detailing test errors and warning messages
- SeleniumHtmlUnit 2.8 Update - Better compatibility with Ajax applications
- savePage - Debugging command
for Selenium saves HtmlUnit's view
of the current HTML and supporting files
- EncryptDPL - Reads encrypted
data from CSV files
- UniqueDPL - Provides unique
values to virtual users in a test
- RDBMS Logging - Logs test and monitor results data to a relational database
- TestMaker Editor improvements and bug fixes, including file save issues, unused resource removal, and options issues, saves chart settings
- Updated Tutorials - 30-minute tutorials on building tests of Ajax applications using Selenium, building tests of SOAP-based services using Selenium, and building tests of Flex-based services using TestMaker AMF protocols.
- Properties file support to dynamically set TestScenario and test settings
- Preference Settings for external soapUI installation location
- Improvements to Selenium IDE - We began to make changes to Selenium IDE to improve its support for Ajax-based Web applications.
- Email Notifications - TestMaker sends emails on test and production monitor status
- J-Unit Style Results Report for integration with Continuous Integration (Hudson, Cruise Control, Bamboo) environments
- Upgrade to Selenium IDE 1.05, Selenium RC 1.0.3, XPather 1.4.3 for Firefox 3.6 compatibility
- Updated Cloud Testing support in Amazon EC2 AMI
- Fix Editor save bug on Windows environments
- Fix soapUI wrong arguments bug
- Upgraded to run soapUI 3.5.1 test suites
- Removed the soapUI authoring environment to reduce the overall TestMaker download size
- Added the Scalability Index Plus to show an application's scalability instantly
- Transformer creates TestScenario for Java tests
- Native Mac OS X application packaging
- Added Ruby and PHP example scripts
- Added reminder for Windows XP users to install a Microsoft patch for the Editor to operate correctly
- Bug Fixes - Dozens of small fixes and improvements.
- Added MaxRampDown timer to force the end of a test ramp down. Set in the Options tab of the Editor
- Added Htmlunit headless Web browser version selector to Options tab of the Editor
- Added Grid support for multiple TestNodes running on one machine. The Grid Configurator utility (in the Tools drop-down menu) accepts a full URL, including port number and path
- Updated to TestGen4Web 1.1.0 for compatibility with Firefox 3.6
TestMaker 5.5 and later requires a special Microsoft patch to Windows XP for the Editor to operate correctly.
TestMaker 5.4TestMaker 5.4 is a major feature enhancement release. The new software includes major feature improvements in the following areas:
- Scriptless Test Development
- The new TestMaker Editor is a fun
easy graphical tool to build and operate tests.
Tutorials - 30-minute tutorials on building tests of Ajax applications
using Selenium, building tests of SOAP-based services using Selenium,
and building tests of Flex-based services using TestMaker AMF protocols.
to Selenium IDE - We began to make changes to Selenium IDE to improve
its support for Ajax-based Web applications.
- Bug Fixes - Dozens of small fixes and improvements.
TestMaker 5.3.1TestMaker 5.3.1 is a minor feature enhancement and bug fix maintenance release.
TestMaker 5.3TestMaker 5.3 is a major feature enhancement release. The project includes contributions from dozens of engineers in the TestMaker open-source community and year of engineering effort by William Martinez Pomares, Luis Carlos Lara Lopez, and Peter Schumacher. The new software includes major feature improvements in the following areas:
- Cloud Testing - TestMaker runs tests in cloud environments, including Amazon EC2, GoGrid, Collab.net Lab Manager (Cubit)
- Rich Internet Application Testing - Flex/Flash testing with the new AMF Protocol Handler, Mozmill for test of Mozilla XUL applications
- REST Testing with soapUI Pro 2.5
- Root Cause Analysis and Mitigation with Glassbox
- PTTMonitor Improvements - Run monitor as a Web application, new Solaris 10 support
- Major feature upgrades to soapUI Pro 2.5, Selenium 1.0, TestGen4Web 1.1, HtmlUnit 2.5
- License Manager - for managing use of TestMaker Enterprise above 200 virtual users
- Goodbye to an old friend, this version retires the MaxQ Proxy Recorder
TestMaker 5.2.5 is a minor bug fix and feature enhancement release. The following is a list of changes.
TestMaker 5.2.4TestMaker 5.2.4 is a minor bug fix and feature enhancement release. The following is a list of changes.
TestMaker 5.2.3TestMaker 5.2.3 is a minor bug fix and feature enhancement release. The following is a list of changes.
TestMaker 5.2TestMaker 5.2 is a major feature enhancement release. The project includes contributions from dozens of engineers in the TestMaker open-source community and 10 months of engineering effort by William Martinez Pomares, Luis Carlos Lara Lopez, Max Enrique Villalobos Blanco, and Jack Cohen. The new software includes major feature improvements in the following areas:
- Results Analysis engine - an entirely new charting package to turn test results data into actionable knowledge
- Selenium for Ajax testing - full support for Selenium IDE, Selenium RC to author tests of Rich Internet Applications (RIA) using Ajax
- Multiple Target Monitoring - analyze CPU, Network, and Resource usage from multiple target application hosts
- Web Debugging Utilities - shows request-by-request interaction between a test and the target Web application
- New Tutorials - step-by-step tutorials with example code for Web application testing, SOAP/WDL testing, and RIA/Ajax testing
- New Data Production Libraries - a HashDPL to enable sharing of operational test data from one test step to the next in a test use case
- TestMaker 5.0 ships with TestGen4Web 0.49 that fails to install on Firefox 18.104.22.168 or later. TestMaker now ships with TestGen4Web 0.50.2-PTT to be fully compatible with Firefox 22.214.171.124 and later.
- TestMaker 5.1 makes it easier to evaluate response data using Agentbase.py. The get and post methods now return the response data. Agent.base now has methods to assert string contains and does-not-contain, and evaluate response data against an XPath expression. (Thanks to Lars Huttar and Gary for the contribution!)
- TestMaker 5.1 changes the way TestGen4Web scripts are run in the TestNode environment. The new script runner uses TestGen4Web's htmlunit-interpreter package. This improves TestMaker's ability to natively run TestGen4Web recorded unit tests.
- TestMaker 5.1 offers many minor improvements and bug fixes to the results and reporting capabilities. The charts for a load test now include a legend of concurrent user (CR) levels. We fixed an issue where load tests greater than 12 hours show negative horizontal axis values in the resource distribution charts. We removed the load test summary report from the output panel. We added new options to the TestScenario definition for a load test to automatically save all charts to the results directory and create an HTML-based summary report.
- We fixed many problems with installing and running TestMaker on Mac OS X, including an installation problem that prevented the Resource Monitor (PTTMonitor) from running.
- We updated the documention by correcting many spelling and grammer problems and including new sections for QA Testers and IT management. Additionally, we added new documentation to show the options to log results data directly to a Relational Database Management System (RDBMS)and added documentation to show how to set TestMaker and TestNode memory settings.
- We did an overhaul of the TestMaker classpath and supporting Java Archive Resource (JAR) files.
- TestMaker runs on Windows XP, Windows 2003 Server, and Windows Vista. TestMaker 5.1 and later is no longer compatible with the old Windows 2000 and NT operating environments because of a bug in Windows that limits the size of a classpath.
- TestMaker 5.0 shipped with a typo in the <delayBetweenStartingUseCases> element in a TestScenario. The element was incorrectly named<delayBetweenStartingUseCasses> (notice the extra 's'.) TestMaker 5.1 accepts the correct and incorrect versions.
- Added the File menu -> Open Scenario command to open TestScenarios.
- Added new capability to TestScenarios to control timeout values for usecases (unit tests) that take too long to operate.
- Added an option for the transaction distribution charts to use a moving data average instead of inserting every data point.
PushToTest maintains an issue/change database at http://bugs.pushtotest.com. The following is a list of changes in TestMaker 5.1:
What Is New In PushToTest TestMaker Version 5.0PushToTest TestMaker 5.0 delivers many new features and makes the environment easier-to-use and much more useful as software transitions from developer, to QA, to IT manager. We were thrilled TestMaker received an "Excellent" rating from InfoWorld. The article touched on something that we knew for a long time: while TestMaker is wonderfully advanced it can also be complex to learn. We focused our efforts on TestMaker in two areas: ease-of-use to build and operate tests, and broaden the feature set to make TestMaker a platform that is useful in software development for unit and functional testing, QA for load and scalability tests, and for IT for patch qualification (regression/integration tests) and service monitoring.
Here is a summary of the architectural advantages in TestMaker Version 5.
- Designed For IT Integration and Continuous Use
- Monitor enterprise Web services to surface performance bottlenecks, missing functions, and issues
- Root cause analysis and remediation through integration with application and database analysis tools
- Governance Platform Integration to manage tests as corporate assets using Registry/Repository
- Automate test operation using Continuous Integration Environments
- Reduces downtime by qualifying patches, updates, hardware changes
- Designed for multiple forms of testing
- Regression, Functional, Load, Scalability, Performance, Service Monitoring
- Designed for environments with XML, Platforms, Applications, and Databases (XPAD)
- Distributed test environment: vertical deployments for load and scalability tests, horizontal deployments to analyze users experiences on multiple networks/ISPs
- Multiple protocols (HTTP, HTTPS, SOAP, REST, AJAX, email) and extensible to add more
- Build tests in a variety of languages, including Java, .NET, Jython, Groovy, PHP, Ruby, and many other
- Designed for multiple user roles
- Software developers - run tests on their development machines
- QA technicians - Recorders and Wizards avoid coding and do not require deep technical skills
- IT managers - operates tests as service monitors to prove SLAs with no extra effort
- Proven deployments
- Commercial licenses at GM, BEA, AMP, European , TIBCO, Raining Data
- Consistent and regular product updates/upgrades since 2002
- Large open-source user community - 130,000 users
- Cost effective solutions
- PushToTest Test and Remediation Services
- PushToTest Global Services
- Less expensive and more powerful than many commercial test tools
- Open source software makes improvements and bug fixes much more rapidly available than commercial solutions.
- TestScenario is a new XML-based system to reuse unit tests as regression tests, functional tests, load tests, integration tests, and service monitors requiring no coding.
- Graphical environment to control tests and view results in real-time.
- Record/play-back unit tests of Web applications using the integrated point-and-click TestGen4Web utility.
- Create test suites of SOAP-based Web Services using the integrated Eviware soapUI utility.
- Create tests of AJAX and REST applications using the integrated HTMLUnit framework.
- Java 6 ScriptEngine (JSR 223) support enables you to write tests in any supported language: Java, Jython, Groovy, PHP, Ruby, and many others.
- Distributed test environment (TestNetwork/TestNodes) to vertically scale tests to thousands of users and horizontally scale tests to operate the test from multiple locations around your network
- Performance Comparison Utility shows changes in performance from one test to another.
- Monitors CPU, Network, Memory utilization as test operates.
- Transaction distribution analysis charts as test operates.
- Automatic archiving of test results in XML file format.
- Log to Relational Databases (RDBMS)
- Data Production Library (DPL) system to create dynamic data as test operates from XML files, Comma Separated Value (CSV) files, and Relational Database (RDBMS)
- Validate service responses using Data Production Library (DPL,) integrated XML parsers, regular expressions, and custom objects.
- Operate PushToTest As A Service (PAAS) by making a SOAP request to TestMaker to operate a test and return the statistics of the test results. Includes fully functional interoperability with SoftwareAG/WebMethods X-Registry repository.
- Start, pause, stop, and change the order of test case operation as the test operates.
- Service monitor panel shows service status dashboard.
- Notification of test operation by email, SMS, pagers.
- Expanded protocol handler support (HTTP, HTTPS, Apache SOAP, Apache Axis SOAP, REST, AJAX, email) and extensible to add your protocols.
- Backwardly compatible with TestMaker 4 tests.
- Call TestMaker from the command-line to use with Continuous Integration Environments.
- Professional documentation, tutorials, screencasts, and reference guides.
We recommend you read the PushToTest TestMaker User Guide to get started quickly.
Changes Since the Previous
- Improved the charting capability to let XSTest show live results in a chart as the test runs. While the test runs, click the Meters And Charts tab in the Output window to see the live Transactions Per Second (TPS) results as they happen.
- TestMaker 4.4 uses jFreeChart. TestMaker no longer supports or uses JOpenCharts, the DatasetOperations objects in TOOL, nor the old MetersAndCharts.py example test agent script. See XSTest or testmaker_home/example_agents/ChartResults.py for examples of the new charting support.
- New test agent script environment variables - ChartPanel and ChartSupport - make it easier to work with the built-in charting package. See testmaker_home/example_agents/ChartResults.py for usage.
- Rewrote all of the example test agent scripts and moved them to testmaker_home/example_agents
- Test agents generated from the Recorder automatically extend from junit.framework.TestCase
distributions no longer come with the testmaker_home/source directory.
Source code is free and available through these services:
CVS - The source code control system allows anonymous check-out the tm44 repository.
Source Browser - Use the cvsweb utility to browse the TestMaker source code repository using a Web browser. Access the utility at: http://cvs.pushtotest.com/cgi-bin/cvsweb.cgi
- The TestNetwork client software has several new features: enhanced reporting, live charts, synchronized test case start protocols, improved reporting, and remote jar file installation.
- Fixed a bug that crashes TestMaker while using Log4J.
- Removed unused jar files and add a newer version of the Java Web Services Developer Pack (WSDP) 1.4.
- Change the simplelogger to default to append mode.
- Fix a bug that prevented logging HTTP respond header values.
- Fix a bug in XSTest where test cases after the first test case would not run or record results.
- Fix a problem where URL encoded GET requests were encoded wrongly.
- Changed the default for recorded scripts to throw exception when a Web host returns a 404 Not Found error.
a problem where
running test agent scripts from the command-line using the -c option
would not work on fast (ie- 64 bit) Unix systems.
- Added new functions to handle multipart MIME attachements in HTTP Post requests. See the new testmaker_home/agents/multipartform.py test agent script for details.
- Changed HTTPProtocol.java to handle 401 Unauthorized responses correctly. The old method would throw an IOException, the new method returns a response object with the 401 response code.
- Fixed bugs in agentbase.py introduced in TM 4.3.2 where agentbase needlessly created an HTTPProtocol object for each connection to the host. It also introduced a workaround to copy cookies from one http call to the next, when the HTTPProtocol handler object does this automatically.
- Added setHeaders() to HTTPHeader.java to fix a problem where non-multipart-form POST commands would throw a Null Pointer Exception. This is fixed.
- Fixed an Null Pointer Exception in the HTTPProtocol object that happens when making a POST with no headers.
- Update to the latest Tagsoup 1.0 rc3
a bug in the TestMaker start-up scripts where the script adds the wrong
TagSoup file name to the classpath. This results in an error when
playing back recorded scripts:
org.jdom.JDOMException: org.jdom.JDOMException: Could not load org.ccil.cowan.tagsoup.Parser: SAX2 driver class org.ccil.cowan.tagsoup.Parser not found: org.ccil.cowan.tagsoup.Parser
This change avoids the error. No change to recorded scripts is required.
- The XSTest default properties.py file included an extra script command that set the cvus variable to . This is fixed.
- Playback of recorded scripts used to identify requests to a host as coming from a 'java' client type. Agentbase.py now sets the default to emulate Microsoft Internet Explorer. The change to agentbase.py makes it easier to set this value as needed. Thanks to Geoff Meankin for this patch.
- Playback of recorded scripts now checks documents received from the host to see if the document is of a text/html type. If not, TestMaker skips parsing the document for <img> tags. Thanks to Geoff Meankin and Bongos Amigos for this patch.
- Improvements to agentbase.py improve handling of cookies and HTTP headers during playback of recorded scripts. Thanks to Geoff Meankin for this patch.
- Unix/Linux startup script now correctly uses the TMCP.sh script to create the TestMaker classpath. Additionally, it now correctly passes in command-line arguments.
the testmaker_home/utils/runagent.sh and runagent.bat
command-line scripts. These were used to run test agent scripts from
the commandline. The new way to do so is to add the -commandline switch
to the testmaker_home/TestMaker.sh and TestMaker.bat scripts. For
instance, the following will run the master.py script from the command
line on a Unix platform:
testmaker_home/TestMaker.sh -commandline /home/myhome/master.py
on Windows use TestMaker.bat in the above example.
- The TOOL HTMLParser was truncating form elements whose name included spaces. For instance, <select name="san jose"> would be truncated to 'san'. This is fixed. Thanks to Bongos Amigos for pointing out the problem and solution.
- Implemented the Plastic look-and-feel using JGoodies. TestMaker now takes a new parameter named lookandfeel that contains the classname for a UIManager-compatible class. The TestMaker.bat and TestMaker.sh scripts now feature this new parameter. By default the look is set to Plastic3DLookAndFeel. Others are possible by setting the lookandfeel parameter to: com.jgoodies.plaf.windows.ExtWindowsLookAndFeel, com.jgoodies.plaf.plastic.PlasticLookAndFeel, com.jgoodies.plaf.plastic.Plastic3DLookAndFeel, com.jgoodies.plaf.plastic.PlasticXPLookAndFeel, or any of the other UIManager compatible class names. Thanks to
- Added the PyHttpTestCase.py base class file from MaxQ to enable MaxQ recorded scripts to run in TestMaker. Thanks to Jerry S. (jer_57 at cox.net) for the suggestion.
- Changes to the proxy host location and port now are loaded from the TestMaker.properties file on start-up. Thanks to K Satish (ksatish21 at yahoo.co.in) for pointing out the problem.
- The HTTP protocol handler object now performs URL encoding correctly. When a GET request to a URL whose path and/or query include the URL encoded value for # - or %23 - then the protocol handler would throw a java.io.exception. When the encoded %23 appeared in the query then TestMaker truncates the query to the characters appearing before the %23. Both are wrong and have been fixed in TestMaker 4.3.1.
- Added JAX-RPC and SAAJ distributions from the Java Web Services Developer Pack 1.4 to the TestMaker distribution.
- Add Control-R (Windows) and Command-R (Mac) keyboard accelerator for Run command.
- Add SOAP header support to SOAP Protocol Handler. Use the SOAPHeader to set the header information. Thanks to Mark Bocco (mbooco at us.ibm.com) for contributing the code to support SOAP Headers. See the SOAPHeader Wiki page for details.
- The Jython project recently received a grant from the Python Foundation. They are working on a number of enhancements. For TM 4.3.1 no update to Jython is planned. The implication is that TestMaker will still not be compatible with Java 1.5. Details on the changes to Jython are found here: http://www.python.org/psf/grants/Jython_PSF_Grant_Proposal.pdf (53K Adobe Acrobat format.)
now treats test agent scripts that end in .xql as XQuery
scripts. The included Kawa
parser compiler parses the XQuery, runs the query and outputs the
results in the output panel.
- Updated the TestMaker documentation to reflect the new user interface. Added the J User's Guide.
- Alt-R (on Windows) and Command-R (on Macintosh) automatically saves all open scripts and runs the current open script file.
- Added a new QuickStart window that appears the first time TestMaker starts. The QuickStart window facilitates easy access to the New Agent Wizard, documentation, and example test agents.
- All test agent script files now use the standard Python .py file extension.
- Previous versions of TestMaker performed an update check to notify you when new versions of TestMaker become available. This is disabled in 4.3 and may return in a future version.
TestMaker 4.3 beta 1
- Replaces the previous text editor with J, a very powerful text editor. J features Python syntax highlighting, easy navigation among method definitions, search/replace, advanced search using Regular Expressions, code indentation, and much more.
- New Quick Start Helper window opens upon start-up to offer immediate choices to create a new test agent script, read documentation, and learn about the included sample test agents.
- Updated the graphic interface for creating new scripts.
- TestMaker 4.3 is distributed under a GPL General License. Previous versions of TestMaker are distributed under an Apache-style license. The GPL license is free and allows you to distribute and modify TestMaker, however, unlike the Apache-style license changes to TestMaker must be contributed back to the TestMaker project.
- Upgraded to tagsoup rc1 to solve problem parsing HTML from Microsoft Word. Older tagsoup would thran exception indicating that <o:p> is not a valid tag. Fixed.
- When playing-back a recorded test agent script a post command with no arguments would throw an exception. Fixed.
- Recorder's Help button now functions properly.
- Rewrote the generate new agent to implement the new jUnit-style TestCase interface.
- Help -> Online Docs now opens to ./Docs/index.html.
- Agentbase.py - used by recorded test scripts - would throw an exception for post commands that contained no arguments. Fixed.
- TOOL simplelogger has new methods to control file naming and open/close status.
- TestNetwork would sometimes throw a "TNMaster" unknown module exception. Fixed.
TestMaker 4.2 beta 2
- Added MaxQExceptionHandler code to MaxQ
- Attached Proxy server and port settings from Prefs
- Save Script As now asks for user approval to overwrite existing file
TestMaker 4.2 beta 1
Using the Tools -> Network Monitor drop-down menu would create a new tcpmonitor object each time. The correct behavior is to instantiate the tcpmonitor object if one already does not exist and to make the tcpmonitor window visible. This is fixed.
Found and fixed a problem where TestMaker would display an alert box saying "Invalid URL" on start-up for users that were either behind a proxy server or without a network connection. The default.properties files pointed the HelpSystem class to http://www.pushtotest.com/ptt an old and now invalid URL. When TestMaker constructed the HelpSystem it would open a java.net.URL object with this URL. The HelpSystem would then open the alert box. This is now fixed.
A few TestMaker users have posted problem reports where TestMaker would display a modal alert box saying "Malformed URL exception" on start-up. We originally thought this was the Update Checker code going wrong. I looked into it today and found that the HelpSystem was causing the alert to appear. The default page for the HelpSystem is set in the TestMaker.properties (and Default.properties) files to http://www.pushtotest.com/ptt. If you are behind a proxy server or without a network connection, TestMaker will throw the error.
System.out and System.err messages are now displayed in the Output panel within the TestMaker application. Previously such output appeared in the console/shell that launched TestMaker.
By default TM now adds the scriptpath value to the sys.path. Scriptpath contains the path to the directory/folder containing the script that is running. This enhancement means it is not necessary to manually add the scriptpath to the Jython sys path to find script modules. Before this change scripts needed to do the following to make import statements for modules work correctly.
TestMaker now ships with Xerces version 2.6.2.
AgentRunner.java now adds ./lib to the Jython path. Test agent scripts created with the TestMaker 4.2 Recorder import a base class that provides several needed methods (agentbase.py) from the.testmaker_home/lib directory.
Added simplelogger.getPrintWriter() method to return the open log file's PrintWriter stream, returns null if no open log.
Added convenience method: response.getContentSize() returns the size of the response content. Same as doing response.getContent().length()
Added conveneience method for ProtocolWrapper.getURL() with no input parameter assumes the transport is http.
Added TestMaker.getOutputWriter() method to return a java.io.writer (actually a JTextAreaWriter) for the Output panel in the TestMaker application. I need this for an enhancement to the new recorder.
Updated results.xsd schema as part of the new recorder output project. Checked-in new JAXB-created bindings for results.xsd. Looks like there will be at least one more change to the schema to fix little details, for example, some fields are coded as strings when they should be int or long.
Added some JavaDoc comments and added openLogFile( String fname ) convenience method to simplelogger.
build.xml clean method now removes the dist directory too.
Build.xml Ant build script now correctly generates JavaDoc API documentation for TestMaker and Tool.
Known Changes and ProblemsWe expect to have 2 beta releases of TestMaker 4.2 to get your feedback, solve bugs, and improve the code. Here is the list of remaining changes we know still need to be made. Volunteers are gladly accepted to the effort.
- Add MaxQExceptionHandler code to MaxQ
- Attach Proxy server and port settings from Prefs
- Save Script As asks for user approval to overwrite existing file
- Why doesn't the status field appear in the RecorderGui after clicking record?
- Remove AgentRecorder directory and files from cvs tree
- Plus, any of your feedback, comments, criticisms that require changes to TestMaker
- Added network monitor (sniffer) utility to TestMaker. The monitor allows TestMaker users to view the network-level conversation with a host. TestMaker automatically saves the monitor configuration between TestMaker uses. The monitor is based on the TCPMonitor utility from the Apache Axis project. Thanks to Sai at ComCreation for doing the majority of the work.
- SOAPProtocol now tests to see if a SOAPBody has been set. If no, it makes a SOAP RPC encoded request. Previously, no SOAPBody would throw a Null Pointer Exception.
1.4.1 and higher throws a "HTTPS hostname wrong: should be..."
exception when it makes a secure connection to a host using the
HTTPProtocol object in TOOL. Java wants to verify the server
certificate. Java now implements a HostnameVerifier class to verify the
server certificate. This release changes the HTTPProtocol handler to
add a new setHostnameVerifier() method. This was originally brought up
Here is an example script to use the new method:
from com.sun.net.ssl import HostnameVerifier
class verifier( HostnameVerifier ):
def verify( self, urlHostName, session ):
print "HTTPS host name verifier. urlhostname: ", urlHostName, ", session: ", session
hv = verifier()
from com.pushtotest.tool.protocolhandler import ProtocolHandler
from com.pushtotest.tool.response import Response
protocol = ProtocolHandler.getProtocol("http")
protocol.setHostnameVerifier( hv )
protocol.setUrl( url )
response = protocol.connect()
print "Response = ", response
handlers now have getURL and parseUrl convenience methods. The
following example script shows the new methods:
protocol = ProtocolHandler.getProtocol("http")
protocol.setHostnameVerifier( hv )
protocol.setUrl( url )
print "The URL is set to:", protocol.getURL()
- The Recorder did not record Web applications that do not return a content-type header parameters. The Recorder now uses a default content-type: test/html when no content-type header is found.
- The TestMaker launcher scripts now pause for a keypress if system environment variables are not found.
- Fixed problem where TestMaker would check for updates and display a browser everytime it opened. TestMaker now performs the check as it was designed: Use the History->Preferences->Updates menu to set the interval between checks, or set it to 0 to disable.
- Changed the classpath statement in testmaker_home/source/build.xml build script to fix a number of typos. Thanks to Peter Schumacher for leading me to the problems.
- Martyn Fagg (martyn.fagg at scluk.com) pointed out that the TestMaker text editor is using a short type array to keep track of lineOffsets. This restricts the largest offset to 32,767 characters. Large test agent scripts bump into this maximum value. Martyn recommended changing to an int type array to allow for 2,147 M characters. Changed.
- Added the testmaker_home/TMCP.bat and TMCP.sh files to consolidate all the needed TestMaker jar files into one place. This is in preparation for a future launcher script that will offer new features, including running test agents as services. Thanks to Todd Bradfute for this improvement.
- Added XSTest - a framework for turning TestMaker recorded functional test agent scripts into load and scalability tests. See the testmaker_home/XSTest dirctory for details or click here.
- The TestMaker recorder was ignoring applications that do not return a content-type header. The recorder now uses a default content-type: text/html when no content-type header is found.
- Modified the underlying Apache SOAP library to provide support for java.util.Calendar data types.
- TestMaker 4.0.7 introduced a bug in the Recorder where recorded test agent scripts would not set the URL of the host in certain HTTP operations. When the recorded agent script is run a "Not Ready Exception" is thrown. This is fixed in 4.0.8.
- Fixed a problem where the preference.java was not saving the Recorder port number.
- Changed the wording of the report a Recorded script gives. The new wording makes it clearer which call to the host caused the error.
- Changed HTTPProtocol to correctly handle HTTP responses that have no content. Previous to this fix a redirect URL with no content would throw an exception. This is fixed.
- Simplified the jCookie cookie handling methods in response to problems report by Jeff Bernardine where the HTTPProtocol object throws "Illegal State: Already Connected" exceptions it follows a series of redirect commands.
- The Lingo class now has a new method getString( int size ) that returns a string of characters of the length determined in the size value containing random characters.
- The Recorder now checks to see if browser activity was actually observed. If not, it displays an error alert indicating that no script will be saved and asks the user to check their browser proxy settings. Thanks to Mazen Sheblak for suggesting this change.
- The Recorder now works with proxy servers. Use the Help -> Preferences -> Recorder command to set the proxy server address and port number. A check-box tells the Recorder to use the proxy. You must restart TestMaker to make the new settings active.
- The TestNetwork StatusFrame is now a free-floating window that may keep it open at any time. Additionally, the utility to define new test scenarios integrates the many separate user interface panels in previous versions into one dialog. See TestNetwork for details.
- Added quick command bar below test agent script editor that features buttons to clear the contents of the output panel, start the Recorder, and to find strings in test agent scripts.
- Find command added (keyboard equivalent/accellerator = Control F)
- Added TestMakerGui script variable so test agent scripts can control the TestMaker gui itself. For example, TestMakerGui.doClearOutput() programatically clears the Text Output panel.
- TestNetwork client is now fully functional. See TestNetwork for details.
a problem that caused TestMaker to throw a jExplorer exception on
Mac OS X systems depending on the last saved state of the file
navigator in the Explorer panel. Also fixed an Explorer panel problem
where a testmaker.properties file moved from one machine to another
would cause TestMaker to throw an exception. Both problems are fixed.
agent scripts created by the Recorder did not instantiate the "error"
variable correctly. This causes playback scripts to fail on when the
host responds with an error. This is fixed.
client is now functional - but not yet feature complete. TestMaker
4.0.3 ships with the code but the TestNetwork feature is disabled. See http://www.pushtotest.com/ptt/TestNetwork/index.html
for information on TestNetwork.
preferences would not save default font. Fixed.
messages now display the TOOL version number on start-up
disabled the syntax highlighting feature in the script editor. We just
did not have enough time to find and solve a problem that causes
TestMaker to throw exceptions for files longer than 800 lines. Any
(In case you were wondering, TestMaker 4.0.2 was an internal release to use the new TestNetwork code for a customer datacenter certification project. See Chapter 15 of Push To Test, Java Web Development and Test Automation for details.)
TestMaker 4.0.1 Final
- Fixed a problem where the Recorder would stop functioning when processing a HTTP Post transactions. The Recorder now functions correctly.
- Test agent scripts created with t he Recorder had a few syntactical errors. For example, the transaction-per-second result value actually showed transactions-per-millisecond. These are fixed. Thanks to Jim Homan for pointing them out.
- The New Agent -> HTML Agent did not automatically insert a command to set the call type to POST. This is fixed. Thanks to Jim Homan for pointing this out.
- The New Agent -> HTML Agent included the URL form parameters in the URL. The new software does not do this.
TestMaker 4.0 Final
Remaining known problems
warning and error messages appear in the shell that launched
messages should appear in the TestMaker graphical interface.
syntax highlighting problems. This is purely a cosmetic bug and does
not effect test agent performance and functionality.
- TestNetwork client functionality is not yet complete. Instead, it shows what is to come and displays an alert indicating the feature is not yet complete. Details on TestNetwork are found in a free whitepaper at: http://downloads.pushtotest.com/TestNetworkOverview.pdf Our plan is to release TestMaker 4.0.1 with the TestNetwork client functionality implemented. However, keep in mind that the TestNetwork servers (called TestNodes) are proprietary and commercially licensed products from PushToTest.
Things we would like to see improved in future TestMaker
- Preference setting: "Clear output when before running an agent."
- File changed (*) indicator in tabs in Editor window.
- Add icons to menu bar entries, see http://developer.java.sun.com/developer/techDocs/hi/repository/
- Add spinning icon during busy times (for example, while operating the New Agent Wizard)
- Nice printing with http://www.fawcette.com/javapro/2002_12/online/print_kgauthier_12_10_02/default_pf.asp
- Pop-up menu to close and refresh opened agent script
- New save as reminder agents end in .a
- Output panel clear command
- How many concurrent copies of this script to run?
- Find/Replace function for the editor
- Tab and Shift Tab indent/outdent selected block of script
- Stopping an agent auto selects another agent in the Running Agent list
- Multiple selections in Running Agent List
- Standard error alert message instead of System.out.println
- Save as asks user before replacing an existing file
- Write LogHandler, DBHandler, HTMLHandler, NotificationHandler, ControlHandle
TestMaker 4 beta 4
- Colors and Font Preferences - Users may choose the font and colors used in the script editor from the Help -> Preferences dialog box. The Colors tab in the Perferences Dialog box displays a color picker. The Fonts tab shows the available fonts, styles and sizes.
TestMaker 4 beta 3
TestMaker now comes with an Agent Recorder. The new Agent Recorder
watches you drive a Web application using your browser and writes a
test agent script for you. The test script is fully functional and
mounts a transactions-per-second test. The new Agent Recorder function
appears in the File -> New -> HTML Agent Recorder menu
The Help -> Preferences dialog box now includes a Recorder tab
set the port number of the proxy service for the Agent Recorder. By
default, the proxy port is 8090. You need to tell your browser to use
the proxy port for the record to function. The Agent Recorder does not
support HTTPS connections. Most of this new function is contributed
from the MaxQ open-source project.
TestMaker 4 beta 2
updated for TestMaker 4. The testmaker_home/docs directory and the docs.pushtotest.com
site are now up-to-date.
- Recent Agents menu included entries titled Recent Agents. Fixed.