Scriptless Test Orchestration and Development

Fun and easy testing using the new graphical TestMaker 5.4 Editor. A few clicks of the mouse is all it takes to get an accurate picture of front-to-back system performance.

Scriptless Test Orchestration and Development

Hi. This is Frank Cohen, the CEO and founder at PushToTest.

PushToTest enables reliable delivery and operation of modern internet applications. Just about every medium and large-sized organization is in the process of adopting rich internet application techniques to deliver a more productive IT-based experience for their customers, employees, vendors, and even partners.

PushToTest delivers software, services, and methodology to ensure reliable customer delivery of their RIA-based IT systems. Our solutions uniquely deliver an ability to author tests, operate them, and derive actionable knowledge to unify the organization’s approach from the business executive level all the way to the development testing and IT member level.

Let me show you how we do that using TestMaker. Here I’ve got a web application that actually ships with TestMaker. This is a simple tutorial service, and in Firefox I can go here and select Selenium IDE. Selenium IDE ships fully integrated with TestMaker. At this point, you can just click over here and know that Selenium is watching my use of this application. Selenium IDE gives you everything you need to record a test. So for example, I could just use this application, and as I am clicking through the various different parts of the application, Selenium IDE is recording the test.

At this point now, I have a completely recorded script using the Selenium IDE tool, and I can play it back in the Firefox browser just by clicking here.

The editor is an entirely new part of TestMaker 5.4 and it provides you with a graphical way of orchestrating a Selenium test to be a functional test, a load test, or a business service monitor. I identified that this is a functional test and I can tell TestMaker to run this test, this Selenium test just once.

So if I click here on Selenium, I can now browse out to the Selenium test that I want to run, and now I’m ready to run the test in TestMaker. If I wanted to take the additional step of data enabling this Selenium test, I can click here under add DPL. DPL stands for data production library and it’s a unique system built into TestMaker that can read data from a CSV file, a relational database, or another data generating object, and inject that operational test data into the Selenium test as TestMaker operates the test. It couldn’t be easier.

If I wanted to run an additional test after this Selenium test, I can just click here and now I could run say a soapUI test or maybe a unit test that I write in Java or one of the other languages.

This test is already set up the test in the local TestNode that was installed when I installed TestMaker on this machine. However, if I had a QA lab full of machines, I could install a whole bunch of different TestNodes and TestMaker will automatically take care of running the tests in your own grid of tests.

PushToTest OnDemand network enables TestMaker to work with cloud machine instances running in Amazon, GoGrid, and Collabnet. In this case, we might tell TestMaker to create 40 different TestNodes. Each one will run a couple of hundred copies of this Selenium test concurrently. TestMaker will communicate with the cloud environment automatically, start those machine instances, run the tests, and then take them down. It couldn’t be easier.

The other two tabs control monitoring of resources at the application tier or the web tier or in the TestNodes, and also, you have various different options for controlling how long the test takes and various different debugging options.

When I’m ready to run this test in TestMaker, I open a controller panel that looks like this. I’ll click here and tell it to start. What TestMaker is now doing is initializing the TestNode to receive the Selenium script. It’s going to send the Selenium script over to the TestNode. Any needed data like the CSV file that will inject operational data into that Selenium script will be sent over to the TestNode automatically.

The TestNode will operate the test step by step and it will keep track of how long each step of the Selenium test takes to operate, and then TestMaker will return the results to the local machine and show me a chart indicating the success or failure of each operation within the Selenium script. If I want to see the actual action as it is happening, I can click here under output and this will show me the different status reports as the test is operating.

This test finished and you can see the checkmark items indicate successful operation for each of the steps within that Selenium script. Plus TestMaker is showing me the duration it took for each step to operate and the overall success or failure of this transaction.

When I’m ready to operate this same Selenium test as a load and performance test, I just need to go back over to the editor, and under the general tab, I can now select load and performance test. While this is running, I’m going to show you one other thing that we can do with TestMaker and that is to repurpose the same Selenium script to be a business service production monitor. In this case, I’ll tell it to operate that Selenium test in the use case every 10 minutes showing me a visual indicator of the success or failure of the application.

In many cases, we see TestMaker users starting in functional testing to make sure that the test itself is configured right and that the application is actually functioning properly. Then they’ll immediately pivot in to load and performance testing. If they encounter performance or functional issues in load testing, they can always pivot back to functional testing just by clicking that one button within the editor. The new TestMaker 5.4 also incorporates several changes that make it fully integrate with continuous integration environments. Now, you can do nightly builds and nightly tests.

TestMaker includes a results analysis engine. So in addition to seeing this scalability index, if I wanted to see reports that show me how well the transactions were operating, which transactions steps, and it can even map resource utilization as the test was operating. That is we can see how the CPU, network, and memory utilization on the application server was behaving. Once TestMaker is done operating this test, TestMaker will display the additional charts that I’ve requested and it will also put the charts into an HTML report that can easily be published.

So with this one tool, I was able to repurpose a test written in Selenium to be a load and performance test, a functional test, or a service monitor. TestMaker also fully supports SOAP and REST-based service testing using soapUI. So for example, I could click here and this will bring up the soapUI tool.

TestMaker ships with a tutorial service called BrewBiz Web. I’ve typed in the WSDL location of that service and you’ll see that soapUI is able to easily interpret the different SOAP findings that are present in that WSDL document and display to me graphically the different requests that I might be able to make to that service.

So for example, here is request number one, which I can view the request to that service in an XML form, but I can also view it within a form view. The quantity will be 450 units to location 1. And if I click here, soapUI will show me the response from that SOAP-based web service.

When I’m ready to add this request to a test suite, all I have to do is name the test suite, name the test case. I can now go here and open up a test case editor, in which case I can add additional steps to this test. All I have to do to add that soapUI test to my use case is to go down here under test type and select, identify the project that contains that soapUI test, and then type in the test suite and the test case name. If I wanted to provide DPL data from a CSV file or relational database, I can use the same DPL mechanism. It couldn’t be easier.

TestMaker 5.4 is now available for free download at www.pushtotest.com. The product comes in a free open source form and there’s also a more advanced Enterprise product that comes with a commercial license and professional support.

Thank you very much for listening.