A short list of companies at the JavaOne Pavilion that look interesting to me:

Krugle is a source code search engine. It spiders your organizations source code repositories and lets you track code definitions. For instance, when you fix a bug in a well used, and shared, code library Krugle shows the other projects that use the same bug-laden code.

Style Intelligence has 101+ dashboards, reports, scorecards, and other visualizations of data.

Skyway Software makes an open-source application development environment. You build everything in their Eclipse plug-in. It's wonderfully well design user interface makes construction of apps very easy. I'm going to look into this one with the hope that I can build a bunch of sample code for PushToTest TestMaker users quickly.

Determyne is an open-source transaction-level performance monitoring solution for JEE applications. I'll be investigating this to incorporate into TestMaker.

-Frank

JavaOne 2008 struggles to retain its importance. Read my full report at:

http://www.theserverside.com/news/thread.tss?thread_id=49332

-Frank

Jeremy Geelan of Sys-Con interviewed me at the AjaxWorld conference last month. He asked me about trends I am seeing in Web 2.0 and Ajax development at enterprise IT shops. I also talk about what its like for software developers and testers to work in Ajax environments.

http://www.sys-con.tv/read/540772.htm

-Frank

I am planning to attend the Redmonk Unconference at CommunityOne on Monday. The Unconference is at Moscone Center in San Francisco on May 5, 2008. I will propose myself to talk about data-driven Ajax performance testing using Selenium and PushToTest.

Last year's Unconference was terrific.  Instead of going to another track planned by Sun to hear from their marketing folks or a JavaOne conference sponsor, the Unconference topics are proposed by attendees and the selection is made by a democratic vote at the beginning of the day. The Unconference worked out splendidly. I am very much looking forward to the Unconference!

Please join me and I will be happy to take you around and introduce you to everyone I know.

-Frank

Java developers and QA testers are challenged to develop automated tests in a time when Ajax, REST, Service Oriented Architecture (SOA), and Web application development is expanding greatly. Open Source Test Automation has emerged as a more affordable and flexible option to the traditional vendors.

Open Source Test Automation vendors Eviware (maker of soapUI)  and PushToTest (maker of TestMaker) will have some exiciting news to announce at the JavaOne conference. The announcement will leverage our abilities to deliver SOA testing best practices using open source tools and methods.

I am finding that customers are increasingly turning to Open Source solutions for their testing needs. We recently wont such large customers as AMD, Amazon, Ford, The Jackson Laboratory and TVGuide. With 160,000 users, PushToTest delivers a single affordable platform to accomplish functional testing, load and performance testing, and business service monitoring. PushToTest integrates soapUI for fast and easy authoring of Web service test suites to run on a set of distributed PushToTest load test injector TestNodes.

PushToTest and Eviware will be exhibiting in the SOA Village in the JavaOne Exhibition Hall in booth 425.

I am available to meet you on Wed., May 7 or Thurs., May 8 at the show.  Please let me know.

-Frank

I'll be giving a keynote talk on SOA Development In The Real World at TIBCO's user conference next week. This will be based on the work PushToTest did to surface the developer productivity and performance differences between TIBCO, BEA, Oracle, and IBM platforms for SOA development. All of the research is available under a free open-source license at http://soakit.pushtotest.com.

If you are attending TUCON then please let me know and I will be glad to meet with you personally.

-Frank

Here are the details:

BPEL Basics for Java Developers Webinar

WHEN: Thursday, April 17, 2008 at 2pm EDT, 11am PDT, 18:00 GMT

WHERE: Online web meeting. Register at https://www1.gotomeeting.com/register/316860522

WHO: Ron Romano, Principal Consulting Architect, Active Endpoints, Inc.

WHAT: Business Process Execution Language (BPEL) Basics for Java Developers. This free, educational webinar is being presented as a service to the Java development community and will introduce developers to the concepts and techniques they need to understand in order to effectively create a new generation of applications based on web services. BPEL for Java Developers maps common Java techniques to their equivalent concepts in BPEL, enabling developers to transition to orchestrating web-services quickly and easily.

WHY: Java is an imperfect development language for creating SOA-based applications. BPEL has been developed as an open standard to address the need for a robust environment to create services-based applications. This webinar introduces Java developers to SOA-based service orchestrations developers by discussing BPEL in terms that Java developers already understand.

Luc Castera blogged about my talk at AjaxWorld. His review is at:

http://dambalah.com/2008/04/13/ajax-world-conference-day-3/

-Frank

This week I was the speaker at SkillsMatter.com in London. I gave a talk on the PushToTest test methodology and free open-source test automation tools to check Web applications, Web services, and Ajax applications for function, scalability, and performance. SkillsMatter is the new European partner to PushToTest to offer training on our products. In June I am back in London teaching our popular Open-Source Test Automation Bootcamp.

Harry Wood attended my talk and posted a blog entry about his experience at http://www.harrywood.co.uk/blog/2008/04/11/unit-testing-soa-and-mule-talks

-Frank

Mule makes an open-source Enterprise Service Bus (ESB) and SOA development platform. Held at the Hilton Hotel in San Francisco over 2 days, Mulecon attracted approximately 400 delegates, including Mule staff. I was very glad to go as the conference let me talk to several Mule ESB users. I had to do a fair amount of begging to get an invitation since PushToTest is the author of the SOA Knowledge Kit, a TIBCO commissioned comparison of commercial SOA/ESB development platforms. (PushToTest will be an exhibitor at TUCON later this month, and I might be in their keynote session.) Maybe someday I will be appointed an ambassador to something. :-)

The presentations varied between Mule staff talking about Mule and the product roadmap and customers talking about their use of Mule. The Mule staff seemed to be singing from the same songbook: Mule claims 13,000 deployments of Mule ESB, Mule 2.0 is not compatible with 1.4 but the migration should not be too difficult as the concepts are the same, Mule Enterprise Edition, Saturn and Mule HQ releases coming late this year.

A Room Full of Architects

It did not take long for me to realize that Mulecon delegates are mostly software architects. Typical questions I heard goes like this:

"Your company inherits a big bank and needs to decide between JMS and Advanced Message Queue Protocol (AMQP) for a universal protocol for your bank. Which would you pick?"

and

"How do you empirically come to a model that will let you uncover the performance characteristics and measurements?"

These are the kinds of discussions that make my wife immediately yawn and roll her eyes! Geeks, there's a lot of 'em in this room.

ESB, Grid, AMQP Approaches

Conversations about hub-and-spoke, virtual circuits, and bus architecture seem to be over. Just about everyone I met expects a heterogeneous architecture where parts of a system are hub-and-spoke and others are bus-based. Easy service interfaces now exist between each type of architecture using daemons and local brokers. Jahan Moreh, VP of Engineering at U1, had a very basic message "Provided the underlying products are good you won't have a problem."

Jahan was more sanguine about guaranteed deliver and performance. In essence "performance and guaranteed delivery are in conflict." Choose one but don't expect both. I very much want to do primary research to determine a standards way to profile performance in a guaranteed delivery setting. Hopefully one of our customers will commission a PushToTest study. (hint, hint.)

I learn about AMQP at Mulecon. AMQP is a protocol-level standards initiative to provide a messaging infrastructure. Jahan told me "AMQP tries to do something very important but may not be attainable. Because they are at the protocol level they may wind up being extensions on the side. I would bet on the JMS side."

POJOs Rule, But Where Is BPM?

Eugene Ciurana of LeapFrog gave a talk about LeapFrog's use of Mule. I just saw Eugene at TheServerSide.com Java Symposium in Las Vegas last week. Boy does Eugene get around!

LeapFrog is marketing a child's toy that features downloadable content – not bad for products that must cost less than a few dollars to manufacture! The backend system to support this effort uses ESB technology to provide flexibility to LeapFrog's software developers and reliability in operating the service and applications.

Many of the question-and-answer sessions at Mulecon include a question on Business Process Management (BPM) such as: "What are you doing for BPM in your architecture?" Eugene took one of these.

This question echoes what I had learned in Las Vegas: Java architects and developers are frustrated looking for Business Process Management (BPM) standards and tools. Brian Sletten's talk at TSSJS was titled "Avoiding ESBs" but could have been better described as "I'm sick and tired of waiting for vendors to give me a decent Business Process Management (BPM) platform!"

At Mulecon, Rory dela Pax of Biogen Idec, told me that "Mule is good at doing some workflow. But, all of us are struggling with BPM. We do a separation of skills strategy rather than overloading the ESB with these tasks."

Over the years I have seen most of the platform vendors try to provide the Java development community with set of products, best practices, framework, and code to implement applications that deliver business flows. I am specifically remembering the failures of JBI, JEMs, BPEL, SCA, and WS-*. None have come together to offer a standard. I sense a lot of anger and disappointment at all the failed attempts. Mostly disappointment from me.

Eugene answered the BPM question by telling the delegates that LeapFrog pushes all of its development to Plain Old Java Objects (POJOs.) They actively separate every application from each other. LeapFrog then uses Mule routers to define the inputs and outputs. Eugene said "We are not doing any transactional data at Leapfrog. Our support is for the toy devices." He added that Leapfrog has a separate side of the IT house for Oracle tools and BPEL.

Mule and Clustering

Rory dela Pax of Biogen Idec gave a brief talk on their migration from BEA WebLogic Integration (WLI) to Mule. The migration came about like many companies adopting an open-source project: A developer mentioned it during a water break conversation. The environment was ripe: Biogen Idec found WLI to be "a bit on the heavy side, unstable, and has not scaled well." They were also leery of the rumors of the Oracle buy-out since late 2006. They found that Mule has a one-for-one match in capabilities, plus they see Mule as lightweight, stable, and scalable.

I can understand lightweight and stable, but the scalability claim seemed to need some proof. Rory tells me they achieve scalability by using WebLogic Server (WLS) 8.1 SP6's clustering. They deploy their Mule application as a WAR file in a cluster of JVMs in WLS. There are no separate JVMs in their environment. I imagine the gaul the WLI product manager must feel at being swapped out. :-)

Still, there is a lot that Biogen Idec needs to learn about clustering. They still need to define the best way to cluster. Rory told the delegates "We don't know how to do that." They would like to define clustering methods in environments where message sizes grow larger. Right now their application uses simple verb+link combinations. The incoming request is to approve an invoice and the response contains a link to bring the user's browser into the application.

Rory told me some of the downsides to using this WLS clustering approach. For example, they use Quartz as a scheduler. In this WLS clustering set-up changing the Quartz schedules means redeploying the application to the cluster.

Moving Up To Mule 2.0

Mule 1.4 uses DTDs to define pretty much everything – settings, configuration, deployments. Mule 2.0 delivers XML Schema based configuration that leverages Spring's "Extensible XML Authoring." The result is less cumbersome class names because of namespace support. They are updating their Eclipse support to provide auto-completion and context-specific help. Mule 2.0 features transport-specific endpoints and connectors and now everything is done through typed properties. There is a lot of extensibility because each module/transport comes with its own schema. Mule 2.0 lets you implemtn your own schema too.

Some additional changes in Mule 2.0: CXF supercedes XFire (XFire is still available,) there is a new expression evaluator framework, streaming improvements, auto-transformation, and lots of bug fixes. Plus, Mule 2.0 has 30% more unit tests over 1.4. Mule uses Spring to implement Beans. Mule uses session and entity (data) beans inside the Mule container or through proxies to external data services.

Mulecon left me a few questions:

1. How important was the Spring-based approach to building services to Mule's popularity and success?
2. Is anyone, I mean anyone on Earth, using OpenESB from Sun? None of the Mule users had any experience with it.
3. How are Mule and Mule users testing Mule for scalability and performance? Mule 2 provides hundreds of unit tests but apparently no performance tests, at least none that I could find.
4. What are the tradeoffs of an ESB versus Gigispaces, Tangosol and other grid environments?

Mulecon impressed me with its quality presentations and delegates. I will keep it in mind when PushToTest does its first user conference, PTTCon, or PushCon, of TestCon, or who knows!?

-Frank

HTMLUnit is an open-source library that implements APIs that act like a modern JavaScript-enabled browser. It is ideal for building unit tests of browser-based Ajax-style applications. We use HTMLUnit in PushToTest to test Ajax, Web, and JavaScript in browser-based applications. PushToTest turns HTMLUnit's functional tests of a Web site into a load and performance test and a Business Service Monitor (BSM.)

I'm happy to spread the news that the HTMLUnit team release version 2.0 today!

The Change Log notes many improvements and bug fixes, including migration to Java 5, implementation of org.w3c.dom.*, better support for incorrect html code, large improvements in JavaScript support (GWT 1.4 tests now pass, and a bunch of bug fixes.

I opened an enhancement ticket to upgrade PushToTest to HTMLUnit 2.0.

Congratulations HTMLUnit team!

-Frank

We've been making very good progress in building TestMaker 5.2. I'm feeling like a proud father and want to show off my child's cleverness. In TestMaker 5 we introduced a new system of ScriptRunners. This is an extensible facility to operating externally created functional unit tests within a PushToTest TestMaker test. We bundled Eviware soapUI and SpikeSource TestGen4Web with TestMaker and provided ScriptRunners for each. As a result of the latest work TestMaker tests share data between ScriptRunners through a Data Production Library (DPL.) For instance, a functional test first makes a SOAP-based Web service request to learn product information codes and then makes a Web application (HTTP) request to make sure customers are finding the products.



PushToTest orchestrates a Functional test by defining the operating parameters and use cases in a TestScenario document in XML format, as illustrated above.

The test identifies a PushToTest Data Production Library (DPL) that will be the exchange medium between the soapUI TestCase and the TestGen4Web recorded test. The soapUI TestCase reads the data value for the SOAP request from a Properties file in the soapUI project. The soapUI TestCase receives the SOAP response and saves the response parameter to the DPL.

The TestScenario instructs PushToTest to then run the recorded TestGen4Web unit test. The test uses the DPL data from the soapUI test to make an HTTP Get request to the Web host. That concludes the functional test.

I am happy to make the above example available for immediate free download at http://downloads.pushtotest.com/tm5/EndToEndTestExample.zip

You will need the latest TestMaker 5.2 (pre-alpha) version to operate the example. See the faq for instructions on getting the code.

Enjoy.

-Frank

I recently received an email asking:

"Is it possible to use PushtoTest to build a test of a 

Web Application that uses Flash?"

It is possible to test Flash applications using PushToTest. We are currently helping FordDirect.com do this. It does not come without some extra work. For instance, by itself there is no record/playback capability to build tests of Flash animations. However, there is a programmatic way to discover the content of ActionScript variables and to activate functions by doing the equivalent of pressing a button or using Flash controls. PushToTest professional support services will instruct you on the approach, how to instrument your Flash animations for testing, and teach our test methodology and testing platform.

-Frank

The session is free and I will be happy to answer your questions. For details and directions

see: http://www.skillsmatter.com/event/java-jee/the-next-step-in-unit-testing-and-java

I was surfing the Web to find an answer to a customer questions and I ran across 

Over the past year PushToTest observed a trend among the 160,000 people in our open-source community: Software developers, QA testers, and IT managers are looking for a better way to develop, orchestrate, deploy, and manage services. The Composition Approach to Building Large-Scale SOA introduces a new service composition approach to building SOA services, explains composition, and provides a methodology and test kit to evaluate today's tools for developer productivity and ease of deployment/management.

Last week, TIBCO hosted me on a Webcast to talk about Service Virtualization, Service Composition, and the SOA Knowledge Kit. The Webcast is now available at:

http://media.tibco.com/video/tibco_031808/index.html

 

-Frank

 

Neal Ford Pushes For Jargon

 

Neal Ford of Thoughtworks gave the opening keynote speech. His talk was mostly to urge Java developers to look at expressing programs using simplified nomenclature. He pushed jargon. He had a bunch of examples of useful jargon. For instance, the StarBucks jargon of:

 

"Iced decaf triple grande vanilla skim with whip latte."

 

In this example there is not reference to this being coffee. You are already in StarBucks so you know the context is coffee. Java developers that do not use jargon wind up with code like this:

 

Coffee latte = new Coffee(Size.VENTI);

latte.setFatContent(FatContent.NON_FAT);

latte.setWhip(Whip.NONE);

latte.setFoam(Foam.NONE);

latte.setTemperature(Temp.EXTRA_HOT);

latte.setStrength(5);

 

Neal told the audience, "Once we establish context, additional references to the context are noise." An alternative way to express the above code is:

 

Coffee latte = new Coffee().lowfat().nowhip().hot().strong();

 

Neal points out that there is no framework needed to achieve the above. Instead the example uses mutators that return this.

 

Neal also covered domain specific languages (DSL.) He includes the use of XML as a DSL. Neal made the point that while these DSLs and the use of XML may look like a good thing from the outset, they are often prone to spin out-of-control as the DSL needs to expand to support uses over time. For instance, Neal pointed to using Groovy to implement DSLs that are more like natural language and carry more meaning when reading a command:

 

recipe.add 1.pound.of("flour")

 

Neal paints Sun as the drug pushers in a kid's playground. He showed a picture of a playground with the words: The first one is <cost>free</cost>. I found this very funny and biting at the same time.

 

Convincing Management That Testing is Important

 

After Neal's keynote I gave my talk on "Unit Testing in Java" to a packed room. I suspect the TSSJS producers had me talk next to set-up the day's test and performance talks. My talk avoided the normal drill-down on how to unit test – I'm not an expert at unit testing - and highlighted the eco-system that has grown up own unit testing. My personal favorite is repurposing unit tests written by developers as functional tests, load and performance tests, and business service monitors. The idea is to let testers and IT managers reuse a software developer's expertise at building unit tests of their code. I propose adding Data Product Library (DPL) capabilities to a unit test to provide operational data at runtime. For instance, a unit test of a sign-in page of a Web site needs the id/password operational data provided by a DPL. Additionally, the unit test needs a protocol handler library to speak the native protocol (such as HTTP) to the service under test. By orchestrating a sequence of unit test calls we repurpose the unit test into a functional test. And by running the sequence in multiple concurrently running threads we repurpose the unit test into a load and performance test.

 

My slides are available at http://downloads.pushtotest.com/200803/ TSSJS08_UnitTestingInJava.pdf

 

 

I talked about building a Quality Engineering Process (QEP) within a Java shop to provide test automation and continuous testing for an organization. The idea is to have the typical ANT build script compile an application and automatically schedule a set of tests with a QEP system. We see PushToTest customer's needing to operate tests automatically but only when the test resources are available. Like many organizations the test equipment is often not the same as their production equipment and often the test equipment is tied-up and unavailable. A QEP system delivers a scheduler, repository and dashboard for running tests. The scheduler detects when the test equipment becomes available and runs the test.

 

The talk elicited several questions about the most effective way to successfully get an organization to adopt QEP and unit test repurposing. My advice is to get your organization to adopt a test-first strategy. This builds the unit test creation into the development schedule and avoids the "we ran out of time to test" syndrome. But I was hearing in these questions a tougher issue: How do I convince management that testing is important? Several attendees came up to me afterwards with an exasperated look on their faces. I recommended that they take the PushToTest 3-Day Test Automation Bootcamp

to learn our QEP methodology and get hands-on experience using soapUI, Selenium, TestGen4Web, and TestMaker. We had AMD at our last Bootcamp and the test architect brought his manager along too. By the end of the Bootcamp the manager knew the approach to testing, how much time to budget for tests, and how test automation makes testing a low-impact part of building and releasing software and services.

 

Description Language Debate Continues

 

In Mark Hansen's talk "Building RESTful Web Services with JAX-RS" an attendee asked, "Why is there pushback on REST adopting a description language?" SOAP has WSDL and Sun is promoting the Web Application Description Language (WADL) for REST. Hansen answered the question by describing two types of user. REST is great for applications where a group needs to quickly build a user base. For example, building an on-line community, an open source community, and a social networking site. For these, scripting languages like Python, Ruby, PHP and Java in this domain are appropriate. Mark said that these applications were mostly mashups. More importantly Mark said in this domain delivering the application with bugs is acceptable. The goal is to build the user base quickly. Bugs are acceptable because the application benefits from community support and the bugs that show up are low cost to solve. Mark said you do not want to worry about data types and message service interface definitions. He told the audience that some really big sites are built that way but did not give an example.

 

In the other camp, Mark talked about mission critical business application where functions that blow-up are the cause of very big problems. In that camp you have a smaller number of developers that need clear interface specs, clear unit tests, clear operations plans. In this case Mark told the audience that it makes sense to use SOAP and WSDL.

 

Mark went on to talk about JAX-RS to build REST-style service interfaces. One interesting point Marc made was about annotations. Mark told the audience that the JAXB group, the JAX-RS group, and others at Sun need a way to express annotations externally where you can't easily recompile the code. Mark indicated there is a lot of talk about doing this and he is expecting a way.

 

Ruby On Rails On GlassFish On Java Looks Solid

 

Arun Gupta of Sun showed Ruby on Rails running on JRuby on GlassFish. He told attendees to get the JRuby on Glassfish module from the Glassfix download site. The module comes with JRuby 1.0.3, Goldspike, and Rails 1.2.6, delivered as a WAR file. Arun recorded a <a href=" blogs.sun.com/arungupta/entry/screencast_web_9_jruby_on "> screencast to show how to install and run JRuby on GlassFish. Then you download and run Rails from the native JRuby environment. The JRuby team at Sun has been at work for the past year on this and the ROR demo looks solid. Still no official word from Sun on providing an official release and Sun support.

 

One-liners I heard in the hallway:

• Everyone is supporting JAX-WS except for IBM.
• Appcelerator and WaveMaker are doing just what jMaki already does.
• Web Application Description Language (WADL) is not part of the JAX-WS JSR.
• JRuby final release candidate out now, expect JRuby release at JavaOne.
• Glassfish app server starts in 863 milliseconds. Less than 1 second is great.
• Asynchronous Web services with callbacks are better suited for OpenESB than JAX-WS.
• Amazon's REST interface to EC2 is an insult to REST in 2008.

 

Avoiding ESBs and Developer Disappointment in BPM Tools

 

Brian Slettin of Zepheria

lead a talk titled "Staying Off the Bus: SOA Strategies Without ESBs" that could have been better described as "I'm tired of waiting for vendors to give me a decent Business Process Management (BPM) platform." Over the years most of the platform vendors have tried to provide the Java development community with set of products, best practices, framework, and code to implement applications that deliver business flows. I sensed a lot of anger and disappointment at all the failed attempts, including JBI, JEMs, BPEL, SCA, and WS-*.

 

Metro Web Services Stack

 

Kohsuke Kawaguchi, Senior Staff Engineer, Sun Microsystems gave a talk on the Metro Web Services Stack. Metro is a high performance SOAP stack for Java. Back in 2002 I found that the first generation of SOAP stacks were not scalable and I published my findings

. People would ask me the cause of the scalability problem and I told them that the marshalling/unmarshalling tasks that SOAP stacks need to perform to move in and out of the Java object environment was the biggest problem. Kohsuke's talk was a refreshing and well though-out discussion of all of the performance issues and the Metro architecture to get around the issues.

 

Kohsuke told the audience that Metro defers the XML binding to as late in the marshalling process as possible. XML arrives as InputStream. Their validator wraps a SOAP message in an object but avoids parsing until later. Metro databinding reads XML in SAX where parsing and validation finally happens. The output is a data binding into POJOs.

 

Metro provides a "handler" that uses AOP messaging techniques in SAAJ, XML read into DOM here. Then a Databinding method reads XML in SAX with a visitor walks over DOM and generates SAX events. Internally the Metro stack uses InputStream, XML stream buffer, DOM/SAAJ, and JAXB POJOs to represent the data internally.

 

Kohsuke spent the rest of his time talking about what they have found are lighter and heavier in their demands for resources and processing time. Many times the choice that requires buffering or a non-streams approach is much slower. For example. Kohsuke told the audience that Metro does Encyrption/Decrypt, Signing/Verification in a streaming fashion, but that these require buffering. Kohsuke said that cryptography is still very costly. Additionally, he told us that Dump is streams-based but requires buffering and validation requires streaming.

 

Kohsuke told us WS-Reliable Messaging and Secure Conversation requires buffering. WS-AtomicTransaction, no conversation however additional round-trips

Handler requires a DOM conversion and this is very expensive. Kohsuke told the audience this axion: "Pick 2 out of 'fast', 'portable', and 'easy'." There is always a trade-off.

 

Kohsuke finished by telling the audience that they are still working on scalability. He said horizontal scalability (aka clustering) is trivial if you don't use stateful web service protocols like WS-AT, WS-RM. He also said his team is working on an AsyncProvider to do call-back based request/response.

PushToTest version 5.2 (which should be out this Spring) will feature Selenium integrated into the PushToTest test automation platform. See http://bugs.pushtotest.com/ticket/136 for details.) Selenium users will create functional tests of Web and Ajax applications using the Selenium plug-in to MS Internet Explorer, Firefox, Mozilla, and other browsers. PushToTest users have two choices to run Selenium tests in the PushToTest test automation environment:

1) PushToTest 5.2 will include a Selenium test runner. This runner instantiates a Web browser in a TestNode (our distributed test environment,) operates the Selenium test, and reports the results. This is a powerful choice to run Selenium tests as functional and regression tests, and business service monitors.

2) PushToTest 5.2 includes a Selenium-to-Java and Selenium-to-Jython transformation utility. The utility creates a Java class (or Jython script) that implements the test steps using the HTMLUnit framework. HTMLUnit acts like a browser, including operating Ajax applications using Rhino (the JavaScript engine from the Apache Mozilla/Firefox browsers.) The is a powerful choice to run Selenium tests as load and performance tests.

For both of the above options, Selenium tests receive dynamic operational data at runtime from PushToTest Data Production Libraries (DPLs.) For instance, a Selenium test that operates a sign-in page receives the account number and password from a DPL that accesses data from a comma-separated-value file (or a relational database or custom DPL.)

Additionally, Selenium tests produce results data that PushToTest Results Analysis engine renders into hundreds of charts. And, these charts correlate Selenium test operation to resource utilization (CPU, Network, Memory, Threads) in the back-end server.

We have an "alpha" quality version of the Selenium-to-Jython transformation utility to give to you today. This is from a contribution from Dominique and Olivier at Denali. All you need to do is ask.

-Frank

I often get asked questions like "What does PushToTest do?" PushToTest is a test automation solutions business. We surface scalability and performance problems in Web-based information systems. Our PushToTest solution drives a Web system like a real user and correlates the activity to identify broken functions and hot spots of performance issues. The same system then operates tests periodically through the day to monitor an enterprise's services and provide proof-of-service reports. See our "white paper", http://downloads.pushtotest.com/PushToTest_WhitePaper.pdf for a more complete answer. I often get asked a follow-up question that is usually asked like this: "I was wondering if you could provide me a sample of the outputs the system generates (or can generate)." The PushToTest software produces a summary report that shows the Scalability Index to the application, the resource utilization as the test operated, and the performance as load increased. This summary report is an HTML document and is easily shared among your team members. "Click here", http://downloads.pushtotest.com/tm5/ExampleSummaryReport/index.html for an example summary report. I often also get asked about the PushToTest methodology and techniques to find and solve performance issues. The question is often asked like this: "Once you run a test and the results are not acceptable then how do I leverage the PushToTest tools to find the code that is running slow?" PushToTest version 5.1 keeps track of the steps for each transaction. For instance, the following snippet of a TestScenario implements a test use case where four steps are required for each transaction.

<test>
   <run name="Open Account" testclass="com.pushtotest.WebAppTest"
method="testOpenAccount" langtype="java" />

   <run name="Transfer Funds" testclass="com.pushtotest.WebAppTest"
method="testTransfer" langtype="java" />

   <run name="Check Transfer" testclass="com.pushtotest.WebAppTest"
method="testValidateTransfer" langtype="java" />

   <run name="Close Account" testclass="com.pushtotest.WebAppTest"
method="testCloseAccount" langtype="java" />

</test>

In the above test PushToTest operates four steps: Open Account, Transfer Funds, Check Transfer, and Close Account. PushToTest keeps track of the total transaction time and the time for each step. The time of each transaction and step is stored in the transaction log file. When a particular step takes longer than expected you can break open the transaction logs for the application server and learn what was going on at that point in time. We have been hard at work over the past two months building an Enhanced Results Analysis engine. The new feature will appear in PushToTest version 5.2. It is already working and we are happy to provide it to anyone that "asks", http://www.pushtotest.com/ask_a_question. The new engine makes it much easier to visualize broken functions and hot spots to performance. Consider the Step Report that visually charts the average duration of each step across 10 equal periods of time of a test. The reporting is great but that still leaves the correlation effort up to a software engineer. And, it is a manual process to go from PushToTest logs to the application server logs. Our planned solution is to incorporate "Glassbox", http://www.glassbox.com into PushToTest version 5.2. Glassbox watches a Java application server for performance hotspots as a test operates and identifies the objects and database connection level issues causing the hotspots of performance. "Click here", http://bugs.pushtotest.com/ticket/61 to learn about the integration project. To learn more about Glassbox "click here", http://www.glassbox.com/glassbox/Download.do?mediaItemId=173 to read the Glassbox user guide. One final question I usually hear goes like this: "Where exactly are your developers located? In other words when you leverage out-of-country developers, where are they and if the customer wanted US based developers can you accommodate that." PushToTest outsources its customer engineering to our Costa Rica-based partner "Avantica", http://www.avantica.net. Costa Rica is a 4-5 hour flight to the United States and Canada. The country is on Central Time (GMT -6) and their English is very good. We normally need about one week to plan for our engineers to be in your office. -Frank

The Selenium founders are organizing a user meeting at Google's Mountain View, California office on Monday, February 25. Among other things they will be describing the push to release Selenium 1.0 and the roadmap and internals of Selenium 2.0.

Details on the Selenium user meeting are at

http://selenium.openqa.org/meetup.jsp

-Frank