Over the past year PushToTest observed a trend among the 160,000 people in our open-source community: Software developers, QA testers, and IT managers are looking for a better way to develop, orchestrate, deploy, and manage services. The Composition Approach to Building Large-Scale SOA introduces a new service composition approach to building SOA services, explains composition, and provides a methodology and test kit to evaluate today's tools for developer productivity and ease of deployment/management.

Last week, TIBCO hosted me on a Webcast to talk about Service Virtualization, Service Composition, and the SOA Knowledge Kit. The Webcast is now available at:

http://media.tibco.com/video/tibco_031808/index.html

-Frank

This morning I am glad to introduce you to a new set of resources to help surface scalability and performance issues in Service Oriented Architecture (SOA.) The SOA Knowledge and Performance Kit is a free open-source resource to show you what it really takes to build services using today's leading SOA development platforms.

The Kit delivers an SOA use case design, source code to the implementations of the use case on Oracle, IBM, BEA, and TIBCO platforms, developer journals describing our experiences step-by-step, a Total Cost of Ownership (TCO) calculator, and performance and scalability tests that leverage the PushToTest test automation platform.

PushToTest looked below the surface-level marketing claims to understand the skill sets, domain expertise, and specialization that it takes to be successful. We discovered a wide variety of experiences: some highly successful and others rife with challenges. We measured the amount of developer effort to implement an SOA use case and summarize the findings in the following total cost of ownership (TCO) comparison chart.

This morning we opened the doors on a new center within the PushToTest Web site for you to learn more about SOA, learn about the new composition approach to SOA development and deployment, and download the kits.

SOA Knowledge and Performance Kit

-Frank

We are getting very close to the PushToTest 5.1 release - hopefully this week! New datasheet and white paper downloads are available to explain PushToTest and open-source test automation:

PushToTest 2-Page DataSheet

PushToTest White Paper: The Importance of Test Automation

Both are in Adobe Acrobat (PDF) format.

-Frank

TestMaker test scenarios generate a lot of useful data. TestMaker 5.0 introduced standard charts to visualize the data into actionable knowledge, including the Scalability Index, Transaction Distribution Charts, and Resource Monitor Charts. TestMaker 5.0 also introduces a result log archiving system and a Performance Comparison Utility to generate charts to compare results between operations of test scenarios.

TestMaker 5.2 will introduce new results analysis functions:

1) Transaction logs will include the "step" times for the operations within a

2) A new charting function will summarize logged results data in new and flexible ways.

TestMaker 5 creates a lot of data that can be put into a chart. Here are the base data series:

• Test Scenario (name)
• Test Node (name)
• Concurrent User levels (CRs)
• Data Index (Message size)
• Transaction (Use Case)
• Sequence name
• Step (run) name
• Pass/Fail status

I put together a new presentation to introduce you to the Enhanced Results Analysis functions. Download the presentation at http://downloads.pushtotest.com/tm5/PushToTest_ResultsAnalysis_20071220.pdf

-Frank

Last week PushToTest hosted the first of a new series of 2-Day Hands-On seminars on Test Automation. The event sold out and the room was packed with eager students. Some traveled from Michigan to the seminar in Silicon Valley, California.

Here are some of my notes from the seminar:

1) The students at the seminar were from a rich cross section of backgrounds and experiences. We had software architects, software developers, QA testers, database administrators, and project managers. This shows me that test automation is a multi-discipline effort. It is not enough anymore to throw testing over the wall to a bunch of testers and not worry about the results.

2) TestMaker is in better shape that I had thought. In the days that lead up to the seminar I was concerned that the usability in TestMaker would make the hands-on portions difficult to follow. I was happy to none of the students ran into any roadblocks that kept them from completed the workbook of tasks. (This is not to say that we can put off usability improvements in TestMaker. Usability is a big issue in my eyes.)

3) The students universally told me they are not finding commercial test automation tools that are appropriate for Web application, Service Oriented Architecture (SOA,) Ajax, Web service, and Web 2.0 testing. As a result they often have one tool for Web application testing and another tool for Ajax testing.

4) Many students liked the informal style of the presentations. They were looking to gain an understanding of testing in the 21st century and take home tactics and methodology they will use in their own environment.

5) Several of the students are working in SOA environments and were interested to learn about TestMaker integration with registry/repository products. See here for details

The next seminar events are in London, New York, and then back in Silicon Valley. For details check http://www.pushtotest.com/Docs/training/seminarform.

-Frank

The Server Side ran our announcement of TestMaker 5 today at:

http://www.theserverside.com/news/thread.tss?thread_id=46598

We also issued a press release on the new product. Download the press release

If you are so inclined, please post a reply asking a question or telling of your experience using TestMaker. The more you post the more attention we get.

Thanks.

-Frank

Perry Donham of Aberdeen gave a short Web conference this morning to talk about his upcoming research report on SOA and Web Service QA and Test. The report covers the basics: why test, popular test methodologies, and top testing challenges. One of the interesting factoids to come from the research is the large number of software developers that are testing SOA and Web Service applications. When asked the question

"Who does the testing?" the answers were:

QA: 79%
Developers: 53%
End users: 37%
External beta testers: 17%

That confirms the research we have conducted in the PushToTest community. SOA, Web Services, AJAX, and Web application testing tools normally require technical skills that QA technicians do not posses. My assertion is that developers play an increased role in the lifecycle to build services. We see developers having an increased role in testing.

Astonishing to me is the answer from 37% of respondents that End Users do the testing. How would you like to be a customer of those companies?

-Frank

The new release is the full and complete TestMaker 5 product. We will continue testing it for bugs and issues and take your bug reports. Depending on the testing results and your feedback we will either issue a second release candidate or declare this to be the TestMaker 5.0 release.

http://www.pushtotest.com/Docs/r5

-Frank

My blog entry elicited a response from Jason at ZapThink. He asked – more or less - what he doesn't get? I also received a request to help parse part of the blog.

My blog shows that I'm bothered by two things: using the word Divorce when talking about SOA and Web Services and iTKO's skeptical position on why they give away Lisa Web Services.

Jason E's blog entry says: "Yes, [iTKO Lisa Web Services] can be useful, but we honestly believe that Web Services testing on its own will never deliver the quality levels required to achieve Trust in your SOA applications."

Perhaps I'm taking this the wrong way but I read that as... We will give you a partial solution for free and wait for you to fail because the real solution - the SOA one - costs $$$. There is a skepticism there that is wholly inappropriate for where we are in the growth of the IT industry. Imagine the developers, QA technicians, and IT managers in a business or organization that adopts both SOA and Web Services. How does offering a free Web Service testing utility solve their needs?

To Jason B.'s report, we shouldn't be using terms like Divorce because of the heavyweight emotional negative context it evokes. What CIO from divorced parents is going to think that Web Services + SOA is a good thing? We should be writing from the perspective of "All this new communication, new rapid integration, and new interoperability built with XML, Platforms, Applications, and Databases is yielding great benefits for our businesses and organizations." It doesn't matter if the approach is SOA or Web Services or some wonderous mix of both.

Jason B. raises a valid point to my previous blog entry. I should have not written that Jason does not get it because that criticizes Jason the person, instead of Jason's actions.

Finally, in my blog I wrote:

"SOA keeps the WS component idea, focuses on composite applications for business workflows, and loses discoverable service idea for statically brokered endpoints, governance for choreography, business issues, troubleshooting, and Quality Of Service (QOS.)"

and a reader asked the question:

"How did you intend to parse [that paragraph]? Did we "lose" governance for choreography? Or substitute governance for choreography? Or "focus on" governance for choreography..."

Let me put it this way…

SOA and Web Services both start with the idea of saving money and effort by reusing software components. SOA uses composite applications and master data management techniques to implement business processes and workflows. SOA is different from Web Services in that it does not require discoverable services at runtime, but instead implements a governance plan where statically brokered endpoints, choreographed components, and defined Quality of Service goals are stated.

And the most important point is… SOA and Web Services work well together.

-Frank

A blog by Jason English at iTKO has me bothered. Jason praises an analyst report from ZapThink's Jason Bloomberg titled Divorcing SOA from Web Services. Neither iTKO nor ZapThink seem to get the vision behind SOA and Web services.

The IT industry has a long way to go to serve enterprises and organizations needing to well manage their efforts through information systems. Service Oriented Architecture (SOA) and Web Services (WS) are entirely appropriate to build information systems. The vision behind SOA and WS help us a lot and in different ways.

Here is a slide I have been presenting at various conferences for the past two years to explain the differences between the SOA vision and the WS vision.

The vision behind SOA and Web Services comes from enterprise and organization needs to save development effort and money by reusing software in the form of components.

The Web Services vision achieves reuse by building service components that autonomously discover at runtime other needed components needed to solve a business process. The SOA vision achieves reuse by aligning new software development projects to business goals through a governance plan. Both expect a registry of services will help avoid building the same software component twice.

In terms of architecture, SOA relies on composite applications and data services while Web Services relies on finely-grained, loosely coupled, discoverable services. A composite application is a piece of software that is able to talk multiple protocols to existing services to give you one view of either the customer or one view of the business process. For instance, you may have been in a situation where you've had to call an insurance company to try and find out two different things: the date they received your most recent payment and how to file a claim.

This example insurance company may have two systems - one system retrieves your most recent payment, but then the operator that you talk to might have to forward your call onto a different operator just to ask the question about filing a claim. With a composite application, you have a single piece of software that is able to speak the native protocol to the service to look-up your payment information. The composite application, typically on the same screen, can even speak the protocol of the claim center to place a claim directly.

As a customer, you are served better because you conclude your business with a single agent on a single phone call. And the insurance business saves money by serving you within that one phone call. Many companies PushToTest serves that have adopted SOA and Web Services find that composite applications are more easily built than trying to get the existing services to interoperate directly. These enterprises have a faster time to market advantage than their competitors.

Management techniques vary between SOA and Web Services. The SOA vision expects an enterprise to define a governance plan. Web services expect composite applications to register themselves with a dynamic repository of services to be dynamically discoverable at runtime. This is where SOA and Web Services diverge the most. Unfortunately, none of the information ontologists showed up to the Web Services party. There is no standard for categorizing composite applications or software components. That is a big challenge for any enterprise or organization to realize the vision of Web Services.

SOA and Web Services vary in an important way for message formats and protocols. The SOA vision says "whatever message format and protocol works is acceptable" whereas Web services mandates XML. This is an important point because it means that SOA may use Web Services.

What we have here is a success at communication! Communication between information systems has never been this good.

SOA and Web Services are useful visions to move us from the current XML, Platform, Application, and Database environments (I call this XPAD computing) into the future. IT has been wanting this kind of interoperability, reuse, and governance for decades, including in efforts like CORBA, OpenDoc, DCE, Client/Server, Web 1, Web 2.0, and Enterprise Web 2.0. Those were all efforts to be able to provide a component architecture where software could be reused to provide an enterprise with a faster time to market advantage and then lastly to provide an enterprise with a better view of the customer.

SOA keeps the WS component idea, focuses on composite applications for business workflows, and loses discoverable service idea for statically brokered endpoints, governance for choreography, business issues, troubleshooting, and Quality Of Service (QOS.)

It is just fine to me that sometimes enterprise architects and technology managers get the terminology of SOA and Web Services wrong. You won't see PushToTest talking about "divorce" in a family of technologies that helps the world become better.

-Frank Cohen

I'm very happy to tell everyone that PushToTest TestMaker version 5 beta 2 is now available for download. The new beta is likely to be the last before we declare TestMaker 5 final and stable.

Beta 2 solves all of the bugs from Beta 1, includes the preliminary documentation, and includes the Java Runtime Environment 1.6.

Get TestMaker 5

Enjoy.

-Frank

We are very excited to make available a screencast and the first "beta testing" release for PushToTest TestMaker Version 5. Both are available right now at:

http://www.pushtotest.com/Docs/r5/PushToTestRelease5.html

The new screencast demonstrates TestMaker 5 to users of TestMaker 4. We are next working on 2 more screencasts to demonstrate how TestMaker 5 turns unit tests into functional tests, load tests, and service monitors. The last screen cast will show TestMaker operating as a service and interoperating with the Software AG X-Registry repository.

The beta version is already pretty stable, so we plan to do 1 or 2 more beta release and the go final at the end of June.

-Frank

InfoWorld now has a product information page for PushToTest TestMaker. The page includes their review, in which they give TestMaker 4.4.1 an "Excellent (8.7)" rating. The InfoWorld page has a section for reader reviews. If you have anything nice to say about PushToTest then please post your comments here.

The review says in summary: "PushToTest's TestMaker is a powerful, professionally crafted package. There are lots of useful test code examples, and its documentation surpasses all expectations. But it has a steep learning curve, and you'll need to know Python to get the most out of it."

This brought an ear-to-ear smile to my face. They think the current release (4.4.1) is excellent and talk about the learning curve. I can't wait until they see PushToTest Version 5! Version 5 delivers a distributed test environment and a much easier learning curve. Version 5 supports 6 scripting languages (Java, Jython, JRuby, Groovy, Rhino-JavaScript, BeanShell, and others.) Script development is made easier with the TestGen4Web browser plug-in to record scripts and integration and bundling of soapUI, a powerful utility for building functional tests of SOAP-based Web services. Lastly, version 5 delivers an easy way to define a test using an XML Test Scenario format.

The InfoWorld review also gave soapUI an "Excellent" rating. The combination of the two is going to be a killer application for software developers, QA technicians, and IT managers.

Details on PushToTest Version 5 are here.

-Frank

JavaOne is Sun Microsystem's conference for Java software developers. Sun and many of the Java tools and platform vendors use JavaOne to announce new products and set the agenda for the next 12 months. I wrote-up a summary of my postings to TheServerSide.com as a first-person analysis of the JavaOne 2007 conference announcements, news, and trends. Topics covered include: Java Business Integration (JBI,) Software Component Architec-ture (SCA,) JavaFX Script, Dynamic Scripting Languages on Java VM, the Java Community Process (JCP,) TIBCO ActiveMatrix, and much more.

Download The RedStripe Research Paper (Adobe Acrobat PDF format, 288 Kbytes)

-Frank

A couple of very cool things for Jython, the scripting language in PushToTest TestMaker

• Oti Humbel is a Jython expert. He led a session at JavaOne today that presents a portion of his amazing knowledge and humor on the Jython danguage. His slides contain a lot of information and are found at http://jython.extreme.st/
• Jython 2.2-beta2 is available for download: http://sourceforge.net/project/showfiles.php?group_id=12867&package_id=12218&release_id=507592

See http://jython.org/Project/installation.html for installation instructions.

This is the second and final beta release towards the 2.2 version of Jython. It includes fixes for more than 30 bugs found since the first beta and the completion of Jython's support for new-style classes.

Enjoy!

-Frank

I recorded a narrated video podcast of a presentation on the new PushToTest release 5 platform. The slide show is for developers, QA technicians, and IT managers with an interest in SOA governance and test automation.

Watch A Briefing On The PushToTest Version 5 Platform
(Flash, 53 Minutes)

Creating the slide show was suprisingly difficult. I would expect dozens of tools to help me create a slide show with voice narration to be available and inexpensive. That's not what I found! Instead I found one needs a variety of products, most of them expensive.

I finally put together a solution that makes it easy to produce and host narrated slide shows. I posted the solution here

Please let me know your comments and feedback on the slide presentation's content, style, and asthetics by commenting on this blog entry.

-Frank

IONA launched their new IONA Repository product today. SearchWebServices.com covered the news story. Unfortunately for SearchWebServices and IONA the article's quotes from analysts were pretty far off the mark. Consider these:

"However, analysts say that the main differentiator for the Iona entrant into the registry/repository game is that it is the newest entry in a somewhat crowded field..."

"Jason Bloomberg, senior analyst with ZapThink LLC., said: "What's most unique about the Iona reg/rep is that it's at version 1.0, while HP Systinet Registry, webMethods Infravio X-Registry and LogicLibrary Logidex are now all quite mature products. Even the IBM WebSphere Service Registry/Repository, a relative latecomer, has been on the market longer."

"However, the analysts are somewhat skeptical that the new product will go beyond being of value to existing Iona Artix customers, at least in version 1.0."

For a news story about a new product launch in an emerging market, the skepticism that comes across is unwarranted. For instance, how many businesses do you know that are actually using an SOA registry or repository in production? Further, how many of these are delivering business value in the form of cost savings or time-to-market advantages today? I don't know of any.

The market for SOA Registry/Responsitory products is emerging and the industry and media should take a chill-pill on skepticism.

For Artix customers, the addition of a means to govern the services they build and deploy makes operating an SOA environment better. How? I don't know for now but I have my opinions. I am willing to bet money that...

1) Software developers will eventually turn to repositories more than version control systems to learn the existance and use cases for a set of services.

2) CIOs will turn to a dashboard to see the score-card on how well the IT infrastructure is able to deliver services against an SLA.

3) Sites like Freshmeat.net and sourceforge.net will become repositories.

I commend IONA for shipping the IONA Registry. I hope it does very well in the marketplace, the emerging one.

-Frank