Matt at Appcelerator pointed me to a discussion of Automated Testing of Appcelerator With Selenium on the Appcelerator user forums. I am very interested in ways to use Selenium for testing Appcelerator applications.

From TestMaker's perspective a Selenium test is a unit test of a Web application or Ajax application. TestMaker already provides Test Runners for TestGen4Web, soapUI, and the Java 6 ScriptEngine support for unit tests written in Java, Jython, JRuby, Groovy, and a bunch of others.

It seems to me that the forum members have already hit upon some of Selenium's limitations and design problems:

a) Selenium was not really developed for Ajax, and vice versa. Ajax frameworks like Appcelerator implement their own eventing mechanism and Selenium has no idea of how this works. For instance, when an Appcelerator log-in page changes the DOM to reveal site content Selenium IDE does not notice the change. Selenium IDE will sometimes record a 'click' when 'clickAndWait' is appropriate. I find myself adding 'waitForElementPresent' commands that look for the logout element in the user interface to appear to signal that we've successfully logged-in. As a tester I need to resolve the differences between the Selenese model of my application and the actual application state. This is not fun, error prone, and difficult to teach. Appcelerator does a good job of providing quality element ID values. Consequently I find myself in Firebug and Firefox a lot more than I should to get my Selenium test to run properly.

b) Selenium has no apparent way to provide a test with operational data. I am in the process of writing Data Production Libraries (DPLs) for Selenium IDE as a user extension. My first DPL reads from comma-separated-value (csv) flat files. The DPL opens a file and provides getNext and getPrevious methods to retrieve data from the csv file. I had to write this entirely in JavaScript and it was not fun. I will contribute these to the Selenium project in the next few weeks.

c) Selenium IDE recorded test scripts need to be exported into a scripting language format to implement business test flows. Conditionals, looping, and branching are possible using community contributions. For example, the Selenium Wiki includes a flow control extension.

For the past month I have been working on a design that enables TestMaker to operate Selenium tests. Here is what we have in mind:

1) Browser compatibility tests using Selenium IDE and Selenium RC.

2) Load and performance testing using Selenese scripts and TestMaker.

3) Service monitoring using Selenese scripts and TestMaker.

If this interests you then please participate! The enhancement ticket at design notes for Selenium in TestMaker are at http://bugs.pushtotest.com/ticket/136

-Frank

I am planning to attend the Redmonk Unconference at CommunityOne on Monday. The Unconference is at Moscone Center in San Francisco on May 5, 2008. I will propose myself to talk about data-driven Ajax performance testing using Selenium and PushToTest.

Last year's Unconference was terrific.  Instead of going to another track planned by Sun to hear from their marketing folks or a JavaOne conference sponsor, the Unconference topics are proposed by attendees and the selection is made by a democratic vote at the beginning of the day. The Unconference worked out splendidly. I am very much looking forward to the Unconference!

Please join me and I will be happy to take you around and introduce you to everyone I know.

-Frank

Luc Castera blogged about my talk at AjaxWorld. His review is at:

http://dambalah.com/2008/04/13/ajax-world-conference-day-3/

-Frank