Selenium's Ajax limitations and new designs
by
Frank Cohen
—
last modified
May 15, 2008 11:49 PM
Filed Under:
Matt at Appcelerator
pointed me to a discussion of Automated
Testing of Appcelerator With Selenium on the Appcelerator
user forums. I am very interested in ways to use Selenium for testing
Appcelerator applications.
From TestMaker's perspective a Selenium test is a unit test of a Web application or Ajax application. TestMaker already provides Test Runners for TestGen4Web, soapUI, and the Java 6 ScriptEngine support for unit tests written in Java, Jython, JRuby, Groovy, and a bunch of others.
It seems to me that the forum members have already hit upon some of Selenium's limitations and design problems:
a) Selenium was not really developed for Ajax, and vice versa. Ajax frameworks like Appcelerator implement their own eventing mechanism and Selenium has no idea of how this works. For instance, when an Appcelerator log-in page changes the DOM to reveal site content Selenium IDE does not notice the change. Selenium IDE will sometimes record a 'click' when 'clickAndWait' is appropriate. I find myself adding 'waitForElementPresent' commands that look for the logout element in the user interface to appear to signal that we've successfully logged-in. As a tester I need to resolve the differences between the Selenese model of my application and the actual application state. This is not fun, error prone, and difficult to teach. Appcelerator does a good job of providing quality element ID values. Consequently I find myself in Firebug and Firefox a lot more than I should to get my Selenium test to run properly.
b) Selenium has no apparent way to provide a test with operational data. I am in the process of writing Data Production Libraries (DPLs) for Selenium IDE as a user extension. My first DPL reads from comma-separated-value (csv) flat files. The DPL opens a file and provides getNext and getPrevious methods to retrieve data from the csv file. I had to write this entirely in JavaScript and it was not fun. I will contribute these to the Selenium project in the next few weeks.
c) Selenium IDE recorded test scripts need to be exported into a scripting language format to implement business test flows. Conditionals, looping, and branching are possible using community contributions. For example, the Selenium Wiki includes a flow control extension.
For the past month I have been working on a design that enables TestMaker to operate Selenium tests. Here is what we have in mind:
1) Browser compatibility tests using Selenium IDE and Selenium RC.
2) Load and performance testing using Selenese scripts and TestMaker.
3) Service monitoring using Selenese scripts and TestMaker.
If this interests you then please participate! The enhancement ticket at design notes for Selenium in TestMaker are at http://bugs.pushtotest.com/ticket/136
-Frank
From TestMaker's perspective a Selenium test is a unit test of a Web application or Ajax application. TestMaker already provides Test Runners for TestGen4Web, soapUI, and the Java 6 ScriptEngine support for unit tests written in Java, Jython, JRuby, Groovy, and a bunch of others.
It seems to me that the forum members have already hit upon some of Selenium's limitations and design problems:
a) Selenium was not really developed for Ajax, and vice versa. Ajax frameworks like Appcelerator implement their own eventing mechanism and Selenium has no idea of how this works. For instance, when an Appcelerator log-in page changes the DOM to reveal site content Selenium IDE does not notice the change. Selenium IDE will sometimes record a 'click' when 'clickAndWait' is appropriate. I find myself adding 'waitForElementPresent' commands that look for the logout element in the user interface to appear to signal that we've successfully logged-in. As a tester I need to resolve the differences between the Selenese model of my application and the actual application state. This is not fun, error prone, and difficult to teach. Appcelerator does a good job of providing quality element ID values. Consequently I find myself in Firebug and Firefox a lot more than I should to get my Selenium test to run properly.
b) Selenium has no apparent way to provide a test with operational data. I am in the process of writing Data Production Libraries (DPLs) for Selenium IDE as a user extension. My first DPL reads from comma-separated-value (csv) flat files. The DPL opens a file and provides getNext and getPrevious methods to retrieve data from the csv file. I had to write this entirely in JavaScript and it was not fun. I will contribute these to the Selenium project in the next few weeks.
c) Selenium IDE recorded test scripts need to be exported into a scripting language format to implement business test flows. Conditionals, looping, and branching are possible using community contributions. For example, the Selenium Wiki includes a flow control extension.
For the past month I have been working on a design that enables TestMaker to operate Selenium tests. Here is what we have in mind:
1) Browser compatibility tests using Selenium IDE and Selenium RC.
2) Load and performance testing using Selenese scripts and TestMaker.
3) Service monitoring using Selenese scripts and TestMaker.
If this interests you then please participate! The enhancement ticket at design notes for Selenium in TestMaker are at http://bugs.pushtotest.com/ticket/136
-Frank
