Ajax Testing with MXUnit, Webdriver, and Firefox

"Release early and often", is an Agile mantra. I usually wait until something is more stable, but I'm particularly excited about this effort and hope to get some feedback quickly - even better, get some help!

One of the goals we have at MXUnit is to be able to perform not just unit testing, but other types of testing as well - all inside of MXUnit and using CFML. One big gap and need I see is the ability to test ColdFusion applications via the the user interface and to test the UI itself. Testing has some catching up to do with the rapid adoption of jQuery, ExtJs, and other Javascript Ajax frameworks. One effort in particular that caught my attention is Webdriver (http://code.google.com/p/webdriver/). Simon Stewart's smart ideas behind it are to create a simple interface and to be able to use existing browsers. Webdriver has the ability to invoke Firefox and Internet Explorer natively as well as leverage HtmlUnit. HtmlUnit's main advantage is that it's fast. However, it does not handle JavaScript in the same way most user's browser do - it can, at best , emulate the browser. So, how can we then accurately automate testing in common browsers?

This effort, CFWebdriver Experimental (for the moment), allows you to write tests that call URLs, find page elements, write data to form fields, click buttons, and verify output. Again, the Webdriver interface is intended to be simple, which is very appealing.

Here's some example code:

<cfcomponent output="false" extends="mxunit.framework.TestCase">
<cfscript>
function doGoogleSearch(){
driver = createObject("component","cfwebdriver.WebDriver").newInstance("firefox");
driver.get("http://google.com/");
q = driver.findElement("q");
q.sendKeys("mxunit");
q.submit();
link = driver.findElementByLinkText("MXUnit Blog");
link.click();
assertEquals("MXUnit Blog", driver.getTitle());
}
</cfscript>
</cfcomponent>

What this does is to start Firefox, invoke Google's home page, find the input named "q", enter mxunit into the input, submit the form, find the link text MXUnit Blog, click that link, then verify the expected text in the title tag. It works nicely on Windows and Ubuntu! Here's a screen shot of MXUnit and Webdriver testing Google's Ajax Search API example (http://www.google.com/uds/samples/apidocs/helloworld.html)

This is included in the experimental download .... repeat ... experimental. ;-) Visit http://github.com/virtix/cfwebdriver/tree/master and click the Download button. I'll get this in the mxunit subversion repository this weekend - having problems accessing it M-F/9-5.

The installation instructions are in the README in the download, but it's basically copying the Webdriver jars into the ColdFusion classpath, and making sure they are loaded first. I've tested this on CF8 multi-instance installations on Windows and Ubuntu, and Railo 3 on Windows.

This is the preferred method of installation as of today:

1. Copy /webdriverlib to WEB-INF/lib/webdriverlib
2. Edit WEB-INF/web.xml adding the path to the webdriver jars above
to the BEGINING of the cf.class.path element:

<context-param id="coldfusion_context_88">
<param-name>cf.class.path</param-name>
<param-value>

./WEB-INF/lib/webdriverlib,./WEB-INF/cfusion/lib/updates,./WEB-INF/cfusion/lib,./WEB-INF/cfusion/gateway/lib,./WEB-
INF/flex/jars,./WEB-INF/cfform/jars</param-value>
</context-param>

3. Restart the cf instance

4. Try the example1.cfc test and the googleAjaxTest.cfc

Notes: The README should have enough info to get you up and running. The Internet Explorer implementation is very buggy when running inside of CF. It works sometimes and other times not. There's 2 CFCs (wrappers). These contains only some of the full functionality of Webdriver but should be enough to give you a good taste. If the CFC wrappers don't do what you need, you can always go down the java route and instantiate the java representations and work with those instead. Or better yet, contribute ;-)

Happy Ajax testing!

-bill

Automating Your Unit Tests: Start-to-finish example

Getting started with unit testing is hard for a lot of people. If you could count the number of people on January 2nd who say "This year I'm going to start unit testing", and then compare that with the number of people on March 1 who actually followed through, you'd get a graph that looked like... well, it'd look like the graph of the people who promised to start exercising, too.

Hard stuff, or stuff that's perceived to be hard, requires lots of different motivators to turn the behavior into a habit. One potentially overlooked motivator to keep you unit testing is test automation. Why? Because it acts as a type of feedback loop. You're reminded every day (or several times a day) of the work you've put in. There's comfort in seeing the Success-to-Fail ratio improve over time. When you get an email from your test automation code that says "Successes: 5" on Monday and then the next Monday you see "Successes: 10"... that's progress. You've written tests. You're getting the green bar. It's basic slot machine psychology.

So how do you automate?

Simple: you spend 10 hours installing a continuous integration server, write a few hundred lines of ANT script, and bam, you're done.

So, get to it!

.........

What? Oh. You don't need that yet. You only have a few dozen tests and your boss is already up your a$$ telling you to quit testing and start programming (tee hee). OK, well, i guess you could just do it the easier way then. It's boring. It's plain-jane. And it certainly won't get you chicks. But it will get you automated tests and notifications and all that stuff.

Without further adieu, here it is: simple test automation using good old CFAdmin scheduled task and some pretty simple code (that you can copy and paste). It'll take you longer to read this documentation than it will to have your tests automated.

Good luck!

--marc