Automating Your Unit Tests: Start-to-finish example

Getting started with unit testing is hard for a lot of people. If you could count the number of people on January 2nd who say "This year I'm going to start unit testing", and then compare that with the number of people on March 1 who actually followed through, you'd get a graph that looked like... well, it'd look like the graph of the people who promised to start exercising, too.

Hard stuff, or stuff that's perceived to be hard, requires lots of different motivators to turn the behavior into a habit. One potentially overlooked motivator to keep you unit testing is test automation. Why? Because it acts as a type of feedback loop. You're reminded every day (or several times a day) of the work you've put in. There's comfort in seeing the Success-to-Fail ratio improve over time. When you get an email from your test automation code that says "Successes: 5" on Monday and then the next Monday you see "Successes: 10"... that's progress. You've written tests. You're getting the green bar. It's basic slot machine psychology.

So how do you automate?

Simple: you spend 10 hours installing a continuous integration server, write a few hundred lines of ANT script, and bam, you're done.

So, get to it!

.........

What? Oh. You don't need that yet. You only have a few dozen tests and your boss is already up your a$$ telling you to quit testing and start programming (tee hee). OK, well, i guess you could just do it the easier way then. It's boring. It's plain-jane. And it certainly won't get you chicks. But it will get you automated tests and notifications and all that stuff.

Without further adieu, here it is: simple test automation using good old CFAdmin scheduled task and some pretty simple code (that you can copy and paste). It'll take you longer to read this documentation than it will to have your tests automated.

Good luck!

--marc

Bad Habit?

Something about unit testing in general that really struck me recently after reading http://homepage.mac.com/hey.you/lessons.html was the fact that as developers we really do unit testing all the time. This is usually in the form, code -> add cfoutput or cftrace -> Alt+Tab to browser -> Ctrl+R to refresh, and make a visual assessment of the output. If it looks good, we go back to code -> remove or comment cfoutput or cftrace -> Alt+Tab to browser -> Ctrl+R to refresh, and make a visual assessment of the output. This is the way I was doing it for most of my career. If you think about it, there is a lot of really good information contained in this activity: (1) why was a cfoutput placed in the code?, (2) what was I looking for?, and (3) what was I expecting to see in the output? Ok, what really struck me, like a ball-peen hammer upside the head, was the fact that all this intellectual property gets thrown away when we go to production. During maintenance, I then repeat the same steps over - code, trace, view output. I'm starting to think this is a bad habit. I have to recall what I was trying to do in the code in the first place and why. With a saved unit test, which takes more time up front, I have an artifact immediately at my disposal that answers these questions and let's me know when my code breaks. It also lasts for the life of the software and serves as really good documentation.

This code illustrates that I want to see that the generated output from this component is in fact XML, JUnit XML, and HTML. I would be looking for specific test result numbers in addition to format info. This is now all preserved in code so I won't forget.

<cfcomponent generatedOn="12-12-2007 4:35:38 AM EST" extends="mxunit.framework.TestCase">
 

<cffunction name="testRun">
  Tests the AntRunner and makes sure the generated content is aok
  <cfsavecontent variable="actual">
  <cfinvoke component="#this.httpAntRunner#"  method="run">
    <cfinvokeargument name="type" value="dir" />
    <cfinvokeargument name="value" value="#expandPath("../framework/fixture/fixturetests")#" />
    <cfinvokeargument name="packagename" value="mxunit.httpantrunnertests" />
    <cfinvokeargument name="outputformat" value="xml" />
  </cfinvoke>
  </cfsavecontent>
  <!--- CF parser doesn't care for xml declaration ... --->
  <cfset actual = replace(actual, '<?xml version="1.0" encoding="UTF-8"?>','','one')>
  <cfset rsDom = xmlParse(actual) />
  <cfset assertisXmlDoc(rsDom) />
  <cfset debug(rsDom.xmlroot.xmlAttributes) />
  <cfset assertEquals(rsDom.xmlroot.xmlAttributes["tests"],8,"Should only be 8 tests in this suite") />
  
  <cfsavecontent variable="actual">
  <cfinvoke component="#this.httpAntRunner#"  method="run">
    <cfinvokeargument name="type" value="dir" />
    <cfinvokeargument name="value" value="#expandPath("../framework/fixture/fixturetests")#" />
    <cfinvokeargument name="packagename" value="mxunit.httpantrunnertests" />
    <cfinvokeargument name="outputformat" value="junitxml" />
  </cfinvoke>
  </cfsavecontent>
  <cfset rsDom = xmlParse(actual) />
  <cfset assertisXmlDoc(rsDom) />
  <cfset debug(rsDom.xmlroot.xmlAttributes) />
  <cfset assertEquals(rsDom.xmlroot.xmlAttributes["tests"],8,"Should only be 8 tests in this suite") />

  
  <cfsavecontent variable="actual">
  <cfinvoke component="#this.httpAntRunner#"  method="run">
    <cfinvokeargument name="type" value="dir" />
    <cfinvokeargument name="value" value="#expandPath("../framework/fixture/fixturetests")#" />
    <cfinvokeargument name="packagename" value="mxunit.httpantrunnertests" />
    <cfinvokeargument name="outputformat" value="html" />
  </cfinvoke>
  </cfsavecontent>
  <!--- Search for this pattern:
    <title>Test Results [12/12/07 06:12:47] [127.0.0.1]</title>   
  --->
  <cfset found = refind("<title>Test Results \[[0-9]{2}/[0-9]{2}/[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}[ ]*.*</title>", actual, 0, true)>
  <cfset debug(found) />
  <cfset assertTrue(arrayLen(found.len) gt 0) />
  <cfset assertTrue(arrayLen(found.pos) gt 0) />
 
</cffunction>


<!--- Override these methods as needed. Note that the call to setUp() is Required if using a this-scoped instance--->

<cffunction name="setUp">
<!--- Assumption: Instantiate an instance of the component we want to test --->
<cfset this.httpAntRunner = createObject("component","mxunit.runner.HttpAntRunner") />
<!--- Add additional set up code here--->
</cffunction>
 

<cffunction name="tearDown"></cffunction>

</cfcomponent>