Since JUnit appeared in 1997, the simple testing framework has enjoyed tremendous popularity. It has spawned a legion of counterparts for other languages (VBUnit, CppUnit, DelphiUnit to name a few) and has been extended to incorporate a variety of test scenarios and criteria e.g. Cactus for testing J2EE components and JUnitPerf for performance testing. However the focus in the JUnit world always seems to be upon implementation specifics rather than the design and structure of the tests themselves. Once you have access to the xUnit implementation for your platform, the question of how to use it effectively to write tests of adequate rigour seems to be left largely to intuition. This article will provide a strategy for structuring unit tests in JUnit that makes tests case selection more methodical and enables you to feel some confidence that you are adequately testing your classes. The strategy is based upon the principled use of Design by Contract in the code under test.
Design by Contract is perhaps the most valuable principle you can employ to produce reliable software. I am continually amazed at how little awareness there is of it in the development community. An introduction to DBC is beyond the scope of this article, but the information is readily available.
Eiffel is one of the few languages to support DBC with native language constructs. In Java, the best we can do is to imitate a subset of DBC using the assert keyword introduced in Java 1.4. With this technique, we can write contracts that represent a method's preconditions and postconditions, however class invariants and the inheritance of contracts cannot be fabricated in Java without resorting to the use of third party pre-processing tools such as JMSAssert.
Ideally we would like to select test cases for a class we have just written, so as to be certain that the class functions correctly. However an absolute proof of correctness requires the use of formal methods, which are too time consuming to be practical in real world software development. Unable to establish a class's correctness with certainty, we can at best hope to establish a high degree of confidence that the class is correct. Employing a methodical strategy for test case selection gives us some confidence that we haven't forgotten to test some particularly critical boundary condition or scenario. DBC provides the basis for such a strategy.
Thinking back to your second year Computer Science lectures, you'll recall that an Abstract Data Type is defined completely by four attributes:
A class, being an expression of an ADT in some programming language, is said to be a correct implementation of the ADT if it respects all four of the above attributes.
Each of these attributes has a programmatic equivalent in Java that we can exercise with our JUnit testing code.
It is at this point that theory and practice first diverge, due to the limitations of Java. Stricly speaking, axioms incorporate the concept of a class invariant, a set of statements that a correct implementation of an ADT should guarantee to be true at any observable instance. That is, the class invariant will be true at the time execution of a public method begins, and immediately after the execution of the method ends. It is effectively conjuncted with each method precondition and postcondition. Java provides no convenient means of expressing class invariants programmatically, so the theoretical gap this exposes defines one of the limits of this strategy.
In practice, the size of this gap is small. The net result is that we can obtain a high degree of confidence that our class is correct by simply verifying that the preconditions and postconditions of each method are correct. Such verification requires that the pre- and postconditions of each method be precisely stated.
Below is a simple class that represents a range of integers with inclusive limits. The pre- and postconditions have been explicitly stated in the method javadoc comment, although in practice you might choose not to be quite so formal. (you would also include the standard javadoc tags, which have been ommitted here for brevity). The implementation of each method has been omitted, because it is not relevant to this sort of black-box testing. We are only interested in externally observable behaviour. Indeed, when constructing test cases, it is useful to deliberately ignore the bodies of methods and focus solely on the correct behaviour of each method from the perspective of client code:
public class Range {
public Range(int aLower, int aUpper) { ... }
public int getLower() { ... }
public int getUpper() { ... }
public boolean intersects(Range anOther) { ... }}
The pre- and postconditions on each method are the key to developing a methodical strategy for test case generation.
The strategy is to construct a JUnit test case for each accessible method in the class under test, where each test case verifies both the preconditions and postconditions of the method.
The general pattern of a test case constructed using this strategy is:
public void test() {
// Test each precondition of by violating it and
// checking that an exception is thrown.
try {
// violate a precondition
fail();
} catch (Exception e) {
// precondition violation was detected by . OK.
}
// Invoke in such a way that the preconditions
// are satisfied. Once returns, test that each
// postcondition is true instance.
Assert.assertTrue(postcondition);
}
This leads to a JUnit test like the following:
public class RangeTest extends TestCase {
private Range mRange;
public RangeTest(String name) {
super(name);
}
protected void setUp() throws Exception {
super.setUp();
mRange = new Range(1, 5);
}
public void testConstructor() {
// pre: aLower < aUpper
try {
new Range(6, 4);
fail("[6,4] should be precluded");
}
catch (AssertionError e) { }
// pre: aLower == aUpper new Range(-3, -3);
// post: aLower == getLower()
Assert.assertEquals("[1,5].getLower() should be 1", 1,
mRange.getLower());
// post: aUpper == getUpper()
Assert.assertEquals("[1,5].getUpper() should be 5", 5,
mRange.getUpper());
}
public void testGetLower() {
// pre: object constructed
Assert.assertNotNull("[1,5] could not be constructed", mRange);
// post: getLower() == aLower provided to c'tor
Assert.assertEquals("[1,5].getLower() should be 1", 1,
mRange.getLower());
}
public void testGetUpper() {
// pre: object constructed
Assert.assertNotNull("[1,5] could not be constructed",
mRange);
// post: getUpper() == aUpper provided to c'tor
Assert.assertEquals("[1,5].getUpper() should be 5", 5,
mRange.getUpper());
}
public void testIntersects() {
// pre: anOther != null
try {
mRange.intersects(null);
fail("intersects(null) should be precluded");
}
catch (AssertionError e) { }
// post: true if two ranges have an integer in common
Assert.assertTrue(!mRange.intersects(new Range(-3, -1)));
Assert.assertTrue(!mRange.intersects(new Range(6, 7)));
Assert.assertTrue(!mRange.intersects(new Range(-2, 1)));
Assert.assertTrue(!mRange.intersects(new Range(5, 7)));
Assert.assertTrue(mRange.intersects(new Range(0, 2)));
Assert.assertTrue(mRange.intersects(new Range(3, 4)));
Assert.assertTrue(mRange.intersects(new Range(4, 8)));
}}
If these test cases pass, then we can be confident that all methods in Range are:
Note that this strategy only tells you what test cases to generate and gives a broad structure to the unit test itself. It doesn't tell you exactly how to verify the pre- and postconditions in each case. In some instances this verification is trivial. Sometimes it results in duplication of assertions across test cases. Sometimes you still have to think a bit about the possible mappings between pre- and postconditions, as in the postcondition checking in testIntersects(). But overall it provides you with a routine way of structuring your test cases that is likely to be at least rigorous enough to give you a high degree of confidence that the class under test is correct.
The key to success with this strategy is to incorporate the explicit statement of method pre- and postconditions into your coding habits, so that you are routinely documenting the contract between each method and it's callers in a precise way thereby increasing the testability of your classes.