The Macaca Test Framework

Writing tests

The Macaca test framework is designed to make it as easy as possible to write tests. It is especially well-suited for writing unit tests, i.e. tests that test a single class. The easiest way to write a test in the Macaca framework is to write a class that extends com.javamonkey.test.AutoTest and that contains one or more public methods that take no arguments and return a boolean and that have names starting with "test". For example a trivial, but well-formed, test class is:

package foo.bar.baz;

import com.javamonkey.test.AutoTest;

/**
 * An example test class.
 */
public class FooTest extends AutoTest {

  /**
   * This test case always passes.
   */
  public boolean testSuccess() { return true; }

  /**
   * This test case always fails.
   */
  public boolean testFailure() { return false; }
}

However, often you want to know more than just whether a test case passed or failed—especially when it failed. To that end, AutoTest defines three methods:

  public boolean success(String msg);
  public boolean failure(String msg);
  public boolean failure(String msg, Throwable t);

Another feature that AutoTest provides is the ability to make test cases depend on other test cases. For example, suppose in our trivial test class above, we only want to run the testFailure() test case if testSuccess() had in fact succeeded. We could write our test class like this:

package foo.bar.baz;

import com.javamonkey.test.AutoTest;

/**
 * An example test class.
 */
public class FooTest extends AutoTest {

  /**
   * This test case always passes.
   */
  public boolean testSuccess() {
    return success("ok.");
  }

  /**
   * This test case always fails.
   */
  public boolean testFailure() {
    require("testSuccess");
    return failure("this always fails.");
  }
}

When this test class is run, AutoTest will invoke the test methods in an arbitrary order but the call to require() will cause the required test method, testSuccess() to be run, if it hasn't already, and then either return normally if it passed or throw a runtime exception (PreconditionNotMetException) if it failed. The later result will cause testFailure() to be reported as a skip.

Note that in the absence of require() calls, the test methods run in an essentially arbitrary order. If you want a series of methods to run in a certain order, you need to chain them together with calls to require(). The idea is that each test method should be be atomic, meaning it tests one thing, and independent, meaning it can be run independently. However those goals are often in conflict: Suppose I have a class that lets me read a file and the read from it. In order for my test methods to be atomic I should have two: one that tests the opening functionality and another that tests the reading. But the read test can't really be independent—it only makes sense if the open test has run and passed. The require() method is designed to solve this problem.

Test results

In the Macaca framework, all test cases should produce some result everytime a test class is run. There are three possible results: PASS, FAIL, and SKIP. PASS means that the given test case ran and performed as expected; FAIL means the test case ran but did not perform as expected; and SKIP means the test case did not run at all, usually because some other test case it depended on did not pass.