Wood for the Trees

ActiveTest: Examination, Part V: Redesigning Weak Areas

Posted by Mathew Abonyi Tue, 02 Jan 2007 14:49:32 GMT

There are three aspects of ActiveTest which really need to be cleaned up: the Subject abstract class, metaprogramming subjects, and the extension of Test::Unit.

The Subject Abstraction

It is pretty obvious now, without a real need for define_behavioral, that ActiveTest::Subject as an interface class is an unnecessary intermediary between Base and the subjects which inherit from it. So trim the fat and move all of Subject’s functionality to Base. The result is a cleaner hierarchy and less temptation to put something in that middle-man model.

class ActiveTest::Controller < ActiveTest::Base
end

This change in design clears up the association between subjects and the base. Base, which was originally a general abstract for any kind of test case, becomes a general abstract for subjects. The change is mostly conceptual. Functionally, it still provides the rudiments of an ActiveTest suite: inheritance and nested setup/teardown. The next logical step, then, is to start using those advantages with a concrete class: Controller, for example.

Subject Metaprogramming

The only concrete subject written so far is Controller and it is a harbinger of doom for the others. It is bloated with metaprogramming, namely succeeds_on and its peers, because of a misunderstanding of where to use macros. Do we need macros in tests? Not to define them—Ruby is clean enough to do it already. If macros clean up readability, then when does something become muddied in a test? Consider the following real situation:

  def test_should_add_quantity_for_product
    @cart = carts(:first)
    @item = @cart.line_items[0]
    assert_no_difference @cart.line_items, :count do
      assert_difference @item, :quantity, 1, :reload do
        @cart.add_product(@item.product.id)
      end
    end
  end

What’s going on? In this example from a simple test on shopping carts, you want to ensure that the quantity of an item in a cart is incremented when a duplicate item is added to the cart. The item and cart objects are associated through the line_items join model. Because the Cart model updates Item, the item instance needs to be reloaded to reflect changes in the database.

Essentially, we want to test changes being propagated down a hierarchy of Record associations. This situation is not so uncommon and we can easily think of more instances. Once we have a repetition of this pattern, we can write a macro to open up the hierarchy:

  expand_hierarchy :cart_to_items, :reload => true do |a|
    a.cart = carts(:first)
    a.line_items = a.cart.line_items
    a.item = a.line_items[0]
  end

  def test_should_add_item_quantity_for_product
    cart_to_items do |a|
      assert_difference a.item, :quantity do
        a.cart.add_product(a.item.product.id)
      end
    end
  end

  def test_should_increment_line_items_for_product
    cart_to_items do |a|
      assert_difference a.line_items, :count do
        a.cart.add_product(a.item.product.id)
      end
    end
  end

The expand_hierarchy macro here helps us remove any extra tests or setup from one test case as well as make it clearer what we are testing for: we want to find, within a hierarchy, the correct change. The easiest way to do that is to expand out the parameters. You can think of it as a ‘setup-on-demand’. Also, when any one instance is updated, the others are out of sync. The :reload flag tells the macro to offer up proxies of each object; when one changes, all of the other objects are refreshed automatically.

expand_hierarchy, then, does two things: it offers a standard interface for accessing hierarchical associations and it removes extra setup within a test case. Both make it easier to read what is happening, I’d say, but don’t effect the documentation quality. In fact, it seems to improve it. So rather than using macros to define the test cases themselves, a better use would be to package non-assertive elements of testing. These programmatic elements do not need to appear in each test case—they belong in a library, which is what ActiveTest can standardise.

Extending Test::Unit

The premise of ActiveTest began to sour when it became a tool to move test cases into stock meta-definitions. The idea of templating is fine, but the meta-programming aspect was a serious flaw. It hid all the guts in a DSL rather than gave a transparent way of writing tests faster and more easily. The reason it went in this direction is that it is the same direction taken by Test::Unit: linear testing.

So the question is: should ActiveTest extend or evolve? When looking at the problems with Test::Unit and the original design of ActiveTest, it may not be so bad of an idea to address the real issue which has been skirted around by many Ruby programmers for a long time now: Test::Unit isn’t flexible enough. I do not wish to trod over Nathaniel Talbott’s work, but rather to address problems that arise in more complicated environments, like Rails, and are not suited to Test::Unit.

The new version of ActiveTest, I think, should live up to its namesake and really be its own testing framework, written to be compatible with Test::Unit, but with a design that can be improved by others without lengthy review of its code. If the only change, in the worst case scenario, is to change Test::Unit::TestCase to ActiveTest::Base, then there is no issue of learning a new DSL. Instead, ActiveTest would use the same language as Test::Unit, but be completely different under the hood and consequently include much more.

Coming Up Next: ActiveTest::Redesign…