Posted by Mathew Abonyi
Tue, 02 Jan 2007 14:49:32 GMT
There are three aspects of ActiveTest which really need to be cleaned up: the Subject abstract class, metaprogramming subjects, and the extension of Test::Unit.
The Subject Abstraction
It is pretty obvious now, without a real need for define_behavioral, that ActiveTest::Subject as an interface class is an unnecessary intermediary between Base and the subjects which inherit from it. So trim the fat and move all of Subject’s functionality to Base. The result is a cleaner hierarchy and less temptation to put something in that middle-man model.
class ActiveTest::Controller < ActiveTest::Base
end
This change in design clears up the association between subjects and the base. Base, which was originally a general abstract for any kind of test case, becomes a general abstract for subjects. The change is mostly conceptual. Functionally, it still provides the rudiments of an ActiveTest suite: inheritance and nested setup/teardown. The next logical step, then, is to start using those advantages with a concrete class: Controller, for example.
Subject Metaprogramming
The only concrete subject written so far is Controller and it is a harbinger of doom for the others. It is bloated with metaprogramming, namely succeeds_on and its peers, because of a misunderstanding of where to use macros. Do we need macros in tests? Not to define them—Ruby is clean enough to do it already. If macros clean up readability, then when does something become muddied in a test? Consider the following real situation:
def test_should_add_quantity_for_product
@cart = carts(:first)
@item = @cart.line_items[0]
assert_no_difference @cart.line_items, :count do
assert_difference @item, :quantity, 1, :reload do
@cart.add_product(@item.product.id)
end
end
end
What’s going on? In this example from a simple test on shopping carts, you want to ensure that the quantity of an item in a cart is incremented when a duplicate item is added to the cart. The item and cart objects are associated through the line_items join model. Because the Cart model updates Item, the item instance needs to be reloaded to reflect changes in the database.
Essentially, we want to test changes being propagated down a hierarchy of Record associations. This situation is not so uncommon and we can easily think of more instances. Once we have a repetition of this pattern, we can write a macro to open up the hierarchy:
expand_hierarchy :cart_to_items, :reload => true do |a|
a.cart = carts(:first)
a.line_items = a.cart.line_items
a.item = a.line_items[0]
end
def test_should_add_item_quantity_for_product
cart_to_items do |a|
assert_difference a.item, :quantity do
a.cart.add_product(a.item.product.id)
end
end
end
def test_should_increment_line_items_for_product
cart_to_items do |a|
assert_difference a.line_items, :count do
a.cart.add_product(a.item.product.id)
end
end
end
The expand_hierarchy macro here helps us remove any extra tests or setup from one test case as well as make it clearer what we are testing for: we want to find, within a hierarchy, the correct change. The easiest way to do that is to expand out the parameters. You can think of it as a ‘setup-on-demand’. Also, when any one instance is updated, the others are out of sync. The :reload flag tells the macro to offer up proxies of each object; when one changes, all of the other objects are refreshed automatically.
expand_hierarchy, then, does two things: it offers a standard interface for accessing hierarchical associations and it removes extra setup within a test case. Both make it easier to read what is happening, I’d say, but don’t effect the documentation quality. In fact, it seems to improve it. So rather than using macros to define the test cases themselves, a better use would be to package non-assertive elements of testing. These programmatic elements do not need to appear in each test case—they belong in a library, which is what ActiveTest can standardise.
Extending Test::Unit
The premise of ActiveTest began to sour when it became a tool to move test cases into stock meta-definitions. The idea of templating is fine, but the meta-programming aspect was a serious flaw. It hid all the guts in a DSL rather than gave a transparent way of writing tests faster and more easily. The reason it went in this direction is that it is the same direction taken by Test::Unit: linear testing.
So the question is: should ActiveTest extend or evolve? When looking at the problems with Test::Unit and the original design of ActiveTest, it may not be so bad of an idea to address the real issue which has been skirted around by many Ruby programmers for a long time now: Test::Unit isn’t flexible enough. I do not wish to trod over Nathaniel Talbott’s work, but rather to address problems that arise in more complicated environments, like Rails, and are not suited to Test::Unit.
The new version of ActiveTest, I think, should live up to its namesake and really be its own testing framework, written to be compatible with Test::Unit, but with a design that can be improved by others without lengthy review of its code. If the only change, in the worst case scenario, is to change Test::Unit::TestCase to ActiveTest::Base, then there is no issue of learning a new DSL. Instead, ActiveTest would use the same language as Test::Unit, but be completely different under the hood and consequently include much more.
Coming Up Next: ActiveTest::Redesign…
Posted in Discoveries, Ruby, Rails, Plugins | no comments | 3 trackbacks
Posted by Mathew Abonyi
Tue, 02 Jan 2007 04:13:42 GMT
DRY and Abstraction: There is a difference between practices of DRY and Abstraction. When you are being DRY, you are either addressing a new repetition or continuing an earlier practice. The primary focus of being DRY is not to create higher level tools for future use. It is a practice of improving past and present code. It replaces, deletes, compresses, encapsulates—it is passive redesign. Abstraction, on the other hand, is active design. When you try to be DRY without a real model, you are actually abstracting. The process of abstraction is a wider programming activity which can lead quite naturally into over-design, because without real models and implementation, it often fails to address real issues.
With that short sermon, let’s make ActiveTest useful again. As I mentioned in the introduction to this mini-series, there are things to take away from the old version of ActiveTest. We’ll now look at them in full and show why they are still useful.
1. Test::Unit inheritance: filtering classes
One of the few patches in the old ActiveTest is to modify Test::Unit’s very well hidden collector filter. The idea of filtration is critical to making inheritance possible. If you have a situation where many ActiveRecord models share many attributes, for example in a Single Table Inheritance for a ‘Content’ model, and they all have a body, title, excerpt, owner, created_at, created_by, updated_at and updated_by, the tests for these attributes will be identical across all model tests. So why not create a ‘ContentTestCase’ suite which each model test inherits? With Test::Unit you can make a ‘StandardContentTests’ module and mix it in, but then you are looking for a place to put them and later looking around for those abstracted tests. Alternatively, if you want to use inheritance, those tests will be run immediately. You want to filter out the abstract class, but still be able to inherit from it. By modifying Test::Unit’s collector filter, it is possible to put anything in the ActiveTest namespace and it will not be run:
require 'test/unit/autorunner'
class Test::Unit::AutoRunner
def initialize_with_scrub(standalone)
initialize_without_scrub(standalone)
@filters << proc { |t| /^ActiveTest::/ =~ t.class.name ? false : true }
end
alias_method :initialize_without_scrub, :initialize
alias_method :initialize, :initialize_with_scrub
end
I won’t go on about the peculiarity of having the filter in AutoRunner, but the inaccessibility of the filter requires a monkey patch like the one above. Just poking into initialize lets us add to the @filters instance variance on AutoRunner and tell it to ignore the ActiveTest namespace. This technique is more about just reading through the code and finding the right place to patch.
2. Metaprogramming techniques (not all of them)
Wrapping define_method has proven to be pretty pointless other than ensuring an unique method name, but developing a lower-level language to standardise class-level macros is a general idea which ActiveTest could keep with some minor adjustments. Ignoring what define_behavioral does, this is a very short macro definition (from ActiveTest::Controller):
def assigns_records_on(action, options = {})
define_behavioral(:assign_records_on, action, options) do
send("call_#{action}_action", options)
assert_assigned((options[:with] || plural_or_singular(action)).to_sym)
end
end
If define_behavioral is made to register behaviors for a better parsing method than using method_missing, then we give it some meaning, but not really enough to keep it. The rest of the method definition, however, is quite clean for creating macros for Subjects, especially if it is compressed in the way mentioned in Part III. Making test macros, however, needs to be dropped in the way it is being used here, or at least used more sparingly. We could keep the setup macro because it sets up useful instance variables and makes reasonable guesses about the test suite’s environment. So, the macro idea should be kept back for special cases, the current implementation entirely dropped.
3. Nested, self-contained setup methods through sorcery method unbinding & stacking
I can’t help but be partial to the way I nested setup and teardown methods. It is my first actually useful innovation in Ruby—a trickery of method unbinding and stacking. Perhaps this bias makes me think it is still useful, but I honestly think it is a useful way to wrap Test::Unit. Let’s have a look at the way it is done in ActiveTest::Base:
class_inheritable_array :setup_methods
class_inheritable_array :teardown_methods
self.setup_methods = []
self.teardown_methods = []
def setup_with_nesting
self.setup_methods.each { |method| method.bind(self).call }
end
alias_method :setup, :setup_with_nesting
def teardown_with_nesting
self.teardown_methods.each { |method| method.bind(self).call }
end
alias_method :teardown, :teardown_with_nesting
def self.method_added(symbol)
unless self == ActiveTest::Base
case symbol.to_s
when 'setup'
self.setup_methods = [instance_method(:setup)]
remove_method(:setup)
when 'teardown'
self.teardown_methods = [instance_method(:teardown)]
remove_method(:teardown)
end
end
end
The first aspect which is useful is allowing setup and teardown nesting without affecting additions to Test::Unit made before or after loading ActiveTest. This is allowed by the code in method_added. Before ActiveTest is loaded, the methods and aliases already exist on Test::Unit; after ActiveTest is loaded, all setup or teardown methods are stacked in a class inherited array and subsequently undefined from the original class. When setup is finally called on a subclass of ActiveTest::Base, it is the ActiveTest::Base method called first. Here is an example stack:
class Test::Unit::TestCase
def setup; puts "a"; end;
end
class Test::Unit::TestCase
def setup_with_fixtures; puts "b"; end
alias_method_chain :setup, :fixtures
end
class ActiveTest::Sample < ActiveTest::Base
def setup; puts "c"; end
end
class SampleTest < ActiveTest::Sample
def setup; puts "d"; end
end
Upon execution, it will output: d, c, b, then a. A caveat of this technique is that you must be absolutely certain that each unbound method setup_with_nesting executes must be rebound to the original class or one of its descendants. Because of the way class inherited attributes work, this rule is not violated: SampleTest can run the setup method from ActiveTest::Sample because it is a descendant, but other classes inheriting from ActiveTest::Sample will not have methods from SampleTest.
4. Specificity of breaking down things into Subjects
This is more of a design note than actual programming. Presently in Test::Unit, it is only possible to add to Test::Unit::TestCase, from which every test suite inherits. There is no way to say that, for example, assert_difference for an ActiveRecord test is slightly different to 'assert_difference for an ActionController test. The functionality may be the same (for example, refreshing the Record or refreshing the Controller request), but implemented differently for each. By creating subclasses of a test suite that provide only the methods needed by that kind of test, the developer can easily encapsulate test methods and not leak public methods across different kinds of tests. It’s just cleaner and will definitely form the backbone of test suites in the new ActiveTest framework.
Coming Up Next: Redesigning Weak Areas…
Posted in Discoveries, Ruby, Rails, Plugins | 1 comment | 1 trackback
