Wood for the Trees

ActiveTest: Examination, Part II: Abstracting Without Basis

Posted by Mathew Abonyi Mon, 01 Jan 2007 04:37:50 GMT

My first mistake started the following cascade effect:

• move all assert_x_success and assert_x_failure to class methods
• begin defining any test case as a class method
• extract the definition of an instance method to the superclass level (ActiveTest::Controller)
• extract those superclass methods to abstract methods (ActiveTest::Subject)
• fragment assert conditions to be passed up the hierarchy (define_behavioural)
• extract fragments to the class level (assert_restful_get_success, method_missing, etc.)
• rinse & repeat for each Subject

Rather than inheriting from a single class with all its assertions self-contained, I now inherited from ActiveTest::Base -> Subject -> Controller. Along the way there were sprinkled methods on the class and eigenclass level which helped define the test cases (ActiveTest::Subject.define_behavioral) or provide extra assertions. I had extracted so much that it was a nightmare to understand what was actually happening (a rare problem when writing in Ruby).

An Extraction Case Study from 0.1.0 Beta (abridged)

module ActiveTest
  class Controller < Subject

    class << self

      def succeeds_on(action, options = {})
        define_behavioral(:succeed_on, action, options) do
          send("call_#{action}_action", options)
          send("assert_#{action}_success")
        end
      end

    end

    ...

    def assert_restful_get_success(action, options = {})
      assert_response :success
      assert_template action.to_s
    end

    ...

    def method_missing(method, *args)
      options = args.last.is_a?(Hash) ? args.dup.pop : {}
      if method.to_s =~ /^assert_(.*)_(success|failure)$/
        action, sf = $1, $2
        if action !~ /get|post|put|delete/
          request_method = self.class.determine_request_method(action)
          send("assert_restful_#{request_method}_#{sf}", action, options)
        end
      elsif method.to_s =~ /^call_(.*)_action$/
        request_method = self.class.determine_request_method(action = $1)
        send("call_request_method", request_method, action, options)
      else
        super
      end
    end

    def call_request_method(request_method, action, options = {})
      options = expand_options(options)
      send(request_method, action, options[:parameters])
    end

  end

end

As you can see, I left out quite a few secondary methods which are being called. The first problem here is in the design of defining a behaviour: it gives too many hooks to the user that allow them to customise. If I am trying to address common cases, this should have been a red flag to a bull—I was blind. The result was a ton of method_missing calls because I extracted every call into a dynamic method, including the assertions in a dynamically defined test case (twice, in fact). Awful.

What Was I Thinking?

When I first began writing ActiveTest, I focused solely on making test cases a one-line business. ActiveTest contained just a simple class method on ActiveTest::Subject to define any kind of test case and a number of classes inheriting from it that defined common groups of assertions. Then, I abstracted the entire process into defining ‘behaviours’, approaching each test case as a behaviour request-response pair (in fact, it is a context if you use BDD terminology). I went through a number of phases looking for the cleanest way to implement this idea of behaviours (I aped RSpec, Test::Rails and others). None of them seemed appropriate, so my own metalanguage was written that looked clean (but only when I did not provide parameters). I thought the metaprogramming of tests with a descriptive class method call would be self-explanatory enough.[1]

Take away from this one single maxim, by which you should live and die when you refactor: do not abstract without real-life need. If you don’t base your abstraction on something real, you are theorizing—and we all know that the problem with theories is that they have not been tested.

Coming Up Next: Code Bloat…

Footnotes

¹ Recently I encountered a situation where parameters were impossible to pass. This unforeseen situation (where I needed RFuzz:http://rfuzz.rubyforge.org/) contributed to my realisation that ActiveTest’s current design is useless for anything moderately dynamic or complex.