Syspec: System Specifications & Tests using a Rake-like batch file

Posted by Mathew Abonyi Sat, 10 Feb 2007 20:45:43 GMT

Syspec lets you write a batch file a lot like a Rake file, but in common HTTP-style language, to test the existence and proper functioning of all the domains, subdomains, their pages and responses. This project is still very much in its infancy, but I decided to get it out there to get some feedback while I use it to track production servers. This is a sample Syspecfile:

check :"www.rubyonrails.org" do
  accept 200
  get "/"
  accept 404
  get "/syspec"
  get "/lacrima"
end

The above will check that http://www.rubyonrails.org/ responds with 200 and http://www.rubyonrails.org/syspec and http://www.rubyonrails.org/lacrima respond with 404. You can have multiple checks with the same domain and it does not have to be a symbol, though I prefer it that way.

Currently, Syspec features the following:
  • accept(*codes) – response code should be among these codes
  • get, post, delete, put (uses fetch(:request_method, url))
Upcoming features:
  • login “username”, “password”
  • body matching
  • forms
You can grab a copy of it directly from my SVN repository here: 
  svn co http://mabs29.googlecode.com/svn/trunk/gems/syspec .
Or download the gem from within the SVN repository (later will be moved to rubyforge): 
  svn export http://mabs29.googlecode.com/svn/trunk/gems/syspec/versions/syspec-0.2.1.gem .

Enjoy!

Posted in ,  | no comments | 2 trackbacks

ActiveTest::Redesign < ActiveTest::Examination

Posted by Mathew Abonyi Tue, 02 Jan 2007 14:55:27 GMT

The following is taken straight from the first two sections of the ActiveTest Redesign Draft. Feel free to comment, ask for features, steer me away from the unnecessary or just say you’re interested.

1.0: ADDRESSING A NEED

This section takes a cursory look at the evolution of ActiveTest.

1.1: The Original Premise

I originally started writing ActiveTest to address problems I had with Test::Unit. It lacked the following for my projects:

  • inheritance
  • modularity (adding)
  • flexible (redefining)
  • granular verbosity
  • observing
  • BDD interface

The most annoying omissions were the first two. I felt that no matter how much documenting my tests were doing, I hated breaking with the habit of not repeating myself. There are simply times when different parts of an application share a common interface or set of functions. That’s all there needs to be said in the tests.

1.2: Problems with ActiveTest 0.1.0

The first beta of ActiveTest was a flop; it was too specific to Rails, especially controllers, and used too many metaprogramming techniques. Consequently, it created doubt about the failures in test cases—were they ActiveTest or real? It mildly discouraged TDD and, as seen in my small series of articles on the mistakes I’ve made, there was too much focus on the One-Liner™.

1.3: Current State of Need

From what I can tell, there are three related schools of thought about testing which have members in the Ruby community: Test Driven Development, Behaviour Driven Development and Story Driven Development. The Agile and XP groups of developers select whichever according to taste. ActiveTest can easily cater to all schools and groups.

Looking at Test::Unit and comparing it to other projects, it is apparent that Rubyists are being left high and dry. Concessions have come in the form of ZenTest for automation, RSpec for BDD, RFuzz for fuzzing, Mocha for mocking, and rcov for coverage. Each of them (except RSpec) are forced to extend an inherently closed model, Test::Unit::TestCase, and the arcane classes which drive it.

2.0: APPROACHING THE REDESIGN

This section explains the purpose and approach for the new ActiveTest framework.

2.1: Purpose

The new ActiveTest is to provide an advanced, backward-compatible replacement for Test::Unit. By this design, it is compatible with many projects already extending Test::Unit, allowing for quick uptake and usage. It is also to be at the forefront of testing theories and advanced testing techniques both within and outside the Ruby community. Ultimately, it’s purpose is to make developers in all other languages very, very jealous.

2.2: Summary

ActiveTest must change from being a Rails plugin to a gem. It must be compatible with other gems and plugins within the Ruby community or, if compatibility is too complicated, incorporate their ideas with due credit. It must take head-on the latest in BDD and SDD styles. It must reuse as much of its old code to this purpose as possible. Finally, it must draw on projects in other languages, most notably Java and Python, to make use of the latest developments.

2.3: Influenctial Ruby projects

2.4: Influential projects in other languages

2.5: Objectives

The following features from Test::Unit:

  • assertions
  • setup & teardown
  • interfaces: Console, Fox, GTK, GTK2, TK
  • auto-running
  • filtering
  • backtrace
  • text report
  • unit tests

The following features from Ruby projects are compatible:

  • mocking and stubbing (through Mocha)
  • fuzzing (through RFuzz)
  • coverage reports (through rcov)

The following features incorporated from other projects:

  • data fixtures (from ActiveRecord)
  • automation (from ZenTest)
  • colorized output (from ZenTest & Testoob)
  • granular test selection (from Testoob)
  • different report formats (from Testoob)
  • test threading (from Testoob & GroboUtils)
  • Test Anything Protocol (from TAP, Test::Harness & Test::More)

The following features are specific to ActiveTest:

  • co-existence of TDD, BDD & SDD testing styles
  • inheritance
  • test type variety: unit, integration, system, performance
  • granular verbosity (backtrace, debug flags)
  • interface: Web/AJAX

Posted in ,  | no comments | 2 trackbacks