Topic: Testing your model code...

I'm a recent convert to RoR, but have been attempting to practice TDD for about a year now in various languages. I'm finding it a little hard to properly test the models in RoR.  I've spent a lot of time developing some custom assertions to help drive thee development of validates_*, and the association links and these are getting closer to feeling acceptable to me.  However I haven't found a good way to TDD the migration in the first place -- ie. if we assume that the database truly is an application database, then it should be viewed as part of the application code-base and the generation of a migration file should only follow a failing test case...  Has anyone else explored possible ways of intelligently driving this aspect of model development?

How do you approach testing use of the built-in validations, and association link?  (has_many, etc)  Do you feel these fall under the category of "trivial" like a lot of accessors and don't bother testing them?  Do you test them indirectly by normal exercise of the class, etc?

Note that this does assume not just the regular agile development practices, but agile data modeling as well...

My RoR journey  -- thoughts on learning RoR and lessons learned in applying TDD and agile practices.

Re: Testing your model code...

I'd be interested whether this is an accepted way of doing things, but I create unit tests liberally and I create at least one to check each validation.  Here's a snippet from a page model I have that requires a filename:

  def test_needs_filename
    p = pages(:about)
    p.filename = nil
    assert_false p.save
    assert_equal 1, p.errors.length
    assert_equal "can't be blank", p.errors.on(:filename)
  end

It's pretty simple and it works well.  It tests that the model can't be saved, that there was one error, and that error was in the right place with the right content.  What it doesn't do is relate to the db migrations like you mentioned.

I believe testing associations and built-in validations is anything but trivial - all my code gets tested.

I also think that in order to tie the migratory state of the db to tests (like you mentioned) we'd have to either 1) always revert to a previous code version when testing against a previous migration or 2) require each test method to only execute if the current schema_info.version is above a certain level.

I haven't personally run into a situation where I needed agile data modeling beyond creating new migrations to handle changes.

Last edited by danger (2006-06-18 02:29:32)

Re: Testing your model code...

I've been moving towards something like

def test_validations
  assert_attribute_required :username, :length=>(4..20), :unique=>true
  assert_attribute_required :password, :length=>(4..20), :confirmed=>:password_confirmation
  assert_attribute_required :email
end

These are all custom assertions that I'm still working on cleaning up.  The tests basically wrap up the method you just showed, with a few other options.  Otherwise I feel that the unit test code gets very non-DRY.

I'm still working on a similar set of assertions for associations.

[PS, any chance we can get a Ruby syntax highlighter for this forum so we can use ruby blocks and not just code blocks?]

My RoR journey  -- thoughts on learning RoR and lessons learned in applying TDD and agile practices.

Re: Testing your model code...

I really like your custom assertions.  For some reason I was under the impression that tests needn't be too DRY.  I tend to repeat myself a lot in tests - maybe I should look into that.

For reference, could you post the code from your custom asserts?

Re: Testing your model code...

Sure, but please understand these are still a work in progress and the work of a neophyte ruby coder.  Plus they are getting complex enough that I want to split them off to a side project complete with their own suite of tests to test the test....  I find it enjoyable coding, but I shouldn't be so distracted from my main development efforts....

I also need to covert them to give better default failure messages, they should probably be changed to use assert_block in many cases, and I think I should extract a wrapper lambda/block to remove the duplicated code for retrieving the classname and deal with caching the attribute hash.

These currently live in test_helper.rb

class Test::Unit::TestCase
  self.use_transactional_fixtures = true
  self.use_instantiated_fixtures  = false
   

  protected
  def assert_unique_on (field)
    klass = class_under_test
    cached = @attr_hash.clone

    cached[field]=get_reference_fixture.send(field)
    current = klass.count
    tmp = klass.create(cached)
    assert_field_invalid tmp, field
    assert_equal current, klass.count

    tmp.destroy
  end
 
 
  def assert_attribute_required(field,opt={})
    klass = class_under_test
    cached = @attr_hash.clone

    cached[field]=nil
    obj = klass.new(cached)
    assert_not_valid obj
    assert_field_invalid obj, field

    assert_required_length(field,opt[:length],opt) if opt.has_key? :length
    assert_unique_on(field) if (opt.has_key?(:unique) && opt[:unique])
    assert_confirmation_required(field, opt[:confirmed]) if (opt.has_key?(:confirmed))
  end

  def assert_confirmation_required(field, confirm)
    klass = class_under_test
    cached = @attr_hash.clone

    cached[confirm]=cached[confirm].swapcase.reverse
    obj = klass.new(cached)
    assert_not_valid obj
    assert_field_invalid obj, field
  end

  def assert_required_association(field)
     field_id = (field.to_s + "_id").to_sym

     klass = class_under_test
     cached = @attr_hash.clone

     obj = klass.new
     assert_respond_to obj, ("build_"+field.to_s)
     field_id = (field.to_s + "_id").to_sym

     cached[field_id]=nil
     obj = klass.new(cached)
     assert_not_valid obj
     assert_field_invalid obj, field_id
   end
   def assert_optional_association(field, opts)
      klass = class_under_test
      obj = klass.new
      assert_respond_to obj, field
      assert_respond_to obj.send(field), :find
      assert_instance_of Array, obj.send(field)
    end
   

  def assert_not_valid (obj)
    assert_block("#{obj.to_s} should not be valid.") { !obj.valid?}
  end
 
  def assert_required_length(field,allowed,opt={})
    klass =class_under_test
    cached = @attr_hash.clone

    failing = [allowed.first-1,allowed.last+1]
    passing = [allowed.first,allowed.last].uniq
    for length in failing do
      populate_field_with_fixed_length_string cached, field, length, opt
      assert_not_valid klass.new(cached)
    end
    for length in passing do
      populate_field_with_fixed_length_string cached, field, length, opt
      assert_valid klass.new(cached)
    end
  end

  def populate_field_with_fixed_length_string(cached, field, length, opt)
    cached[field]='a' * length
    cached[opt[:confirmed]]=cached[field] if opt.has_key? :confirmed
  end
 
  def get_reference_fixture
    return class_under_test.find(1)
  end
 
  def class_under_test
    /([A-Z][A-Za-z_]*)Test/.match(self.class.to_s)
    $1.constantize
  end
 
  private
  def assert_field_invalid(obj, field)
    assert_block("#{obj.to_s}'s #{field.to_s} should be invalid.") {obj.errors.invalid?(field)}
  end
end


Using them, at present requires one thing -- you have to define a @attr_hash in your test setup.  This hash should have a parameter list that can potentially create a new, valid entity, but one that is NOT already in a fixture.  Many of the tests take this known good parameter list and then tweak it to create a should fail case.  Either by blanking out a field, or grabbing a duplicated value from an arbitrary fixture, etc

Please ignore the "assert_[required|optional]_association" assertions those are initial essays into the solution space, but are no where near solid enough yet.

Last edited by NielsenE (2006-06-18 02:59:57)

My RoR journey  -- thoughts on learning RoR and lessons learned in applying TDD and agile practices.

Re: Testing your model code...

danger wrote:

I also think that in order to tie the migratory state of the db to tests (like you mentioned) we'd have to either 1) always revert to a previous code version when testing against a previous migration or 2) require each test method to only execute if the current schema_info.version is above a certain level.

That's not quite what I meant by driving the migrations.  What I'm trying to find a good way to express in test code is the notion of "this class must include the following persist-able attributes"  The simple fact that the class respond_to? the names of the various accessors/mutators isn't enough to imply that those attributes are round-tripped to the DB.  As well as the notion that some attributes aren't persisted -- take a stereotypical User class.  It'll have to deal with "raw" password and password confirmation fields that aren't persisted and the hashed password that is persisted, but set indirectly, etc.

I've started writing a "baby-steps" approach to this, but I'm not happy with where its going, so I'm letting it stew while I hope for inspiration... It needs to be updated to my current level of understanding,  but ....  http://www.verticalexpressionsoftware.c … s-and-tdd/  the back 2/3rd are most applicable to what I'm struggling with.

danger wrote:

I really like your custom assertions.  For some reason I was under the impression that tests needn't be too DRY.  I tend to repeat myself a lot in tests - maybe I should look into that.

I first learned TDD from the Astel book, and he was extremely aggressive about refactoring test code.  And I've normally found that being aggressive in refactoring test code keeps you more honest in refactoring production code.   In some sense its the "Broken Window" effect that the Pragmatic Programmers talk about, once you let one place get sloppy, it rapidly degenerates.  In the book for instance he'll have a separate test case for testing a zero element list, versus testing a one-element list.  Both test cases needed a custom setup, that was shared by multiple test methods, thus leading to their separation.  Furthermore often the two setups are slightly similar so you end up extracting a common superclass for them both to inherit from, etc..  The test code quickly gets very DRY.  However it does make naming even more important, a lot of your test case setup (called a fixture in other languages, but not too exactly equal to Rails fixtures) can get pushed to super classes and you might not be able to tell at a quick glance exactly what the state of the object was in pre-test.  If your naming is clear, that solves that problem.  (Rails conventions and those of Test::Rails on top, can get slightly in the way here as well).

Rails makes it slightly hard to refactor functional (or controller under Test::Rails) tests as some of the automated plumbing can get confused, but I've been working through that most of today, and have a workable solution -- ie one that meets my needs and is clean, but not one that is releasable as a plugin/patch.

Last edited by NielsenE (2006-06-18 03:34:26)

My RoR journey  -- thoughts on learning RoR and lessons learned in applying TDD and agile practices.

Re: Testing your model code...

NielsenE wrote:

[PS, any chance we can get a Ruby syntax highlighter for this forum so we can use ruby blocks and not just code blocks?]

You wanted something? smile

vinnie - rails forum admin

Re: Testing your model code...

So I'm back working on these assertions again, just applied the library to a second project and using that to help triangulate in on some gaps -- my length tests currently only work for double bounded ranges, and I think would get confused even with a 0..20 type range as it would try to create a -1 length string to test, etc.

I'm also struggling to find a good way to extend the assert_uniqueness_of to work for scoped uniqueness.  I have individual tests that work, but they require a more complicated pre-configured fixture.... so moving the tests into the generic code will place additional demands on the conventions surrounding the test suite....  The current requirements I think are reasonable, but adding more might make it less friendly.

Finally I'd like to create an assert_format_of to piggyback on the main assert_attribute_required, via a :format=>/ / add-on.  However, I'm drawing a blank at a good way to dynamically create strings that don't match a given regexp -- other than something brute forice like building a random length random string from a large characters set and looping until it doesn't pass the regexp, and then testing it in the model class.  Generating 5-10 such strings within the test to hopefully exercise more of the possible failure modes.  However that adds non-determinism into testing, which I don't like doing...  Otherwise I'd need to basically write a parallel regexp parser that creates a list of known failing words by going through the regexp and generating an invalid character at each point in the expression while keeping the rest legit....  Anyone know of any existing code that might help me with this?

My RoR journey  -- thoughts on learning RoR and lessons learned in applying TDD and agile practices.