Topic: Trouble with Mocking

Out of curiosity, how often do you use mocks in your tests/specs? Perhaps I'm doing something wrong, but I'm finding it severely limiting.

Since switching to rSpec over a month ago, I've been doing what they recommend in the docs where the controller and view layers do not hit the database at all and the models are completely mocked out. This gives you a nice speed boost and makes some things easier, but I'm finding the cons of doing this far outweigh the pros.

Since using mocks, my specs have turned into a maintenance nightmare. Specs are meant to test the behavior, not the implementation. I don't care if a method was called I just want to make sure the resulting output is correct. Because mocking makes specs picky about the implementation, it makes simple refactorings (that don't change the behavior) impossible to do without having to constantly go back and "fix" the specs.

I'm very opinionated about what a spec/tests should cover. A test should only break when the app breaks. This is one reason why I hardly test the view layer because I find it too rigid. It often leads to tests breaking without the app breaking when changing little things in the view. I'm finding the same problem with mocks.

On top of all this, I just realized today that mocking/stubbing a class method (sometimes) sticks around between specs. Specs should be self contained and not influenced by other specs. This breaks that rule and leads to tricky bugs.

What have I learned from all this? Be careful where you use mocking. Stubbing is not as bad, but still has some of the same issues.

Railscasts - Free Ruby on Rails Screencasts

Re: Trouble with Mocking

I took the past few hours and removed nearly all mocks from my specs. I also merged the controller and view specs into one using "integrate_views" in the controller spec. I am also loading all fixtures for each controller spec so there's some test data to fill the views.

The end result? My specs are shorter, simpler, more consistent, less rigid, and they test the entire stack together (model, view, controller) so no bugs can slip through the cracks.

I'm not saying this is the "right" way for everyone. If your project requires a very strict spec case then it may not be for you, but in my case this is worlds better than what I had before using mocks. I still think stubbing is a good solution in a few spots so I'm still doing that.

I thought I would never go back to fixtures, but I'm realizing they do have their place as "filler" data as long as the tests don't rely too heavily on them. The models still don't use fixtures which will probably stay that way.

How's the speed? You would think loading all fixtures for every controller spec would be a problem, but the fixtures are small (usually only two records). I have 33 controllers and 39 models. It used to take 3.7 seconds to run the full suite with mocking on my intel imac, but now it takes 6.5 seconds. Still perfectly acceptable I think.

Railscasts - Free Ruby on Rails Screencasts

Re: Trouble with Mocking

I'm using some ideas from Beast to load all fixtures, and I do find that it slows things down. I don't mind waiting an extra second or two for tests to run, though, and it's better than having tests fail because you're missing fixtures.

I hear that you can speed up testing by doing it with SQLite, and wasn't Geoffrey Grosenbach doing something where you can run tests in memory?

I hate fixtures just as much as the next guy, but if you keep them up to date (TDD, test-first) then it's not so bad. I don't know what the alternative would be, honestly.

Re: Trouble with Mocking

If you're running edge rails you can do this to load all fixtures:

fixtures :all

Saves from having to add to the growing list when you add a model. This may work 1.2.3 as well, I haven't tested it fully.

Railscasts - Free Ruby on Rails Screencasts

Re: Trouble with Mocking

Cool. I'm checking out the changelog here:

http://dev.rubyonrails.org/changeset/6227

... and it looks like this is the default behavior now. Nice!

Re: Trouble with Mocking

Well I am experimenting all kind of troubles with mocking objects.
Maybe it will be easy for me to begin my firsts BDD works  with the approach proposed by Ryan.

Ryan, I really appreciate, if you could post a example of your specs for a simple controller generated with scaffold_resource, using real objects and integrating the views.

And even, I think this could be a good theme for one of your fantastic railscasts.
I really enjoy them.

Thank you very much.

Re: Trouble with Mocking

It might look something like this:

describe ItemsController do
  fixtures :all
  integrate_views
 
  it "index action should render index template" do
    get 'index'
    response.should render_template(:index)
  end
 
  it "show action should render show template" do
    get 'show', :id => 1
    response.should render_template(:show)
  end
 
  it "new action should render new template" do
    get 'new'
    response.should render_template(:new)
  end
 
  it "create action should render new template when model is invalid" do
    item = Item.new
    item.stub!(:valid?).and_return(false)
    @controller.stub!(:fetch_model).and_return(item)
    post 'create'
    response.should render_template(:new)
  end
 
  it "create action should redirect to index action when model is valid" do
    item = Item.new
    item.stub!(:valid?).and_return(true)
    @controller.stub!(:fetch_model).and_return(item)
    post 'create'
    response.should redirect_to(items_path)
  end
 
  it "edit action should render edit template" do
    get 'edit', :id => 1
    response.should render_template(:edit)
  end
 
  it "update action should render edit template when model is invalid" do
    item = Item.find(1)
    item.stub!(:valid?).and_return(false)
    @controller.stub!(:fetch_model).and_return(item)
    put 'update', :id => 1
    response.should render_template(:edit)
  end
 
  it "update action should redirect to show action when model is valid" do
    item = Item.find(1)
    item.stub!(:valid?).and_return(true)
    @controller.stub!(:fetch_model).and_return(item)
    put 'update', :id => 1
    response.should redirect_to(item_path(item))
  end
 
  it "destroy action should destroy item and redirect to index action" do
    delete 'destroy', :id => 1
    response.should redirect_to(items_path)
    Item.find_by_id(1).should be_nil
  end
end

This won't quite work with scaffold_resource because it relies on there being a method in the controller called "fetch_model" which returns the model. I stub this out sometimes so I can return whatever model I want in the test.

Also, there's quite a bit of duplication here, I refactored this out into a module so it's easier to test these 7 actions.

Railscasts - Free Ruby on Rails Screencasts

Re: Trouble with Mocking

jmcervera wrote:

And even, I think this could be a good theme for one of your fantastic railscasts.
I really enjoy them.

I'm so quoting that one.

jmcervera wrote:

Thank you very much.

And that one.

Re: Trouble with Mocking

still rspec nuby:
Thanks for bringing up the issue what I have as well. Moreover I'm holding dev and test data in the same fixtures (so I can reset everything easily, just haven't think out better place for dev data). Therefore fixtures holds only "good" data and everything else I plan to stub/mock out.

Don't know if it's good way yet but let's see what practice will bring.

Estonian ruby coders => http://ruby.ee

Re: Trouble with Mocking

My big gripe with rspec controllers is that it seems like the default controller tests are testing _Rails_ code instead of my application code. And I agree that we should be testing the behaviour instead of the implementation. Whether or not a certain method is being called really should be no concern of our tests.

As for fixtures, I've started using them again only for fixed data like "categories". I find that maintaining associations with fixtures is a major headache, even with only one or two objects. What works for me is setting up lots of methods named "valid_xxx_attributes". This way I can create the objects as needed in my setup method and let the proper associations be created for me. I suppose this could slow things down once there are a lot of descriptions. Anyone have any experience with this?