Topic: TDD vs BDD

Okay it's confession time...

I do not test..ever. I know it's wrong and that every time I write a line of untested code, God kills three kittens, etc etc..

But to be honest, the very concept of writing tests is, well.. boring to me.

I have really been trying to motivate myself to do this because I do see the value of testing, (resilience, refactoring and stability) so I been poking around the net and in my searches I came across this excellent bit of writing.

I had heard the BDD buzzword before but not paid much attention... now I am wishing I had!

Is anyone else hip to this stuff? If so, are you using Rails' baked in tests, or something third-party like Rspec or ZenTest?

Do tell

Re: TDD vs BDD

I use the built-in test framework + rcov.  Religiously.  I'm striving to maintain 100% test coverage + write regression tests every time I find a bug.  Sure, it's boring.  Yeah, I spend almost as much time writing test code as production code.  The upside?  Getting 90%+ test coverage and supreme confidence I haven't broken anything in about 30 seconds.  Oh yeah, I've found 10-20 bugs just by improving test coverage.  Ruby's dynamic nature and duck-typing is a curse as well as a blessing.
I almost never write unit tests.  I strive to get all of my coverage through functional tests.

BrewControl.com - Brewery and Brewpub management powered by RoR

Re: TDD vs BDD

bkrahmer wrote:

I almost never write unit tests.  I strive to get all of my coverage through functional tests.

Interesting. I generally do the opposite. I find unit tests are easier to write than functional tests, and keeping the business logic in the models makes it ideal for unit testing. My functional tests are usually small and are mainly used for coverage.

Out of curiosity, when a test breaks do you find it difficult to determine the problem since functional tests are at a higher level?

Railscasts - Free Ruby on Rails Screencasts

Re: TDD vs BDD

As a PHP developer who struggles daily with "testing" using a browser and hitting re-fresh a million times Rails' built in testing is a god send.

I use Autotest to constantly run my tests as I go and stick to TDD.  I always write tests first and then produce the code later.  Plus with auto test it's super simple to see if your recent change to some model logic broke your application.

The only time I open my browser is towards the end when I'm tweaking layouts and CSS etc. Thanks to my tests I know my app will work as intended. So much better.

Re: TDD vs BDD

Ryan, I have been thinking a lot about your post.  One conclusion that I came to is that I may be placing too much of my business logic in my controllers.  That is hard to quantify, though.  Even handling both get and post cases in the same actions, my average actions are 10-15 lines of code.  Most of my model methods are one-liners, usually doing semi-complex find_by queries. 
The way most of my code is structured, if I were to write a unit test first, I would get test coverage on the model, but then I would be lacking coverage in the controller.  If I were to add a functional test, I would now have overlap on my model coverage, making the time writing the unit test wasted.  To me, it's essential that I have 100% functional test coverage, because that's how the user uses the application.
I generally don't have trouble finding the source of test breakage.

BrewControl.com - Brewery and Brewpub management powered by RoR

Re: TDD vs BDD

bkrahmer wrote:

If I were to add a functional test, I would now have overlap on my model coverage, making the time writing the unit test wasted.

There is some overlap when it comes to coverage, but coverage isn't everything. Testing the behavior is equally important IMO. This is what I use unit tests for. However, not all applications are heavy on business logic so the models are fairly small and the behavior isn't as critical. Your app may fit into this.

Railscasts - Free Ruby on Rails Screencasts

Re: TDD vs BDD

Don't get me wrong.  I don't write tests just for the coverage.  Correct behavior is definitely the most important thing.  My point is that you cannot have tested all of the functionality until at least your controllers are 100% tested.  On the flip side, just using rcov, you cannot determine that you have 100% functionality/path coverage just because you have 100% code coverage.  That's why I write tests to expose any bugs that I might find while playing with the app / doing manual testing.
Another reason I tend to avoid writing unit tests is that if you have a unit test for a function, and that function ends up stranded as dead code, you may never notice it, and it may get dragged along for quite some time.  I'm usually able to spot dead code that I forgot to remove at refactoring very quickly.

BrewControl.com - Brewery and Brewpub management powered by RoR

Re: TDD vs BDD

To get back to the original question, BDD, I use BDD whenever possible. It's a much more positive way of describing stuff. I wrote about rSpec here:

http://calicowebdev.com/blog/show/2
http://calicowebdev.com/blog/show/7

I've just started on a new project and everyone else is using Test::Unit. My functional tests look like:

def test_create_with_post_and_good_data_should_succeed
  post :create, :setting => {:setting_name => 'first_setting', :setting_value => 'first_value', :setting_type => 'string'}
  assert_response :redirect
  assert_redirected_to(:action => 'index')
end

which is nowhere near as nice as the rSpec version (which also tests the flashes and is, therefore longer):

describe "A SettingsController new/create action" do
  it "should succeed with POST and good data" do
    post :create, :setting => {:setting_name => 'first_setting', :setting_value => 'first_value', :setting_type => 'string'}
    response.should be_redirect
    flash[:info].should_not be_blank
    flash[:error].should be_blank
    response.should redirect_to(:action => 'index')
  end
end

The whole point of including this code is by way of illustrating that the Test::Unit syntax says, "if these things happen ok then we're fine." BDD says, "a working widget should behave according to these rules." The distinction is subtle, but when you run the spec:doc Rake task, you'll see a slightly stronger pattern emerging:

A SettingsController in general
- should succeed and show 2 items by default

A SettingsController new/create action
- should create entry form
- should fail with GET
- should succeed with POST and good data
- should fail with POST and bad data

A SettingsController edit/update action
- should succeed with POST and good data

The spec is typically finer grained than most Test::Unit code I've seen. So, in this one, I've tested the bulk of the controller, using mocks to isolate it from the database, but changing them per example.

Have a look at rSpec.

Re: TDD vs BDD

There's a good interview with the creators of rSpec. Definitely work a watch.

I'm fully convinvced of BDD, but I don't think I'm ready to jump on the rSpec bandwagon yet. It's just too unstable. Maybe once it settles down a little.

Railscasts - Free Ruby on Rails Screencasts

Re: TDD vs BDD

I've been using BDD lately, with rspec - and I don't see why people say it's unstable. I've been having a blast. Especially with unit testing, it's complete bliss.

Ryan, have you played with it to note the 'unstableness' of it?

http://danielfischer.com - Personal Web-Technology-Blog, Los Angeles.

Re: TDD vs BDD

I'm referring to the stableness of the development. They are constantly changing the syntax and structure of how to build specs. I'm waiting for it to settle down a bit. I'm also waiting for it to get a little more Rails friendly. Last I heard it doesn't fully support functional tests and I'm a little bit too tied up with the test/unit framework. But I plan to play around with it in the near future.

Railscasts - Free Ruby on Rails Screencasts

12

Re: TDD vs BDD

You're right, Ryan, they are making changes. But changing all my specs across 5 projects over the last 6 months to track this has probably cost me less than an hour. And, it's made me look at my specs one more time which can only be a good thing. What's more significant to me is how hard they are thinking about the semantic nuances that will guide developers in writing better tests. That, IMO, is why they keep tweaking it.

WRT: Rails-friendly, I'm using rSpec on all my Rails projects and it kills me to have to use Test::Unit. Others may feel differently, but for me it just feels more natural to use. What could be more natural and Rails'ey than:

Product.should have(4).records

or

response.should render_template('user/login')

Re: TDD vs BDD

cwd wrote:

response.should render_template('user/login')

Looks like a functional test, is this fully supported by rSpec now? If so I'll have to check it out. I heard somewhere that rSpec didn't support functional tests, but maybe that was out of date.

Railscasts - Free Ruby on Rails Screencasts

14

Re: TDD vs BDD

rSpec supports (and has for some time) unit (model tests), functional (controller tests), and view tests. It also has a mocking framework that  is (IMO) extremely good.

Look over my blog posts on rSpec at:

http://calicowebdev.com/blog/show/2

and

http://calicowebdev.com/blog/show/7

Re: TDD vs BDD

Oh, cool. Thanks for that. I think it's time to get off my lazy bum and learn rSpec. Your posts should help me a lot because they are more up to date than some others I've read.

On a side note, has anyone tried test/spec? I looks a lot like rSpec but it wraps test/unit so it might work better for me. Also, the dudes on err the blog seem to prefer it.

I'll give both a try.

Edit: just noticed the docs are much better than when I saw them last time. Looks like rSpec has matured a lot since then.

Last edited by ryanb (2007-05-11 13:33:52)

Railscasts - Free Ruby on Rails Screencasts

Re: TDD vs BDD

Well, after spending a day with rSpec, I must say, it's impressive! Right now I'm redoing some old test/unit code in rSpec. It's more verbose, but it feels so much more organized and easy to read. I'm also relying much less on the database and there's no more fixtures - been wanting to do that for a long time! Thanks for convincing me to try it out.

Railscasts - Free Ruby on Rails Screencasts

17

Re: TDD vs BDD

Don't forget, you can have separate contexts (the old term) for specs. Like:

describe "An authenticated user in the FooController" do
  setup do
    controller.stub!(:authenticated?).and_return(true)
  end

  it "should sail right to one of the protected pages smile" do
    get :very_secret_page
    response.should be_success
  end
end

describe "A visitor (not authenticated) in the FooController" do
  setup do
    controller.stub!(:authenticated?).and_return(false)
  end

  it "should make the user log in or else" do
    get :very_secret_page
    response.should be_redirect
    response.should redirect_to('user/login')
  end
end


The reason for stubbing the authenticated is that the database lookup would have nothing to do with the controller's behavior. It would couple the workings with the controller's behavior. Your tests will also run lickety-split if you decouple model from controller testing, isolating them.

Last edited by cwd (2007-05-12 14:35:09)

Re: TDD vs BDD

RSpec does look pretty cool... I am torn between it and test/spec

Check out this article on Err

http://errtheblog.com/post/4268

19

Re: TDD vs BDD

FWIW, I'm pretty sold on rSpec because it has active development and more than just one person. Actually, quite a number have worked on rSpec. Additionally, it has mocking/stubbing baked in. If you go with one of the other frameworks, you'll have to use Mocha/Stubba or FlexMock.

I'm also sold on the kind of thinking that goes into the rSpec syntax. SimplyBDD is good, but I don't think with all his other responsibilities, Rick is going to devote the same level of effort to it that rSpec is getting. test/spec is ok, and is my middle choice if I have to run under Test::Unit, but I find the wrapper around the asserts very thin.

Write specs in each, check out mailing lists to see what kind of traffic they get, then decide.

Oh, did I mention rSpec also integrates with Watir and Selenium, even going so far as to put screenshots of failed specs in the output?

Re: TDD vs BDD

Could anyone help explain mocking and stubbing to me? I really have no idea what it's used for, besides making some structure so it doesn't use the DB?

Confused, I would appreciate help.

Thanks!

http://danielfischer.com - Personal Web-Technology-Blog, Los Angeles.