Topic: Fixtures "best practice"

Hello all.

I have a question about the use of fixtures. Unit and functional tests obviously need to ensure that bad data does not get into the database, by ensuring that the model (and view) refuses to allow it. I am wondering what peoples opinions are regarding the use of fixtures... Presumably they should only contain valid data, because they are going to be written directly to the database. If that is the case, then the convenience of using fixtures is lost when you want to test bad data, e.g. for testing validation - you'd have to code the bad data directly into your test methods.

I am thinking about creating two different test suites, each with their own set of fixtures - one for testing good data and the other bad. However, surely there's a potential problem with loading bad data into the database via fixtures, because there may be parts of the app that require good data to function properly. This looks like a maintenance nightmare.

Does anyone have any advice or pointers about the use of fixtures for testing functionality that requires bad data (e.g. validation)?

Many thanks.

Steph

Re: Fixtures "best practice"

I don't think fixtures should be used in validation testing. You are trying to test input variables, not data which is already loaded or in the database.

On a related note, see this tutorial on refactoring a validation test. I don't see how loading fixtures would make validation testing easier?

Railscasts - Free Ruby on Rails Screencasts

Re: Fixtures "best practice"

Thanks for your reply.

When I first read about fixtures I jumped to the conclusion that they were a way of abstracting test data out of the test methods. Now I'm starting to see them more as a time saving device for populating the database with valid data, so that it's in a state comparable to a live application.

So.. only good, valid data in fixtures.

The tutorial you linked looks useful- I'll certainly apply the principles to my tests. However, even following this advice, we're going to be in a situation where half of the test data is in fixtures and the other half is in the test methods.

With fixtures providing a mechanism for storing test data, I can't help but think I'm missing something about their potential use. Or have I completely missed the point? Should fixtures even be considered test data?

Re: Fixtures "best practice"

IMO fixtures should be used as little as possible, especially on bigger projects. As a project grows, the fixtures become this long list of data that is trying to fit the needs of every test. When you add a new test, the question constantly comes up: "Should I try to use an existing record that *kinda* fits the job, or should I add yet another record to the fixture?". Then you have to juggle around which fixture records are related to other fixture records and it can become a nightmare.

I also think tests should be self-contained, where they don't rely on external data (like fixtures). The problem with relying on fixtures is that if you ever try to change the fixtures (some times just adding a record) it can break existing tests.

Oh yeah, fixtures are slow too, and they're a pain to add. wink

Unfortunately the alternative is not much better. Creating the test data in code is ugly. However, if you apply refactoring and mocking/stubbing, you can usually clean this up quite a bit.

Railscasts - Free Ruby on Rails Screencasts

Re: Fixtures "best practice"

I agree that fixtures should represent the database in a "good" state. All fixtures should be valid, all the time!

But fixtures are should not be considered "test data". I look at them as a way to bring your application into a usable state. A minimal set of data that is required to keep testing sane.

In your tests, it's useful to create data in setup. Sometimes I will use a fixture in my functionals to grab a hash of values (which I know are valid), and post that back to a form. Invalid or 'edge case' data does not belong in fixtures.

By using ruby code in YAML (uh, erb), you can generate data at test time dynamically, which makes for less brittle fixtures. With the fixtures loaded into the development db, your site should be totally usable. This exercise really helps me to visualize what the tests see.

Re: Fixtures "best practice"

I'll just add in, try to play with mocha (mocking and stubbing) it's hard to grasp at first, but once you get a hang of it; you start to see the light.

http://danielfischer.com - Personal Web-Technology-Blog, Los Angeles.