Topic: Testing without a test database?

I'm working on a very large, very complex legacy database (Oracle, > 4GB of text data in 1000s of tables).  For this situation, dropping tables and populating them from fixtures just isn't feasible: I can't load 500MB of data every time I want to test my app.  I also can't muck with the dev database, since it is used by other developers for other projects, and only gets cloned from the production DB once every few months.  Is there a way to short circuit the typical testing methods to have them open a transaction, load test-specific fixtures, run the tests and then close the transaction without dropping the tables first?

Thanks! smile

Re: Testing without a test database?

I am in a similar boat and have a lot of fun doing tests.

Mocking and stubbing become your friends very fast.

But this soon stops working if you have legacy stored procedures you need to hit as well.

If you do come up with any ideas or solutions, please let me know by posting back here!


Re: Testing without a test database?

Your test database should be TOTALLY separate from any production or development DB. As a result, you don't need to load huge masses of data whenever you need to test: just make your test DB separate (I like to use SQLite, which makes it a local file). Rails will load the fixtures into it, run the tests, then drop that test DB with no impact on your development or production databases).

Some people prefer not to use fixtures, but either way, you really should have a test DB of some sort if you want to test functionality that relies on the database.

Re: Testing without a test database?

manitoba98, either you didn't read the thread or misunderstood the intent behind it smile

The problem with the Rails Way (in this SPECIFIC case) is that loading and dropping all the tables takes TIME.

I have several HUNDRED tables in my legacy DB.  With LOTS of rows in them.

Just __starting__ my test suite (before it even starts with any test) can take up to 10-20 seconds depending on the phase of the moon.

I think this has less to do with dropping and lifting tables and more with Rails bootstrapping the environment with by reading in the schema.rb file and doing all the reflection needed... but anyway...

Maybe I can patch autotest not to load the schema all the time..


Re: Testing without a test database?

My point was: do you necessarily need LOTS of rows to test? In many cases, you only need a handful of rows per table to run a test suite, making sure the code functions. Why do you need hundreds of megabytes for functionality testing?

Re: Testing without a test database?

Ah, I see what you mean, but my test_db is empty.  I have only a few rows to load in my fixtures (which I actually barely use)

It still takes a long time.  If you get a huge complex data base like this, try it.  We are talking like 5000 + line long schema.rb files here.  The original poster would probably have a 10000 line + schema.rb file.

I think the problem is that autotest and rake spec have to re-init every time they run, but the re-init the whole environment, not just the classes that have changed and that is the slow down because every AR class needs to do it's reflection discovery.

Anyhow, If it annoys me too much, I'll go hunting through autotest smile


Re: Testing without a test database?

Manitoba, you're right that I don't need lots of rows to test.  But there's more to it than that.

I don't really own the database.

It's part of a large Oracle-based forms package, with lots of other users & developers.  I can't ruin the test environment by dropping tables and not re-populating them because people use the test environment for other things.  For example, some of our archival tables run into the millions of rows.  If I'm working with one, re-creating it is just not an option.  My best bet is to open a transaction, put in my fixtures, run the test and then rollback the transaction.  It means that I will lose the final state of the dabatase if my tests fail, but that's the price I have to pay for supporting a huge legacy db.

Mikel's also right -- part of the constraint is time.  If there weren't millions of rows, in theory I could create my fixtures from the test db, and have those load.  But at millions of rows, the overhead is prohibitive.

So I'm back to my original question: does anyone know of a way to run tests this way, which isn't quite the rails way?

Re: Testing without a test database?


I'm not sure how this would go, but what if in the test helper you wrapped calls to the DB with a direct call to the database and execute a begin transaction and then after the tests finish, kill the transaction?

Transactions, from how i understand, set a state for the user, so you can just do a begin, then basically do anything, and then rollback after you finish.

You could try this out by overriding something in ActiveRecord or Test::Unit parts that Rails creates....

Just a thought.