Topic: All mongrels crash after intensive data gathering request begins

Hi everyone,

I have a 512Mb slice on Slicehost.  I've got 3 Mongrels running on nginx. If I start an intensive data gathering process (specifically, gathering data from Wikipedia via their API), all other requests to the box are not serviced.  Eventually, nginx times out and I have to restart the Mongrels to get requests to be processed again.  It seems to me like all 3 Mongrels are being used by this intensive process, but can that be right?

I've tried making the intensive task less intensive by putting in 3 second sleep periods between each API request, but to no avail.

Has anyone any ideas?

Thanks very much.

Re: All mongrels crash after intensive data gathering request begins

Hmmm ...

How are you starting this process?
Mongrel/Rails is single threaded, so you are going to block the I/O channel while you are using the API.

Toby Hede
===================================================
FiniteStateMachine - Software Development for Social Networks
===================================================