Topic: Recommended number of Mongrel instances

I am on slicehost and my website is starting to pick up traffic. I haven't seen a slowdown in response time yet but it is picking up more traffic each day. My setup is Apache with 2 mongrel instances, does anyone recommend increasing the number of mongrel instances to say 3 or 4 or ? to handle more traffic?

JiggyMe
"Videos that Matter to You" powered by Ruby on Rails

Re: Recommended number of Mongrel instances

What size of a slice do you have? You can probably get away with 3 mongrel instances on 256, but I wouldn't go any higher. Just pay careful attention to your memory usage to make sure it's not peaking.

Railscasts - Free Ruby on Rails Screencasts

Re: Recommended number of Mongrel instances

Actually I am running top now, and it is showing 257MB used, 3MB free, but only using 1MB of swap, so it seems to be maxing out memory usage without too much swapping, so increasing it to 3 could push it over the top (no pun intended).

JiggyMe
"Videos that Matter to You" powered by Ruby on Rails

Re: Recommended number of Mongrel instances

Actually if anyone can share their optimal configuration for a typical 256MB slice, using Apache, Mongrel, MySQL, and Rails, that would be great. Thanks in advance for any help.

JiggyMe
"Videos that Matter to You" powered by Ruby on Rails

Re: Recommended number of Mongrel instances

I'm running 3 mongrel instances with Apache on a 256mb Slicehost slice with no problems. I use MySQL for the database.

Are you running anything like RMagick, that takes up a bit of RAM?

Last edited by learc83 (2007-10-05 22:45:26)

Re: Recommended number of Mongrel instances

Ok cool, I will switch to running 3 Mongrel instances, I am also using MySQL. I am not running RMagick but I noticed that the 2 Mongrel instances are both sucking up 70MB each and MySQL using about 29M. I will monitor the Mongrel memory usage when I restart it next time to see how fast they grow.

Last edited by bngu (2007-10-05 23:04:19)

JiggyMe
"Videos that Matter to You" powered by Ruby on Rails

Re: Recommended number of Mongrel instances

Do you even "need" 3? Are you having trouble with queuing http requests?

http://danielfischer.com - Personal Web-Technology-Blog, Los Angeles.

Re: Recommended number of Mongrel instances

With increase in traffic, I noticed overall slower performance but I haven't identified where the bottlenecks are yet, hoping that 3 instances of Mongrel might help speed things up.

JiggyMe
"Videos that Matter to You" powered by Ruby on Rails

Re: Recommended number of Mongrel instances

A mongrel instance will serve up to two requests per a second. Do you really have more than that?

http://danielfischer.com - Personal Web-Technology-Blog, Los Angeles.

Re: Recommended number of Mongrel instances

imho, the first order of the day would be to replace apache with nginx.  i did that with my 256 slice with 4 nginx workers (4 is recommended for slicehost) and things are snappier now/i don't go over my 256MB.  i also have 3 mongrel instances, which from the forums, seems to be the sweet spot for 256MB slices.  oh, and if you're not using innodb, you can turn that off as well - my MySQL usage went down from 11% to 4%.

Re: Recommended number of Mongrel instances

I'm curious ... why doesn't Mongrel do it's own proxying?

Given the reason for requiring multiple Mongrels is Rails, why not have Mongrel itself manage the Rails containers?

Toby Hede
===================================================
FiniteStateMachine - Software Development for Social Networks
===================================================

Re: Recommended number of Mongrel instances

Because Nginx (and others) already does it and very well, so why spend time on duplicate code that could badly perform?

Re: Recommended number of Mongrel instances

DFischer wrote:

A mongrel instance will serve up to two requests per a second. Do you really have more than that?

i haven't optimized my code, but I get about 8 requests per second.  i thought 8 was pretty bad, but then i saw you have 2! i'm curious about peoples experiences with requests per second on a mongrel...

Re: Recommended number of Mongrel instances

I'm still a bit stumped about the need for multiple mongrels.

I'm looking at the possibility of JRuby and a Tomcat container.

Toby Hede
===================================================
FiniteStateMachine - Software Development for Social Networks
===================================================

Re: Recommended number of Mongrel instances

Hi tobyhede.  can you elaborate on why you're confused about multiple mongrels?  i've been reading about deployment and scaling up and it seems fairly straightforward.  if 1 mongrel can do 50 requests per second, then 3 can do 150?  that way you can scale up or down depending on traffic.

i use Tomcat for another application with GWT and it works well.  i wanted to know why you're considering Tomcat+JRuby - the nginx+mongrel+capistrano is straightforward and works well (for me) so i'm curious why you're considering something else.

Re: Recommended number of Mongrel instances

The reason we need clustered mongrels is because of shortcomings in the way Rails handles threads. No other framework/system I have  used has required clustering except in a physical sense (clustering across physical nodes). However, because of these limitations we need to deploy with non-physical clustering just to get throughput with all the complexity and extra management overhead this entails (apache/nginx requirements, mod_proxy, multiple servers). Not to mention the extra server capacity required (50mb RAM for each instance).

PHP apps don't require this extra setup and configuration, neither do Java or .Net ... or ColdFusion or even good old-fashioned original recipe ASP.

Toby Hede
===================================================
FiniteStateMachine - Software Development for Social Networks
===================================================

Re: Recommended number of Mongrel instances

sthapit wrote:

if 1 mongrel can do 50 requests per second, then 3 can do 150?  that way you can scale up or down depending on traffic.

Unfortunately that is not the case, it would have been too good to be true. Having too many mongrel instances can actually lead to weaker performance.

Last edited by Johnson (2008-03-15 07:35:47)

Re: Recommended number of Mongrel instances

Johnson wrote:

Unfortunately that is not the case, it would have been too good to be true. Having too many mongrel instances can actually lead to weaker performance.

is this true even if you have enough memory for multiple mongrels?  i don't see why it would be if you have nginx as a proxy for example which can handle many hundreds of requests per second and each mongrel has no idea of the existence of other mongrels.  i'm no deployment expert though so i'm just thinking aloud smile

Re: Recommended number of Mongrel instances

sthapit wrote:
Johnson wrote:

Unfortunately that is not the case, it would have been too good to be true. Having too many mongrel instances can actually lead to weaker performance.

is this true even if you have enough memory for multiple mongrels?  i don't see why it would be if you have nginx as a proxy for example which can handle many hundreds of requests per second and each mongrel has no idea of the existence of other mongrels.  i'm no deployment expert though so i'm just thinking aloud smile

No it's not true if you have enough memory. If you have enough memory more mongrels = better performance, barring other bottlenecks. However, sometimes mongrels can get out of hand and eat up crazy amounts of memory. There is a great resource on how to manage mongrels here

http://www.pragprog.com/titles/fr_deploy

Chapter 6 on Mongrel management is a free excerpt.

Re: Recommended number of Mongrel instances

Zed used to have plenty documentation about these issues, but the website has changed and all the links are now broken...