Topic: Scalability Question

I've spent 9 LONG years running the IT for our own small businesses, always using COTS applications back ended by MS SQLServer.  One app evolved when networks were slow and databases were wimpy, now that we run it on current hardware and fast networks,  the app is remarkably scalable.  When I finally went to a users conference for this particular app,  I was astounded at the user loads that others were handling on really small and underpowered SQL Servers.

Over 9 years, a lot of code was developed to customize this one COTS app, it's all expressed in SQLServer stored procedures (10%),  and SQL Server functions (90%). 

So then I start on Web 2.0 for the businesses and I end up here.  And I have a dilemma.  Phase one of my Web stuff doesn't connect to the back office,  phase 2 will.   The bulk of my legacy code is SQL statement executions and simple string manipulations.  I took a stab at re-creating one bit of old code in RoR,  it ended up as three lines of code in a controller,  it was about 45 lines of SQLServer scalar functions and a 10 line stored procedure. 

RoR clearly won the day WRT total lines of code.

But the SQL traffic shot up substantially. 

So which way to turn?   

I really do NOT want to do any further development in SQLServer.  But on the other hand,  there is potential application of what I'm doing to very large scale operations.  I want to make sure it's extremely scalable, and portable to other databases.   RoR wins the 'portability' argument, no contest.  But the scalability issue I'm stuck on.

I think it boils down to the scalability of all the string manipulations.  I think a benchmark would answer the question,  but that's too much work in my case.

Database engines are very tight pieces of highly optimized code written in C, C++, even assembler.  RoR is a much looser stack of cooperating processes,  with lots CPU cycles and network bandwidth needed to cooperate.   

TWO questions:

How efficient is the Ruby interpreter at string manipulations vs a generic database engine's capability?

Is it heretical to even WORRY about the overhead of a stack like RoR, or the raw efficiency of executing string manipulations?

The last time I really thought about and solved these kinds of issues was Pre-Web, 1988 - 1995.

COTS - Commercial Off The Shelf

Last edited by BradHodges (2010-06-11 20:39:57)

Joe got a job, on the day shift, at the Utility Muffin Research Kitchen, arrogantly twisting the sterile canvas snout of a fully charged icing anointment utensil.

Re: Scalability Question

Moore's Law says your kit will double in speed every 18 months for the same cost, something you've already noticed with the performance of your existing database calls. So you have to work out which is the cheaper TCO in the long run; your time coding and debugging your SQL server procedures and bespoke string manipulations, or the extra server resources you are going to consume with the more modern solution. Don't forget to factor in the ease of adding new functionality with RoR - just download a Gem and start using it vs. rolling your own...

Re: Scalability Question

With RoR I have found that whilst active record makes like easy for development it can sometimes make you lazy.

I've found i use it to get the data i need instantly then revisit the active record select's at a later date. You can soon find that using active record your performing 100's of selects to get some simple data from the DB.

I always review my debug logs and then go back and create custom sql selects to cut down on the number of requests to the DB.

Dump_Weed Customizable RSS Aggregation