Topic: Million Dollar Question?

I've been knocking on the door of this question for the last two weeks and no one seems to open?
Two weeks ago, I posted http://railsforum.com/viewtopic.php?id=3521 on the March 1st and received 80+ views and ZERO responses.
Obviously, I'm a newbie and I do very much appreciate the character and quality of those I've meet through RonR, but why isn't this the "Million Dollar Question"?
After all, doesn't everyone realize that the days of running Win32 desktop applications is going to fade into the sunset with multi-user web applications taking their place? Look at QuickBooks now and many others.
The key to entering this market and prospering is to learn the 'bones' of handling many distinct user databases INSIDE one monolithic database (such as Basecamp).
If I can get some help here, I'd love to create a new category of this forum called "Splitting the Monolith".
I see on page 313 of Dave Thomas's AWD where he explains "Compositing Data with Aggregations". I've got to believe this is the heart of the solution. Building upon this idea, I frame my question. (What throws me in his explanation is that he puts all his code on one "Aggregation.rb" file).
I use the term, "project" as the distinct sub-data set for each user's universe. The PROJECT table will be the top hierchical table (Integer key) in this monolithic database with every other table in the database having (some type of) FK relationship. Even though we've been taught in the many RonR books that we can transcend through any level of parenting, my 'gut' feeling is that this might be very slow. Thus, I'm thinking that 'Aggregations' might speed things up by forming 'compound fields" with the two integers (Project_id and AnyTable_id). Then when a user wanted to perform a Tablename.Find(:first) they could simply pass the session[project_id] at this to find the first row in this table for this project.
I've also contemplated a 'front end' design to allow the user to login and securely access their PROJECT(s).
If there's anyone out there that is trying to accomplish this type of endeavor, I would be grateful for any reply.
Thank you,
David

Re: Million Dollar Question?

BraveDave wrote:

I've been knocking on the door of this question for the last two weeks and no one seems to open?
Two weeks ago, I posted http://railsforum.com/viewtopic.php?id=3521 on the March 1st and received 80+ views and ZERO responses.

Your post was easy to start reading and get bored.

BraveDave wrote:

Obviously, I'm a newbie and I do very much appreciate the character and quality of those I've meet through RonR, but why isn't this the "Million Dollar Question"?

Because this has already been figured out elsewhere.

BraveDave wrote:

After all, doesn't everyone realize that the days of running Win32 desktop applications is going to fade into the sunset with multi-user web applications taking their place? Look at QuickBooks now and many others.

Web 2.0 and all of that...

BraveDave wrote:

The key to entering this market and prospering is to learn the 'bones' of handling many distinct user databases INSIDE one monolithic database (such as Basecamp).

*Screeching tires* Whoa.  Get the idea of having a bunch of user databases inside some other database out of your head.  It's just not necessary and it's not how anything else works.  I get the impression that in your head that's what Basecamp is doing but it's not, I can't think of any web application that works in such way.


BraveDave wrote:

I use the term, "project" as the distinct sub-data set for each user's universe. The PROJECT table will be the top hierchical table (Integer key) in this monolithic database with every other table in the database having (some type of) FK relationship. Even though we've been taught in the many RonR books that we can transcend through any level of parenting, my 'gut' feeling is that this might be very slow. Thus, I'm thinking that 'Aggregations' might speed things up by forming 'compound fields" with the two integers (Project_id and AnyTable_id).
Then when a user wanted to perform a Tablename.Find(:first) they could simply pass the session[project_id] at this to find the first row in this table for this project.
I've also contemplated a 'front end' design to allow the user to login and securely access their PROJECT(s).

Before you mentioned having distinct databases, now you mention one database and a bunch of relationships. Which is it?

Getting into compound fields and session variables etc. etc. is just asking for trouble.  I can't tell what your application is supposed to do (since you seem to be trying to keep it hush hush) but it doesn't sound like it needs to be nearly as complicated as you're making it out to be.

BraveDave wrote:

If there's anyone out there that is trying to accomplish this type of endeavor, I would be grateful for any reply.
Thank you,
David

I've been working on a web application (not in RoR) which has many relationships and tables with information that pertain to just one of many users.  I can see where you want to use a bunch of user databases and link them all together in some other parent database but you're just asking for trouble with something as convoluted as that.

Re: Million Dollar Question?

There isn't a built-in way to do this in Rails.

If you're committed to the design you're going to have to do some monkeying around to get it to work. I see two means to accomplish it:

1. You're going to have to manually establish connections to each users database then run raw queries against it. You lose most of the benefits of ActiveRecord.

2. You build an entirely seperate application for each user. The first one is done manually the latter ones are done by auto-generating the right config files and a few .rb files. Each user gets their own database and webservice application, your main application makes calls to the correct userapp based on the username which then acts as a front for its own user specific database.

Both of those options are going to be difficult to develop, even harder to maintain, and really likely to be hard to debug and test. Rails is one of the more rigid frameworks I've used. Rails is also the first and only one that I've used where if something is hard to do within the framework then you're probably doing it wrong. My team and I have learned time and time again that in Rails if we require any more than 5-10 lines of code to accompish a discrete task then its wrong. Rails just does things simply and its not a good idea to add a layer of complexity on top because you immediately lose all the advantages of Rails.

** Your primary goal seems to be the recovery of user data. What are your users doing and what kind of data are they working with that it needs to be recovered so often that this is an issue? **

Post your answers here for the best peer review a RoR programmer could home for. Your idea is getting cut apart pretty badly, but we're doing it because we want to see Rails used successfully. Bending Rails as hard as you need to will probably break it and thus not be a successful project.

Re: Million Dollar Question?

Mr. Bartels,
Thank you for the kind reply. I've been feeling alot of iyre from my questions on this forum and have thought I'd go to the Google one or elsewhere.
DNN used a strategy to maintain many users inside his Basecamp database, but everytime I ask questions that circle this fact, I get "Beyond the known world, there are Dragons!"
I don't want to do things the hard way, but I can't believe no one has a web based application where users only see their own data.
I'll just keep knocking on that door.
Thank you for your kind words.
David

Re: Million Dollar Question?

BraveDave wrote:

DNN used a strategy to maintain many users inside his Basecamp database, but everytime I ask questions that circle this fact, I get "Beyond the known world, there are Dragons!"

I don't think anyone knows EXACTLY how Basecamp is setup but I doubt it's complicated and convoluted.

BraveDave wrote:

I don't want to do things the hard way, but I can't believe no one has a web based application where users only see their own data.

I've built many a web application (in PHP, but the language doesn't matter) where there is one database with multiple users and each table has data for multiple users but each individual user only sees THEIR data.  This isn't something that requires using one database per user.  Out of curiosity where did you get the idea that multi-user web applications would need to store user data in different databases in order to create a situation where "users only see their own data".

Is there some other piece of the your puzzle that I'm missing here?

Re: Million Dollar Question?

clayton wrote:

This isn't something that requires using one database per user.  Out of curiosity where did you get the idea that multi-user web applications would need to store user data in different databases in order to create a situation where "users only see their own data".

From his earlier posts he was offering the example use cases for some sort of application/userbase where there may be a high demand for rollbacks, backups, and restores. Isolating a single users data to its own database would better facilitate using standard database  utilities to proctect the data.

Its an interesting idea if one is stuck in an environment where a user needing all of their data backed up and restored is a common enough occurence to warrant such seperation. It also may lead to some speed improvements as it means each database is only being used by one user, then again it could kill performance as one server has to run a lot of databases.

Dave, any chance you can clue us in to what kind of application you're working on? What kind of environment is this targeted at that you need that kind of seperation? What is the square root of -1? Is there some reason that writing the application once and then cloning it for each user wouldn't work? Are users people or are users another organization or piece of software?

I still think that seperate user databases aren't the right path, but its proving to be an interesting academic exercise.

Re: Million Dollar Question?

Mr. Bartels,
I sent you a private email to you to more fully describe my project.
In summary, for any reader reading this thread, there is a distinct need to be able to maintain a large 'cluster' of related domains (each having a distinctive domain (not subdomain)), without copying each RonR application to it's own application directory and 'cloning' a small portion of it.
David

Re: Million Dollar Question?

jbartels wrote:

From his earlier posts he was offering the example use cases for some sort of application/userbase where there may be a high demand for rollbacks, backups, and restores. Isolating a single users data to its own database would better facilitate using standard database  utilities to proctect the data.

Thanks for answering since he stopped talking to me big_smile

I still think that you could build this type of functionality into your application without needed to use one database for each user. Using one database per user would certainly be EASIER from the DB Admin's point of view, dumping and restoring one database is a lot easier to do than dumping a certain slice of a larger database.

However, in another thread I mentioned that I faced the exact same problem with an application I am currently developing and through proper design and some additional functionality I was able to allow users to make a "backup" of their data which they can restore as needed.