Topic: Huge memory saving in rails 2.3.4

Hello,

I am working with a large application where a fresh mongrel uses 220MB of memory (REE 1.8.6 with 37signal GC settings). I wanted to find out what uses all that memory, so I started to profile all the main steps in Rails::Initializer.process method. With the help of the trashed plugin, I discovered the load_view_paths step consumed 63MB. My application has almost 1000 templates and partials, but the total size of all those files is little over 1MB. So, how 1MB of files turns into 63MB of memory?

I'm using config.action_view.cache_template_loading = true, which causes rails to find all the templates and partials, construct ActionView::Template objects and load them. The loading is done by declaring a bunch of memoized methods and then freezing each template instance. Memoization aliases freeze and primes the cache for all the memoized methods. It does so like this,

def prime_cache(*syms)
  syms.each do |sym|
    methods.each do |m|
      if m.to_s =~ /^_unmemoized_(#{sym})/
        if method(m).arity == 0
          __send__($1)
        else
          ivar = ActiveSupport::Memoizable.memoized_ivar_for($1)
          instance_variable_set(ivar, {})
        end
      end
    end
  end
end

Turns out this prime_cache method is responsible for about 52MB of memory! Somehow calling methods on each template instance is not great and then the =~ uses lots of memory as well. I don't really understand why this memory is not freed?! But trashed continued to show all of this allocated even after an explicit GC.start.

So I changed the load! method of ActionView::Template (vendor/rails/actionpack/lib/action_view/template.rb:210) from,

def load!
  freeze
end

with a version that explicitly invokes each method that has been memoized and by-passes prime_cache,

def load!
  format_and_extension
  mime_type
  path
  path_without_extension
  path_without_format_and_extension
  relative_path
  filename
  source
  method_segment
  handler
  method_name_without_locals
  
  __send__(:variable_name) if respond_to? :variable_name
  __send__(:counter_name)  if respond_to? :counter_name

  freeze_without_memoizable
end

Boom ... 52MB saving!! This seem incredible. Am I crazy or is this actually true. I'm curious whether someone else can reproduce these results. Of course, your mileage will be relative to the number of templates and partial your application has.

Last edited by pkmiec (2010-08-18 02:36:42)