Topic: Working with large fixture datasets

As my application grew, I found it increasingly tedious to try to generate hand-coded fixtures.  Some searching revealed the following chunk of code which lets me take a snapshot of my current development database, and save it into fixture files (yamls).  http://snippets.dzone.com/posts/show/2525.  The flip side of that coin is this little nugget: http://www.railsweenie.com/forums/2/topics/778.  This will load your database from a set of fixtures, just like what your tests will see.  I modified the extract_fixtures task to extract to a different directory so that I can maintain control over what gets migrated into my permanent set of fixtures.
One drawback that I found with the extract_fixtures task is the fact that the yaml file it produces is not sorted in any way.  Imagine a diff of a yaml file that has 400 records in it!  I finally today got off my arse to produce this little gem which runs as part of the extract_fixtures task.  It not only sorts the records by id, but it also strips unnecessary quotes, and orders the fields in each record by it's column name.

class YamlObject

  def initialize(name)
    @name = name
    @lines = []
  end
 
  def add_line(line)
    fields = line.split(':')
    if !fields.empty? && fields[0].strip == 'id'
      temp = fields[1].strip
      @ar_id = temp[1..temp.size-1].to_i
    end
    @lines << line
  end
 
  def write(file)
    file.puts @name
    @lines = @lines.sort_by { |o| o.split(':')[0] }
    @lines.each do |line|
      file.puts line.sub(/\"([0-9]*)\"/, '\1')
    end
  end
 
  attr_accessor :ar_id
end

$dir = 'test/dev/fixtures/'
Dir.open($dir) do |dir|
  dir.each do |file|
    next if file !~ /yml$/
    filename = $dir + file
    in_object = false
    objects = []
    object = nil
    globals = []
    File.open(filename).each do |line|
      next if line =~ /^---/
      next if line =~ /^\s*$/
     
      if line =~ /^<%/
        globals << line
      elsif line =~ /^[^\s]/ #start of object
        if in_object
          objects << object
        else
          in_object = true
        end
        object = YamlObject.new(line)
      else
        object.add_line(line)
      end
    end
    objects << object if in_object
    if !objects.empty?
      objects.each do |x|
        puts filename if x.ar_id.nil?
      end
      objects = objects.sort_by { |o| o.ar_id }
      file = File.new(filename, "w")
      if !globals.empty?
        globals.each do |line|
          file.puts(line)
        end
      end
      objects.each do |o|
        o.write(file)
      end
      file.close
    end   
  end
end

BrewControl.com - Brewery and Brewpub management powered by RoR