Topic: ActiveRecord Float precision? How to represent $10 000.12

Hi Guys

I just viewed some weird behaviour. I have a model which includes a float. It seems that the precision of a floating point record is not very good. Pathetic actually. I use the model to keep track of money (mostly), and it loses precision pretty quickly: Check it:

Variable.create{:numval => 100.12}  #works fine
puts Variable.last.numval returns
>> 100.12

Variable.create{:numval => 1000.12}  #works fine also
puts Variable.last.numval returns
>> 1000.12

BUT
Variable.create{:numval => 10000.12}  # STUFFS OUT MISERABLY
puts Variable.last.numval returns
>> 10000.1

and
Variable.create{:numval => 100000.12}  # even worse sad
puts Variable.last.numval returns
>> 100000


I cant have an accounting solution that cant handle $10 000.12 ?????

I am aghast. My app has been deployed with 20 companies and now this?? WTH

PLEASE PLEASE tell me there is something simple I am missing?

Your kind assistance, as ever, will much appreciated.

Re: ActiveRecord Float precision? How to represent $10 000.12

Check Rails AP, ActiveRecord::ConnectionAdapters::TableDefinition and look for 'precision' option in 'column' method, as it depends on the DB you use.
Normally, you define the precision in your migration file, something like this:

class CreateOperations < ActiveRecord::Migration
  def change
    create_table :operations do |t|
      t.integer :account_id
      t.string :operation_type
      t.date :value_date
      t.date :close_date
      t.decimal :sum, precision: 10, scale: 2
      t.decimal :rate, precision: 5, scale: 2
      t.decimal :withholding, precision: 5, scale: 2
      t.integer :duration
      t.decimal :interests, precision: 10, scale: 2
      t.decimal :total, precision: 10, scale: 2

      t.timestamps
    end
  end
end

You can also check 'schema.rb' file to see all the tables columns definitions.
Hope this helps.

Re: ActiveRecord Float precision? How to represent $10 000.12

Hi Javix

Thanks for the pointers. I got it right!

Interestingly enough there were 2 issues: Floats are 7 bytes, so they lose resolution above 10000 if you want at least 2 least significant digits.

The first option was to use a double, ie 14 bytes, but:

Another issue I found when googling is this :

Try (10.12*100).to_i

It returns 1011! Nasty. The solution, apparently, is to use decimal, as you have done.

Cheerio!

Re: ActiveRecord Float precision? How to represent $10 000.12

Ruby 'to_i' method does not around by math rules, just cuts its decimal part:

1.9.3p327 :001 > 10.12*100
 => 1011.9999999999999 
1.9.3p327 :002 > x=10.12*100
 => 1011.9999999999999 
1.9.3p327 :003 > x.to_i
 => 1011 
1.9.3p327 :004 > 

Last edited by Javix (2013-05-14 04:24:16)