Wednesday, March 2, 2011

Ruby Floating floats like a Dead Fish

I was so serious in attempting to solve a problem that Thoughtworks usually gives to its new recruits to test their Object Orientation skills. In that there is a Sales Tax problem which involves lot of math library, floating point calculations rather than using our brain memory in it.

While when I do the rounding of a sales tax calculation to the nearest 0.05, I faced the problem in my ruby
Though I agree that this is my first time where I am doing intense floating point calculations and round off, could someone explain me why this inconsistency happens with ruby?

I need at least two decimal places right in my calculations, rather than implementing a work around to round off a 8 decimal float number .

Also it should be noted that round(n) as written in Float class in ruby, which rounds to 'n' decimal places doesn't work with Jruby. Any clue on this???


Peter Vandenabeele said...

Use BigDecimal for this:

$ ruby -v
ruby 1.9.2p180 (2011-02-18 revision 30909) [i686-linux]
$ irb
ruby-1.9.2-p180 :001 > require 'bigdecimal'
=> true
ruby-1.9.2-p180 :002 > price_no_tax ="10.32")
=> #
ruby-1.9.2-p180 :003 > price_no_tax - 10
=> #
ruby-1.9.2-p180 :004 > (price_no_tax - 10).to_s
=> "0.32E0"
ruby-1.9.2-p180 :005 > tax = price_no_tax *"0.21")
=> #
ruby-1.9.2-p180 :006 > total_price = price_no_tax + tax
=> #
ruby-1.9.2-p180 :007 > "%s" % total_price
=> "0.124872E2"
ruby-1.9.2-p180 :008 > "%d" % total_price
=> "12"
ruby-1.9.2-p180 :009 > "%5.2f" % total_price
=> "12.49"
ruby-1.9.2-p180 :010 > rounded_price_to_5_cents = (total_price*20).round * BigDecimal("0.05")
=> #
ruby-1.9.2-p180 :011 > "%5.2f" % rounded_price_to_5_cents
=> "12.50"

Gabriel Sobrinho said...

You can use BigDecimal class to calculate with precision:

BigDecimal('10.32') - BigDecimal('10')
BigDecimal('10.52') - BigDecimal('10')
BigDecimal('10.43') - BigDecimal('10')

This will solve your problem :)

Jim Croft said...

This problem is the same for every language that uses a binary / IEEE-754 representation for floats. That includes, C, Java, Javascript, PHP, Python and, yes Ruby too.

Basically binary representations of floats are only ever approximations.

For accurate (but usually much, much slower) representation and computation of floats you need to use a non binary / IEEE-754 form like BigDecimal.

Anonymous said...

This is consistent with any programming language you'll use that uses floating-point notation. For example, in Java:

System.out.println(10.97 - 10.93);
System.out.println(10.92 - 10.93);

Returns 0.040000000000000924 and -0.009999999999999787.

So, as other people said, use BigDecimal. Is optimized for precise computations of decimal numbers at the price of speed.

Yorick said...

This is just the conversion between base 2 and base 10. BigDecimal stores the numbers as binary-coded decimals so it doesn't have this problem.

vapid babble said...

Seems not all implementations have this:
ruby -v
ruby 1.9.2p0 (2010-08-18) [i386-mingw32]
ruby -e "puts 10.32-10"

ruby -v
ruby 1.8.5 (2006-08-25) [x86_64-linux]
ruby -e "puts 10.32-10"

Lonny Eachus said...

Actually, if you want to meet accounting standards, those are the wrong answers. That is, BigDecimal might do it, but that is not a standard way of doing monetary calculations.

Programs that do math on currency typically use at least 4 decimal points (hundredths of pennies), and used scaled integer math. So 10.52 - 10.00 would work like this:

((10.5200 * 10000) - (10.0000 * 10000)) / 10000
=> 0.5200

This is why the .NET framework has a "currency" data type, which is a scaled integer. Math using the currency type works this way.

Lonny Eachus said...

Actually, since BigDecimal is (in a sense) an integer representation, it might meet accounting standards for these calculations. I am not sure. But they are slow... whether doing the scaling yourself as I showed or using standard BigDecimals is faster is an interesting question.

Lonny Eachus said...

(Note: My example of currency-type math was more schematic than actual, since the numbers are stored and manipulated as integers, and are just displayed as floats. So the example I gave would actually be, internally:

X = 105200 - 100000

and the scaling doesn't take place until you want to display X. But this scheme has a lot less overhead than BigDecimal. You could create your own currency type, but to get the advantages of it you would also need to define the arithmetic functions for it.)

Jarmo Pertman said...

I have written a blog post about that topic long time ago by using 19 different programming languages. I have also some good links there with explanations and so on. Just give it a read at