Welcome to Atalasoft Community Sign in | Help

Floating Point Errors

I just wrote a chunk of code to do unit conversion between scalar types.  This is not a big deal to do, but being the nerd that I am, I really wanted the code to do something like this:

double conversionFactor =
    LookupConversionFactor(sourceUnits, targetUnits);

return new Measurement(MyValue * conversionFactor, targetUnits);

I like this because the code is tiny and it reads well.  It's just a matter of writing LookupConversionFactor(), which should be a simple table lookup.  So for example, the entry for yards to feet will be 3.0 and the entry for feet to yards will be 1.0/3.0

The problem with doing this is that conversions won't be clean.  If this were put mathematics, it wouldn't be an issue, but in computer land, you can't get an accurate representation of all fractions.  The result is that when you convert 3 feet to yards, you get 0.99999999989 yards (or something like that).  Needless to say, this is unacceptable and my unit test caught it right away.

Lessons: floating point errors are important.  Multiplying by a reciprocal is not always the same as division.
Published Wednesday, June 21, 2006 11:46 AM by Steve Hawley


No Comments
Anonymous comments are disabled