issues in float division a/b

Question

Hello everyone If a and b are both float (or double I think it does not matter?) and I want to calculate a/b. I am wondering whether there will be any issues which make the calculation result in-accurate compared with the calculation result by-hand (on a paper). :-) For example if a is too big or if b is too small? If such issue exists what is the best practices to calculate a/b? thanks in advance George

George George · Answer

Cool Alan! Question answered. So int rounds to zero and float/double rounds to nearest int? regards George

Alan · Answer

Different rounding schemes are used for different purposes. When you convert a double or float to an int the rounding scheme is towards zero. Thus if you convert 51006.9999694824 to an int the answer is 51006. If the number had been negative the answer would have been -51006 rather than -51007. This can be confirmed by running this simple program: using System; class Program { static void Main() { int i = (int)51006.9999694824; Console.WriteLine(i); int j = (int)-51006.9999694824; Console.WriteLine(j); Console.ReadLine(); } }

George George · Answer

Hi Alan I am confused why 51006.9999694824 could rounding two different results? IMHO no matter how length internal storage will be used to store it the result should always be 51007. In what situation the result should be 51006? In my understanding of rounding >=.5 should be round-up and regards George

Alan · Answer

My understanding is that internally (i.e. at CPU register level) floating point arithmetic is carried out to 10 bytes of precision. However when the result is stored in managed memory it will be rounded to the precision of the variable which occupies that memory slot - either float (4 bytes) or double (8 bytes). Now the result of multiplying 51.007f by 1000.0f is (rounded to 16 digits) 51006.9999694824. When this is rounded to float precision (7 digits) the result is clearly 51007. However as the double type can accomodate 15 or 16 digits of precision it is not rounded up to 51007 when stored in a double memory slot. Thus when it is subsequently converted to an int and the fractional part is dropped the result is 51006.

George George · Answer

Thanks Alan I studied your reply carefully. Great! Two more comments 1. the result (to 10 byte precision) is very slightly less than 51007 Seems the intermediate result which uses 10 byte precision has nothing to do with original/destination type (float) right? (since float is not 10-byte precision). What is the rule of how precise the intermediate result will be? 2. so it doesn't get rounded up to 51007. Morover when the result is then cast to an int it's rounded down to 51006 I am confused. In what situation should we round-up and in what situation should we round-down? regards George

Alan · Answer

I've managed to find the code for one of those anomalies which was posted originally back in 2004. However the anomaly still occurs in the latest version of the C# compiler and runtime: using System; class FunnyFloat { static void Main() { float f = 51.007F * 1000.0F; double d = f; int x = (int)d; Console.WriteLine(x); int x2 = (int)(double)(51.007F * 1000.0F); Console.WriteLine(x2); Console.ReadLine(); } } /* output 51007 51006 */ This one doesn't depend on the order in which operations are carried out but the actual operations which are carried out. If I remember correctly when the two floats 51.007F and 1000.0F are multiplied together the result (to 10 byte precision) is very slightly less than 51007. When this value is assigned to the float variable 'f' the difference is well outside the float's range and so it gets rounded up to 51007. Assigning this to the double variable 'd' makes no difference nor does assigning it to the int variable x and so 51007 gets printed out. Now moving on to the second set of operations as before the floats are multiplied together and the result is very slightly less than 51007. However this time the result is cast immediately to a double and is just within the double's range and so it doesn't get rounded up to 51007. Morover when the result is then cast to an int it's rounded down to 51006 assigned to the int variable x2 and so this is what get's printed out! I think you'll now be able to see that all sorts of permutations are possible when operating on floating point values and if you can't tolerate these occasional anomalies (for example in most financial applications) then you should use the decimal type instead.

George George · Answer

Thanks Alan -------------------- I can't produce one off the top of my head and the site where I saw several of these anomalies posted (GotDotNet) has now been shut down. However they do occur but only affect the umpteenth decimal place and so are not normally significant. -------------------- I do not need to exact sample. I am just interested in the basic idea why order impacts the calculation result. Could you just briefly describe the idea then it should be enough. :-) regards George

Alan · Answer

I can't produce one off the top of my head and the site where I saw several of these anomalies posted (GotDotNet) has now been shut down. However they do occur but only affect the umpteenth decimal place and so are not normally significant. I mean to test whether the divisor is zero before making the calculation. In practice this usually means that either there's another problem which is causing the divisor to be zero or that some special value should be assigned in this scenario rather than doing the calculation.

George George · Answer

Cool Alan! Two more comments 1. this can sometimes lead to strange anomalies when displaying values depending on the order in which operations are carried out. Could you show me a sample about why/when result depends on order of calculation please? :-) 2. an explicit test needs to used to avoid this What do you mean explicit test? You test whether the result is NaN or you test whether the divident/divisor is zero before calculation? regards George

Alan · Answer

Unlike integer arithmetic floating point arithmetic (using the float and double types) never produces overflow underflow or divide by zero errors. Instead five special values are supported: positive and negative infinity positive and negative zero and NaN (not a number) which cater for these situations. For example: double d1 = 0.0; double d2 = 0.0; double d3 = d1/d2; Console.WriteLine(d3);// NaN Whilst it's comforting to know that exceptions will not be thrown you may still need to watch out for these sort of situations particularly if values are going to be displayed to the user. Personally I regard the float type as more or less useless and nearly always use double instead (unless a method requires a float parameter of course). This means that for all practical purposes I don't need to worry about overflow or underflow. However I do try to avoid dividing by zero where possible and sometimes an explicit test needs to used to avoid this. One thing that can't be avoided when using floating point arithmetric is the possible loss of precision when values need to be rounded off to a given number of places and displayed to the user. The float type (4 bytes) is accurate to 7 digits and the double type (8 bytes) to 15 or 16 digits. However arithmetic is carried out internally to greater precision (10 bytes I believe) and this can sometimes lead to strange anomalies when displaying values depending on the order in which operations are carried out. In many applications this loss of precision doesn't matter but if you are unable to tolerate it then the decimal type (16 bytes) should be used instead which is accurate to 28 digits but slower in operation.