Someone in the lab recently came to me asking for help in diagnosing a strange problem. The time output from their program was drifting inexplicably — that is, they were adding a certain increment every iteration, but the time was changing by some value slightly different from that increment. It turns out that this was because they were using single-precision floating-point (a.k.a “float”) variables. The problem did not show up right away. In fact, it would not have been an issue were they not adding an increment so very many times.

To diagnose this problem, (because I haven’t dealt with this issue in a long while), I wrote this little piece of code:

#include <stdio.h>

int main(void){

  float float_v, fdt;
  double double_v, cross_v, ddt;
  int i, found;

  float_v = 0;
  double_v = 0;
  cross_v = 0;
  found = 0;

  ddt = 1e-5;
  fdt = 1e-5;
  for(i=0; i< 10000; i++){
    double_v += ddt;
    float_v += fdt;
    cross_v += fdt;
     if(i%100 == 0){
       printf("Float: %5.10f ", float_v);
       printf("Doubl: %5.10f ", double_v);
       printf("Cross: %5.10f ", cross_v);
       printf("Diff : %5.10f\n", double_v - float_v);
     } 

/* use this to find how many iterations before it screws up
    if(double_v - float_v > 1e-8 && !found){
      printf("Found discrepancy of %5.10f at step %d!\n", \
               double_v - float_v, i);
      found = 1;
    }
*/
  }

  return 0;
}

(You can download the code here: add_test.c)

This code tests three combinations:

1. Float counter with float increment
2. Double counter with double increment
3. Double counter with float increment

The little commented section (starts with /* and ends with */) will (if uncommented) stop the program when the difference between the all-double value and the all-float value exceeds 10^-8. On Mac OS X on a 64-bit Intel architecture, this happens after 912 steps:


Found discrepancy of 0.0000000101 at step 912!


Here’s the output from 900 iterations (printed every 100):

Float: 0.0000100000 Doubl: 0.0000100000 Cross: 0.0000100000 Diff : 0.0000000000
Float: 0.0010100005 Doubl: 0.0010100000 Cross: 0.0010100000 Diff : -0.0000000005
Float: 0.0020099971 Doubl: 0.0020100000 Cross: 0.0020099999 Diff : 0.0000000029
Float: 0.0030100048 Doubl: 0.0030100000 Cross: 0.0030099999 Diff : -0.0000000048
Float: 0.0040100124 Doubl: 0.0040100000 Cross: 0.0040099999 Diff : -0.0000000124
Float: 0.0050100200 Doubl: 0.0050100000 Cross: 0.0050099999 Diff : -0.0000000200
Float: 0.0060100276 Doubl: 0.0060100000 Cross: 0.0060099998 Diff : -0.0000000276
Float: 0.0070100352 Doubl: 0.0070100000 Cross: 0.0070099998 Diff : -0.0000000352
Float: 0.0080100335 Doubl: 0.0080100000 Cross: 0.0080099998 Diff : -0.0000000335

As you can see, after 100 iterations (first line) everything is OK. However, after 200 iterations, there’s already a difference (see “Diff: ” column) of -0.0000000005 between the double and float counters. Note that the doubles all end in 5 zeroes (as they should) while the float and cross variables end in some random assortment of digits. This is what first alerted the user to the problem.

This is one of the things I like about programming. If you’re not sure how something will work, it’s often easier to cook up a little test program and find out than to look around for an answer. Ultimately, it’s important to understand the difference between floats and doubles, but if the system doesn’t conform to IEEE standards, you might have to test it to find out anyway.

The moral of the story is: use doubles when you’re doing a lot of iterations with a counter. Don’t use floating-point values as increments at all. (See Rob’s comment for the correction).