Numerical Half-Truths

Numerical programs are based on mathematical algorithms. The fun part is that mathematical truths are not quite so true on a computer.

Here is a simple mathematical statement:

1 + ε > 1,     for all ε > 0

On a computer this inequality will be false when ε is small.

For example, using ε = 2^-53 gives this equality using double-precision:

1 + 2^-53 = 1.

Note that 2^-53 is nowhere near the smallest positive number on a 64-bit computer, which is 2^-1074.

On the other hand, 64-bit computers behave properly for these values:

1 + 2^-52 > 1.

Look here for an explanation why 2^-53 is special in this case.

Numerical programmers must keep in mind that math is only sometimes true on a computer. The cautionary rule for programmers: code can be logically correct and work for most cases, but then fail for some special values. Design your tests appropriately!