Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have a problem that exasperates me:

When I take the Windows 7 calculator in standard mode, if I do

4, then (square root), the result is 2

Fine.

But, at this point, if I do

- (minus), then 2, the result is -1,068281969439142e-19 instead of 0!

OK, I know about ϵ (machine epsilon), and yes, -1,068281969439142e-19 is less than the 64 bits ϵ (1.11e-16), so, we have an arithmetic underflow, in other words in this case: 0.

Great, my computer is able to represent subnormal numbers instead of just flush to zero when this happens, and it seems that it is an improvement!

Subnormal values fill the underflow gap with values where the absolute distance between them are the same as for adjacent values just outside of the underflow gap. This is an improvement over the older practice to just have zero in the underflow gap, and where underflowing results were replaced by zero (flush to zero).

BUT:

  • this result is false!
  • when you try to explain the concept of the square root to a child and you end up with this kind of result, it only complicates your task...
  • what is the point to represent subnormal numbers in a standard, non scientific calculator?

So, is there a way to fix this?

share|improve this question
1  
Damn, Raymond Chen led me to believe that Calcuator fixed such things (blogs.msdn.com/b/oldnewthing/archive/2004/05/25/141253.aspx). "The standard IEEE floating point library was replaced with an arbitrary-precision arithmetic library. Today, Calc's internal computations are done with infinite precision..." –  Ian Boyd Apr 1 '12 at 19:17
    
Yes, Windows 7 simple calculator is better than before, and better than a lot of other simple calculators software, but this point of sqrt(4) - 2 = -1,068281969439142e-19 is really annoying... –  Pascal Qyy Apr 2 '12 at 7:21
    
It seems to be fixed. But I don't found the KB… –  Pascal Qyy Oct 26 '12 at 16:43
    
i still get -8.1648465955514287168521180122928e-39 (calculator version 6.1.7600.16385). You can paste 4 @ - 2 = into Calculator to easily repeat the test. –  Ian Boyd Oct 27 '12 at 12:06

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.