Problems with NaN (not a number)

  1. ‹ Older
  2. 3 years ago

    Jean-Yves P

    27 Jan 2017 Testers, Xojo Pro Europe (France, Besançon)

    with the str thing, I can check for NAN but also easy the same way for +INF or -INF ...

  3. Michel B

    27 Jan 2017 Testers, Xojo Pro
    Edited 3 years ago

    @Robert W I think you're confusing 'complicated' with 'reliable.'

    Don't be upset. There is never one unique way of doing things. If people don't adopt your method, it is not the end of the world.

  4. Markus W

    8 Mar 2017 #JeSuisHuman New Zealand, Auc...
    Edited 3 years ago

    @Robert W I don't understand obsession of doing a string conversion. Why make it more complicated than necessary?

    I just ran a comparison between my IsNaN function, and Christian's suggestion of using "x<>x". Here are the results:

    Invalid
    Operation   x*0<>0   x<>x
    
    log(0)      True     False
    log(-1)     True     True 
    sqrt(-1)    True     True 
    1/0         True     False 
    -1/0        True     False 
    exp(1e40)   True     False 
    
    Valid
    Operation   x*0<>0   x<>x
    
    100         False    False 
    0/5         False    False 
    2+4         False    False 

    The x<>x comparison fails in many cases, apparently because it doesn't detect +inf or -inf

    No.

    Let's just look at passing 1/0 to your function (just follow it in the debugger):

    1/0 is INF.
    INF * 0 is NaN.
    NaN is <> 0 so you return TRUE for the IsNaN function.

    But the argument passed to your IsNaN function was INF, not NaN, so you return the wrong value.

    Christian's function works (even if you pass 1/0 aka INF or -1/0 aka -INF to it):

    Public Function IsNaN(x As Double) as Boolean
      return x<>x
    End Function
  5. Robert W

    8 Mar 2017 Western Canada

    Yes, I pointed that out in my earlier post; it returns true for NaN, inf and -inf, which is what I find most useful for my needs. If you need to distinguish between these three results, then Christian's suggestion of using the x<>x test will return true only for NaN's.

    I also need to post a warning about using these functions. Michel was correct about reliability. I was testing these again today, and discovered that both my function And Christian's function fail in 64 bit builds. I came up with a workaround for my function, but haven't yet attempted a workaround for Christian's.

    Public Function IsNaN1(x As Double) as Boolean
      #If Target32Bit
        return (x*0<>0)
      #Else
        Return not(x*0<>0 or x*0=0)
      #Endif
    End Function

    But please note that this has only been tested on Macintosh builds. So, if you wish to use it, then it should be tested on whichever hardware it's to be deployed on.

    Because of this inconsistency, I decided to look at other ways of testing for NaN's. The most direct is to look at the bit pattern of the result of the mathematical operation. So, I came up with this alternative function:

    Public Function IsNaN2(x As Double) as Boolean
      'Test for invalid number: NaN, +inf, -inf
      static m As new MemoryBlock(8)
      m.DoubleValue(0) = x
      Return Bitwise.BitOr(m.Int64Value(0),&h800fffffffffffff)=-1
    End Function

    This can easily be modified to distinguish between NaN, inf and -inf.

    For comparison purposes, I also put together a string based function:

    Public Function IsNaN3(x As Double) as Boolean
      return InStr(str(x),"N")>0
    End Function

    Speed testing these three functions by running each one 10 million times, I get the following results:

    Function     Time (32 bit)     Time (64 bit)
    --------     -------------     -------------
    IsNan1        2.773251 sec.     0.5279456 sec.
    IsNan2        2.395691 sec.     1.862914 sec.
    IsNan3       12.01183 sec.      9.931013 sec.
  6. Markus W

    8 Mar 2017 #JeSuisHuman New Zealand, Auc...

    @Robert W Yes, I pointed that out in my earlier post; it returns true for NaN, inf and -inf, which is what I find most useful for my needs.

    You should not call your function IsNaN then … the name should always clearly reflect what it does.

  7. Jean-Yves P

    8 Mar 2017 Testers, Xojo Pro Europe (France, Besançon)

    @Robert W Christian's function fail in 64 bit builds

    isn't it a xojo bug instead of a christian bad method ?

  8. Markus W

    8 Mar 2017 #JeSuisHuman New Zealand, Auc...

    @Jean-YvesPochez isn't it a xojo bug instead of a christian bad method ?

    Depends. Christian's method relies on NaN not comparing to itself. I don't think that is documented anywhere.

  9. Robert W

    8 Mar 2017 Western Canada

    In my opinion, it's a Xojo bug.
    The IEEE 754 specification which defines the standard double floating point format, and which defines NaN's, also specifies how NaN's should behave with comparison operators. Assuming that Xojo follows the IEEE 754 specification, then there must be a bug in either the 32 bit build or the 64 bit build, because the 32 and 64 bit builds should be consistent if they conform to IEEE 754.

  10. 2 years ago

    Steve K

    3 Oct 2018 Melbourne, Australia

    Hey guys, I really really would prefer not to be a part of this thread because it's beyond my understanding, but I do need to know where things are at with this issue.

    I've read this post a few times but there is no clear answer, (except maybe for Robert Weavers information in the preceding post.) There seems to be a lot of ideas thrown around, but no clear cut solution. Although Kems solution was looking good, then is seemed to be disregarded because of an apparent flaw.

    I'm certainly not offering a solution. I just need an answer.

    I've had the issue before (about a year ago) and it wasn't a big deal because it was a "display" issue and never affected any important calculations. I got around it something like this:

     // detect if nan or Inf
      If InStr (showCurrentGs,"Inf") <> 0 Then
        showCurrentGs = "0.0"
      End If
      
      If InStr (showCurrentGs,"nan")<> 0 Then
        showCurrentGs = "0.0"
      End If //

    However, now I have a situation where a number is calculated and sent to an external device. I believe I'm getting (-Nan) and the external device becomes corrupt. Although it can be reset (via software), this is not an acceptable scenario.

    I haven't done any more work on my software for the last week because I this issue. I've done some reading but want to make sure the solution is as foolproof as possible.

    I guess I could try InStr as before - or is Kems solution better?

  11. Robert W

    3 Oct 2018 Western Canada

    The Regex solution appears to be more intended to check the validity of user input. If your variable showCurrentGs is the direct output of a numeric calculation, then it will either be a valid number, or it will be a NaN, or inf. So your test will work fine as is. In fact you can simplify it to a single test to look for the character "n" which would never appear in a normal number.

    // detect if nan or Inf
      If InStr (showCurrentGs,"n") <> 0 Then
        showCurrentGs = "0.0"
      End If

    This is essentially the same code as the IsNan3 function that I posted above.

  12. Steve K

    4 Oct 2018 Melbourne, Australia

    Thanks Robert.

    It turns out that there were two places that the error was occurring which made it hard to track down.

    The main issue was displaying the calculated result from a double (calAverage). So I've used your simplified method:

     calAverage = round((total / (calibrationArray.Ubound-1) - calibrationPointNEW)) // Displayed Calibrated/Tare
    
      If InStr (Str(calAverage),"n") <> 0 Then
        calAverage = 0
      End If 
    
      lblShowCalibratedData.Text = Str(Format(calAverage/1000, "-00.000")) //Display formatted result

    I haven't seen (-Nan) since. The poor dear... :)

    Also, I think it was just was I was "seeing" and the (-Nan) string was never sent to the external device like that, or if it was, it didn't cause an issue. It was just internal calculations that became corrupt. I tried to check my external serial device via the Arduino IDE to check the data, but starting that IDE resets the device and therefore I could never tell.

    All looking good now though. Thanks Robert.

    Cheers.

  13. last year

    Garry P

    17 Apr 2019 Testers, Xojo Pro Europe (Torquay, UK)

    This is such an interesting yet infuriating problem. I'm porting a physics library from C# and so came across a need for a function to return True if a number is a NaN but False if it's a real number or positive/negative infinity.

    Using String or Text comparisons is unacceptable as the library makes thousands of these checks and I can't tolerate the performance overhead.

    I've been trying out its of the solutions above. I think there must be a bug in Xojo's comparator operator for Doubles.

    macOS 64-bit

    Public Function IsNan1(x As Double)
      Return x <> x
    End Function
    
    Dim r1 As Boolean = IsNan1(10) // Real number
    Dim nan1 As Boolean = IsNan1(Sqrt(-1)) // NaN
    Dim nan1b As Boolean = IsNan2(0/0) // NaN
    Dim log01 As Boolean = IsNan1(Log(0)) // - Infinity
    Dim negInf1 As Boolean = IsNan1(-1/0) // - Infinity
    Dim inf1 As Boolean = IsNan1(1/0) // Infinity
    Dim exp1e41 As Boolean = IsNaN1(Exp(1e40)) // Infinity

    Results:
    r1 = False (correct)
    nan1 = False (wrong)
    nan1b = False (wrong)
    log01 = False (correct)
    negInf1 = False (correct)
    inf1 = False (correct)
    exp1e401 = False (correct)

    Public Function IsNan2(x As Double)
      Return Not (x = x)
    End Function
    
    Dim r2 As Boolean = IsNan2(10) // Real number
    Dim nan2 As Boolean = IsNan2(Sqrt(-1)) // NaN
    Dim nan2b As Boolean = IsNan2(0/0) // NaN
    Dim log02 As Boolean = IsNan2(Log(0)) // - Infinity
    Dim negInf2 As Boolean = IsNan2(-1/0) // - Infinity
    Dim inf2 As Boolean = IsNan2(1/0) // Infinity
    Dim exp1e402 As Boolean = IsNaN2(Exp(1e40)) // Infinity

    Results:
    r2 = False (correct)
    nan2 = True (correct)
    nan2b = True (correct)
    log02 = False (correct)
    negInf2 = False (correct)
    inf2 = False (correct)
    exp1e402 = False (correct)

    As you can see, IsNan2 works correctly, IsNan1 fails. What is strange is x <> x should give the same result as Not (x = x) but it doesn't. Surely this is a bug?

  14. Dave S

    17 Apr 2019 San Diego, California USA
    Edited last year

    Looks like it may be a bug in Xojo...
    I duplicated your exact test in SWIFT and it produced the results you seemed to have been expecting

    func isNAN(_ x:Double) -> Bool { return (x != x) }
    func test() {
        print("---- START TEST ----")
        print( isNAN(10)        )
        print( isNAN(sqrt(-1) ) )
        print( isNAN(0/0)       )
        print( isNAN(log(0))    )
        print( isNAN(-1/0)      )
        print( isNAN(1/0)       )
        print( isNAN(exp(1e40)) )
        print("---- END TEST ----")
    }

    ---- START TEST ----
    false
    true
    true
    false
    false
    false
    false
    ---- END TEST ----

    if produced the SAME results if I changed the isNAN function the same as you had

    func isNAN(_ x:Double) -> Bool { return !(x == x) }
  15. Robert W

    17 Apr 2019 Western Canada
    Edited last year

    I deliberately made my versions of IsNaN functions to return true for NaN +Inf and -Inf. The memoryblock based function is probably the fastest and most reliable. It can be modified to return true only for NaNs and false for +/-Inf, since the function is looking directly at the NaN bits in the number. Here is a modified function which I put together rather quickly and haven't had a chance to test yet.

    Public Function IsNaN3(x As Double) as Boolean
      'Test for invalid number: NaN, but not +inf, -inf
      static m As new MemoryBlock(8)
      m.DoubleValue(0) = x
      Return (Bitwise.BitOr(m.Int64Value(0),&h800fffffffffffff)=-1)_
         and (Bitwise.BitAnd(m.Int64Value(0),&h000fffffffffffff)<>0)
    End Function
  16. Jürg O

    17 Apr 2019 Testers, Xojo Pro

    @Garry P This is such an interesting yet infuriating problem... I think there must be a bug in Xojo's comparator operator for Doubles.

    Just recently I've noticed that when it comes to trying to "compare" NaN, the Xojo Framework behaves differently on Target32Bit and Target64Bit. And also some differences between Platforms...
    I can't say if that's related to your situation - but it's maybe worth being aware of how Xojo handles this currently...:
    Feedback Case #55370 NaN: Different behavior and comparison results in Target32Bit and Target64Bit

  17. Robert W

    17 Apr 2019 Western Canada
    Edited last year

    I've now had a chance to test the IsNaN3 function I gave in my last post, and it appears to work correctly:

    log(-1):   True
    ASin(500): True
    sqrt(-5):  True
    log(0):    False  (Inf but not NaN)
    1/0:       False
    (-1)/0:    False

    The method of putting the double being tested into a memory block should be reliable, because we can then directly test the bits that determine whether the number is a NaN, or an Inf, and not have to guess at the internal logic of Xojo's comparison operators.

    The format of a double is
    SEEEEEEE EEEEFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF
    where S=sign bit, E=exponent bits, F=fraction bits (also called mantissa or significand)

    If all 11 bits of the exponent are set to 1, it indicates an abnormal value: either a NaN or an INF. My original function simply checked these bits to see if they were all 1's, because for my purposes I didn't need to distinguish between NaN's and Inf's. However, to distinguish between a NaN and Inf it's necessary to check the fraction bits as well. If they are all 0's then it's an Inf. (In the case of an Inf, the sign bit determines whether it's a +Inf or a -Inf.) Otherwise it's a NaN. So, the second bitwise test checks the fraction bits for all 0's.

  18. Tim H

    17 Apr 2019 Portland, OR USA

    I agree that testing the bits is the best approach. You can optimize that by using the AND and OR operators instead of the Bitwise functions (no function call overhead).

  19. Robert W

    17 Apr 2019 Western Canada

    I thought AND and OR were just shorthand, and still called the bitwise functions.

  20. Tim H

    17 Apr 2019 Portland, OR USA

    No, they compile direct to machine code. Much faster, and the compiler can further optimize if possible.

  21. Robert W

    17 Apr 2019 Western Canada
    Edited last year

    Thanks. In that case, here is the simpified code using the AND and OR operators:

    Function IsNaN3(x As Double) as Boolean
      'Test for invalid number: NaN, but not +inf, -inf
      'using bitwise ops to detect the actual IEEE-754 NaN bit pattern
      static m As new MemoryBlock(8)
      m.DoubleValue(0) = x
      Return ((m.Int64Value(0) OR &h800fffffffffffff)=-1) AND ((m.Int64Value(0) AND &h000fffffffffffff)<>0)
    End Function

or Sign Up to reply!