EncodeHex() in Windows crashes with Einhugur binary string

Targets Windows
1

<https://xojo.com/issue/29250>

This bug is only a problem in Windows, not in Mac, Linux or Web Edition. It is not an error in Einhugur’s plugin (I used the latest v10.2.1) but in Xojo. I was using Xojo 2013 b5, but it was there in previous non-beta versions. It was not a problem in RS that I remember as I have this code in the current app.

Convert a string to an encrypted binary string using Einhugur’s plugin:

Dim returnString, tempString As String
Dim tempInt, tempInt2 As Integer

returnString = Encrypt(Salt + OriginalText, PassKey, 0) 'This part works OK
returnString = EncodeHex(returnString, True) 'this is where Windows gives an ‘XYZ application has stopped working’ error. EncodeHex inserts spaces and makes the Hex uppercase

I am not sure if there is something funny in the Einhugur binary string, but it seems OK in the debugger and as I said above it works fine in Mac, Web and Linux.

Workarounds:
The workaround is to Hex the binary string manually — this works in Windows:

'EncodeHex crashes in Windows, so manually Hex the binary string
returnString = “”
tempInt2 = LenB(tempString)
For tempInt = 1 to tempInt2
returnString = returnString + Right(“0” + Hex(Asc(Mid(tempString, tempInt, 1))), 2) + " "
next
if right(returnString, 1) = " " then returnString = left(returnString, len(returnString) - 1) 'remove trailing space

2

Maybe you ask Björn about how he builds the string…

3

Your case occurs for any string or just one specific string? EncodeHex() hangs even with EncodeHex(returnString, False) ? As you know the result of the Encryt(), can you replace this line (to check if the guilt comes from there) by a DecodeHex() to obtain the same result and check if the following EncodeHex() will hang?

4

The string is raw binary data without encoding since its encrypted. I am guessing the EncodeHex function does something internally that assumes encoding. So you would get random crashes passing in binary data to it.

5

I recall reporting something similar on the mailing list a few months ago.

6

Well I have no idea whats wrong but the following encodes every byte value just fine
This suggest something else is going on - but I’m not sure what

Dim returnString As String

dim mb as new memoryblock(256)

for i as integer = 0 to 255
mb.byte(i) = i
next

returnString = EncodeHex(mb, True)

break

7

Could be great trying to isolate the origin as I told, with few tests.
If we can’t reproduce the problem without the Encrypt() part (a specific content causing the problem), Björn will need to check the case more deeply.

8

Is usually character combination that means something in some text encoding and not a specific byte code that creates such crash. Win32 API calls are very sensitive to getting different encoding than expected.

9

I suppose he could give us the hex codes of the string that crashes ?

And better yet, try to execute that string on his system. A string created from the hex codes.

10

Remember this only has a problem with Windows (tests OK on Mac, Linux and WE).

Try the following under Windows:

tempString = Encrypt("!VRkZ$$t4#Rstest123", "v6mX??+q0Q)B", 0) 'This returns ‘??{%<??bJ??t??*??z+?’

tempString = EncodeHex(tempString, True) 'this returns Hex ‘CF CF 7B 25 3C D1 81 62 4A 04 BB 74 BD 89 2A C9 F2 7A 2B BD’ on Mac, but crashes under Windows

11

In the debugger, what is the encoding of tempString before you try to run EncodeHex?

(Both platforms would be helpful)

12

No encoding, just:

Dim returnString, tempString As String

I tried to place a DefineEncoding command after the encrypt plus a MidB(tempString, 0) but neither helped.

13

For a bug report and attach a sample project. I’d really like us to see this in action.

14

I have added an attachment to the FeedBack report. Run it under Windows and click the ‘EncodeHex’ button a couple of times and Windows will complain that the app has stopped working. Clicking the ‘non-EncodeHex’ button does a manual EncodeHex().

Interestingly, placing the code directly in the button rather than calling a method doesn’t seem to cause the crash.

It might also be the particular Salt, PassKey and password I am using, but I don’t think so, since they are normal ASCII characters.

15

Remove the calls to the plugin & see what happens
If you get no crashes without using the plugin then you’ve probably isolated it to something about using the plugin

The code I pasted above I’ve run in a loop about 100000 times and it encodes it all possible byte values each time in that loop with out problems.

Some thing else is the issue