Why does ConvertEncoding fail when source encoding is nil?

General
1

I have wasted an afternoon trying to figure out why my Win API call failed and no found the culprit. My question is if this is a bug in Xojo or not?
The API expects the argument in UTF16 and I stored the data to be passed as a base64 encoded value. Doing a simple DecodeBase64 -> ConvertEncoding did not work! After fighting this I saw that ConvertEncoding don’t work in the source’s encoding is Nil!!! I thought the purpose of ConvertEncoding was to… well… convert encodings…

This simple test demonstrates this (fails in Xojo 2014r2.1 on Windows 7 SP1 64-bit)

[code] dim s as String = “Hello World”
dim s2 as String

break
// Check the encoding of s - UTF8 as it is the default
// Now we EncodeBase64s
s2 = EncodeBase64(s)
// Now we DecodeBase64s2 back - s will have Encoding nil
s = DecodeBase64(s2)
// Let’s try to set encoding (will FAIL as s’s Encoding=nil!!)
s = ConvertEncoding(s, Encodings.UTF16)
// OK - try again and now specify the target Encoding for Base64Decode
s = DecodeBase64(s2, Encodings.UTF8)
// Do the conversion (this will work!)
s = ConvertEncoding(s, Encodings.UTF16)
[/code]
If someone could just say that this is the expected behaviour I wont file a bug but instead file a request that docs should be updated to reflect this.

2

OK - found this 4 year old report <https://xojo.com/issue/12524> so I guess it is the intended behaviour.
I will file a bug report to get the docs updated.

3

To set the encoding on an “incoming” string use DefineEncoding, not ConvertEncoding:

... s = DecodeBase64(s2) s = DefineEncoding(s, Encodings.UTF8) // insert this line s = ConvertEncoding(s, Encodings.UTF16) ...

4

Convert encoding converts from one known encoding to another
At least thats what the docs say
http://documentation.xojo.com/index.php/ConvertEncoding
Provides a quick way to convert a string of known encoding to some other encoding, without having to create a TextConverter object.

To take text from an outside source you should use define encoding

5

[quote=141559:@Norman Palardy]Convert encoding converts from one known encoding to another
At least thats what the docs say
http://documentation.xojo.com/index.php/ConvertEncoding
Provides a quick way to convert a string of known encoding to some other encoding, without having to create a TextConverter object.

To take text from an outside source you should use define encoding[/quote]
One could argue that I know that the encoding is Nil… :slight_smile:

6

I suppose YOU could - but the framework obviously would disagree about what “known encoding” means
The framework doesn’t treat NIL as a “known encoding”
Those are all the ones in http://documentation.xojo.com/index.php/Encodings
Nil isn’t in that list

7

An empty string has an ASCII encoding. Tested it.
Discussed here : https://forum.xojo.com/16512-encoding-trouble-mysterious-empty-added-row

8

This isnt an empty string though
This is a bunch of bytes recovered by decoding base 64 data
Debug the code the OP posted and you’ll see that

9

[quote=141550:@Mattias Sandström]// Now we DecodeBase64s2 back - s will have Encoding nil
s = DecodeBase64(s2)[/quote]

How did you find s was nil encoding ? Try this to make sure of the encoding of the string :

[code]// Now we DecodeBase64s2 back - s will have Encoding nil
s = DecodeBase64(s2)

msgbox encoding(s).internetname[/code]