Crypto and international characters / encoding / sqlite problem

General
1

Sorry for that stupid title, maybe I’ll have a better one sometime later. My problem is this:

Using the Crypto methods RSAEncrypt and RSADecrypt works fine - until the string you encrypt contains international characters like and stuff. They will come back as ?? instead of or or . Well … similar with encodeBase64.

So I “wrapped” the string up before encoding it - put a “encodeHEX” around it. Works fine. I even found a way to lift the “length” barrier by dividing strings that are longer than 32 chars into several chunks that will be de & encrypted separately. I can now encrypt long sentences WITH special characters (like or or or WHATEVER) and decrypt them - works fine. In memory.

THEN I wanted to write that encrypted string into my cubeSQL tables. Looks like it works, too (at least I get a very similar pattern of funny characters in my table’s field) - but when I read from the database, the decryption fails because the string is not long enough or too long. I tried with “text” fields in the database as well as with a “blob”.

I am assuming at the moment that this might have something to do with the encodings. After the encryption the string has a “nil” encoding. When writing to a cubeSQL (and also realSQL I assume) - and reading it back from the table - the string is a UTF8. No way to get it back into a “nil” encoding - at least I don’t know a way.

That’s where I’m lost. I have been even trying to encodeHEX the string again after encoding (which makes it perfect to write into a database) - but … I’ll never get a clean string again to decrypt afterwards - again, encodeHEX also only works with UTF8 encoding - thus produces a string with UTF8 and I cannot NIL the encoding again, can I?

Arrgh. It’s been 10 hours now. Sorry for babbling so much - got to put some air in my brain and try again tomorrow. Or … do you have an idea?

Cheers,

Jan :slight_smile:

2

[quote=199433:@Jan Redepenning]Sorry for that stupid title, maybe I’ll have a better one sometime later. My problem is this:

Using the Crypto methods RSAEncrypt and RSADecrypt works fine - until the string you encrypt contains international characters like äöü and stuff. They will come back as ?? instead of ä or ö or ü. Well … similar with encodeBase64.[/quote]

Looks like you need to DefineEncoding the string you get from Crypto or Base64.

3

I would start here. Everything else is just muddying the waters.

Using DefineEncoding. you should be able to get the string back intact.

Crypt yields a bunch of bytes - a string with no encoding.
Decrypt expects a bunch of bytes and should ignore encoding. But it also can’t restore the original encoding, you have to provide that with DefineEncoding.

4

First ConvertEncoding to UTF-8, than crypt.
After decrypt use DefineEncoding.

5

Thank you for your answers; unfortunately this doesn’t do the trick (or I do it wrong). Here’s my encrypt routine:

Function mEncrypt(mS As String) As String
Dim mEnc As String = ConvertEncoding (mS, Encodings.UTF8)
Dim mRet As String

Dim mData As New MemoryBlock (mEnc.Len)
Dim mEncryp As MemoryBlock

mData.StringValue(0, mEnc.Len) = mEnc

mEncryp = Crypto.RSAEncrypt(mData, keyPublic)

mRet = mEncryp.StringValue(0, mEncryp.Size)

Return mRet
End Function

and the decrypt method:

Function mDecrypt(mS As String) As String
Dim mRet As String

Dim mData As New MemoryBlock (mS.Len)
Dim mDecryp As MemoryBlock

mData.StringValue (0, mS.Len) = mS

#Pragma BreakOnExceptions False

Try
mDecryp = Crypto.RSADecrypt (mData, keyPrivate)

Catch Err As CryptoException
mRet = “<cryptoexception:” + Err.Message + “>”
Return mRet
End Try

#Pragma BreakOnExceptions False

If mDecryp <> Nil Then
mRet = DefineEncoding (mDecryp.StringValue(0, mDecryp.Size), Encodings.UTF8)
End If

Return mRet
End Function

If I do this:

initKeys

Dim mEnc As String = mEncrypt (“schöön”)
Dim mDec As String = mDecrypt (mEnc)

Say "decrypted: " + mDec

I get “schö#” with # standing for a “broken” character. If I do it only with “schön” I get back an “schö” … which hints me to yet another encoding problem since for each umlaut there’s a character cut off…

Well … I surrounded that nicely with the hexencoding. But still I cannot get the encoded string written to the cubesql DB and then deecrypted. I always get an exception when decrypting that the string length does not fit the key (which does not happen in my other, longer routines when the string has never “touched” anything else but Xojo.

Maybe I’ll try it with the Xojo.Core.MemoryBlock instead (here I don’t have the problem with the encrypting & decrypting umlauts - that works fine there) - but still I cannot get the block of bytes to the SQL database.

Next step I’ll try is to write the MemoryBlocks content to a file on disk and upload that file to the DB. Maybe this works ?!?

Best regards,

Jan

6

Finally! Got it - maybe it’s an awkward detour but: use the Xojo.Core.Memoryblock. Trouble here is that I cannot use that standard “ConvertDataToText” because with special characters it raises a runtime-exception if you set “allowlossy” to false (“The data could not be converted to text with this encoding.”)

Now … what I do now is read the BYTECODES from the MemoryBlock (using Int8Value(x)) - convert each byte into hex, add this up in a string, write it to the database, read it, build a MutableMemoryBlock from that Hex-String, decrypt it -> Works.

Thanks for your input! Tim Hare’s words “Crypt yields a bunch of bytes - a string with no encoding.” helped me to re-think!

Bests,

Jan

7

In your code above, the error is that you’re not using the B versions, particularly here:

dim mData as new MemoryBlock( mEnc.Len)

Should be mEnc.LenB

Memoryblocks are byte oriented.

8

New week, new thinking. You’re right - and … that solves all. Even the UTF8 thing works now as expected.

Thank you very much!

Jan :slight_smile: