Base64 Encoding/Decoding issues

Hi,

I have an iOS app that needs to exchange binary data with a desktop app on a timely basis.
As it’s binary I’m encoding the data with Base64.
Data is sent/received using TCP socket (classic on desktop size, Xojo.Net.TCPSocket on iOS.

Sending data from iOS to Desktop is working real fine.
Sending the data back from Desktop to iOS does NOT work because of Base64 issues.

Here’s the process:

[code]Sender side:

  • read SQLite database file data using BinaryStream
  • encode data to Base64
  • write data to TCP socket

Receiver side:

  • read data from TCP socket
  • decode Base64 data
  • use BinaryStream to write decoded data to folderitem --> SQLite database file
    [/code]

I’m testing with the same data on both sides for tests.

It’s not straight forward as TCP uses string on desktop and memoryblocks on iOS.

On Desktop side:

  • TCP.Write takes a String, I assume it’s UTF8
  • TCP.ReadAll/Lookahead are requiring encodings, I’m using Encodings.UTF8

On iOS side:

  • TCP.WriteData takes a memoryblock, I’m using TextEncoding.UTF8.ConvertTextToData(dataToSend)
  • TCP.ReadData takes a memoryblock, I’m using TextEncoding.UTF8.ConvertDataToText(dataReceived)

I’ve tried with both iOSkit and M_Text encoding/decoding methods and have the same problem.
I found multiple posts on this forum (most dated years ago) but don’t find a solution.

Using Xojo 2019r1, macOS (Mojave).

Any ideas ?

Thanks !

This is what I do using Kem Tekinay’s M_Text

TextArea1.Text=xojo.Core.TextEncoding.utf8.ConvertDataToText(M_Text.DecodeBase64(xojo.Core.TextEncoding.utf8.ConvertDataToText(Content)))

I see now you are working with binary data. I am working with Text. So this may not work for you.

[quote=434748:@Steve Koger]This is what I do using Kem Tekinay’s M_Text

TextArea1.Text=xojo.Core.TextEncoding.utf8.ConvertDataToText(M_Text.DecodeBase64(xojo.Core.TextEncoding.utf8.ConvertDataToText(Content)))[/quote]

Thanks Steve, but this is exactly what I do, in my example dataReceived is already created , detail is

Dim tcpData As Text =  Xojo.Core.TextEncoding.UTF8.ConvertDataToText(content)
dataReceived = Xojo.Core.TextEncoding.UTF8.ConvertDataToText(M_Text.DecodeBase64(tcpData))

This returns an empty text on iOS side.
If I use allowLossy = True, as

Dim tcpData As Text =  Xojo.Core.TextEncoding.UTF8.ConvertDataToText(content, True)

it works but the data is corrupted, full of “?” (Unicode replacement character (U+FFFD)

What I’m sending back and forth are very small SQLite databases (< 100Kb), it may be due to the DB content but I have no control on these as they are created on other systems. The tables in the DBs contain only integer, real and text columns. No blobs.

I would contact @Kem Tekinay see if he has any ideas?

Let’s see if @Kem Tekinay has an idea.

I have a workaround, it’s another technic I’m using for syncing more complex dbs, using dictionaries and json to pass data between devices. But I found it overkill in this case as I just need a kind of backup and restore.

Thanks anyway.

Maybe I missed something, but once you convert a content (from TCP) to text (ok is EncodedBase64); then decode this text and you should write it to the file as binary, you have not to see as text.

Hi Antonio, thanks for catching up. I just left the office and will check again tomorrow. …
But I’m using the same modules on both desktop and iOS side and iOS to desktop is working fine.
The process is in fact a bit more complex than explained above as the tcp data is part of a structure containing the binary data and other informations I need to process as text.
More tomorrow…

I was having same issue long time ago and leave it for some other time.
Only working way is to by pass base64 and use Hexdecimal encoding (which is bigger in size).

Thanks Bogdan.
I gave up, spent too many time searching and I’m afraid the solution won’t come from Xojo as there are a lot of posts already, for years, about base64 and still no Xojo iOS native solution.

Hex is not an option as the size is much bigger and because the desktop app is dealing with other systems.

I have no issues with base64 in the classic framework, the same modules are used in desktop and web apps on multiple platforms.
I had to use the new framework because of iOS.

As it seems to work with non-binary data, I’m using another technique, i.e. converting the records to dictionary -> json then encode in base64. The operation reversed on the desktop side.

I’m a bit confused. Does the data being encoded/decoded represent human-readable text? If not, forcing it into the Text type will not work, not should it be attempted.

I need to send/receive binary data to/from desktop/ios devices. The existing desktop app uses BinaryStreams and TCPSockets, classic framework, which are using String types for reading/writing data.
I agree binary non-human-readable data shouldn’t be forced to text, but I have to deal with strings on desktop side.

on desktop strings are “text” but in this case just a memory block.
On iOS (text or data)=>encodeBase64=>text=>TCP(text as data)=>RVCTCP(data)=>text=>decodeBase64=>data (IE memoryBlock) now you have to choose if it was text or data to make the final conversion.

On Desktop (string or data, but data is string)=>encodeBase64=>string (real string)=>TCP(string but is data)=>RCVTCP(string but is data)=>decodeBase64=>string (IE memoryBlock) but you see directly if is a real string or data.

The problem is that string is more or less the same as memory block. On iOS you have Text and this is not a memoryBlock.

In any case is more difficult to explain than to code. I use it and it works on all environments.

Strings are fine on the desktop side. As Antonio said, they work like convenient MemoryBlocks. But Text only holds legitimate human-readable text, so on the iOS side, you’d have to use a MemoryBlock to hold and access that data.

@Olivier Colard I run in same situation and that is why I told you to stop running for it.
I even wrote about it and submit tickets for it and till now I didn’t get anything about it.

@Antonio Rinaldi
Can you share with us some working concept and testing code where you can exchange any binary data in both directions using base64:

  1. iOS App -> Mac OS App
  2. Mac OS App -> iOS App
    ?

Let say, 100KB, 250KB, 500KB and 1.5MB in size of data

Thanks!