memoryblock vs binarystream vs string.

General
1

I have a binary data file loaded into a string.
I have a the following methods for handling this data from a file but not a string.

Sub fromFile(fi as FolderItem, debugOn as Boolean=false) dim stream as BinaryStream stream = BinaryStream.Open(fi, False) if stream = nil then dim err as new IOException err.Message = "Unable to open file." Raise err end if fromBinaryStream(stream, debugOn) End Sub

Sub fromBinaryStream(stream as BinaryStream, debugOn as Boolean=false) dim dada as string if stream = nil then dim err as new IOException err.Message = "Unable to open file." Raise err end if dada = stream.read(128) ... ... End Sub

So given that the contents of the file are in a string and not in a file or binarystream, what should I do?

fromBinaryStream(new BinaryStream(myString), true)

This seems right, but I’m afraid that there is an actual memory copy involved here and that that will slow the process down.
(I’m thinking in C/C++ here, because I doubt a string can be cast into an object and that there is no reference to a string in Xojo)

2

IMHO, instead of passing BinaryStream, pass MemoryBlock to the function. Function signature then will be

Sub ProcessData(data as MemoryBlock, debugOn as Boolean=false)

Now, you can use the method to process data from a file

dim bs as BinaryStream = BinaryStream.open(fi)
dim mb as MemoryBlock = bs.read(bs.length)
ProcessData(mb)

Or from a string (in memory data)

dim s as String = ""
ProcessData(s)
3

My datasets can be quite large .
I’m very concerned about sloshing data from the string to the memory block and vise versa.
Why can’t these three be ‘casted’ without copying the physical bytes.

4

If you change

Sub fromBinaryStream(stream as BinaryStream, debugOn as Boolean=false)

to

Sub fromBinaryStream(ByRef stream as BinaryStream, debugOn as Boolean=false)

You wont be making a copy, but passing a reference to the BinaryStream you created in fromFile.

EDIT: Hmm, memoryblock isn’t intrinsic so it will be handled ByRef internally anyway, so how you have it shouldn’t be making a copy of the daya anyway.

EDIT2: Just double checked and there is no 2nd copy as expected so you should be fine doing it the way you are.

5

[quote=373805:@]If you change

Sub fromBinaryStream(stream as BinaryStream, debugOn as Boolean=false)

to

Sub fromBinaryStream(ByRef stream as BinaryStream, debugOn as Boolean=false)

You wont be making a copy, but passing a reference to the BinaryStream you created in fromFile.

EDIT: Hmm, memoryblock isn’t intrinsic so it will be handled ByRef internally anyway, so how you have it shouldn’t be making a copy of the daya anyway.

EDIT2: Just double checked and there is no 2nd copy as expected so you should be fine doing it the way you are.[/quote]

Each object variable is passed byref.

I created simple test. It’s much more faster working with BinaryStream than MemoryBlock.

Memoryblock test
================
BinaryStream.Open : 0.042 ms
mb = bs.Read(bs.Length) : 346.53 ms
new BinaryStream(memoryblock) : 0.004 ms

String test
================
String to MemoryBlock : 262.87 ms
new BinaryStream(string) : 0.016 ms
6

How large is large? For MBs load everything at once. For GBs you had better read in pieces.

It doesn’t matter if you start with memoryblock or string you just need to make sure that you don’t copy from one format to the other.