BinaryStream backed by MemoryBlock with size 0

I found the following example in the docs for RealStudio 2011:

This example backs a BinarySream with a MemoryBlock that is declared 0-sized:

Dim mb as New MemoryBlock(0)
Dim bs as New BinaryStream(mb)
bs.WriteLong(4)
bs.WriteDouble(3.14)
bs.Close
MsgBox Str(mb.Long (0))

(the code is supposed to be quoted too. Is there a way to format this post accordingly?)

In the current docs there is still the same example.

I would expect the first Write call on the stream to fail with an error. This seems not to be the case.

Alternatively I would expect the special behavior, which is suggested by the introduction to the example, to be documented somewhere. I looked it up in BinaryStream.Constructor. in the notes and example section for the BinaryStream class, in MemoryBlock.Constructor, and in the notes and example sections for the MemoryBlock class.

If there is some magic behind “a MemoryBlock that is declared 0-sized”, why is it not documented? Or where is it documented? Where should I have looked, or what should I have searched for in order to find the relevant documentation after having looked in all obvious places was unsuccessful?

From the docs bottom of the MemoryBlock page:

MemoryBlocks supports creating a zero-length MemoryBlock

It has always been this way. The binarystream writes the content when .close is called if I remember correctly.

I don’t remember if it was documented somewhere, by my observation has been the MemoryBlock will be enlarged to at least 32 bytes on the first write, and thereafter doubled in size when a subsequent write would exceed the existing length. Calling the Close() method truncates any unused bytes.

1 Like

I have to say that I don’t even know what this, a BinaryStream opened (?) to a MemoryBlock, means. Can someone explain briefly.

That’s very reminiscent of how lists and StringBuilders are auto-expanded in .NET. There’s some default starting size and then when capacity is exceeded it doubles the backing store. In the case of List, which behaves similar to a Xojo array, a backing array is created with the new size and the original contents copied to it; this overhead can be avoided by setting the initial capacity in the constructor, if you have an idea how large it needs to be.

I wonder if there isn’t a similar issue if the memory block can’t be expanded in place – it would need to be copied someplace else to remain contiguous. As such … it might be better to size it correctly to begin with, or if the required size is fuzzy, create it with an estimated / average initial size to avoid the earlier reallocations.

One wonders if this auto-expanding behavior of memory blocks is how Xojo manages array storage under the hood.

Of course all of this should be documented to begin with so we don’t have to guess, lol. Of course nothing says that the exact initial size and the resizing logic have to be part of the language / runtime spec and can’t remain an “implementation detail” subject to change, but certainly the auto-expanding capability and its possible performance implications and the reasons why you might want to specify an initial size rather than let it auto-grow, should still be in the docs, IMO.

The BinaryStream will write to the MemoryBlock, enlarging it as needed, when you call BinaryStream.Write. Resizing an existing MemoryBlock is usually much more efficient than concatenating two strings together.

e.g., suppose you want to build a large string by combining lots of smaller strings. A naive implementation uses the + operator to combine the strings:

' runtime 1262ms
Dim s As String
For i As Integer = 0 To 99999
   s = s + Str(i)
Next

And it works fine. But using a MemoryBlock+BinaryStream will execute an order of magnitude faster:

' runtime 64ms
Dim s As New MemoryBlock(0)
Dim bs As New BinaryStream(s)
For i As Integer = 0 To 99999
   bs.Write(Str(i))
Next
bs.Close()
3 Likes