I have a small amount of variable length data I want to read and write from a memoryblock
I’m assuming that a Int8 is 1 byte, and a double is 8 bytes.
My code is this:
dim mb as new MemoryBlock(9*(me.inks.LastIndex+1))
dim bs as new BinaryStream(mb)
//Int8 inkid double dot
for i as integer = 0 to me.inks.Ubound
bs.WriteInt8(me.inks(i))
bs.WriteDouble(me.dots(i))
next
Before the loop the memoryblock is shown as 9 bytes as expected.
A breakpoint at the end of this shows that the byteposition of the BinaryStream is 9. This is expected as I’ve written a Int8 and a Double.
BUT the memory block is now 41 bytes long and padded with zeros.
If I reverse the code (to read the data back in later)
dim mb as new MemoryBlock(decodeddata.Bytes)
mb.StringValue(0,decodeddata.bytes)=decodeddata
dim b as new BinaryStream(mb)
//data is
//Int8 inkid double dot
redim me.inks(-1)
redim me.dots(-1)
while not b.EOF
me.inks.Append(b.ReadInt8)
me.dots.Append(b.ReadDouble)
wend
As the memory block is now 41 bytes, the loop count is incorrect.
I can’t work out why the MemoryBlock is 41 bytes. I can add another writeint8 with the number of inks/dots but it should not be necessary.
Regards,
Lee
Create a sample app (just the small simulation of the problem), zip it, and post here. People will find out what’s wrong from it.
I made a simple console application which contains only the following code:
dim mb as new memoryblock(9)
dim bs as new BinaryStream(mb)
bs.WriteInt8(-5)
bs.WriteDouble(100.0)
break
Inspecting the properties shows that bs.position is 9, but mb.size is now 41
Regards,
Lee
bs is 9. mb is 41. Maybe mb attached to a binarystream increases in blocks reserving spaces to more writings, or its some bug. You should attain to the bs length and ignore the mb length right now, but a Xojo explanation is needed.
I’ve updated the code to add a read test.
dim mb as new memoryblock(9)
dim bs as new BinaryStream(mb)
bs.WriteInt8(-5)
bs.WriteDouble(100.0)
dim mbs as string = mb.StringValue(0,mb.size)
dim mbo as new memoryblock(mb.size)
mbo.StringValue(0,mb.size)=mbs
dim bso as new BinaryStream(mbo)
while not bso.EndOfFile
print str(bso.ReadInt8)
print str(bso.ReadDouble)
wend
break
This is the output
iMac:~ lee$ /var/folders/pb/mb5b5frn52j590j38v717_1c0000gn/T/memoryblock.debug/memoryblock.debug
-5
100
0
0
0
0
0
0
0
0
That’s the properties of the binary stream I think, what’s the memoryblock size though?
Maybe some kind of “buffer increasing in advance” property or bug. It demands more explanation from Xojo. The Memory block expansion from 9 to 41 only occurs when you fulfill the 9 bytes (i.e. after bs.WriteDouble )
It’s certainly strange.
If I truncate the memoryblock to 9 using
dim mbs as string = mb.StringValue(0,9)
and then try to read it back in, the ReadDouble gets 0.0, not 100.0
For the meantime I’m using a WriteInt8 count first, but it means that there is much more data being saved in the memoryblock than required. I may be reading/writing millions of entries of the data, so it would make a difference to the overall performance to keep it as a minimum.
Regards,
Lee
What if you do it without a binary stream?
mb.int8Value( 0 ) = -5
mb.doubleValue( 1 ) = 100.0
1 Like
The original spec required a loop, so a binarystream tracks the position without me having to keep a pointer. One object may have only 1 entry for each Int8 and Double, but another may have up to 8 of each.
dim mb2 as new memoryblock(9)
mb2.Int8Value(0)=-5
mb2.DoubleValue(1)=100
Gives a size of 41, with lots of padded zeros.
Wow. That seems a bug. Report that.
1 Like
As Rick mentioned above, it would seem that the memoryblock increases in size in increments of 32 bytes when it gets near full which is probably some performance side effect of the binary stream e.g.
Dim mb As New MemoryBlock(0)
Dim bs As New BinaryStream(mb)
bs.WriteDouble(100.0)
Break
Allocated 32 bytes where as using
bs.WriteDouble(100.0)
bs.WriteDouble(100.0)
bs.WriteDouble(100.0)
still remains at 32 bytes
but
bs.WriteDouble(100.0)
bs.WriteDouble(100.0)
bs.WriteDouble(100.0)
bs.WriteDouble(100.0)
jumps to 64 bytes, double what it needs.
Its a bit odd that it doesn’t tell you this in the docs so it might have been an unintended side effect of a change that resulted in a bug. In your example, you allocate 9, then 32 more is added when you get to the end taking it to 41.
But as Lee reports, that expansion happens even without a BinaryStream bound to it. A fixed block handling should keep it fixed, and an out of bounds access should fire an exception.
Ah then its a generic memory block issue for performance I guess, but I can’t really see that allocating just 32 bytes would improve speed that much, ^2 would usually be done for that so yeah its probably just a bug.
Using a while loop to get to the end of the memory block now loops an incorrect number of times. 1 write, gives 5 reads as shown in an above post of mine.
For I/O it kind of makes sense, you increase blocks instead of bytes because you expect more bytes coming and you can avoid continuous allocation, you just use pre-allocated memory. But I would expect that even for this case, the framework should hide such details. Length should be the expected and another property like RealLength could have the real memory taken size. Without I/O, an allocated memory should be constant unless resized by code. There’s a terrible bug here. Please report.
The BinaryStream will increase the size of the underlying MemoryBlock automatically when you reach the end of the current size. This is by design for performance reasons. The extra bytes in the MemoryBlock are truncated when you Close() the BinaryStream.
i.e. MemoryBlock.Size reflects the number of bytes currently allocated and BinaryStream.Length reflects the actual number of bytes that have been written so far. The Size will always be >= the Length until the stream is closed.
