Weird WriteUint8 behavior

Hello

I want to parse a BinaryStream file and change bytes that way:

var Track As UInt8

D81Image.BytePosition = 400128

Do
  MainWindow.Console.Value = MainWindow.Console.Value  + "Read before: " + D81Image.BytePosition.ToString + EndOfLine
  Track = D81Image.ReadUInt8
  MainWindow.Console.Value = MainWindow.Console.Value  + "Read after: " + D81Image.BytePosition.ToString + EndOfLine

  If Track = 0 Then
    MainWindow.Console.Value = MainWindow.Console.Value  + "Write before: " + D81Image.BytePosition.ToString + EndOfLine
    D81Image.WriteUInt8( &hFF )
    MainWindow.Console.Value = MainWindow.Console.Value  + "Write after: " + D81Image.BytePosition.ToString + EndOfLine

    Exit

  Else
    // Jump over to next sector
    D81Image.BytePosition = D81Image.BytePosition + 255
  End If

Loop Until D81Image.BytePosition >= 409600

The output to the console is this:

Read before: 400128
Read after: 400129
Read before: 400384
Read after: 400385
Write before: 400385
Write after: 404481

In second loop iteration the code Track is 0 and heads to “D81Image.WriteUInt8( &hFF )”. The weird thing: Before writing the BytePosition is at 400385 (correct) but after writing a single byte the BytePosition is set to BytePosition 404481 instead of expected 400386. Either there is a bug or I am missing something trivial.

Any ideas about this?

I see no bug here, you’ve done a good job of adding effective logging.

Does it behave this way when you step through the debugger?

Glad to hear it’s not my bug…

Yes, that’s the output running in debug mode, the output is sent to a textarea item. I compiled it as well but same result.

I never worked with the debugger, have to find out how to step through it.

Xojo for Mac, V2025, R3.1

I tried this, but doesn’t show the same problem:

var f as FolderItem = GetFolderItem("test.jpg")
Var b As BinaryStream = BinaryStream.Open(f, true)

b.Position = 123456
Var p1 As Integer = b.Position
b.Write "Hello World"
Var p2 As Integer = b.Position
System.DebugLog str(p2-p1)+" bytes"

Var p3 As Integer = b.Position
b.WriteUInt8 123
Var p4 As Integer = b.Position
System.DebugLog Str(p4-p3)+" bytes"

Var p5 As Integer = b.Position
b.WriteUInt16 123
Var p6 As Integer = b.Position
System.DebugLog Str(p6-p5)+" bytes"

Var p7 As Integer = b.Position
b.Writeint8 123
Var p8 As Integer = b.Position
System.DebugLog Str(p8-p7)+" bytes"

Break

b.Close

Works fine:

     : 11 bytes
     : 1 bytes
     : 2 bytes
     : 1 bytes

Thank you for taking time and testing, appreciated! I see you do it another way as I did so far. I will test your way as well.

I have reduced my sample to simply do first a BinaryStream.ReadUInt8 and then right after a BinaryStream.WriteUInt8. That seems to cause the problem (at least on my side). Your sample does only writes only which is working on my side too. Can you try once to first do a ReadUInt8 and then right after a WriteUInt8?

If that works on your side I guess it’s a project problem on my side. In between I have updated to latest OS version and also to latest Xojo version (2026/1).

By any chance – is the file you are writing to shorter than you expect? Perhaps its size is right around 400,385 bytes?

if you can show it in a small sample project, please report the issue here.

I have created a small demo app to reproduce the case. So, it looks like a bug to me.

// Open stream file
Var File As New FolderItem(SpecialFolder.Documents)
File = File.Child(“testfile”).Child(“emptyfile.d81”)

var Stream As BinaryStream
Stream = BinaryStream.Open(File, True)

// Set Stream to specific position
Var TestByte As UInt8
Stream.BytePosition = 399360
TestWindow.Output.Text  = TestWindow.Output.Text  + "BytePosition: "  + Stream.BytePosition.ToString + EndOfLine

TestByte = Stream.ReadUInt8
TestWindow.Output.Text  = TestWindow.Output.Text  + "ReadUInt8 value: "  + TestByte.ToString + EndOfLine
// → Byte read at 399360 has value 40 which is correct

// → Expected: After reading a single byte the Stream.BytePosition should be 1 higher, so should be now at 399361
Stream.WriteUInt8(255)
TestWindow.Output.Text  = TestWindow.Output.Text  + "BytePosition: "  + Stream.BytePosition.ToString + EndOfLine
// → Bug here: BytePosition should be at 399360 but it’s at 403456 and the written byte (255) is also written there
//                         (side note: the written byte is always and exactly 4096 after where it should be written)

// Close binary stream
Stream.Close

The key problem: after ReadUInt8 you do a WriteUInt8 and that will cause the issue.

The binarystream file has the size of 819’200 bytes. After writing into it (causing the bug) the file has still the same size. While you mention this… It’s dividable by 4096 (=200) and the position of doing the writeUInt8 is at position of 4096 bytes higher than it should be.

Yeah, the 4096 value caught my eye. If you have a BinaryStream that is backed up by either a string or a MemoryBlock and you write past the end of it, the BinaryStream will automatically extend the string or MemoryBlock to accommodate more data – I thought that might have been the case here.

This bug happens on my machine as well and is very serious. I have created a bug report:

https://tracker.xojo.com/xojoinc/xojo/-/issues/81210

1 Like

Your case might be related to mine. So it’s good you have reported is as well. In my case I change a byte in between, not attaching at the end.

I tagged you in my report and gave you credit because I didn’t know you created a report already. :grin:

1 Like

I wonder if the sector size is contributing to this.. how low level is Xojo’s writing?

A 4096-byte (4K) sector size, known as Advanced Format, is the modern standard for hard drives (HDDs) and solid-state drives (SSDs), replacing the traditional 512-byte sector.

Also, the point about memoryblock backed file access is a good one.

Does the same thing happen if you back with a memoryblock and do the work there before writing back to disc in one go?

That’s a very interesting theory. I was wondering what the significance of 4096 was in this context and wasn’t coming up with any good explanations. However, I doubt Xojo gets down that far into the technical details of how drives are managed.

It’s totally fine you reported as well. That way Xojo devs can see different problems probably caused by the same issue.

Valid point with the memory block size! :+1:

Regarding memory block: I don’t think it will affect the memory block. There is no stack pointer like BytePosition on Bitstreams. Furthermore, as far as I’ve seen it, memory block manipulators always needs a direct position value (index) or an offset.

Take a look at the BinaryStream documentation. You can do this:

Var stream as BinaryStream

Var backingMemoryBlock as MemoryBlock

backingMemoryblock = new MemoryBlock(1000)

stream = new BinaryStream(backingMemoryBlock)

The BinaryStream now reads and writes to the MemoryBlock just as though it were a file. It will even auto-expand the MemoryBlock if you try to write past the end of it via BinaryStream.Write*.

That is unnecessary. BinaryStream supports reading and writing using the same object.

All of this is fine but unrelated to the issue at hand, which is the BytePosition of the BinaryStream and not the size of the file.

2 Likes

Sorry, it seems that I goofed !

And for READ/WRITE: what a strange idea I had. For some reason (unknow), I took the Boolan as Read OR Write.

I am quite sure that I never use the same reference to read AND write. (code clarity ?). I do not know.

Thank you for extensive explanation. I see it the same way as @Eric_Williams. I think you just mixed:

  • How is the file system structured OS-wise
  • How do/should ReadUInx/WriteUIntx behave

The manual doesn’t go that deep and just mention this regarding Read/Write in BinaryStream:

“This property is automatically incremented by all of the Read and Write methods.“

Independent of the OS file structure in the background as developer you’d assume BytePosition is automatically incremented by the byte value size you read or write.

I do make heavy use of BytePosition in my project before read or write operations and never had an issue (all within the same file). Now, new: I read the first byte and then wanted to write to the next byte (without using BytePosition). And there all came up posted here. My workaround now will be to set the BytePosition as well before doing the write.

1 Like