File compare via checksum

Getting Started
1

We established from here that checksum is the way to compare two files (https://forum.xojo.com/43383-binary-file-compare).

When I review the documentation on checksum (i.e. MD5) it operates on memory blocks.

Am I correct to assume that doing this operation in memory blocks will limit the size of the files you can compare (to size of available memory)?

If so, what is the way to compare large files?

2

If you have the MBS plugins, see:

https://www.monkeybreadsoftware.net/plugincontent-mbsencryptionplugin.shtml

Great stuff and easy.

3

Use MD5Digest.

http://developer.xojo.com/md5digest

4

MBS HashFile functions can be used on a thread and calculate the hash in a preemptive thread.
This way you can be quicker with multiple threads.

5

[quote=360969:@Christian Schmitz]MBS HashFile functions can be used on a thread and calculate the hash in a preemptive thread.
This way you can be quicker with multiple threads.[/quote]
Can you explain what you mean by “calculate the hash in a preemptive thread”? Preemptive to what?

6

You can run 8 threads to checksum 8 files individually at the same time.
This can cause 8 CPU cores to be busy.
And your Xojo app still is responsive.

7

[quote=360955:@Kem Tekinay]Use MD5Digest.
http://developer.xojo.com/md5digest[/quote]

So, for this simple code:

Dim m As New MD5Digest
Dim s As String
m.Process(fileContents)
s = m.Value

How do I prepare/package the file contents for the fileContents variable above?

8

see example project

http://www.monkeybreadsoftware.net/example-encryption-threadedencryptionandhashes.shtml

9

[quote=360972:@Christian Schmitz]You can run 8 threads to checksum 8 files individually at the same time.
This can cause 8 CPU cores to be busy.
And your Xojo app still is responsive.[/quote]
OK - understand. Thanks.

10

[quote=360973:@Mark Pastor]So, for this simple code:

Dim m As New MD5Digest
Dim s As String
m.Process(fileContents)
s = m.Value

How do I prepare/package the file contents for the fileContents variable above?[/quote]

Create a new instance of MD5Digest, then read your file in chunks and call Process on each chunk. When done, read the Value. I do this in Kaju and it is very quick.

11

I haven’t done a lot of chunking/memoryblock work - would the process look like this…

While not having yet read total file 
    Read x number of bytes of file as BinaryStream into MemoryBlockA
    fileContents (As String) = MemoryBlockA
    m.Process(fileContents)
    increment position in file
    End While

S As String = m.Value
12

BinaryStream.Read will increment the file position for you.

while not bs.EOF
  dim chunk as string = bs.Read( 1000000 )
  m.Process( chunk )
wend

dim hash as string = m.Value

MemoryBlocks convert to Strings (and back) automatically.

13

[quote=361018:@Kem Tekinay]BinaryStream.Read will increment the file position for you.

while not bs.EOF
  dim chunk as string = bs.Read( 1000000 )
  m.Process( chunk )
wend

dim hash as string = m.Value

MemoryBlocks convert to Strings (and back) automatically.[/quote]
Thank you!