# Large memoryblocks under 64 bit **URL:** \ **Category:** macOS\ **Created:** [August 5, 2018, 12:08am UTC](https://forum.xojo.com/t/large-memoryblocks-under-64-bit/44651 "2018-08-05T00:08:59Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Peter\_Stys](https://forum.xojo.com/letter_avatar_proxy/v4/letter/p/e0b2c6/32.png) [@Peter\_Stys](https://forum.xojo.com/u/Peter_Stys)\ **Post date:** [August 5, 2018, 12:08am UTC](https://forum.xojo.com/t/large-memoryblocks-under-64-bit/44651/1 "2018-08-05T00:08:59Z")
I’m in the process of porting my app to 64 bit, ran into some issues and investigated with this simple code snippet: dim n as integer = 5e9 ’ allocate 5GB dim m as new MemoryBlock(n) for j as integer = n-1 downto 0 m.UInt8Value(j) = 128 ’ OutOfBoundsException at the first iteration next Unless I’m missing something obvious, there is something really amiss. - allocating a 5GB block is seemingly successful, but as shown the first iteration throws an outOfBounds - allocating a 4GB block succeeds and the loop runs fine, but curiously the size of the memblock is reported as negative in the debugger - allocating \<= 2GB succeeds, the loop runs fine, and the size of the memblock is correctly reported in the debugger So it seems that: a) m.Size is still an Int32 even under 64 bit, even tho ‘integer’ should be a 64 bit Int64 b) m is limited to 4GB (32 bits) in size (at least the indexing of its elements, the block itself can be allocated bigger it seems) under 64 bit Even tho the docs clearly state “There is no practical limit for 64-bit apps” . What am I missing? If I can’t even get this to work, no hope for porting to 64 bits! (and yes, my compiler is certainly set for 64 bit and I have 32GB RAM tho that should not matter). Thx for any insights, Peter. ---
**Author:** ![Greg\_O\_Lone](https://forum.xojo.com/user_avatar/forum.xojo.com/greg_o_lone/32/49_2.png) [@Greg\_O\_Lone](https://forum.xojo.com/u/Greg_O_Lone)\ **Post date:** [August 5, 2018, 12:19am UTC](https://forum.xojo.com/t/large-memoryblocks-under-64-bit/44651/2 "2018-08-05T00:19:36Z")
The size parameter is of type Integer, which is 64-bits when built as 64-bit. Now, one thing to consider… even though you have 32GB of RAM, you may not have 5GB of contiguous memory available. You may be able to check this by rebooting. Also, right after creating the Memoryblock, try this: ```auto dim tmp as uint64 = m.size ``` I’d be curious if you’re getting a smaller block than you requested. ---
**Author:** ![Mike\_D](https://forum.xojo.com/user_avatar/forum.xojo.com/mike_d/32/266_2.png) [@Mike\_D](https://forum.xojo.com/u/Mike_D)\ **Post date:** [August 5, 2018, 12:47am UTC](https://forum.xojo.com/t/large-memoryblocks-under-64-bit/44651/3 "2018-08-05T00:47:50Z")
I ran the test in the latest Beta [Edit: run as a Debug, not compiled app] and msgBox str(m.size) gives 705032704 which is a weird size: 672.3615 MB Seems like a bad bug to me: allocating a 5GB memory block on a 64 bit machine with 16GB physical ram should “just work”. ---
**Author:** ![Mike\_D](https://forum.xojo.com/user_avatar/forum.xojo.com/mike_d/32/266_2.png) [@Mike\_D](https://forum.xojo.com/u/Mike_D)\ **Post date:** [August 5, 2018, 12:51am UTC](https://forum.xojo.com/t/large-memoryblocks-under-64-bit/44651/4 "2018-08-05T00:51:28Z")
Aha: 5e9 - 4GB is exactly equal to 5e9-4_1024_1024\*1024 = 705032704 which is the size of the memoryBlock I’m given. So it looks like some sort of 32 bit overflow or rollover in the memoryBlock constructor? ---
**Author:** ![Mike\_D](https://forum.xojo.com/user_avatar/forum.xojo.com/mike_d/32/266_2.png) [@Mike\_D](https://forum.xojo.com/u/Mike_D)\ **Post date:** [August 5, 2018, 12:59am UTC](https://forum.xojo.com/t/large-memoryblocks-under-64-bit/44651/5 "2018-08-05T00:59:15Z")
Thanks Peter, I submitted \<[https://xojo.com/issue/52898](https://xojo.com/issue/52898)\> as a bug against the most recent beta. ---
**Author:** ![Peter\_Stys](https://forum.xojo.com/letter_avatar_proxy/v4/letter/p/e0b2c6/32.png) [@Peter\_Stys](https://forum.xojo.com/u/Peter_Stys)\ **Post date:** [August 5, 2018, 1:43am UTC](https://forum.xojo.com/t/large-memoryblocks-under-64-bit/44651/6 "2018-08-05T01:43:09Z")
Thx Greg. I don’t think contiguity of memory matters these days because I was under the impression that the caller receives a logically contiguous block of memory, and the memory manager takes care of the actual physical locations that can be scattered all over various pages in physical RAM and VM. But I could be wrong. Regardless, I did what you suggested, rebooted and the results were the same. To answer your tmp as UInt64 question, if I request 5e9 bytes, tmp shows 705MB but I think this is wrong, I think its simply a 32 bit integer overflow and wrap because as it turns out, 5e9-2^32 is exactly the value that both m.Size and tmp show. If I request 4e9 bytes, tmp = 18446744073414584320 which is also wrong, simply reflecting the cast from m.Size = -294967296 to a UInt64. Now even more curious, if I do: `dim p as Ptr = m for j as integer = n-1 downto 0 p.UInt8(j) = 128 next ` It works. Sort of. The loop runs for a while then crashes: Crashed Thread: 0 Dispatch queue: com.apple.main-thread Exception Type: EXC\_BAD\_ACCESS (SIGBUS) Exception Codes: KERN\_PROTECTION\_FAILURE at 0x0000000114e8bc5f Exception Note: EXC\_CORPSE\_NOTIFY Termination Signal: Bus error: 10 Termination Reason: Namespace SIGNAL, Code 0xa Terminating Process: exc handler [0] VM Regions Near 0x114e8bc5f: CoreUI image data 0000000114c59000-0000000114c6b000 [72K] rw-/rwx SM=PRV –\> mapped file 0000000114c6b000-0000000114ea2000 [2268K] r–/rwx SM=COW Object\_id=7e90559f CoreUI image data 0000000114ea2000-0000000114ed9000 [220K] rw-/rwx SM=PRV So was the memblock allocated or not? It seems yes, because when I use my C plugin fn: ```auto fastFillPtr8(p, 127, n) ``` that does what its name says, it returns with no problem. Now checking the values of p in the debugger: [https://www.dropbox.com/s/c0t6j5egz0eifka/PtrBug.png?dl=0](https://www.dropbox.com/s/c0t6j5egz0eifka/PtrBug.png?dl=0) Many bytes report as 00 when they should all be 7F. So were the values written incorrectly? No, because my other plugin fn: ```auto dim b as Boolean = checkPtr8values(p, 127, n) ``` reports that all 5e9 bytes were in fact 7F. Bottom line: new memoryblock allocates correctly, but accessing the bytes is totally messed up for large blocks. Peter ---
**Author:** ![Greg\_O\_Lone](https://forum.xojo.com/user_avatar/forum.xojo.com/greg_o_lone/32/49_2.png) [@Greg\_O\_Lone](https://forum.xojo.com/u/Greg_O_Lone)\ **Post date:** [August 5, 2018, 2:07am UTC](https://forum.xojo.com/t/large-memoryblocks-under-64-bit/44651/7 "2018-08-05T02:07:47Z")
[quote=399317:@Michael Diehr]Aha: 5e9 - 4GB is exactly equal to 5e9-4_1024_1024\*1024 = 705032704 which is the size of the memoryBlock I’m given. So it looks like some sort of 32 bit overflow or rollover in the memoryBlock constructor?[/quote] Either that or in the Size method. ---
**Author:** ![Greg\_O\_Lone](https://forum.xojo.com/user_avatar/forum.xojo.com/greg_o_lone/32/49_2.png) [@Greg\_O\_Lone](https://forum.xojo.com/u/Greg_O_Lone)\ **Post date:** [August 5, 2018, 2:10am UTC](https://forum.xojo.com/t/large-memoryblocks-under-64-bit/44651/8 "2018-08-05T02:10:12Z")
> [@399318:@Michael Diehr](#): > > Thanks Peter, I submitted \<[https://xojo.com/issue/52898](https://xojo.com/issue/52898)\> as a bug against the most recent beta. I have changed it to a non-beta bug though as this problem is not new to 2018r2. ---
**Author:** ![anon20074439](https://forum.xojo.com/letter_avatar_proxy/v4/letter/a/ac8455/32.png) [@anon20074439](https://forum.xojo.com/u/anon20074439)\ **Post date:** [August 5, 2018, 4:57am UTC](https://forum.xojo.com/t/large-memoryblocks-under-64-bit/44651/9 "2018-08-05T04:57:17Z")
Confirmed it also happens in Windows. Hopefully this is fixed at the same time as \<[https://xojo.com/issue/52211](https://xojo.com/issue/52211)\> ---
**Author:** ![Sam\_Rowlands](https://forum.xojo.com/user_avatar/forum.xojo.com/sam_rowlands/32/265_2.png) [@Sam\_Rowlands](https://forum.xojo.com/u/Sam_Rowlands)\ **Post date:** [August 5, 2018, 5:17am UTC](https://forum.xojo.com/t/large-memoryblocks-under-64-bit/44651/10 "2018-08-05T05:17:09Z")
As a workaround, can you use the new framework? `Dim mb as new xojo.core.memoryBlock( 5e9 ) Dim tSize as Uint64 = mb.size` Returns the correct value here (2017r3, macOS 10.11.6). ---
**Author:** ![Peter\_Stys](https://forum.xojo.com/letter_avatar_proxy/v4/letter/p/e0b2c6/32.png) [@Peter\_Stys](https://forum.xojo.com/u/Peter_Stys)\ **Post date:** [August 5, 2018, 5:28am UTC](https://forum.xojo.com/t/large-memoryblocks-under-64-bit/44651/11 "2018-08-05T05:28:21Z")
Yes Xojo.Core.(Mutable)MemoryBlock seems to work, but I don’t see how I can get a Ptr to this block, both for faster access from within Xojo and to pass to my many plugin functions. Is this possible like with classic Memoryblocks? If not, a non-starter for me unfortunately. p. ---
**Author:** ![Sam\_Rowlands](https://forum.xojo.com/user_avatar/forum.xojo.com/sam_rowlands/32/265_2.png) [@Sam\_Rowlands](https://forum.xojo.com/u/Sam_Rowlands)\ **Post date:** [August 5, 2018, 5:38am UTC](https://forum.xojo.com/t/large-memoryblocks-under-64-bit/44651/12 "2018-08-05T05:38:02Z")
From what I recall. ```auto Dim mb as new xojo.core.memoryBlock( ptr, 5e9 ) ``` Should do it, no? ---
**Author:** ![anon20074439](https://forum.xojo.com/letter_avatar_proxy/v4/letter/a/ac8455/32.png) [@anon20074439](https://forum.xojo.com/u/anon20074439)\ **Post date:** [August 5, 2018, 5:50am UTC](https://forum.xojo.com/t/large-memoryblocks-under-64-bit/44651/13 "2018-08-05T05:50:56Z")
> [@399334:@Peter Stys](#): > > I don’t see how I can get a Ptr to this block `Dim mb As New Xojo.Core.MutableMemoryBlock(5e9) Dim p As Ptr = mb.Data Dim tSize As UInt64 = mb.Size` ---
**Author:** ![Peter\_Stys](https://forum.xojo.com/letter_avatar_proxy/v4/letter/p/e0b2c6/32.png) [@Peter\_Stys](https://forum.xojo.com/u/Peter_Stys)\ **Post date:** [August 5, 2018, 6:35pm UTC](https://forum.xojo.com/t/large-memoryblocks-under-64-bit/44651/14 "2018-08-05T18:35:10Z")
But the LR states that p = mb.Data is a read-only Ptr. This would imply that I can neither write to this Ptr in Xojo, nor pass it to a plugin to modify the contents. And Sam, your suggestion creates a xojo.core.memoryBlock from a pre-existing Ptr. What is needed is a R/W Ptr to a newly created memoryblock, like we can do with classic mb’s. Generally speaking, what is the fundamental difference between classic mb’s and xojo.core.memoryBlock’s? When should we use one vs the other? Seems redundant to me. ---
**Author:** ![Peter\_Stys](https://forum.xojo.com/letter_avatar_proxy/v4/letter/p/e0b2c6/32.png) [@Peter\_Stys](https://forum.xojo.com/u/Peter_Stys)\ **Post date:** [August 5, 2018, 6:55pm UTC](https://forum.xojo.com/t/large-memoryblocks-under-64-bit/44651/15 "2018-08-05T18:55:13Z")
BTW Greg, I also noticed that Ptr addresses are reported incorrectly in the debugger: their values are truncated to 32 bits (8 hex digits) even tho the Ptr is 64 bits (convert the Ptr to an integer and display in hex and you’ll see what I mean). You may want to fix that while you’re at it. ---
**Author:** ![anon20074439](https://forum.xojo.com/letter_avatar_proxy/v4/letter/a/ac8455/32.png) [@anon20074439](https://forum.xojo.com/u/anon20074439)\ **Post date:** [August 6, 2018, 12:00am UTC](https://forum.xojo.com/t/large-memoryblocks-under-64-bit/44651/16 "2018-08-06T00:00:49Z")
> [@399377:@Peter Stys](#): > > But the LR states that p = mb.Data is a read-only Ptr. This would imply that I can neither write to this Ptr in Xojo, nor pass it to a plugin to modify the contents. That just means you cant alter the pointer, which would be a good thing. ---
**Author:** ![Sam\_Rowlands](https://forum.xojo.com/user_avatar/forum.xojo.com/sam_rowlands/32/265_2.png) [@Sam\_Rowlands](https://forum.xojo.com/u/Sam_Rowlands)\ **Post date:** [August 6, 2018, 12:47am UTC](https://forum.xojo.com/t/large-memoryblocks-under-64-bit/44651/17 "2018-08-06T00:47:06Z")
> [@399377:@Peter Stys](#): > > And Sam, your suggestion creates a xojo.core.memoryBlock from a pre-existing Ptr. What is needed is a R/W Ptr to a newly created memoryblock, like we can do with classic mb’s. It was my understanding that this memoryblock operates the same as the ‘classic’ memoryblock, but I have to confess that I’ve never actually used this version. > [@399377:@Peter Stys](#): > > Generally speaking, what is the fundamental difference between classic mb’s and xojo.core.memoryBlock’s? When should we use one vs the other? Seems redundant to me. One was meant to replace t’other one. I’m not at my ‘work’ computer, so I don’t have Xojo to hand, but a quick Google, reveals that you may want to look at the “MutableMemoryBlock” instead. [http://developer.xojo.com/xojo-core-mutablememoryblock](http://developer.xojo.com/xojo-core-mutablememoryblock) it’s constructors are the same, but in theory having mutable in the name, means that it can be edited. This is one thing from the Obj-C world that I dislike, is having two sets of classes, mutable and immutable. I’ve heard the advantages and technically understand them, but I still prefer having one class to rule them all. ---
**Author:** ![Greg\_O\_Lone](https://forum.xojo.com/user_avatar/forum.xojo.com/greg_o_lone/32/49_2.png) [@Greg\_O\_Lone](https://forum.xojo.com/u/Greg_O_Lone)\ **Post date:** [August 6, 2018, 12:48am UTC](https://forum.xojo.com/t/large-memoryblocks-under-64-bit/44651/18 "2018-08-06T00:48:20Z")
> [@399379:@Peter Stys](#): > > BTW Greg, I also noticed that Ptr addresses are reported incorrectly in the debugger: their values are truncated to 32 bits (8 hex digits) even tho the Ptr is 64 bits (convert the Ptr to an integer and display in hex and you’ll see what I mean). You may want to fix that while you’re at it. It would be helpful to have a separate bug report as these fall under vastly different areas. ---
**Author:** ![Peter\_Stys](https://forum.xojo.com/letter_avatar_proxy/v4/letter/p/e0b2c6/32.png) [@Peter\_Stys](https://forum.xojo.com/u/Peter_Stys)\ **Post date:** [August 6, 2018, 4:13am UTC](https://forum.xojo.com/t/large-memoryblocks-under-64-bit/44651/19 "2018-08-06T04:13:50Z")
\<[https://xojo.com/issue/52902](https://xojo.com/issue/52902)\> ---
**Author:** ![Peter\_Stys](https://forum.xojo.com/letter_avatar_proxy/v4/letter/p/e0b2c6/32.png) [@Peter\_Stys](https://forum.xojo.com/u/Peter_Stys)\ **Post date:** [August 7, 2018, 1:03am UTC](https://forum.xojo.com/t/large-memoryblocks-under-64-bit/44651/20 "2018-08-07T01:03:01Z")
More followup. So I played with MutableMemoryBlock: [code]Dim mb as new Xojo.Core.MutableMemoryBlock( 5e9 ) Dim tSize as integer = mb.size dim p as Ptr = mb.Data fastFillPtr8(p, 127, tSize) b = checkPtr8values(p, 127, tSize) for j as integer = tSize-1 DownTo 0 p.UInt8(j) = 127 next [/code] Here the size is reported correctly, my C plugin fills the bytes, and checks the values correctly (even for ridiculously large blocks of 20e9 bytes). However, the XOJO loop crashes after a few seconds (see the crash log at the end). If I reduce the size to 2e9, the xojo loop completes correctly and all bytes verify. So MutableMemoryBlock allocates correctly, returns the correct size, returns a valid Ptr in mb.Data, but there is an addressing problem also for large MutableMemoryBlocks. As an aside, my C plugin fills 2e9 bytes in 0.1 seconds (in fairness, that’s the second time around, the first time it takes about 10x longer, probably the memory manager is mapping requested pages into RAM or something) vs the xojo for-next loop that takes 52 seconds. I did include these pragmas to give xojo its best chance: `#pragma BackgroundTasks Off #pragma BoundsChecking Off #pragma BreakOnExceptions Off #pragma NilObjectChecking Off #pragma StackOverflowChecking Off ` While I expected xojo to be a bit slower than C, but 500-fold? P. System Integrity Protection: enabled Crashed Thread: 0 Dispatch queue: com.apple.main-thread Exception Type: EXC\_BAD\_ACCESS (SIGBUS) Exception Codes: KERN\_PROTECTION\_FAILURE at 0x000000011b125fff Exception Note: EXC\_CORPSE\_NOTIFY Termination Signal: Bus error: 10 Termination Reason: Namespace SIGNAL, Code 0xa Terminating Process: exc handler [0] VM Regions Near 0x11b125fff: CoreUI image data 000000011aedb000-000000011aeef000 [80K] rw-/rwx SM=PRV –\> mapped file 000000011aeef000-000000011b126000 [2268K] r–/rwx SM=COW Object\_id=7e90559f MALLOC\_LARGE 000000011b126000-000000011b234000 [1080K] rw-/rwx SM=PRV Application Specific Information: Performing @selector(performClick:) from sender XOJButton 0x6040001513b0 Thread 0 Crashed:: Dispatch queue: com.apple.main-thread 0 memblock2Integer.debug 0x000000010e788d28 Window1.Window1.PushButton3\_Action%%o\o + 3192 (/Window1:97) 1 memblock2Integer.debug 0x000000010e78ce24 Delegate.IM\_Invoke%%o + 52 2 memblock2Integer.debug 0x000000010e78ce74 AddHandler.Stub.15%% + 52 3 com.apple.AppKit 0x00007fff38eb375a -[NSApplication(NSResponder) sendAction:to:from:] + 312 4 com.apple.AppKit 0x00007fff38959933 -[NSControl sendAction:to:] + 86 5 com.apple.AppKit 0x00007fff3895985b \_\_26-[NSCell \_sendActionFrom:]\_block\_invoke + 136 6 com.apple.AppKit 0x00007fff38959761 -[NSCell \_sendActionFrom:] + 183 7 com.apple.AppKit 0x00007fff3899aa18 -[NSButtonCell \_sendActionFrom:] + 97 8 com.apple.AppKit 0x00007fff38957fca -[NSCell trackMouse:inRect:ofView:untilMouseUp:] + 2438 9 com.apple.AppKit 0x00007fff3899a75f -[NSButtonCell trackMouse:inRect:ofView:untilMouseUp:] + 777 10 com.apple.AppKit 0x00007fff38956a64 -[NSControl mouseDown:] + 965 11 XojoFramework 0x000000010e91b337 0x10e843000 + 885559 12 com.apple.AppKit 0x00007fff39052959 -[NSWindow(NSEventRouting) \_handleMouseDownEvent:isDelayedEvent:] + 5891 13 com.apple.AppKit 0x00007fff3904f5b0 -[NSWindow(NSEventRouting) \_reallySendEvent:isDelayedEvent:] + 2359 14 com.apple.AppKit 0x00007fff3904e85c -[NSWindow(NSEventRouting) sendEvent:] + 497 15 XojoFramework 0x000000010e926b45 0x10e843000 + 932677 16 com.apple.AppKit 0x00007fff38eaf617 -[NSApplication(NSEvent) sendEvent:] + 307 17 XojoFramework 0x000000010e9165cb 0x10e843000 + 865739 18 memblock2Integer.debug 0x000000010e6f5ec5 Application.\_CallFunctionWithExceptionHandling%%op + 181 19 XojoFramework 0x000000010ea96e53 CallFunctionWithExceptionHandling(void (\*)()) + 262 20 XojoFramework 0x000000010e916546 0x10e843000 + 865606 21 com.apple.AppKit 0x00007fff38710d9d -[NSApplication run] + 812 22 XojoFramework 0x000000010ea9517f RuntimeRun + 40 23 memblock2Integer.debug 0x000000010e750a53 REALbasic.\_RuntimeRun + 19 24 memblock2Integer.debug 0x000000010e792bde \_Main + 846 (/#main:87) 25 memblock2Integer.debug 0x000000010e792123 main + 19 26 libdyld.dylib 0x00007fff62b0d115 start + 1 [Next page](https://forum.xojo.com/t/large-memoryblocks-under-64-bit/44651.md?page=2)