You need a more reliable way to determine if an arbitrary string is image data or not. Norman just confirmed that PictureValue is not a good way to do so. You could petition xojo to implement a workaround for the BM example, but there would be no guarantee that some other prefix wouldn’t cause a problem. You’d just be patching issues as they arise.
Nonetheless, it is hard to figure how checking to see if a record’s picture value is not nil would throw an OutOfMemory exception error only on a text string that begins with the letters BM (and only in Windows) and work every other time.
Of course, I will have to find an alternative method to accomplish what I need (perhaps Norman’s suggestion) as it’s already an established feature of my app that I can’t kill off without running afoul of my current customers. You guys need to keep in mind that we end users don’t know as much about the nuts and bolts of the IDE as you guys do and the Language Reference is frequently a bit too parsimonious on issues like this.
[quote=23275:@Carl Hogue]Nonetheless, it is hard to figure how checking to see if a record’s picture value is not nil would throw an OutOfMemory exception error only on a text string that begins with the letters BM (and only in Windows) and work every other time.
[/quote]
It would probably be easier if you had the source code to that. But you don’t. They use native libraries for things like this, so it wouldn’t surprise me at all if the Windows one has some different error reporting scheme than the Mac one, and is particularly sensitive to “BM” which I’d suspect from how you described the problems means a native Bitmap format.
At any rate, you could “throw” the call to .PictureValue into a try/catch block and deal with any exception locally, then proceed full speed. This isn’t a crashing problem, is it?
I’m not one of “you guys” 
and I do understand your need to preserve the feature. It’s just that the record doesn’t actually contain a picture. It contains a bunch of bytes and has a method that attempts to interpret that data as a picture. You’re right about the documentation being silent on this point.
It might be enough for you to just check for “BM” first and skip the PictureValue check. At least until something else causes an issue. But if you’ve been successful up to this point, it might be good enough.
Alternately, you can check for the signatures of the most common image files. That would cover the majority of cases without your users noticing that you’re being more selective.
And Brad just suggested try/catch, which might handle it, assuming the memory gets released.
Yes, the LR is definitely very terse in spots. When you discover something like this issue and figure out a workable solution, hopefully that can be boiled down to a couple of sentences, you should start a thread and/or a feature request to ask that a note be added to the LR. At various times, there have been people outside of Xojo/RS who could update the LR.
But understand that when people write APIs and document them, it is next to impossible to imagine all the ways developers might try to use them. Your use is “novel”, to say the least. Not saying it’s wrong, but I’d never have imagined it. So please be patient with finding and documenting a solution.
I agree with Brad in that I wouldn’t have thought of using the method this way, and it seemed wrong somehow, but given the silence of the documentation one way or the other, I can see how one might just as easily assume that it should work.
[quote=23275:@Carl Hogue]Nonetheless, it is hard to figure how checking to see if a record’s picture value is not nil would throw an OutOfMemory exception error only on a text string that begins with the letters BM (and only in Windows) and work every other time.
[/quote]
Thats checking to see if the Picture is nil - not the column
It has to convert the column data to a picture to check if the picture is nil
There’s a couple ways you can tell what type the data is (in RB terms not DB terms)
You can check the Recordset ColumnType to see what kind of column it is on each row so you can tell
And you will get a type 0 (NULL) if the column in that row is null
You might also try the Recordset.Field (or idxField).NativeValue as that wont try to convert it to anything
It turns out the solution to this problem was rather simple. I opened every picture file I had on my computer in a hex editor and confirmed that the first character in any picture file is never an alphanumeric character. Now I pattern match the first character of every record for an alphanumeric character and set a boolean switch based on the result. If there’s a match, I bypass checking for a nil PictureValue and read the record in as a string. If there’s no match I create a thumbnail from the PictureValue. If the thumbnail is nil I read in the record’s NativeValue.
I was just heading for bed and saw this one. Did you open a Windows Bitmap file and check the first two bytes? As Andrew L mentioned BM are the first two bytes, followed by a DWORD that defines the size of the BMP file and it is possible that the DWORD (4 bytes) could appear to be a very large number causing the out of memory exception in Windows if Xojo tried to allocate memory for it. You can get some BMP samples from here . Even if you never use BMP files yourself could it be that Xojo sees the ‘BM’ in the header when you are testing for a Picture and thinks it is a valid BMP file?
As I mentioned in previous posts, it was actually a false OutOfMemory exception error that occurred when RecordSet.PictureValue was tested against a text string beginning with the letters BM in Windows versions prior to Windows 8. An actual bitmap file would not cause the error.
The attempt to convert it would mean that framework read a value for the size that was either enormous hence the out of memory or had a dimension < 0 which also causes out of memory exceptions since sizes are never negative
It kind of sucks that BMP’s do use such a short signature - we’d have to validate the entire header to be really sure it’s a bmp.
A lot of formats use much longer signature blocks as the first handful of bytes.
[quote=23571:@Norman Palardy]The attempt to convert it would mean that framework read a value for the size that was either enormous hence the out of memory or had a dimension < 0 which also causes out of memory exceptions since sizes are never negative
It kind of sucks that BMP’s do use such a short signature - we’d have to validate the entire header to be really sure it’s a bmp.
A lot of formats use much longer signature blocks as the first handful of bytes.[/quote]
System monitoring during testing showed that memory usage never even reached 50% whenever the OutOfMemory exception error occurred, so I don’t know what was actually causing that error to be thrown.
In any case, as I mentioned earlier, matching the first character of a record for an alphanumeric character first eliminates the need to use RecordSet.PictureValue to test for a picture and it also improves speed performance.
An RB/Xojo app can use a max of just over 3GB of memory, so if you have more than 4GB, your app will throw an exception well before you run out of physical memory.
I wasn’t aware of that. The computers I use for testing have either 4GB or 8GB.
[quote=23593:@Carl Hogue]System monitoring during testing showed that memory usage never even reached 50% whenever the OutOfMemory exception error occurred, so I don’t know what was actually causing that error to be thrown.
[/quote]
I just told you why it would get thrown
Picture CANNOT have 0 x 0 dimensions and there are limits
We check those before we try to actually create the picture (since we know it WILL cause an error)
[quote=23593:@Carl Hogue]
In any case, as I mentioned earlier, matching the first character of a record for an alphanumeric character first eliminates the need to use RecordSet.PictureValue to test for a picture and it also improves speed performance.[/quote]
But a lot of pictures DO put alphanumeric characters in as their first few characters as a signature
You can thank Windows & its memory management for that
Depending on the OS, it’s actually worse than that. There are limitations on the “Largest Single BlockSize” you can allocate, as well as the “Total Allocation Size” you can make.
See <https://xojo.com/issue/12300>
Here's a re-test using 1MB block size on different OSs
os 32/64 RAM max avail
-----------------------------------
10.6 32 6000 3279
Win7 32 1024 1844
Win7 64 4000 1816
32/64 : version of the OS running
RAM = physical ram in pc
Avail = amount of 1MB blocks that could be allocated.
Code:
dim m() as MemoryBlock
for i as integer = 0 to 4095
try
m.append new MemoryBlock(1024*1024*1) // 1 MB
catch
MsgBox "allocated " + str(i) + " megabytes"
end
next
Conclusion: the 1800 limit in win32 seems to not depend on the OS version or on how much RAM exists. The 3200 limit in OS X also may be a hard limitation of REALbasic?[quote=23660:@Michael Diehr]Win7 64 4000 1816[/quote], this one is very strange, i get on my Win8-64 these numbers with your test
Win8 64 8000 3725
It seems to me that the memory of the Win7-64 pc is fragmented by the use of other programs or data in the memory.
Those test results were run in 2010, so I was probably using RB2010R1 or something - xojo 2013 may be better? Or maybe Win8 is better? Maybe I should re-run them…
Ok, re-ran some tests using xojo 2013 R2:
os 32/64 RAM max avail
-----------------------------------
10.8 64 16GB 3300
Win7 32 1GB 1844
Win7 64 1GB 3735
Win8 32 6GB 1823So, it seems that 64-bit windows does a much better job of giving you nearly 4GB of memory, even in a case where the PC only has 1GB of RAM installed. Interesting.
[quote=23655:@Norman Palardy]I just told you why it would get thrown
Picture CANNOT have 0 x 0 dimensions and there are limits
We check those before we try to actually create the picture (since we know it WILL cause an error)[/quote]
Not knowing the details of how the IDE implements RecordSet.PictureValue, I expected that a NilObjectException would be thrown in such a case rather than an OutOfMemory exception error, thus I was checking for a nil PictureValue as a way to find out if a RecordSet contained a picture before attempting to process it.
Checking for an alphanumeric character is just one idea I had. The default picture value is PNG within my app, and I have yet to encounter a PNG file that leads with an alphanumeric character. I will have to do a lot of testing for stability and reliability before I implement it in a release. The problem I’m faced with is how to effectively determine whether or not a record contains a picture first before I do anything with it.
I use Windows 8, Windows 7 and Windows Vista on computers with 4GB of memory for most of my Windows testing. On the Windows 7 computer with 4GB of memory and the system monitor showing memory at 47% of capacity, that would mean that less than 2GB of memory in total were in use by all processes at the time of the OutOfMemory exception error, would it not?