I have run into a significant performance difference between Windows 8 and Windows 7 and Vista. I have a database app that is blazing fast in OS X, fast in Windows 8 but very slow in Windows 7 and Windows Vista. Aside from being slow in the older versions of Windows, when iterating through large database record sets, it will throw an OutOfMemoryException error before it reaches 200 iterations.
I am encountering the problem consistently on several different machines and the amount of available memory has no actual affect on speed and the system information shows ample available memory when the OutOfMemoryException errors are thrown. I have to assume that the problem is the result of either a significant change in memory management between Windows 8 and its predecessors or there is a problem with the Windows framework in Xojo that is causing memory problems in the older versions of Windows but not Windows 8.
This happens alot with MSWord (well at least it did for me when I was writing a book) - I solved it by changing the virtual memory setting and the amount of space pagesysfile could use. If the c: drive is getting full pagesysfile struggles a little and has to pass over to another disk (if present) - this is how it was explained to me by microsoft. So the upshot is take a look at how win7 is managing the virtual memory. I may be well off here but a least it worth a look
(I’m sure you know this but incase you don’t to change the virtual memory size type : sysdm.cpl in the start menu then Advanced > Settings > Advanced > Change) - hope it works
Strange that you should get an out of memory exception though, if you are just iterating and the record sets don’t vary hugely in size then you would expect the memory consumption to be fairly flat. Does Task Manager show the memory usage rising and never falling? May be a silly question but are you holding on to the record sets for each iteration? Or maybe accumulating a large amount of data? You may expect performance to drop as Windows pages out to disk in order to try and satisfy the allocation of new memory when it is tight. As Nick states, increasing the amount of virtual memory can help but it is going to be slow if this is necessary and not very user friendly if this becomes a requirement to run the application. I would even consider decreasing it to try and track down and fix the problem.
The app limits the number of records that are displayed in a listbox to 100. If a table contains more than that, they are displayed in blocks of 100 with the user being able to control which record sets are displayed. Scrolling to the beginning or end of a block will automatically bring up the next or preceding block, depending on scroll direction. If the starting record to be displayed is not the first record in the set, I use a While/Wend loop to move the record set to the correct starting point before retrieving the record set data for display, Then I use a For/Next loop to retrieve the desired data starting from the appropriate record set number.
Nothing is being stored except for an integer value for the record set position. The retrieved record set data are replaced each time the block changes, so there is no great memory load there. The problem specifically seems to be with the loop count. It always generates an OutOfMemoryException error after 188 iterations on every Windows 7 and Windows Vista box I have, but it works perfectly in Windows 8 and OS X. That’s the mystery.
This happens alot with MSWord (well at least it did for me when I was writing a book) - I solved it by changing the virtual memory setting and the amount of space pagesysfile could use. If the c: drive is getting full pagesysfile struggles a little and has to pass over to another disk (if present) - this is how it was explained to me by microsoft. So the upshot is take a look at how win7 is managing the virtual memory. I may be well off here but a least it worth a look
(I’m sure you know this but incase you don’t to change the virtual memory size type : sysdm.cpl in the start menu then Advanced > Settings > Advanced > Change) - hope it works
Good luck[/quote]
Thanks for the tip. I will adjust the virtual memory settings and see if that solves the problem.
Adjusting virtual memory didn’t resolve the problem, but I did find the source of it. I’m checking the record set for a picture first before text, like this:
If RecordSet.Field("Name").PictureValue <> Nil then
//Make a thumbnail
ElseIf RecordSet.Field("Name").StringValue <> "" then
//Get the text
End if
If I only retrieve text without checking for pictures the app runs very fast and there are no OutOfMemoryException errors. It fails here:
If RecordSet.Field("Name").PictureValue <> Nil then
I’m testing this with a table that contains only text, so RecordSet.Field(“Name”).PictureValue is always Nil, but that line of code is where the OutOfMemoryException error occurs consistently in Windows 7 and Windows Vista, but not in Windows 8 or OS X.
In looking into the problem further it appears that the reason it was always failing at the same iteration was because data in the record set, non-English text containing diacriticals, were causing RecordSet.Field(“Name”).PictureValue <> Nil to be true. In other words, Windows 7 and Windows Vista read non-English text containing special characters as a picture. Deleting the two records from the table that were causing the error resolved the OutOfMemoryException error problem.
I finally found the specific cause of the problem. For reasons which defy explanation, if a record contains a string that begins with the letters BM, the following code results in a false OutOfMemory exception error in Windows 7, Windows Vista and Windows XP but not in Windows 8 or OS X:
If rs.Field(ColumnName(i)).PictureValue <> Nil then
That makes sense, although the problem is occurring with any text string that begins with BM. Pictures are stored in binary format in my databases. I’m guessing there’s a bug in the Windows framework from the IDE that’s causing this.
While there may be a problem with the underlying framework, there is also a problem with the way you’re using the function. You seem to be using it as “does this field contain a picture?”, whereas it really means, “interpret this field as a picture”. You shouldn’t call PictureValue unless you know there is a picture in the field or the field is empty.
I have no idea what an individual user will put in any particular record set and I handle pictures differently from strings when it comes to displaying them in the listbox for the table data, so I need to check the record for a picture before I process it. If I do something like this for every record (which takes longer):
SomePicture = rs.Field(ColumnName(i)).PictureValue
If SomePicture <> Nil then
//Create thumbnail
Else
//Get String
End if
I still get the same false OutOf Memory exception error if the record contains a string that begins with BM.
I should also mention that this problem only occurs if the record contains a string that begins with BM, and it occurs every time PictureValue is used with such a record.
Oh, so you’re trying to create a generic routine to display a database that you don’t control? In that case, don’t try to display pictures. It’s not worth it. Or give the user the option to mark the column as a picture and then display the image. In other words, if you don’t know what’s in the field, don’t make any assumptions.
Of course, this is a commercial database app for general use and I have been displaying pictures as thumbnails in a listbox with this app since I released v1 three years ago. If I remove that feature from the upgrade I am currently working on, my existing customers are not likely to be very happy. It’s hard to figure how such a specific, consistent error could be solely the result of the way I’m using PictureValue or why it occurs only in Windows versions prior to Windows 8.
Is this a regression that’s appeared in xojo? Or have you just not encountered strings beginning with “BM” before? If it worked before and doesn’t now, that would be a legitimate bug. If you go back to previous versions and strings beginnig with “BM” cause the error, then xojo would have a strong case in advising you that you’ve been using it wrong. MBS probably has a method to take an arbitrary string and determine if it’s valid image data. Or you may be able to find something on google.
[quote=23242:@Carl Hogue]I have no idea what an individual user will put in any particular record set and I handle pictures differently from strings when it comes to displaying them in the listbox for the table data, so I need to check the record for a picture before I process it. If I do something like this for every record (which takes longer):
SomePicture = rs.Field(ColumnName(i)).PictureValue
If SomePicture <> Nil then
//Create thumbnail
Else
//Get String
End if
I still get the same false OutOf Memory exception error if the record contains a string that begins with BM.[/quote]
Yeah that call is not “checking to see if the data IS a picture” - really never has.
That call assumes you know what you’re doing and tries to interpret the data as a picture because you basically said it IS picture data.
You might be able to grab the raw data and use the Picture.FromData method to see if it IS a picture
I first noticed this problem in 2012r1 as that’s when I first started using the test database where the problem manifested. I initially thought that the OutOfMemory exception errors where legitimate so I did some major revisions to make sure that processing a large amount of table data wouldn’t run the system out of memory. I only recently discovered that the problem was with two specific records in a table in the test database that contained a column for country codes. The country code for Bermuda is BMU. My staff subsequently narrowed the problem down to text strings that begin specifically with the letters BM. That is the only condition under which the problem occurs.