Excessive(?) Memory Usage with DateTime.FromString(dateStr, Locale.Current)

General
1

I have an app that reads a 5 MB file with about 32,000 records in it. The app reads the file and inserts each record into a database, so it does not retain any of the records in memory. When processing the file the app’s memory usage grows from about 25 MB to 35 GB during the processing and then returns to about 25 MB once the file has been read in. I narrowed the memory increase down to this line of code:

theDate = DateTime.FromString(dateStr, Locale.Current)

I am using Locale.Current because the date format in the data files is “mm/dd/yyyy”. If I use some dummy date strings in SQLDate format there is no increase in memory usage (and it is considerably faster) using:

theDate = DateTime.FromString("2022-01-01")

I have never run into this before, but I haven’t ever tried to process 60,000+ dates before either. I have a work-around so it’s not a show stopper for me.

2

Can you create a sample project and open an Issue at xojo.com/issues ?

3

Sounds like locale.current is using autoRelease internally, but Xojo isn’t flushing the pool until your processing completes.

I would suggest storing locale.current in a variable at the beginning, and using that variable during processing. If this stops the leak, then you have your answer.

Report it by all means, but don’t rely on it ever being addressed (past experience).

4

Already reported here
https://tracker.xojo.com/xojoinc/xojo/-/issues/73155

5

Could the DateTime Class have some internal private counter and pool with a threshold limit, and every New DateTime increment a counter until like 3000, and once reaching it, it would fire the “Xojo collector” to release the current disposable instances and reset the counter to 0? At the end of the loop it would execute the final release.

1 Like
6

That’s what Normal proposes to do as workaround in the ticket. I still need to do the auto release pools for my AppleScript with properties like this:

dim AutoReleasePool as new NSAutoreleasePoolMBS

if MailCountForAppleScript mod 50 = 0 then
  AutoReleasePool = nil
  AutoReleasePool = new NSAutoreleasePoolMBS
  DelayMBS 0.1
end if
7

Instead of calling Locale.Current every time, have you thought of grabbing it up front:

Dim loc as Locale = Locale.Current

And then using that?

theDate = DateTime.FromString(dateStr, loc)

8

People said that a mere DateTime.Now causes such garbage accumulation here: https://tracker.xojo.com/xojoinc/xojo/-/issues/67558

9

@Greg_O I will give that a try here in a little bit. @Sam_Rowlands suggested that as well.

10

Putting Locale.Current into a variable before the loop executes does not reduce the memory usage.

I looked at the tracker issue @Jeremie_L referenced above, and it seems there are no plans to change the behavior. I guess mine is an edge case anyways with the volume of date instantiations I was attempting.

I don’t really need to convert the date string into a DateTime object, so I can just keep the date string as it is.