IPCSocket leaks data buffer on .write call under Cocoa

Targets macOS
1

This seems like a bad leak - basically every call to IPCSocket.Write() is leaking the buffer for the data sent. I believe this is a beta bug / regression (although I’m not entirely sure, since I know there was some experimentation with CFNetwork-based sockets in prior betas).

<https://xojo.com/issue/34107>

2

I tried the test also with:

App.Open
  System.EnvironmentVariable("XOJO_USE_NEW_IPCSOCKET") = "1"

Results:

  • much slower (since the socket data is sent in 4KB packets)
  • still leaks the same amount of memory
3

[quote=102328:@Michael Diehr]This seems like a bad leak - basically every call to IPCSocket.Write() is leaking the buffer for the data sent. I believe this is a beta bug / regression (although I’m not entirely sure, since I know there was some experimentation with CFNetwork-based sockets in prior betas).

<https://xojo.com/issue/34107>[/quote]

I doubt it’s a beta bug as this code hasn’t been touched this cycle. Have you tested this in 2014r1.1?

4

Ok, this is odd.

What appears to be going on is that in Carbon builds, the memory leaks, but hits a ceiling and seems to stabilize in certain situations.

In Cocoa builds, the memory leaks, and keeps on leaking, and quickly crashes the app. This behavior doesn’t seem to depend much on the version of Xojo being used.

Neither situation is good, though…

  • In Carbon apps, about 1.80 GB is leaked, leaving your app quite hobbled. Also, CPU usage by the socket itself is near 100% while sending. Also, if you send the data too quickly, it crashes (presumably, if you over-run the internal IPC buffer)?
  • In Cocoa apps, the leak will kill your app after about 3-4GB have been sent cumulative. However, the CPU usage is way better (around 2% or so) while sending.

Any way this could get addressed?

Things I’ve tried:

  • wrapping IPCSocket.Write in a NSAutoreleasePool : no change
  • calling IPCSocket.Flush after ever write: no change
  • calling IPCSocket.poll after every write: no change.

This will really put the hurt on me for Cocoa if there’s no workaround, as it basically means that any process that sends a non-trivial amount of data continuously will soon crash.

5

Is this documented?

6

No.

7

More testing:

In Cocoa, if I send the data really slowly, e.g. by using a Timer that fires 10 times per second and only send 1MB per action, then there’s no leak.

If I try the same trick with a timer that fires 100 times per second, there’s an enormous leak.

So what appears to be happening is this:

  • Calling IPCSocket.Write, while the socket internally still has some data to send, causes the data to leak.

So, I can imagine some sort of workaround, where the app maintains a separate buffer, and then only spoon feeds data to IPCSocket.Write at a pace that it can accept. This would probably burn a bunch of CPU and is not an elegant solution.

8

Joe, if you have a sense whether this issue could be found understood & fixed in this beta cycle, I’d be grateful - if you think it’s unlikely, I’ll go ahead and start working on a workaround…

9

Have you tested this in 2014r1.1?

10

Nor is it intended to be
Its literally so we can get people to test it in certain situations outside a beta cycle as it will take more than a single cycle to solidify this code because of the wide variety of uses
This gives us a way to have it present & test it without having to make it “the live code base”

11

As far as I can tell, there is no functional difference between:
*2010R1
*2012R2.1
*2014r1.1
*2014r2beta 5

The only difference seems to be Carbon vs. Cocoa - both leak, but the leak in Cocoa is much worse.

12

A workaround for the memory leak is to add a check for IPC.BytesLeftToSend before .Write(). Fixes the memory leak, but kills performance (slowing down by a factor of 25x or more)

Thread1.Run
data = ... 
while true 
     while ipc.BytesLeftToSend > 0
        ipc.poll
        app.SleepCurrentThread(1)
     wend
   ipc.write(data)
wend
13

[quote=102376:@Norman Palardy]Nor is it intended to be
Its literally so we can get people to test it in certain situations outside a beta cycle as it will take more than a single cycle to solidify this code because of the wide variety of uses
This gives us a way to have it present & test it without having to make it “the live code base”[/quote]
Understood. Thanks for explaining.