Some users of my application are starting to report IOExceptions to me, with the error code 24.
According to https://osstatus.com/search/results?platform=all&framework=all&search=24, it looks like the most logical reason is “Too many files open”. I’ve managed to reproduce this on a Catalina setup, and when looking in Activity monitor, getting info on the application and then “Open files and ports” it shows numbers 19 ~ 2559 as being blank.
Obviously there is something wrong here and I suspect it’s yet another bug in Catalina, but any ideas or suggestions?
I don’t know how many files I opened, but I can assure you that it wasn’t 2,600!
I was able to reproduce it by performing the same task multiple times (which reads 5 files a time). On one run, it encountered the error the second time, another time it took 5 times to run into this problem.
I assume so, in the list of open files and folders, there was never once listed a file that I had opened. In different places, I use different functions for it. CGImageSource, CGImageDestination, TextInputStream, TextOutputStream and binary stream.
I am going to experiment to see if I can narrow it down to which function, and then see if I can find a workaround.
Crapolina has been a nightmare for our Autosave routine, various problems across different versions. I am so sick of it.
Edit: I’ve gone through all my code and CGImageSource, CGImageDestination, TextInputStream, TextOutputStream and binaryStream is properly closed.
I must be something that your code is doing. My app reads lots of files from Mail. I know of a customer who had a million emails. That equals a million of files.
You may want to use “sudo lsof” to get a list of all open files on your Mac. This should help to get a clue what’s wrong.
You can look via “ulimit -a” how many files can be opened from a process. In case you need more the count can be increased via “ulimit”. See “man -s3 ulimit” for details.
Note that macOS is really limited for the open file count since the kernel does allow just about 12k open files for all processes! See “sysctl -a |grep files” for the hard limits of the OS.
[quote=488079:@Thomas Eckert]You may want to use “sudo lsof” to get a list of all open files on your Mac. This should help to get a clue what’s wrong.
You can look via “ulimit -a” how many files can be opened from a process. In case you need more the count can be increased via “ulimit”. See “man -s3 ulimit” for details.
Note that macOS is really limited for the open file count since the kernel does allow just about 12k open files for all processes! See “sysctl -a |files” for the hard limits of the OS.[/quote]
Thanks Thomas.
I was using Activity Monitor as it also shows what files are open. I noticed a couple of them had no path, just hex values. I tracked these down to the NSFileHandle, which I get from a NSPipe to use with NSTask. Updating my code to use the Newer Catalina API saw these getting closed when they should have. Yet the thousands of files with no path or hex identifier… Can only have come from using TextOutputStream.append, but even then I highly doubt it was creating that many log entries.
I also found another issue with NSFileHandle on Catalina, calling [NSFileHandle waitForDataInBackgroundAndNotify] for the second, third, forth time, etc (on a new NSFileHandle each time) was leaking threads on Catalina.
There’s been so many file issues with Catalina, I can’t begin to fathom what’s going on. I mean surely Apple employees must be using Catalina right?
Alls good at the moment, thank you @Beatrix Willius and @Thomas Eckert for your help.
I would imagine that most of Apple’s internal “apps” are web based as they scale better. Which of course would explain why Apple employees are not finding these issues. Still you’d hope that this would make them optimize Safari, as it’s still rather slow compared to FireFox.
I’m not using Catalina for anything other than testing my apps, which is probably why I didn’t see these bugs. I tried Catalina for a couple of weeks, but spent most of that time trying to figure out if my app was wrong or I’d found a bug in Catalina, turns out I’d found several bugs in Catalina (which have been confirmed by Apple’s DTS).
The difference between Xojo and Apple, is the billions of dollars that Apple make in profit. They can afford to do a better job with their software and hardware, but have gotten themselves into a negative spiral. As unit sales of their products falls, they’re appearing to be cutting resources invested in their products, to keep “profit” up, but alas it only increases the speed of declining sales.
If Apple were to stop worrying about the stock price and invest a large chunk of that cash into fixing the problems that are costing them sales, sure their share prices would go down for a while, but would ensure the longevity of the company, which means the share price would go up in the long term.