[quote=438656:@Rick Araujo]Not wrong, just semantics as Ivan said. Xojo have a non-preemptive, cooperative, non parallelized sequence of execution; time slicing with cooperative yielding. When you are doing an App.val=1, it’s guaranteed that no other “thread” will do an App.val=2 at the same time, because there is no “the same time” in Xojo.
[/quote]
They are full blown threads
They are schedule cooperatively
But that doesnt mean they are not full blown threads
P-threads have an api that lets the user decide how they are scheduled IF you want
They do not have to be preemptive
[quote=438656:@Rick Araujo]
Yes. This is the excuse for not having it, and looks like we will never will see one. [/quote]
If you’d like maybe 1 or 2 engineers to do nothing for possibly a year other than make all the frameworks preemptive then I suppose you could call it an excuse
I dont
Esp since it really isnt that hard to write a helper app or two that you CAN actually debug stand alone (which you cannot do with preemptive threads since you do not control yield points)
Personally IF I had a say I’d like to see some time invested to make it easier to create helper apps
Basically so from you one project it will have a new project item type … maybe a Helper … that now can access the same code (not variables etc) and when you compileyou get your main app and a second helper app - and maybe even some code in both places ot make marshalling data from the main app tho the helper and back simpler
That would go a long way to making it possible o write helper more simply
Helper apps WILL NEVER be the same, or as good as, using all the processing power with direct shared resources instead of expensive IPCs. The request for preemptive threading exists for, I don’t know, one decade? No hope. Just talking about the fact, not requesting again. Our colleague Amando died waiting for it. Helper apps must be used when it its the only way. Never performant as using a “real thread” in another core sharing and seeing all the memory, public variables and contexts directly without sending and receiving data in an intercomm.
But the second you hear every fan on a 18 Core Mac Pro spin up (and in fact when you hear them spin up on several machines because helpers can be distributed across machines) you get way more than you would with threads
And every one of those machines is absolutely maxed out running every core at 100%
Apparently those aren’t real
Enjoy the wait for preemptive threads
PS Amando did not die because there were not preemptive threads and throwing that in here is just BS for no other reason that to toss it in as an emotional plea. Please don’t
I just wrote another Xojo test program. This one lets me set the size of an HTTPSecureSocket pool, the number of times to call a test URL, and then reports the results. I’m calling a test web API on a client’s staging server that I know is very lightweight (i.e. the server can handle in the low thousands of requests per second) and returns very little data. I double checked this with a simple PHP script that echos a few bytes uploaded to my own server, and I got similar results.
I have to retract something I said earlier: it seems in a situation where you have many calls returning very little data it can make sense to have hundreds of in flight sockets or threads. This is not going to make sense if you’re downloading larger amounts of data and each connection, or just a few connections, can saturate your Internet connection. But for small amounts of data I underestimated the impact of network latency and the benefit of issuing many requests while waiting for some to return.
Best time on macOS comes with roughly 200 in flight sockets, completing 1,000 calls in 2.6-3 seconds. Going a little higher (i.e. 300) does not increase performance, and going much higher starts to drag it back down. More calls with a 200 socket pool slows things down a bit (~4s per thousand) but I’m almost certain that’s the server throttling requests. With 200 in flight sockets the core spikes fairly high. I would guess 200-300 is the limit for a single core in this scenario. (Norman or someone from Xojo would have to confirm, but looking at Activity Monitor it does not appear that HTTPSecureSocket is preemptively threaded across multiple cores under the hood.)
Windows didn’t perform as well. Best times came with roughly 100 in flight sockets, with the total time for 1,000 calls ranging from 6-9 seconds. There was no obvious reason for the large variance. What I found interesting is that on Windows 100 in flight sockets didn’t seem to spike the CPU. Just a moderate increase in activity. That implies something else is limiting the performance on Windows.
In both cases the machines were on Ethernet (not wireless), and both machines have quad core i7’s.
So I was wrong about never benefitting from a large number of in flight threads/sockets. But it’s equally wrong to say that Xojo cannot do this, and that it’s multiple cores which lead to the performance gain. I can’t say for certain that no scenario would ever benefit from having multiple cores, i.e. thousands of in flight requests to many servers. But my guess is still that you would hit another bottleneck first.
Create a class which maintains a pool of sockets. Have that class call the URLs in your list. As each socket completes use it to call another URL from the list. Do not process the data in the socket events. Store the data and leave it up to separate threads to process said data.
If your per-download data processing is heavy then you’re going to run into the limitations of a single core. Architect helper apps or stick with a language that supports preemptive threads. If you’re just saving the data then you should be able to keep any where from 50-200 sockets “in flight” at any given time.
In some scenarios helper apps are better, i.e. faster and more efficient. Shared resources have to be managed or threads crash, and that management is expensive as well. It depends entirely on the scenario as to which is better.
For the record I would love to see preemptive threads in Xojo. There was a time, many years ago, when I sold a plugin that allowed you to execute simple REALbasic code in preemptive threads, across processors, on Mac. As I recall MBS did the same. And my guess is he stopped supporting it for the same reason I did: what you could safely do became increasingly narrow. To the point where you were limited to some math ops.
The Xojo framework will never be fully reentrant and thread safe. Neither is Cocoa, Win32, or .NET. It would be nice to see just the essentials made thread safe in the framework so that we could perform some basic math/string/memory processing in preemptive threads.
I think it was straightforward to do in .net since 2002, the example app I mention was written in 2008, since wasnt realistic to do it in RB. It has being working the same for more than a decade now. (just a few changes when the the api is updated, and using more cores now)
I believe no one is waiting for Xojo, you just use the best tool for the task. If someone enjoys to waste more time and resources creating multiple apps, and using work arounds it is their choice.
