Hi, Is there a Limitation about downloading several files, one file per socket?
I have a String array with 758 Links
And I created a HTTPSecureSocket array that append as many items as items has the String Array.
And I tested it, checking the Ubound of the HTTPSecureSocket
and truly its working.
But dispite of know the number of sockets created and with its respective Handlers, Only Downloads 496 of 758 Links.
I wonder if Is it a Xojo Limitation, or is a Server Limitation.
Have anyone faced similar situation?
There are system limits that might affect this, but I’m pretty sure it’s a bit higher than 758. Did you have a lot of other things open at the time?
Is it possible that something else is limiting your connections? Sometimes ISPs impose connection limits, firewalls might do that?
Does it always stop at the same number? I wouldn’t expect it to be that deterministic if it’s some kind of system or network limit. I’d expect it to vary a little bit at least.
Are you logging socket errors? Anything interesting there?
[quote=344507:@Kevin Windham]There are system limits that might affect this, but I’m pretty sure it’s a bit higher than 758. Did you have a lot of other things open at the time?
Is it possible that something else is limiting your connections? Sometimes ISPs impose connection limits, firewalls might do that?
Does it always stop at the same number? I wouldn’t expect it to be that deterministic if it’s some kind of system or network limit. I’d expect it to vary a little bit at least.
Are you logging socket errors? Anything interesting there?[/quote]
Hi Kevin, In fact I’m realizing that I’m using a For each to parse all my HTTPSecureSockets, but one curious thing happens when I put a Msgbox at every Item on “FOR”, and when I do this, all the 758 files are downloaded.
Might be is right that you told me, I’ll be investigating.
How are you creating the sockets? Is it possible that some of the sockets are being allowed to go out of scope and the reference counter is reclaiming them before they’ve finished their work?
Don’t know what OS you are running, but on Windows there is a limit in the maximum file handles that can be opened. Maybe you are running into this with so many downloads. When adding a msgbox some downloads may close the file handle and allow others to continue? Just thinking…
On Windows an application normally only supports a maximum of 512 open files at the same time, so maybe you need to do some throttling (remember that the EXE itself opens many support files).
I suggest that you don’t attempt to transfer that many at once as you will more than likely run into limitations / restrictions either on the client, firewall or server. 10 - 20 concurrent transfers may be a good number to start at.
FYI. The last time I used the built-in HTTP(s) classes they performed poorly both in performance and CPU overhead when performing multiple transfers. I switched to the MBS CURL plugin which made things a lot better.
With all of that said, remember that your computer can only handle so much bandwidth anyway, especially if you are writing these files to disk. There may also be limits on the number of concurrent connections from a sinhle IP on the server as well.
Try having only 3-4 sockets going at once and you’ll probably see nearly the same overall speed as trying to do them all at once.
[quote=344588:@Greg O’Lone]With all of that said, remember that your computer can only handle so much bandwidth anyway, especially if you are writing these files to disk. There may also be limits on the number of concurrent connections from a sinhle IP on the server as well.
Try having only 3-4 sockets going at once and you’ll probably see nearly the same overall speed as trying to do them all at once.[/quote]
Ok, I’ll gonna with 4 sockets, one queue Download per socket.
