If you get stack overflow then what can the socket do? Nothing, because there’s no longer any stack space. So the socket’s error event can’t fire because that needs stack space. There is no recovery possible.
I don’t even do that. The only thing my DataAvailable event handler does is to resume the thread that is wating for data. Then the thread does the ReadAll. But then I’m not having several hundred sockets open at once.
If you use threads then you can set the stacksize of each thread.
The process I supplied was given to me by Joe Ranieri when I was first designing Xojo Cloud and was the basis for the massive speed increases in the Xojo Web 2 server increases. The pattern doesn’t require threads meaning that you’re not creating any more contexts that the app needs to switch in and out of.
Don’t get me wrong, your processing code still needs to be as fast as practically possible, and if you’re still running into trouble, I suggest compiling using Aggressive to see if you can squeeze a little more speed out of it.
Don’t be tempted use #pragma statements to help you here either, they’ll look like short term wins while testing and lead to long term losses when you have lots of sockets with large incoming data packets.
Something else that can help if the incoming data can be truly large (like file uploads) is to make the incoming buffer be a binary stream. If you know the size of the stream ahead of time, you can decide whether to back the binary stream with a memory block or with a file and then just write to it. IIRC, at Xojo I made a custom class that had the ability to keep track of two pointers, one for reading and one for writing so the incoming data could always be written to the end of the buffer while the data processing method could read from anywhere to try to find a complete packet of readable data.
1 Like
i have created an app.arrayData of strings where i store the data that arrives in every tcpSocket
i use readall method of tcpsocket when datavailable event is generated, then add this data/string to app.arryData and this must liberate the buffer of the TCPsocket , but with large flux of incoming data the listen server (app)crash , may be for same problem of stack data
Testing your project files (Link) on macOS Ventura 13.6.6 (Intel) under Xojo 2024 R1, running in the IDE Debugger:
- I get 231 connections
- I don’t see any lockups or errors (after running for about 5 minutes).
Running Built apps:
- I get 142 connections
- I don’t see any lockups or errors (after running for about 5 minutes).
The fact that the # of connections (142 or 231) is much less than 1000 seems like some sort of bug, but since I’m not seeing any lockups or crashes, I wonder what the differences are. What OS and CPU and XojoVersion are you using?
now this is my code in readall
app.arrayDati.Add(me.ReadAll)
me.Write("reply")
set the aggressive mode
and the probem is the same , there is a stack exception in dataavailabe
try with 5 client that call the same server
build 5 clients app (every client must open 200/300 connections )
then start trasmissions
im using xojo 2024r1.1
w10 pro
i7 quad core
16 gb ram
but the problem is present using othe machines
Ideas:
- we may be seeing differences since I’m testing on macOS and you are on Windows
- your code does user interface manipulation inside the socket events, e.g.:
Sub DataAvailable() Handles DataAvailable
...
Window1.lblPingRicevuto.Text=str(Window1.nDati)
...
End Sub
A safer thing to do is create a method, Window1.UpdateUI() which does the user interface updates, and then call this later using Timer.CallLater:
Function AddSocket() Handles AddSocket as TCPSocket
Dim ret As TCPSocket=New skAscolto(socketTotali)
socketTotali=socketTotali+1
system.DebugLog currentMethodName + " " + str(socketTotali)
//lblTotSocket.Text=str(socketTotali)
Timer.CallLater(16, AddressOf UpdateUI)
Return ret
End Function
Public Sub UpdateUI()
lblTotSocket.Text=str(socketTotali)
lblPingRicevuto.Text=str(nDati)
End Sub
Try that and see if it helps? I tried it on macOS and it makes no difference (but I’m not seeing the crashes that you are…)
i have try with reducing the code in dataavailable at 2 commands
readall and write (withouth interact with gui or others )
but the problem is the same
Good testing! How about moving me.Write(reply) to a timer callback also? I seem to remember there are some weird behaviors (bugs?) when you are writing from within a a socket’s DataAvailable event.
1 Like
i dont kow how i can do this
Blockquote moving How about moving
me.Write(reply)to a timer callback also
if i must reply quckly how can delagate a timer to reply ? (then the sequence of reply can be
could be made wrong if there are timer that respond …or not ?)
Perhaps try this:
Sub DataAvailable() Handles DataAvailable
Window1.nDati=Window1.nDati+1
timer.CallLater(16, AddressOf Window1.UpdateUI)
timer.CallLater(0, AddressOf me.SendReply)
End Sub
Public Sub SendReply()
me.Write("reply")
End Sub
1 Like
ok, now i have tried with only
app,arraydati.add(me.readall) instruction in dataavailable
and with no code in datavailable
and the result is the same non interceptable exception …
now, with one instruction o with zero instruction in dataavailabel, i think that this issue is a problem of TCPSocket
1 Like
Don’t use a property on app. Doing this will cause all of your sockets to were to the same cache. Instead, subclass TCPSocket and add the property there.
The problem Is present even without code in dataavailable, therefore the app.variable Is not the problem that causes stack exception
When you run this on both client and server on the same machine, your resources are too limited to to this correctly. Try running the client on different machines like 200 clients on a simple linux server or such. Then your server should be able to handle (depending on your system resources again) theoretically 281 million sockets. The serversocket should NOT be limited to the number of ports (65535) but number of file pointers/descriptiors. Note that this is only theoretical since the descriptiors may be limited by kernel etc.
One thing is that your Xojo app may be limited in resources (memory, cpu timeslice etc). But in my tests about 5000 or so sockets can be created at the same time on a machine. You may want to try to connect (and keep them alive a minute or so) via a service that tests server load, then you can test the capacity of your server the correct way.
The stackoverflow should not be happening, i’ve never seen that before so perhaps there is an error in your project somehwere?
Would you please try to set the MaximumSocketsConnected as here to a high value and see what happens?
I have tried on my machine and on a system with server and client separated from internet
The opened connection Ares 250 for each client, if i use 4 clients the Total connection on server Ares under 1000
The problems Ares not numero of connection
The stack exception Will be generated After that client send a string tò server continusly, after a loto of minutes
In the example project that i have attached tò issue case, the code is very simple and the exception Will be genetaed also without code in server tcpsocket dataavalaible and this esclude a problem of code
If you try tò execute server project (then press listen button ) and for/ five instance of the client (then press connect, and transmit button) you Will see that the server Will crash After a lot of minutes
I try
