EasyTCPSockets: Multiple Connections and File Transfers

I’m trying to build an app that can communicate to the same app on another computer. I intend to send strings and files between the apps.

I have it working where Computer A Listens and Computer B Connects and Sends a Message. It then Sends Messages back and forth quite nicely.

Question 1: What’s the best way for Computer A to Listen so multiple computers can connect? Is there an example file to see this in action?

Question 2: I’d like to send Files. Should I convert the files to strings and send chunks of the strings via SendMessage?

1: ServerSocket

  1. Absolutely convert it to a string and break into chunks. Make sure you have a way to determine if the file you received is legit by getting the file hash and checking it afterwards.

Hi there Hal , a forum post that i can answer woot .
there is an example in the xojo template for a socket server , but i found that it a few inherent problems such as it would keep loading up sockets when a new client connected, the server socket sample uses the server socket this works buy keeping tcp sockets in its buffer it requires 10 in its buffer as soon as the server socket listens it runs the event handler that must return a socket.

just a bit a blurb about my sample , im in the middle of a project please see ware i am at and i have similar problem with the sending file , so about my sample to help . when the 25 sockets load it loads 25 timers for the buffer , it loads a third timer with 1.5 second delay if the client doesn’t send the corect bits of information it kills the connection ,

i have built in RSA password by trading of public keys but i have not finishes the signing the rsa part but it does do encrypted passwords
there two buttons on the client is send a basic message and send a RSA cryro message (for testing)

when a client disconnects it closes the sockets and adds it back to the pool of sockets that can be gived to the server socket to allow for more connections .

to send to all clients look though to 25 and just if connected

im working on loading the file as base64 with utf encoding , im my example you see the store string and get string functions my hope is is to send the filename and size and hash check as the first part. then spiting the base64 of the file in to something like 250 char long segments stored in my store string with the part number therefore reassemble at the other end loop though the segments and write to the file

have a good look my samples will make more sense its one of my first projects i would love some criticism or improvements
Client https://www.dropbox.com/s/ulj2x4dyayh6ifq/Client%20Socket.xojo_binary_project?dl=0
server https://www.dropbox.com/s/enql0d7ct0zddyj/ChatServer.xojo_binary_project?dl=0

Regards Clev

Thanks Bob & James!

James, I’ll check out and test your samples and see if I understand them and share back anything I can! :slight_smile:

Super star. just made a bit of progress with the load file in to the program and send as string it only works with small files at the moment, can any else improve on this so you can load the files in segments?

  Dim SF as string
  
  Dim f As FolderItem = GetOpenFolderItem("") // as defined in File Type Sets Editor
  If f <> Nil Then
    If f.Exists Then
      SF = "FLEN=" + str(f.Length) + ":"
      sf  = sf + "FNAME=" + str(f.Name) +":"
      
      // Be aware that TextInputStream.Open coud raise an exception
      Dim t As TextInputStream
      Try
        t = TextInputStream.Open(f)
        't.Encoding = Encodings.utf8
        
        sf = sf + "FDATA="  + ConvertEncoding( EncodeBase64(t.readall), Encodings.utf8 ) +":"
        TextArea1.Text  = sf
        
      Catch e As IOException
        t.Close
        MsgBox("Error accessing file.")
      End Try
    End If
  End If

Hi there guys sorry to post twice in a row , but ive got it working only with small files at the moment around 4 to 6kb but it works
see samples here

https://www.dropbox.com/s/yp4n2a0anlmlqce/RSACHATandFILEserver.xojo_binary_project?dl=0 server
https://www.dropbox.com/s/k06axb4ts4espvx/RSACHATandFILECLIENT.xojo_binary_project?dl=0

server will save in my documents so dont unpload from document .i havent done any hash checking but its doing images word dock other files ok but i say only under 6kb , until i work out a way to slow it down or read say 8kb at time

I know I’ve done some work getting ready for a training video where I was transferring multi-GB files between computers. let me see if I can dig that up.

Okay. So it appears I’m not breaking them up into smaller chucks. I’m using JSON to create a structure to my messages so I can see what the command is, FileName, Hash, Length, and finally the file data. I make sure to have a terminator so I can tell on the other end when the command is complete.

Here’s my code for selecting and sending an entire folder:

dim fFolder as folderitem = SelectFolder
if fFolder = nil then return

[code]for i as integer = 1 to fFolder.Count
dim f as folderitem = fFolder.item(i)

if f.Visible = false or f.Directory = true then continue
if f.name.left(1) = "." then continue
if f.name.InStr("/") > 0 then continue //don't send files with obvious file name issues.

dim sData as string
dim s as string
dim bs as BinaryStream = BinaryStream.open(f)

s = bs.Read(f.Length)

sData = CompressBZip2MBS(s,9)

dim oItem as new JSONItem

oItem.Value("Command") = 101
oItem.Value("FileName") = f.name
oItem.Value("FileHash") = f.MD5Hash
oItem.Value("FileLength") = f.Length
oItem.Value("Data") = sData

arsMessages.Append oItem.ToString + EndOfLine.UNIX

txtSocket.AppendText "Added file: " + f.name + " to message queue." + EndOfLine

NextMessage

exit //only send one file for now.

next

[/code]

NextMessage is looking to see if I have anything to send. If so, I wait until the Socket.BytesLeftToSend is zero. Only then do write to the socket with the next bit of data.

[code]Sub NextMessage()
if arsMessages.Ubound = -1 then return

if TCPSocket1.IsConnected = false then
TCPSocket1.Connect
end

if TCPSocket1.IsConnected = false then return

if TCPSocket1.BytesLeftToSend > 0 then return

dim sMessage as string = arsMessages(0)
arsMessages.Remove 0

TCPSocket1.Write sMessage

End Sub
[/code]

James, what are the two timers for? I see an Auth_Timer and a Buffer_Timer…

hi there Hal , the auth timer starts as soon as the client connected if it does not send that “garbage string” and the version and the clients public RSA key within 1.5 seconds it will boot the client from the server , its just a security check !
because xojo sockets when you read has the potential to only read some of the buffer or 2 messages or more at once the buffer timer loops though the buffer string and handles one message at a time its index therefore you know what socket it has come from
does this help !
cheers Clev

It does help. Thanks! The Auth_Timer is a neat idea.

For the BufferTimer, can you not trust the DataAvailable Event to have a complete message? It seems like you’d only need to buffer if DataAvailable fires with partial messages…

Would it be safer to not use a semicolon for your end of message? What if someone sends a semicolon in their message?

Would it be better to send messages as XML? Like:
MSGHello
m = message wrapper, t = type, and c = content

Then you’d know you have a complete message arrive when you see

For sending files you could then do this:
FILE

12:403

Hello

p = part, so this would be part 12 of 403. That would let you show a progress bar when receiving files.

Hi there Hal ,from what i remember im unsure of the answer to that question ,
what justified the buffer timer was when i was testing if the client send 2 pieces of data in quick succession the server would recive the both and the data receive would only run once therefore messing up my handing
the xojo page has limmted information apart from the fact it says in the notes
http://documentation.xojo.com/index.php/SocketCore.DataAvailable
"It is your responsibility to read the data from the buffer using Read or ReadAll methods. "

on the second post , already a head of you that was the storestring and getstring methods are for if the message has one of the chars that we use as “econding” (term used losley) it will convert them in to a %N and when you get the string it will return them back to their original value .

on that note i do agree with your xml i also though of having a look with JSON to see how that would work
i wrote simmlair thing in VB6 and it was good 14 years ago , becouse you call the strings by a string you can look though them with can be really handy if you doing a list ! but i assume you can do the same with these “newer” formats

Clev

DataAvailable doesn’t mean that it has the entire message. It just means there is data available to be read.

And it’s possible that DataAvailable has the end of the first message and the start of the next message. You have to keep track of it.

You cannot. But I wouldn’t use a timer. In DataAvailable, examine what you have, strip out the complete message, leaving the rest in the buffer, and process that message. Be sure to check if you have more than one message in the buffer, of course.

There’s nothing inherently wrong with using a Timer. I just prefer another way.

Hi Tim , in my example the data available only feeds the buffer its the timer that then reads the buffer one message at a time , regardles of what has been sent to the program

i know the timer might not be the best solution i did think of using a thread but the interfacing thing between the thread and sending data out did steered me away from that . any ideas of something better ?, im assuming that a class with a never ending loop inside might be an idea but ive only had xojo for about 2 weeks not sure ware to start on classes and what they can be used for !

Clev

The way you’re using the timer is fine. I was just saying that as an alternative you could process the messages directly in the DataAvailable event. Reading back, it looks like you had trouble with that and the timer is a good solution. No need to change what you’re doing.

Thanks Tim ,
just to give everyone an update and see if anyone has any good ideas i bet there is genius that has done this before.
been playing around with this most of morning
using the binarystream you are able to set the position of the read and how many chars you want to read the same works with the write , this was hurdle one overcome.
now i have tried two or three different ways and with many little variations of speed and size if sending the file and each time slowing down the connection is becoming a problem
first way i test was just send 500 chars of the data of the data until socket.bytes left to send was 0 before it sent the next bit.
this way worked but crashed the server if more than 2 or 3 parts !. because the server could not handle that amount of data

ok so the next way i tried was putting in to a timer even with a one second delay on the timer the server crashed no errors would just hang .

i also come have come across another problem but i only happened once so cannot be sure at the moment, the file sent fine but the recompile at the server side was 2 kb but the first 1500 bytes was all blank apart form the last 500 received

my last attempt is that when you click the send file button is that it sends the filename , Path and length of the file to the server
the server will respond to the client requesting the first 500 chars of the data ,
when the server then receives it will write it to the file
then it requests the next part until Eof = true then

my hope is that this will give a natural throttling and only go as fast as the server is able write to the disk and the socket is able to receive

Clev

Well after about the the last 8 hours working on it
i could not get the binary write function working but doing as the textoutstream and using the append methd works
the only thing that is annoying is that its very slow . takes about 2 min for half a meg, small files it works a treat , i though an error if you try and do an mac package file but video pdf and the rest are all working there is alot of random code from testing all these different ways but its ere.
client tells the server what file it wants to send
server requests to client to send the first 1000 bits ,
one receive of the first 1000bits ask client to send next 1000bits
so on and so forth until the end of the file

https://www.dropbox.com/s/skisxccm5mfkii5/RSACHATandFILECLIENTV2.xojo_binary_project?dl=0
https://www.dropbox.com/s/nbqxx798lq5d2mv/RSACHATandFILEv2server.xojo_binary_project?dl=0

let me know what you think Hal i hope this helps let me know if you improve on it
cheers clev

The fastest I was able to get file transfers going between two machines on a 100Mbs network was 70Mbs. I used 6 connections between the machines with 256KB blocks of data each with a MD5 checksum. Of course there was also positioning data to get the chunks back in the correct place in the receiving file, and a MD5 checksum for the entire file to validate it on completion.

I arrived at 6 connections & 256KB by experimentation.