HTTP very large SetPostContent

General
1

I have to send a very large string of data via HTTP to a server… it can be over a gig. Basically I am doing something like:

theString = “a very long string……"
myHTTPSocket.SetPostContent(theString, “”)
response = myHTTPSocket.post(theURL, 3600)

but if the string is long enough I get a hard crash on the last line… no exception is raised.

In talking with Jason and others it almost certainly a memory issue… no big surprise… but the question is how to work around it.

I am thinking about using a TCP socket and implementing my own HTTP protocol, “chunking” the data, and using the SendComplete event to know when to send the next chunk.

Has anyone tried something like this? Any advice or suggestions?

Thanks,
Jim

2

How about setting theString to “” before calling Post? Getting rid of one of the copies of the string might just fix it for you.

3

For what it’s worth, the SendComplete event simply means that the data has been handed off to the socket for transmission. You may still run into the same problem.

Are you able to compress the string before sending it?

4

Have you thought about using CURL? It can do the post and I think it has the capability to handle larger sets of data. MonkeyBreadSoftware has written a very nice CURL class…

5

Greg… I have set the string to “”… it helps but I am still running into problems… and I cannot compress the string… it is usually jpeg data and I doubt it is going to compress much anyway.

Jon… I did play with MBS CURL but also got a crash on Windows… but I plan to look into that some more.

6

A JPEG that is over a gig?

Considering that consumer upload speeds are so much slower than download speeds, have you considered how long it’s going to take to upload this data? I’m not sure you will be able to reliably hold an http connection open that long.

7

[quote=132570:@James Meyer]Greg… I have set the string to “”… it helps but I am still running into problems… and I cannot compress the string… it is usually jpeg data and I doubt it is going to compress much anyway.

Jon… I did play with MBS CURL but also got a crash on Windows… but I plan to look into that some more.[/quote]

I would definitely look at CURL or some other protocol for that large an amount of data.

8

My app communicates with a enterprise system… it is not a consumer system… this data runs over a LAN… I also have no control over what protocol or method I use… I am stuck with the system’s published API.

Some of the data is jpeg but the biggest files are video… mpeg, mp4…

I am going to try MBS CURL again.

9

I still have some testing to do but it looks like using MBS CURL has fixed the problem… thanks to those that posted suggestions.