Really weird http socket bug?

I have a really old app I use to serve http responses. I use it to control my electric gates, retrieve security camera images etc and serve them via NAT to my iPhone.
All was well for 5 years. I added an electric door to my workshop and decided I needed to update the server so I could open and close the door from my phone.
After compiling the same app with XOJO I started having all kinds of weird issues with camera images intermittently no longer being sent. With HTTP request for the gates and doors failing to get a response. I’ve been troubleshooting it for months on and off.

Today I went back to basics and created a 2 line app that duplicates the problem I see

Dim http As New HTTPSocket MsgBox(http.Get(("http://192.168.200.12/cgi-bin/viewer/video.jpg"), 3))
I run this on a desktop Mac, on my local network. (192.168.200.12 is one of my cameras). Intermittently it fails to get an image (even if I extend the timeout higher). The cameras are not dropping connection, I know this as a fact. The network is all Gb and wired. Loading the same URL in a web page or HTML viewer in XOJO works every time.

Weird thing is, if I compile using RealStudio 2011 v4.3 (I happened to still have a copy at hand) it works just fine every time. If I use XOJO 2015 ( I tested as far back as XOJO 2014 R1.1, the oldest I had installed still) then it sometimes fails.

The ‘error’ code is always 102, which is also what I get if the image does load. I expect the camera to (102) drop the connection once the image is served.

Something changed with http sockets between ~2011 v4.3 and ~XOJO 2014R1.1 and I’m snookered to figure out what that is and how to work around it. I’m going to install a packet sniffer next, but wondered if anyone had any ideas before I did.

I would implement the Headers Received event and see if you’re getting that and what the server is telling you in that. A 102 error just means that the server has disconnected you doesn’t it? You’d get that whenever the connection is closed on purpose or otherwise. I’m guessing that the content length of the headers is somehow confusing the calculation that the page was fully received before the error happened. If it had thought it had gotten the entire page it would ignore that error and present you with the data. I would want to see the headers. You might be able to get them even in the error event if anything was received at all via the PageHeaders method and then print them out. you may be able to see in there why it’s not sending the image, or why HTTPSocket is choking on realizing that it’s not gotten the image.

If nothing in there makes any sense I would temporarily mock up my own TCPSocket version. It’s not hard to write some simple headers asking for the page to the stream and just msgboxing what comes back to see if it’s returning errors or data and whats different about it from the connections that work.

As for changes to HTTPSocket over the last few years:

Thank you James.
Implementing the “Headers Received” event I’m getting a header:
“HTTP/1.0 200 OK
Date: Tue, 15 Dec 2015 07:15:06 GMT
Server: Boa/0.94.14rc21
Accept-Ranges: bytes
Connection: close
Content-type: image/jpeg”
Then the “Receive Progress” event fires four times, with zero bytes received of zero bytes each time.

Greg,
I installed Feedback to view these cases. One was private. The there don’t seem to fit the problem I am seeing.

Alrighty, I’ve tracked down what I thing the bug relates to.

My wife pointed out it was only certain cameras not working. They consistently fail.
It looks like any camera using HTTP version 1.0 fails. Cameras that reply

HTTP/1.1 200 OK Content-type: image/jpeg work.

These cameras do not have any newer firmware to install.

Any thoughts on why a camera that sends a header like this works:
“HTTP/1.0 200 OK
Date: Tue, 15 Dec 2015 07:15:06 GMT
Server: Boa/0.94.14rc21
Accept-Ranges: bytes
Connection: close
Content-type: image/jpeg”

But cameras that send this do not:

HTTP/1.1 200 OK Content-type: image/jpeg

They both worked in early versions of Real Studio

sorry, I got that a little mixed up.
this works

HTTP/1.1 200 OK Content-type: image/jpeg

this does not

[code]“HTTP/1.0 200 OK
Date: Tue, 15 Dec 2015 07:15:06 GMT
Server: Boa/0.94.14rc21
Accept-Ranges: bytes
Connection: close
Content-type: image/jpeg”

[/code]

Interesting. The Accept-Ranges header is clearly an http/1.1 header. That said, there’s no Content-Length header on either of those. I’m surprised that they work at all.

You should definitely try the new Xojo.net.httpsocket class. It uses more modern frameworks and my just work.

Fairly common on network cameras where it streams MJPEG images.

I should have been more clear. Surprised that it works with our socket. To be fair, the headers here say that the mime type is image/jpeg, so I wouldn’t expect them to be mjpg streams. Our old framework sockets are only 1.0 protocol and definitely can’t handle a stream.

Like I said, I’d try the new sockets though. I bet it’ll work there.

Greg,
This is not an MJPEG stream. The ‘URL’ allows the user to grab a static image. I simply poll the camera and send a single JPEG I send outside my network to my iPhone client… The client views the image statically with pinch, zoom rotate, etc; or it constantly polls the app to get a continuous feed of JPEGs for a ‘live’ video feed.
The cameras support static images as a JPEG (via that URL) or H264 encoded streams using a different URL. I only use the single JPEG method.

I’ve been out of the loop with XOJO for a year or so. What are ‘the new sockets’?

dang it, sorry. just saw your previous post.

I wish we could edit posts…

You can. In the upper right corner of the post there’s an edit button. You can also delete posts.

I only have the option to Quote my post, no editing or deleting. Unless I’m missing something.

I believe editing and deleting are time limited.

edit:

http://developer.xojo.com/xojo-net-httpsocket

Ugh! and down the rabbit hole we go. I implemented Xojo.net.httpsocket for grabbing the images, and it grabs them correctly.
However, Xojo.net.httpsocket doesn’t seem to be truly asynchronous.

When my app receives a TCP connection it examines the request and acts accordingly. If the request is for an image, it now creates a Xojo.net.httpsocket GET request to the camera. In the Xojo.net.httpsocket PageRecieved event I set a variable

app.pageHasBeenReceived = true

Whilst my TCP connection from the client is still active I poll this variable to see if we got an image in a reasonable time (if not I send an image that says “camera did not respond”)

dim i as integer for i = 1 to 60 // poll for 3 seconds then give up DelayMBS 0.05 if app.pageHasBeenReceived then exit //never gets called! end next

Even though the Xojo.net.httpsocket has received a page, the event doesn’t fire until after my original TCP connection from the client closes.

Using a httpsocket instead of the new Xojo.net.httpsocket means the PageRecieved event does fire just fine.

I’m guessing a complete rewrite and the use of Timers is in order. Unless someone has a bright idea?

httpsocket is truly async
dont poll it seeing if it has received an image
implement the events and they will tell you when it has fully received the image

I’ll explain the app flow, and hence why I believe I can’t easily implement what you are suggesting.

My app listens on port 8080 for an incoming connection using the old school networking stack. Once the listening tcpsocket fires a DataAvailable event I parse the header and look for a GET request, it then looks for the actual request, like"/getimage20.jpg". I then uses this to create a Xojo.net.httpsocket request like this Xojo.net.httpsocket.send (“GET” , http://192.168.200.12/cgi-bin/viewer/video.jpg"). Whilst still in the event look for original tcpsocket.DataAvailable from my client, I need to wait for either the camera to return an image, or for it to time out. So I poll the variable set in Xojo.net.httpsocket.PageRecieved for 3 seconds, if I get an image I send it, if I don’t I send an image that is a warning we couldn’t connect to the camera.

Maybe it is the way XOJO handles threading between old and new networking, but the Xojo.net.httpsocket.PageRecieved event never fires until after I return from the tcpsocket.DataAvailable event, by which time it is too late.

I’ve decided to just use MBS and Curl instead. It does what I need.

I’ve decided to just use MBS and Curl instead. It does what I need.

Now it may be my poor coding skills :wink: but I’m seeing considerably more throughput using MBSSCurl over Xojo.net.httpsocket or even the old networking stack. So much so that I’m recoding a similar app I use to monitor cameras at my animal hospital to use MBSSCurl