Need Control to Rapidly Update Pictures

Targets Web
1

Hey all,

I need a control that can update/refresh an image on a web app rapidly - preferably on the order of 250 milliseconds or less.

There are problems with Xojo’s built-in controls:

1.) WebImageVIew - The update process for a new picture takes WAY, WAY, WAY to long. I’ve seen it take seconds and I need it a lot faster than that. And hitting it with an image change while it’s attempting to update itself basically starts the whole process over again and the image never updates.

2.) WebCanvas - I’m sure this would work better than a WebImageView for what I am trying to do but it caches every picture it gets. So if I am sending several pictures a second and a page is running this for any length of time, then it’s going to get ugly.

Before you all jump on me and start criticizing what I am doing and why I would want to take so much bandwidth, etc for a web page - relax. This web app runs primarily on a LAN for a very specific application. I really don’t care about bandwidth usage or any of the normal stuff you care about with a regular web page.

I am basically trying to capture some images off a device on my LAN as rapidly as possible and I want to display them in the browser as rapidly as possible. If I could use a WebCanvas w/o it caching, that would be great. But I think the limitations of the current controls is just a problem.

Does anyone have any ideas?

Jon

2

Preload the image into offscreen, invisible WebImageViews – call them “preload views”. Then set the url property of the target view to the url property of your preload views.

3

You can try caching the images per Brad’s suggestion, but that might still be too slow. Changing between cached images in a WebImageView takes half a second in my tests, and that’s running client and server on the same Mac.

If you need it to be faster (250ms) I believe you can accomplish this with the jCanvas control in Web Custom Controls. This control supports two forms of animation. The one you would need is the one where you can queue frames to play.

If you don’t already have a license I would want to gather more information from you to be sure that you can animate as quickly as you would like. (Drawn graphics or real photographs? How complex/large? LAN bandwidth/latency? Etc.) With sufficient bandwidth/latency it is possible to sustain 30 or even 60 fps with drawn graphics. Without sufficient bandwidth/latency even 15 fps can be jumpy, unless you can queue the entire animation at once.

I haven’t stress tested it with photographs. With photographs you would need to download and cache all the photographs to the client browser before starting the animation using the WebImageLoaderTD control (which would allow you to progress report the download). Beyond that it’s a question of how quickly the browser can draw the images you’re asking it to draw, which I imagine varies based on whether the image is a 20 KB PNG or a 100 MB JPEG.

Definitely try Brad’s suggestion and see if it works for you. If not and you decide you want to try jCanvas animation, contact me.

4

Brad’s suggestion won’t work as the control in question is a custom subclass of WebImageView (or whatever I can use to display the image). It contains lots of other important code bits and isn’t just something displaying an image. So I can’t cache multiples of these off screen.

Daniel - I was hoping your control would do what I want. The image is a bitmap pulled from a device on the LAN. The bitmap is converted to JPEG and I scale the image so that actual size of the image is quite small. We are talking a canvas size of roughly 115 x 60 pixels. I’m also talking about being on a LAN. If the user isn’t on the LAN, the a reduced frame rate is just fine. So all the heavy lifting is going to be done at the server level. The browser should be getting very small sized images.

I don’t have a license for your control. How much queuing is needed? Are we talking seconds or fractions of seconds worth?

Thanks,

Jon

5

You’re still at the mercy of the browser and the speed at which it can request the images. I suppose you could create a custom image control that would use a timer and only increment the image # that it’s requesting when the previous one is received (it’d need to be all client side to be fast enough though).

6

Brad’s suggestion is related to getting the images down to the client browser before you try to display them, so you could use a plain WebImageView for caching purposes. (Though I’m not sure WebPicture.Cached is a reliable indicator that the image has actually downloaded. This was being discussed in another thread.)

Even small JPEGs are going to take enough time to download that they disrupt your animation. Ideally you cache all the images before starting the animation. Of course this can be difficult for a long animation to impossible for a continuous (i.e. no fixed length) one. In that case it may be possible, with small images and a LAN, to keep the client browser ahead of the animation with simultaneous image downloads. Most browsers will keep multiple HTTP requests in flight to compensate for latency issues. Another potential work around is to download many images in one HTTP request by combining them into a single larger image, and have the client side code display pieces from the large image.

As for the jCanvas control itself, the frame queue is a way to get around the latency involved in client/server interactions. 60 frame requests a second will never work. One request per second that packages 60 frames can work. The depth of the queue is something you can control and depends on the network situation. With a LAN it can be more shallow (i.e. 2-4 requests a second, 15-30 frames at a time).

7

Well what is interesting is that sometimes the existing WebImageView control works fine and updates the image and does it well. But it does not do it reliably. So I’m not sure what makes it work great for a while and then it doesn’t work so great.

This is not something I can download ahead of time. We are talking about what is effectively a live image capture and download. I"m certainly not expecting it to be as fast as the desktop. But the problem with the current WebImageView is that it appears that if it’s loading a picture and another image comes in then it resets everything. Hmm…this has given me an idea. Maybe I can set a boolean in the picture change event so that the new picture property is assigned only after the image is updated. I may lose some images, but frankly, that’s fine…

8

Latency issues and their impact on download time (even over a LAN).

Worth a try. If WebImageView works some of the time, and you’re OK with dropped frames, then your main issue may simply be that if WebImageView falls behind it stops working completely as it tries to flip between partially downloaded images. Making sure it has completed an image before changing it might work around that.

9

Hello Jon,

Is this what you are after ? :
http://www.radfac-innovations.com/#about

Click on the 'GO! button.

(don’t bother about the website itself, it’s under construction, I’m using the page I currently work on as a test container)

Refresh time is set to 100ms but here I get ~250/300ms (both servers and I are in France but not on the same location)

If you are oversee, refresh time may obviously be slower.

Cheers,
Guy.

10

Well, I guess this is not what you are after, sorry about that.
I’ll remove the test from the mentioned page then.

Cheers,
Guy.

11

[quote=77754:@Guy Rabiller]Well, I guess this is not what you are after, sorry about that.
I’ll remove the test from the mentioned page then.

Cheers,
Guy.[/quote]
I just now saw your message. Could you re-emable the test? I’d like to see…

12

I have an update here on this need of mine.

I have discovered that the build in Image Well for Xojo can actually do this rapid update fairly nicely. The problem I was having when it was not updating was that I was not specifying the session context in my code. That is a newer addition to the web framework. Without it, it seems like images just don’t update well. It’s not yet perfect but it’s getting close.

The biggest issue I now seem to be having is this session shutdown thread that keeps popping up even though I have the sessions set to never time out… Grrrrr… >:(

Here is a link to a video on YouTube of what I am doing…
Image Updates in Action

I have a method in my custom sub-class of WebImageView called Refresh. Here is some of the crucial code:

Sub Refresh(Optional s as Session)
  // Code for some of my picture handling here...
....
//
  
  Dim wc as WebSessionContext
  
  If s <> Nil Then
    wc = New WebSessionContext(s)
  else
    wc = New WebSessionContext(MySession)  // MySession is a property of the control
// MySession is set to App.SessionForControl(me) when the control is first shown
  End If
  
  Try
    
    If wc <> Nil Then
      If wc.Session.CurrentPage IsA MainPage Then  ' Only do this if the page is the main page of the web app
        MainPage(wc.Session.CurrentPage).MSContainer.TVs(Indx).SourceSelect.Picture = DisplayPic
        Call MainPage(wc.Session.CurrentPage).MSContainer.TVs(Indx).SourceSelect.Picture.Preload
      End IF
    Else
      me.Picture = NIl
      me.Picture = DisplayPic
    End If
   
    me.ProtectImage = True
   
  Catch
    Return
  End Try
End Sub

So by referencing the session context, I get my updating images.

13

I’ve recreated it in a small standalone app available here:
http://192.99.36.132:8080

Click on the Go button. The timeout is set to 100ms but the server is in Canada so depending where you are you may not get this speed.

The idea is simple, it is to have a little javascript script that download an image and set it to a canvas. The canvas is a WebCanvas Xojo control. When you click on the Button it just executes the Javascript function with the Canvas Id as an argument.

Here is the Javascript code:

<script type="text/javascript"> var x=0, y=0; var canvas, context, img; function timedRefresh(timeoutPeriod,cid) { canvas = document.getElementById(cid); context = canvas.getContext("2d"); img = new Image(); img.src = "/special/seq_image.jpeg?t=" + new Date().getTime(); img.onload = function() { context.drawImage(img, 0, 0); setTimeout("timedRefresh("+timeoutPeriod+",\""+cid+"\")",timeoutPeriod); } } </script>
You can put it in the App.HTMLHeader (or in a WebPageSource).
The WebCanvas control has no special code.
The Button gets this code in its Action event:

Me.ExecuteJavaScript( "timedRefresh(100,"""+ Self.CanvasSeqImages.ControlID +"_canvas"");" )

‘CanvasSeqImages’ being the name of the WebCanvas on the page.

The trick now is to return the images to the Javascript function. For this you use the App.HandleSpecialURL event:

If Request.Path.InStr("seq_image.jpeg") = 1 Then Select Case m_seq_counter Case 0 Request.Print( m_img_seq01.Data ) Case 1 Request.Print( m_img_seq02.Data ) Case 2 Request.Print( m_img_seq03.Data ) Case 3 Request.Print( m_img_seq04.Data ) Case 4 Request.Print( m_img_seq05.Data ) End Select m_seq_counter = If( m_seq_counter = 4, 0, m_seq_counter + 1 ) Return( True ) End If
m_img_seq01, m_img_seq02, etc… are WebImage App properties created in the App.Open event as such:

Me.m_img_seq01 = New WebPicture(seq_01) Me.m_img_seq02 = New WebPicture(seq_02) Me.m_img_seq03 = New WebPicture(seq_03) Me.m_img_seq04 = New WebPicture(seq_04) Me.m_img_seq05 = New WebPicture(seq_05)
You may have noticed the Javascript function always looks for the image ‘/special/seq_image.jpeg’ with the time in ms added as a suffix. This is to prevent the browser from caching the image.
The App.HandleSpecialURL only look for the image name part (Request.Path.InStr("seq_image.jpeg"))

Hope this helps.

Cheers,
Guy.

14

Please tell me when you have checked the test, because everyone is clicking on the Go button and at some point I’ll have to remove the test in order to keep my server responsive ; - ) Thanks.

15

Yes, I checked the app. You can shut it down. That’s what I am trying to do but unfortunately, I don’t have a URL for where the images are coming from. I think I am pretty close to having the issue solved though after I started using the WebSessionContext properly.

I appreciate you putting this up.

16

Well, with the approach I’ve mentioned it does not matter if you have an URL or not, you just set one arbitrarily and handle the Request inside the App.HandleSpecialURL. There you can send whatever images you want, from files or from memory, one you just draw or not.

Cheers,
Guy.

17

[quote=80136:@Guy Rabiller]Well, with the approach I’ve mentioned it does not matter if you have an URL or not, you just set one arbitrarily and handle the Request inside the App.HandleSpecialURL. There you can send whatever images you want, from files or from memory, one you just draw or not.

Cheers,
Guy.[/quote]
OK. Interesting. I’ll have to delve into this more. I have multiple Web Containers that hold the different ImageWells or Canvases I want to paint to. Your way of doing it might end up being a lot smoother than the way mine if working right now. And I like the fact the browser is not caching the images. That is what I don’t want. Thanks, Guy!