Preemptive Thread Debugging

General
1

Hey all,

I’m starting to experiment with preemptive threads in my app and so far, the results are amazing. In the application where I am using these, I am grabbing image data from multiple remote devices on the local network and painting them to a canvas. Think something like where you have a bunch of security cameras and you are getting their image data and showing them all on the screen in a matrix.

I’ve determined that the biggest culprit taking the most amount of CPU time is the conversion of the downloaded bitmap or JPEG string data (could be either one) into a picture. That operation is taking about 10 mS. By putting that and some other operations into a preemptive thread, everything works better. Until it doesn’t.

I’m seeing my app sometimes run for a long period of time doing this and there’s no problems. Other times it runs for a bit and the app silently crashes. No exceptions thrown nothing - it just quits. On a Windows machine where I have tested, I sometimes see runtime errors.

I’m assuming I’m not doing something correctly and that my code isn’t well enough protected with the preemptive threads and at some point, one thing steps on another and it all blows up. I wouldn’t be surprised. I think I’ve spent maybe 10 minutes with coding on this so far.

But where can I start to look or should I look to figure out where this silent fail is happening? If an exception was thrown, yeah, that would make it much simpler. But the silent fail is a challenge. So I’m just looking for ideas here on how to track that down…

Thanks

2

How does the picture get from the thread to the canvas?

Are you re-using the same picture object or do you create a new one each time?

Does the thread code call out to any global methods that modifies global properties?

Do you have any other data being shared between the threads / main thread.

Maybe you have hit an issue in the Xojo framework. Have you tried removing the network code and faking it with files?

3

Yes, I also noticed this. Very hard to debug. I looked very hard why this can happen but there is no reason why it should crash in my case.
In my case the crash is rare but it does happen now and then.

FWW in each thread I do some changes to a picture. And I am 100% for sure no other thread or main thread is using/accessing the picture.

4

I’ve been experimenting with preemptive threads in my app, too. I have one thread only just to see what happens. I get at least 3 different crashes, but I get nice crash logs.

I’ve made a public bug report. Then I tried to simplify my project until I was fed up with that and made a private bug report with the remaining project.

As wonderful as the idea of preemptive threads is they need something way better to fix the “leaks” than only crashing.

1 Like
5

I have multiple objects in code that are all analogs of the actual real world objects that I am pulling the image data from. There’s a thread running in each of these objects. The picture is created from the image data received over the network. I save that picture to a property of my object. The canvas then paints with that picture that is stored as a property.

I have the thread code semaphored at the point where I call the thread to start and then when the thread ends. So it’s not possible for a second thread to even spin up on the same object.

There’s a picture object created in the method processing the image data. But I use the same picture object property each time. Perhaps I am writing to it while the canvas is trying to paint in th main thread? That’s a possibility. I’m assuming the adduserinterfaceupdate method uses a timer to execute the code in the main thread. So it would certainly be possible for the next thread to be running and “saving” the picture while the canvas is trying to paint. But then everything is inside th semaphore - or since the code running on the main thread is likely launched from a timer, does the semaphore still apply?

I don’t think so but I would have to look deeper…

Potentially. Since we are dealing with sockets, I supposed there could be a clash where the main thread is processing something with the socket and the thread is trying to read data from the socket. Seems like that should be a protection that should be in place in the framework…

Would be a challenge to fake it with files since what I do is so dependent on the network code, but I could try that. Not a bad idea. I am wondering if it might be something in the framework since silent crashes many times are due to something in the framework. The runtime error I see occasionally leads me to believe there’s a potential framework issue.

6

Im continuing to do some testing here. One place where I thought an issue could happen is if I release the semaphore at the end of my code in the thread, but before the frame work ends it’s code, I happen to reinstantiate the preemptive thread elsewhere in my code. That could cause a silent crash. So I put the semaphore release back in the main thread via Timer.CallLater. Yeah, that doesn’t work so well. It runs for a while but then everything freezes. Seems like somewhere along the line the semphore doesn’t get released via the timer and the app just hangs waiting…

7

You be should activating/deactivating the Semaphore around the code that needs to be protected, and within the same thread, be that preemptive or the main thread.

If you are trying to draw out of the same picture the threads are updating, I don’t think that will work well. In the ThreadPool example I posted a few weeks ago, I solved the problem by making a copy of the image to read from, then sent back copies of sections that the main thread could draw back into the original before updating the GUI. Would something like that work for you?

The project and example is here if you want to take a look:

8

How do I find out which code needs to be protected? My app has one thread and I still get random crashes.

9

Any common resource like a Picture, MemoryBlock, Dictionary, etc., that is being accessed from the thread and main thread should be protected.

10

How do you protect an object or property? Blocks of code are easy.

11

You’re protecting access to that object or property.

MySemaphore.Signal

CommonDictionary.Value( 1 ) = 2
CommonDictionary.Remove( 3 )

MySemaphore.Release

As long as the Main Thread or another preemptive Thread is using the same Semaphore in this way, you should be fine.

However, I still had a problem writing to a Picture from multiple threads even using Semaphore, so I came up with that “slicing” scheme.

12

OK. I was activating the semaphore in the main thread and releasing it in the preemptive thread. I thought that would be a good way to force the app to wait if the thread was still executing before attempting to start it again.

I think that would work. I will take a look at your example. But I’m not sure of how making a copy would help.

I create a picture object in the thread. Now I need to paint this on the canvas. But even if I copy this to a different object, the main thread would still have the chance to be attempting to paint the picture while the thread is updating the copy. I’ll have to look at what you did but it’s not clear w/o looking how to completely isolate the two things…

13

If you create the Picture in the Thread, then send it back to the Main Thread using AddUserInterfaceUpdate once the Thread is done with it, you shouldn’t even need a Semaphore.

14

Any code sequence that needs tro be atomic, should be protected. In my case, I might pass a work item from a thread to a server thread. I do this by adding an item to a work-items array and then resuming the server thread. Then in the server thread I might do this:

nextworkitem = workItemsArray(0)
workItemsArray.RemoveAt (0)

WIth cooperative threads this is safe enough. But with preemptive there’s the risk that some other thread could get in between these two statements and (say) do its own .RemoveAt (0). So one would acquire and release a semaphore around them.

1 Like
15

Ah…

OK. I see…

In the thread where I would call AddUserInterfaceUpdate, I would make a New copy of the image and then pass that in the dictionary/pair to the main thread…

Aha…

16

Half of my app is in the thread. As far as I can see there is no common resource.

17

@Beatrix_Willius - Have you read the blog posts with tips for Preemptive Threads? This should give you enough areas to take care of when working with Preemptive Threads.

Cooperative to Preemptive: Weaving New Threads into your Apps

Preemptive Threads Are Here, and They Are… Pretty Good

Using CriticalSection to Manage Resources

Using Semaphores to Manage Resources

18

Ouch. I’m not a beginner. There are no common resources. Preemptive threads are flaky as heck for me and I don’t see any method of finding out what is crashing.

19

Post a sample crash log?

20

Not even getting those!