Looking for advice with threads and URLConnections

General
1

I have a web app that hits up multiple different APIs to collect data and add it to a database. Each API’s data goes into its own staging table in the DB for later processing. The app has a timer that launches a thread to get the data through a single URLConnection and each API is hit up sequentially.

I want to modify the app so that each API has it’s own thread. What are the pros and cons of having the URLConnection as a thread property vs. keeping an array of URLConnections at the app level?

1 Like
2

The events will fire on the main thread anyway so you might as well keep the array.

2 Likes
3

That is, unless SendSync is used in the thread. Then you won’t get events at all.

1 Like
4

yes good question, thanks for asking., ram management/ cpu usages ?

5

so this got me thinking, because i need to do the same thing, i read fast at first, you meant you init url connection/ threads one by one, and you wait for server result to triger other data query ?

6

Currently the part of the app that does all the real work is in a single thread. That thread uses a single URLConnection to get the data from a single API, then the thread processes that data and inserts it into a database. Once it’s done with the first API, it then does the same thing with the next API. The calls to the various APIs are hard-coded into the Run event of the thread.

1 Like
7

@Thom_McGrath I actually am using SyndSynch with the current single-thread version of the app.

1 Like
8

Then I’d say you want at least one thread. You don’t want to be blocking your main thread while waiting for the request.

1 Like
9

Since Xojo Threads are cooperative and do not leverage multiple cores, a more efficient model might be to use Workers.

2 Likes
10

I had thought of that, but I have not worked with workers yet. I think I’ll try the threads first and if I don’t get what i’m looking for i’ll give the workers a try.

1 Like
11

Because of Thread context switches, you’d probably get better results from this scheme:

  • Array of URLConnections that use Send.
  • Start a Timer that checks an array, ReceivedData.
  • The ContentReceived event adds its data to ReceivedData.
  • The Timer processes the ReceivedData array.

Or something like that.

2 Likes
12

In any case, when processing the data if you run the risk of having to run more than a few milliseconds, make sure you call App.DoEvents or App.YieldToNextThread periodically to give active sessions the time they need to communicate with their connected browsers.

1 Like
13

OOPS. DoEvents should only be used if it’s a console or web app.

14

Currently, workers are super primitive:

  • There is no way to check if the workers are still running.
  • If the result of the workers is too large the app will choke on the data. This bug is scheduled to be fixed for 2022r3.
  • The app and the workers are directly coupled which makes integrating workers into a complex app really not nice.
1 Like
15

Thanks for all the advice, everyone. A lot of good info that I hadn’t really thought of. There is so much I don’t know :slight_smile:

2 Likes