Web app < - > client communication

Hi folks,

Here’s the scenario:

  1. Web app running on a LAN pc with (hopefully) fixed ip.
  2. Desktop apps ( vb or c#) apps installed on multiple pc’s connected to same LAN as web app.
  3. Desktop apps collect data from locally connected sensors (e.g. humidity, temperature, pressure etc)
  4. Desktop app sensor data needs to be acquired by & displayed by web app.
  5. Multiple users can view web app from any pc - also on same LAN.
    Each user will have the choice of which sensor(s) to view - i.e. their web page will be personalised to them, so they will need to log in as themselves & web app will remember their settings (which sensors to view).

If the web app was a desktop app i could whip this up in no time, but not having any web experience, the prospect is confusing.

I go around in circles trying to decide the best method of desktop app <> web app communications method should be in this case & I’m hoping somebody here has already done similar & is kind enough to share - at least the method if not actual code.
Some say UDP, some TCP… others say HTTP.

Being a hardened VS desktop developer, my preference for comms would be a polling method, where web app polls each desktop app in turn. This is a very familar method (with having years of modbus experience), but it may not be suitable in the web arena?

Another aspect I’m at a loss with is the user log in.
Are there any xojo examples of implementing a user log in / username, password storage, etc.?

Thank you

Two immediate thoughts to your post;

  1. Yes, there are many examples out there, including at least two Login Example web apps right in the Web Examples folder that comes with Xojo.

  2. If you are more comfortable with desktop apps, why do this as a web app? Build the whole thing as a desktop app (on both the client and server sides) and when a user wants to view data, the server provides it to the client where it is displayed in the native app.

Web apps are great for situations where you need to allow for a variety of platforms to connect and/or you need to avoid installing anything on the clients. It sounds as if you know (and maybe even control) the connecting devices and you’re going to be installing something on them. Why bother with a web app in that environment?

The desktop app to web app communication will be somewhat hard to do (surely someone will be right there to prove me wrong!). JSON, sockets etc. are the tools for that purpose.

Now, on a LAN, it would be relatively easy to have a central database where all the desktop apps connect to dump their data. The web app would connect to the same database to retrieve the data and display it. Security, detail level and such are relatively easy to manage by user also.

As was said, why not have a reporting module right in the desktop app? The app collects data from the sensors, dumps it, and can report on local sensors and remote sensors as well. The only reason I see to have a web app for reporting would be that people reporting are distinct from people collecting the data.

I have one application Desktop and One Web run both on same server with postgresql db, now you can use raspberry pi for the sensor and send data to same server to postgresql.

because the web app is a central location and if multiple users are going to access the app, it’d be better to pull the data once instead of every time a user logs in.

First of all, you might want to consider using console apps for connecting to the sensors. That way, you can run them as a service or daemon depending on the platform and they’ll still work even if no user is logged in. Those applications talk to the web app through the App.HandleSpecialURL or App.HandleURL events to send/store data whenever something new is available. You’d probably want to store a data type, a timestamp and then whatever data is new. Store it in a database and you’ll have historical data to work with as well.

Then as a web app, users can log in and get to whatever data they need, whether it’s current data or historical.

This forum is buzzing with great info. & ideas.
Much appreciated.

I have experience with MySQL, MS local dB & server & SQLite.

However, please note this will (hopefully) be a commercial system & I really don’t like the idea of customer having to install & configure a server database.
Minimal configuration is preferred, but I understand the benefits of a server dB.

I would prefer a local dB, so it’ll be up to the web app to drive this.
Having said that, i have no experience with PostgreSql. Is this easy to install & deploy? (I assume it is a server dB)

I’ve previously had handlespecialurl working with a single client connection.

However, i didn’t get around to investigating the performance & reliability of handlespecialurl in a realistic environment.

For example; how are multiple, possibly simultaneous requests being sent by clients handled by the web app handlespecialurl?
I don’t understand the workings of this method.
Are all requests qued until accepted? With timeouts?
Is this a race condition… who gets there first gets served?
Do some requests get lost?

Going back to a polling method:

Handlespecialurl is effectively just listening & not polling.

How can you send a request from web app? I.e. poll the clients rather just listen.

Thanks

You can use EasyTCPSocket for the polling. I made a similar program that polls modbus data from different remote locations. I used ConsoleApps to perform all the requests and receive the responses using EasyTCPSocket. Then my WebApps sessions connect to the ConsoleApps via EasyTCPSocket (only the ones that the session requires) to display the live data.

Hi Derek,

Did you write your own, true modbus tcp master/slave functions or did you pass modbus rtu over tcp?

I did not design the sensors if thats what you mean. I am sending my commands using standard modbus function codes and either reading or writing to registers that are on the sensors and devices that i’m controlling and monitoring. For each modbus network I am using a separate console app that is I automatically spawned after a new job is set up and the types of sensors that are being used are set. Each site can be adjusted on the fly of coarse using the TCPconnection between the WebApp and console app. For my device side im using a LTE radio that has a serial port and is passing the data through a serial - rs485 converter. We are running these on remote locations that won’t typically have there own network, middle of ranches and what not. These are oil field locations.

I Would prefer using an MQTT server like Mosquitto (simpler) or RabbitMQ. Then use the XOJO MQTT library developed by Zaatar Digital to create a client that listens to all the sensors on various topics. You can have XOJO clients running as Console Apps, Web Apps or Desktop Apps.

A While back I made a PHP script and a XOJO Console App that ran as service listening on a topic on the MQTT server and logged all the data to MySQL. I Then used a XOJO Webapp to display the data to the end user.

*** Unfortunately I’m not at liberty to share the code for my projects but I can confirm that the Github project for XOJO MQTT works well. We where able to use it in a number of ways.

https://github.com/zaatardigital/mqttlib-demo

I think that with Xojo you can do this quite easily (imho).
Collect data with desktops and send data to webapp. Webapp can present data in tables or graphs and can serve webpages to any authorized user in the lan…

This sample here should still work:
http://osswald.com/koblenz16/myservice.zip

… I am on vacation, cannot test myself right now :slight_smile:

Dereck - you said you are using Xojo EasyTcpsocket for your modbus transport.
The reason i queried this is because it sounds like you are transferring modbus rtu packets over tcp, which, as odd as it sounds - isn’t proper modbus TCP, it’s modbus rtu over tcp - there’s a difference!

Rudolph - your post has prompted me to read up a little on mqtt this evening. Looks interesting & specifically for IoT but somewhat intimidating to implement.
Not sure if this is right for me.
Is it fairly straightforward to implement? Costs involved?

Oliver - easily said, & it’s been suggested that “handlespecialurl” be used in the web app to receive the data… but… I’m not sure how this works when there could be 10’s if not hundreds of clients sending data (~100 bytes) to web app at same time. How will handlespecialurl cope this this amount of data?

Oh, and Oliver - how dare you be on vacation! Get home at once to try the code! lol :wink:

Rudolph,

The more i read about mqtt the more attractive it seems.

I shall try to get a system working at my workplace over the next few days.
I’m hoping that i can install & run mosquito broker on my Win 7 desktop machine & take it from there.
Sounds like a bit of a fuss to install, however.

Thanks for the interesting lead!

[quote=395694:@Darren Logan]Rudolph,

The more i read about mqtt the more attractive it seems.

I shall try to get a system working at my workplace over the next few days.
I’m hoping that i can install & run mosquito broker on my Win 7 desktop machine & take it from there.
Sounds like a bit of a fuss to install, however.

Thanks for the interesting lead![/quote]

The whole puzzle might look intimidating but it becomes really straightforward. I Worked with it on Linux, Win 7, Win 10 and Win Server 2012R2. So I know it works and is very stable. But as a test you can just use one of the public servers if you get stuck. Just keep it mind that anybody can read your topics on a public server. https://test.mosquitto.org/.

RabbitMQ also works well but it intimidated me in the beginning :-). Both EMQ and RabbitMQ gave me hassle a month into my experiment by crashing frequently while Mosquitto just never crashed.

ps. @Eric de La Rochette the guy that started the Github library for XOJO wants use the code not only to create a clients but also wanted to make a XOJO crafted MQTT server.

[quote=395665:@Darren Logan]<…>
Oliver - easily said, & it’s been suggested that “handlespecialurl” be used in the web app to receive the data… but… I’m not sure how this works when there could be 10’s if not hundreds of clients sending data (~100 bytes) to web app at same time. How will handlespecialurl cope this this amount of data?[/quote]

http://documentation.xojo.com/index.php/WebApplication.HandleSpecialURL

[quote]Because this is an application-level URL, the event is handled without a session context. Unhandled exceptions will be handled by WebApplication.UnhandledException. If you return False from WebApplication.UnhandledException this event will output Internal Server Error html. This will not cause your app to terminate.
Calls to this event are threaded so that simultaneous calls from clients can be handled properly.
In CGI mode, the headers sent with the WebRequest object have the same limitations as they do in WebSession.Header.
Session is not available from this event handler.[/quote]

First of all I would create the webservice as a Xojo standalone webapp (not as a cgi).

For instance, I run several siblings of a webapp on one linux server behind a haproxy loadbalancer. This setup can handle high loads.

In your case you first must make sure that the pc which is running the webapp is powerful enough(ssd and memory) and then it most likely can handle the amount of requests you mentioned.

It also depends on the work which the webapp has to execute. If this is a time-consuming process, then you may want to hand over all incoming information to a method running in a thread and quickly return true from handleSpecialURL.

In short, whether it will work for you depends on how you design your webapp as well.

Thanks, Oliver.

So HandleSpecialUrl is “threaded”, but how does this work, exactly?
How does this guarantee to handle every request coming in at once? (e.g 1000 requests coming in at exactly the same time… what happens?)

Sorry, I don’t understand what a “haproxy loadbalancer” is - could you explain what this is, please?

In general, I’d really appreciate a clear, layman’s explanation of what happens when multiple requests are made. Perhaps even a diagram? :slight_smile:

Rudolph,

I managed to install mosquito at work today and used the provided _sub and _pub consoles to send/receive messages.

Looks good so far.

I note that there are also .NET Mqtt libraries which I will also have a play with.

Again - thanks.