Architecting new Web project rebooting my knowledge

I have an opportunity to rebuild an application from scratch, and want to refresh my Xojo Web skillset to take advantage of improvements in 2020 and unlearn bad habits of the past.

There are always a million ways to approach an app, but many times someone has already done the work and can save other developers from having to repeat mistakes - which is why I love this forum!

In my particular app there are several hundred devices spread around the globe. Each of these devices needs to accept commands via a centralized multi-tenant web service (xojo web app).

Each of these devices would already be bound to a users account, and depending on certain parameters these devices would ā€˜reach outā€™ to the xojo cloud, perhaps with a special url (?) and ask the central server if the tenant needs to talk to it (send commands, get access to immediate local readings, etc.).

With my current knowledge level Iā€™d likely architect this solution where the field devices do this:

  • Ping the server at a set interval, calling a special URL of sorts, passing in current readings and requesting to know if there are commands waiting for it to execute.
  • If the tenant is online and wants to control the field device in near real-time then the field device starts polling the server in a more rapid interval, retrieving commands and posting results back to the xojo server.

From the tenants side, they log into a xojo web app and do this:

Select the field device theyā€™d like to interact with.

  • View recent readings (that were posted during the devices polling intervals).
  • Click-select a device for ā€˜controlā€™ (which could take some waiting period as the polling device is between polling intervals).
  • And then send commands and review responses as needed.

In a scenario like this, what connection method would you recommend from the field device, back to the xojo cloud server? Each field device is running an embedded RPi with Xojo application running on it - we have full control over the ā€˜clientā€™ portion of this solution.

I want to avoid spawning a session each time the field device sends a ā€˜pingā€™ back to the server. I merely want to receive the ping, store the passed values as ā€œcurrent readingsā€ in MySql (per field device), so theyā€™re instantly viewable by the owner/tenant.

Only once a user needs to actively connect to their selected device do I want the round-trip communications to accelerate into more real-time.

Thanks to anyone willing to toss me some design advice on this!

Best regards to all :slight_smile:

Hello William,

I have a project that may be similar, at least in theory to your project.

Mobile App (for sales reps) performing stock counts, ordering to model stock levels and using merchandising plan-o-grams. Mobile App (a non Xojo App) uses a local SQLite database to store the last X number of transaction data locally. If internet is available. It will push a transaction to a Xojo Web (service) ā€˜handle special urlā€™ (I have a switch statement to detect transaction type, stock count, order etc).

The Xojo Web App is also used by call centre staff to enter telephone orders, corrections etc. The mobile user will get alerts if their local data has not been pushed to the Xojo Web service within a certain time. The Xojo web app is a transaction service as well as the app for office based staff. Over time as the app was used more heavily, I separated the Xojo Web Service (handle special URL) from the Xojo Web (customer service app). The only sessions created are for the office staff.

I crammed a whole bunch of logic into handling the special url. Limits to orders per day, transactions limits, popular stock item ordering limits, deviations from normal ordering, credit limit, opening hours limits, warehouse cutoff times (78 rules in all). Break the rules, you get a call from customer service to discuss the order with rep and customer.

Relying on sessions, keeping sessions alive, polling, rapid polling ā€“ I donā€™t believe are great approaches for this style of work.

From your description, your clients are really desktop clients (being RPi) not web clients, they donā€™t need to establish sessions with your web app to push data to it. Just a timer, that sends a reading or to handle special URL when required (of course identifying the device). The answer it gets back could be a success message with waiting instructions, but certainly no session management. When you need more realtime interaction you could use push notifications to the remote client to wake it up to listen for immediate instructions (maybe see https://blog.xojo.com/2017/03/06/sending-notifications-with-pushover/ - worth exploring). I havenā€™t used notifications on RPi though.

If you do end up polling set a property to increase/decrease polling within limits based on activity:

So if regular polling @ 20 mins and you receive something, go to 10 secs, then if you donā€™t receive further data on next poll go to 30 secs, 60 secs, 120 secs, then if nothing jump back at 20 mins. Just limits polling madness.

I hope that helps.

1 Like

Hi Andrew,

Every insight from others like yourself is a big help - thanks! Iā€™m hoping to do a fair amount of studying and head-scratching before committing to a set path.

These devices will be walled off from being reached, and weā€™ll be relying on their ability to deliver and retrieve information to a central server.

You mentioned HandleURL(), and that feels like a great direction. If I keep poking around I hope to stumble into some other great insights. Speed, security, and flexibility are key, so any suggestions that help me configure the end-points to securely report their data is important as well.

Thanks again for all youā€™ve written - Iā€™ll be reading it thru several times to catch it all.

Best regards!

William, almost forgot, I have another project where we have mobile soil stabilisation test equipment taking measurements from roads projects and building sites. The test equipment connects via bluetooth to the operators mobile (that ability shipped with the device). The test results are collated (they include GPS coordinates and test results) are sent via SMS to a Twilio account. I connected Twilio to Zapier. From memory we have a zap to update mysql, a zap to post to a Xojo Web App (handle special url) to collate and create a .pdf report of the results, a zap/or Xojo code to send the report to the client via gmail with personalised commentary. A Xojo Web App allows us to create a job spec prior to ordering the job. We have the ability to sms the operator, new job instructions, a request for additional tests, or to report job complete. Iā€™m a little hazy, but when a new job is created or maybe when an operator is selected, a database trigger messages the job spec to the operator. Just another model to consider.

Andrew, do you do your own hosting, or do you host on Xojo Cloud? (just curious)

Hello William, both, however I remain a fan of Xojo Cloud. Particularly for the ease of deployment and headless admin. I consider it good value for a premium service, as you can concentrate more on development than server admin. So even if you were planning to deploy elsewhere, the entry level package of Xojo Cloud is a good place to dev / test for web. @Jason_Parsley is very efficient and knowledgeable in supporting Xojo Cloud. Just a tip about making your code portable between servers, just include some if statements to handle if on Xojo Cloud do this, if on Linux do this (mostly around opening ports in the firewall on Xojo Cloud to access external servers, databases etc [if you need to]).

2 Likes

You could also use Xojo Web on the edge devices. That way the central Web server can query an edge device using a socket querying the edgeā€™s HandleURL, and the edge can send monitoring data to the central server using the Central serverā€™s handle URL. This avoids all polling. Of course for privacy reasons the traffic might need to be encrypted.

Can HandleURL be encrypted, or make use of https?

If you launch your app with a secureport and access it via https, yes. There is no ā€œforce sslā€ option, but I did request it <https://xojo.com/issue/51252>. Alternatively you can use nginx or Apache to force SSL, but unless you forward to the secure port of your app Session.Secure will be false.

1 Like

As far as Iā€™m aware thereā€™s no security with HandleURL. You can encrypt the data for or use a VPN or both.

For encrypting data, MonkeyBread has some shared secret key encryption classes and Xojo has a Public/Private key generation Crypto class.

For VPNs, RPis can be a client or a server or both. They involve a bit of setup however.

Here is the link for SecurePort Tim mentioned: https://documentation.xojo.com/topics/application_deployment/web/deployment_details.html
However SSL is slow for this kind of thing IMHO so it depends on how ā€œrealtimeā€ you want the solution to be.