Periodic data processing best practice

Targets Web
1

I’m completely new to Web programming so I’d like to have your opinion on which of the options would fit best for what I’d like to do.

In general I would like to pull data from an API service and write that data into my SQL database on the same server. This has to be done once a day at a predefined time, probably around 9 AM European time.

Now the question is, how to do that:
a) run the WebApp the entire time, with a timer in that WebApp that starts the process every 24 hours or
b) the server triggers the start of the WebApp every 24 hours, the WebApp runs the code and closes itself?

I don’t know if b) is even possible but what would you say how to do it?

2

Use CRON on your webserver for b?

3

Why that must be on a Web app??? Just for consume an API and write to a Database you can do it with a desktop app or a console one.

3 Likes
4

It doesn’t have to be a Webapp necessarily but I don’t know how to do console apps. And what would I have to do to make sure the desktop app runs automatically? I don’t want to have to start the desktop app manually each day.

5

If you use a web app you can use Lifeboat to upload and schedule the cron job :upside_down_face: Just call Quit at the end of your routine.

Personally, either a console or web app on the server sounds like a better solution. I wouldn’t want to go with running an app 24/7 and using a timer – that wastes a lot of resources when there are options like cron. Additionally opening your database to outside connections can pose security risks.

I use a scheduled cron job and console app to collect the latest plugin version information from Einhugur and MBS for Plugins Pro check for updates feature.

6

a WebApp is usually a service application. at linux it is called Daemon.

i think what you are looking for is a triggered ConsoleApplication.
at my raspberry pi i use a cron ui app for settings a cron job.

a desktop app without a user login does not work.
means no desktop no ui, the app will not run.
my experience at linux.

for personal use a desktop app would work.
on business use better a command line app (which can also run as service application)

very easy to use for config cron jobs at Debian GNU/Linux:

Ashampoo_Snap_Mittwoch, 2. Februar 2022_22h15m46s_001_RaspberryPI3 - RaspberryPI3 - Remotedesktopverbindung
Ashampoo_Snap_Mittwoch, 2. Februar 2022_22h15m46s_001_RaspberryPI3 - RaspberryPI3 - Remotedesktopverbindung757×705 86.3 KB

1 Like
7

Since a web app is a console app and you’re not comfortable with console apps, I’d say use a web app that really doesn’t do anything other than collect information, write it to the database, then quit. Set it up in cron to run everyday at 9am. Put all your code in app.open.

1 Like
8

web app as Daemon (24/7) have the benefit you could use it as inspection by browser,
seeing what is going on, monitoring, made a config via Web Page etc.

its more fiddly work to update this exe.
(at least at windows the service must stop, the exe can exchanged and the service must switch on again.)

1 Like