Mulling over dynamic code management

Getting Started
1

I would like to write a program to gather data from various web sites. The data will be processed similarly for all the sites, but (of course) the way the data is gathered is different for each site.

In order to make the code more elegant (and make it easier to add/remove web sites in the future) I am pondering making the program generic by allowing the per-website code to be loaded from a database.
I’ve had a quick look at xojoscript and seen (unless I’m mistaken) it doesn’t do sockets. I could always write some generic web-access code and get & run the xojoscript code before and after the socket call to get round this.

I am making the assumption the xojoscript can be inserted into the DB as a blob and read and run at a later date?

Does this seem like a sensible idea to anyone, can it be done this way, or is there a better way of achieving the same thing?

I have to add that these are currently only high-level thoughts and that for the majority of this I haven’t tried it or even know if it would work…!

Thanks for any input.

2

Look into the XojoScript Context property. In essence you can build a class with a private variable HTTPSocket. Create a wrapper around it that you can then call from your XojoScript.

3

I’ve done a very similar thing for my “Comic Downloader” app: it loads web-parsing scripts for different websites. This allows me to edit the scripts as needed (if a site changes their HTML and breaks my parsing). I don’t grab the scripts from a database, but there’s no reason why you couldn’t. The actual web downloading code is part of the main app, not the scripts.

I wrote an article about my process in xDev 10.1 (you can download the code for free, but you have to buy the issue to read the article). It might be a good starting point for you.

4

Thanks, Marc - I’ll take a look at that.