Recommendations for Database

One can be insulting, arrogant, and still correct. It’s tough when those align. The SQLite authors themselves say its not optimal so I’m not sure why you want to shoot the messenger(s).

How you say what you say is often as important as what you say

But, the SQlite folks don’t say “it will clause problems” - they point out why it might be a problem.
And in the event the network file system DOES implement fcntl correctly and you get no locking problems everything would be happy.
I’m just leery of hoping it does work right
And the SQlite docs don’t have any comments about “this system from vendor X does work properly here” so it would all be trial & error at best

[quote=73039:@Norman Palardy]How you say what you say is often as important as what you say

But, the SQlite folks don’t say “it will clause problems” - they point out why it might be a problem.
And in the event the network file system DOES implement fcntl correctly and you get no locking problems everything would be happy.
I’m just leery of hoping it does work right[/quote]

Thank you Norman, that is what I thought. I appreciate the response and clarification. To me the documentation isn’t clear, it says in high concurrency situations you probably want to explore other options, but what is “High”. It also says that “Some” NFS systems have locking issues but which?

So faced with that I did quite a bit of testing. I took several PCs (win7) and placed the database on our network server and pounded it hard to see if I could cause data corruption. In my case I had one PC writing numerous records then while at the same time had the others attempt to write at the same time and had others doing lookups. In all cases I experienced proper file locking. Sure I had to make it so my app would recognize this and attempt retries. We even have one of our buildings attached via a wireless bridge from 1/8 mile away. It works and can’t really say I’ve been able to break it. Overall I am quite impressed with SQLite for our application and the ease of integration with Xojo.

This is quite fascinating. Is your test suite available?

Sure, but why do both when being correct and respectful works just fine? Have you ever met an insulting arrogant person you like, regardless if they’re correct? I prefer to align with people of higher morale character. If a person wants to portray themselves as an expert on the subject then lead with expertise, not insults. To me, it’s just unprofessional and immature, quite childish. It also reflects poorly on the community.

Kem,

Honestly reading the SQL documentation and the references to non-locking NFS’s and OS’s looked to be a bit outdated because they were referencing Win 95/ME/98 and FAT file systems. The reported issues are not referencing any modern OS’s and my searches related to anything current didn’t really reveal a lot.

Because I did not know the initial implications of using SQLite in this scenario I did the initial testing as part of the product.

I just created a dummy DB and mounted that on a shared drive on the server. I created routines in my app tied to a button that would add 1,000 records, with the option to commit after every add (Slow) or add all then commit. I created other buttons that would fill list boxes with the records (SELECT) in the same table and different tables. I also created routines that would add a single record tied to a button. Basically the app had three buttons (Add 1000, FILL ListBox, Add 1 Record) Using three PCs I started adding the 1,000 records on one while reading on another and attempting to add the single records using the third. I would log any errors to another list box. When completed I analyzed the data in the DB to check for corruption. I did this several hundred times until I felt comfortable with the results.

Joseph, the reason I give away the server without so much as a request for an email address is so nobody in our community has to make this rookie mistake because they won’t buy something or can’t set up something complicated. If you want to file share SQLite databases over your home network for your own private data, be my guest. You’re doing a great disservice for others recommending that to others. If pointing that out makes you not like me, it’s your loss, buddy. If it makes you pay Marco rather than take me up on a free offer, I’m seriously happy for him.

In every community of client/server devs I have ever had my foot in, there is a contingent who think they can file share a database over a network. There is also a contingent that gets burned doing it. Same contingent. Same underlying problem.

[quote=73047:@Joseph Evert]
I just created a dummy DB and mounted that on a shared drive on the server. I created routines in my app tied to a button that would add 1,000 records, with the option to commit after every add (Slow) or add all then commit. I created other buttons that would fill list boxes with the records (SELECT) in the same table and different tables. I also created routines that would add a single record tied to a button. Basically the app had three buttons (Add 1000, FILL ListBox, Add 1 Record) Using three PCs I started adding the 1,000 records on one while reading on another and attempting to add the single records using the third. I would log any errors to another list box. When completed I analyzed the data in the DB to check for corruption. I did this several hundred times until I felt comfortable with the results.[/quote]
I’d have all three machines write records simultaneously as reads are not likely to cause corruption but simultaneous writes are.
Turning on the multiuser flag on the sqlite db enables WAL which can help in the event of a crash or corruption

But, personally, I’d still use a server for this

Joseph, I appreciate the testing you’ve done, and it sounds like that, combined with your experience so far, has convinced you that it’s a safe solution. And it probably is, but as a consultant, I would still recommend against it and concur with Norman and Brad’s opinions here. Even if it’s all working nicely today, a bad OS update (or other unforeseen circumstance) tomorrow could send it all south, and it’s just not worth the risk, not when there are proven, reliable, and recommended solutions for the problem. The worst part is, you may not know that corruption has occurred for days, weeks, or months after the actual event, and unless you’re testing the integrity of your data daily (as you must when you’re using an unrecommended procedure), you’re setting yourself up for a huge headache.

To be clear, this may never happen and we may be wringing our hands on your behalf for naught, but it’s only because we care. :slight_smile:

Joseph

I know that Brad can come across as a little harsh sometimes but I have been a user of Studio Stable for some while now and it is a truly great product.

Please swallow your pride and give it a go, you might be surprised.

It certainly isn’t a pride issue. I simply do not do business with people like that, regardless if I need a product like his or not. If I do in the future, there are many other choices. His product may indeed be the best, but sorry, there is quality of character of the people I deal with in business and that type of behavior is never acceptable in professional business. I could care less if the product is free or not. I just don’t deal with the little snide remarks and put down’s - it exposes an emotional character flaw in an individual.

I simply shared my experience of what I was doing that may benefit the OPs request. If you read the OPs original request he was asking if SQLite was suitable for one application writing ambient room temperatures to a database and having multiple applications reading it. This would be a VERY low concurrency situation, if any concurrency at all. I suggested it was good candidate for SQLite and gave an example of how I was using it with success. I also suggested if he needed something more robust where he can access his data anywhere he should explore MS Azure SQL (Which is a fabulous product by the way). Or maybe a WE app, I don’t know.

Regarding my particular situation I appreciate the input and will review it but it is nothing I have to do overnight as we haven’t had any issues thus far (2 Years). As things progress and user base expands we’ll weigh our options. We do back up, but haven’t had to use one since we implemented. Having said that, if I was a consultant I would recommend exactly what Kem said, because there is absolutely no down side to the consultant, only additional cost for the customer. In this case I am the customer and the consultant so my liability is mine, but so is any time and cost savings…

Well, you should have known better than to say that out loud…

I know, I know… Tomorrow morning KAABOOOM!!

[quote=73056:@Kem Tekinay]…The worst part is, you may not know that corruption has occurred for days, weeks, or months after the actual event, and unless you’re testing the integrity of your data daily (as you must when you’re using an unrecommended procedure), you’re setting yourself up for a huge headache.
[/quote]

BTW, for those who may be particularly concerned about data integrity, the 9.3 version of Postgres has a block checksumming feature that can be turned on to allow some level of verification for stored data.

[quote=73057:@Simon Berridge]Please swallow your pride and give it a go, you might be surprised.
[/quote]

Probably best he doesn’t. If he ever wanted/needed to purchase anything, I’d reject the payment. He looks really high maintenance.

Oh boy. Can we get this conversation locked now please?

[quote=73068:@Joseph Evert]
Regarding my particular situation I appreciate the input and will review it but it is nothing I have to do overnight as we haven’t had any issues thus far (2 Years).[/quote]

I don’t have any product recommendations, but I can say every single network file sharing solution I ever used over the last 25 years, eventually came undone and usually at the most inconvenient time. And usually the backups were useless as well, because they included some degree of the original corruption as well.

So unless it is a just for fun program (which most of my XOJO stuff is), I’d be moving PDQ to but a proper multiuser solution in place, especially if I was supporting a larger user base or selling the application.

With pleasure