An engineering question about saving files

General
1

I’m working on a game-making app for novices. It has an IDE a little bit like Xojo with a side bar navigator for the parts of the project and a central code / image editor.

Internally I plan on a project being composed of a bunch of text files (representing scripts for game objects, images, etc), similar to how the Xojo Text Project format works.

What I’m unclear about (and curious how an app like Xojo handles this) is when a user clicks an object in Xojo’s navigator, it obviously reads the contents of the text file on disk for that object and presents it to the user in the code editor. The changes made are not written back to the text file though unless I save the entire project. Does this mean that Xojo (and my app potentially) needs to keep in memory the contents of all text files after reading them once and only write them back to disk upon saving the project?

2

No, you could hold those/changed/altered contents in temporary files or a (temporary) Database(file). :wink:

3

I would be surprised if Xojo didn’t read all the files into memory when you open the project. Lazy loading is really only useful for projects approaching a GB or more.

4

That was my feeling too, just operate on in-memory file contents until the user saved but I wondered if that was the best approach.

5

Xojo loads in memory all text files from a project.
It might also load all pictures.

For your IDE I would recommend loading all text files and lazy loading pictures when needed.
Pictures could then be kept in a memory cache for speed improvements, such as a dictionary, using a UUID as the key.
Along with that, keep an in memory sqlite db that holds the picture’s UUID, filepath, size, access count, last access time and file modification date.
If the image cache grows a lot, clean it up every 5-10 minutes to remove big items that aren’t accessed frequently or haven’t been accessed within a reasonable amount of time.

When a picture is edited on disk, your IDE can compare the in memory (sqlite db) modification date to the actual file’s modification date to know if it needs to reload the file or if the cache is still up to date.

Edit:
Writing this post made me realise that my caching system in Packr isn’t that good and I’m off to writing a new one :slightly_smiling_face:

2 Likes
6

But with such IMO (“in-mem-only”) Systems, how do you restore changes after a crash?

7

You can either save frequently in a temporary place, or use snapshots and event logs that helps your app to recover the previous state.

8

I hope I remember all of the steps when explaining files :), and here it goes…

The older engineered way of handling files deals with three aspects: 1) Saved Disk File (Hello.doc), 2) Temporary Disk File (Hello.tmp), and 3) Memory File (Hello.mb for memoryblock). The purpose is to have a working record of at least one file when there may be a sudden shutdown (power outage, etc). If a power outage occurs, opening the file will provide the user to retrieve one or two copies of save files.

Opening a file:

  1. Check if both a Hello.doc and Hello.tmp exist - if so, the system shut down incorrectly and give the user the option to view and save one or both files.
  2. Open Hello.doc and immediately make Hello.tmp
  3. Load Hello.tmp into memory
  4. Show user the data in a TextArea control (or your control of choice)

Working with file open to user

  1. Immediate user changes are saved to Hello.mb
  2. User changes are saved from memory to the Hello.tmp file every minute or so
  3. When the user presses save-file, Hello.tmp is copied, then saved to Hello.doc
  4. reload Hello.mb from Hello.tmp so that Hello.mb, Hello.tmp, and Hello.doc all have the same information

Closing a user file

  1. Hello.mb is saved to Hello.tmp file
  2. Free memory of Hello.mb
  3. Save Hello.tmp to Hello.doc
  4. If Hello.doc saving is successful, delete Hello.tmp
3 Likes