Caution about using Windows Roaming directories

When deciding where to store user data, you may be tempted to use SpecialFolder.ApplicationData, which stores the data here on Windows:

\Users\UserName\AppData\Roaming\

The potential problem with this, is the “Roaming” portion of the path. If this is non-essential data, such as certain preferences or app settings that are appropriate to be overwritten when files change, then this may be good. Here is some more information about this location:

https://msdn.microsoft.com/en-us/library/ms995853.aspx

The problem I have with this location is that I’m storing data files where my application may make one or more changes to a single piece of data in the whole file. Consider a flat file format or even something complicated like a SQLite database. Consider the following scenario:

  1. A user launches the app and the data is read in from this roaming folder.
  2. The user makes a few changes to the data, and saves the file back out again.

That doesn’t sound too harmless until you add the “roaming” aspect into it. Since the OS may replicate this data to the other user profiles, this means that the file itself is copied to the other users devices. That also doesn’t sound too bad, and almost sounds like a nice file sync solution. While it is true that it is syncing the data across profiles, it is doing it as one large file. That means, if this large file represents many individual records, like in the case of a SQLite database, there could be a problem when writing this data to the other profiles, i.e. roaming. The problem is introduced when the user is using the app on multiple computers/devices.

Now, consider the following:

  1. A user launches the app on Computer A and modifies some data.
  2. The OS replicates this to Computer B
  3. The user launches the app on Computer B and sees the data from Computer A.
  4. The user doesn’t realize they are offline, and makes some changes on Computer B.
  5. Later returning to Computer A, they make some more changes.

Here is where things get messy. There are now modified files on both Computer A and Computer B, and the OS is going to replicate the “latest modified” file to both computers. So, if the last changes were done on Computer A (step 5), and both computers connect to the network and try syncing, it is likely that changes from Computer A (step 5) are going to overwrite the changes made on Computer B in step (4).

That’s the trouble.

Now, if the data is a simple setting where the user would expect that their “last change” was the surviving change, and they don’t lose data in the process, then all is well, and this may have been a good case for using the “roaming” location. But, if this is not okay, which is most flat file formats and SQLite databases, then this would be a big problem.

So, what’s the solution? To use the “local” flavor of this AppData folder… the problem is, that isn’t built in to Xojo. So, here’s what I wrote to help solve the problem for me:

First I wrote this Declare:

[code]Private Function GetFolder(csidl As CSIDL) as FolderItem
Declare Function SHGetSpecialFolderPathA Lib “Shell32” (hwnd As Integer, pszPath As Ptr, csidl As Integer, fCreate As Boolean) As Boolean

dim space As new MemoryBlock(1024)
dim spacePtr As Ptr = space

if SHGetSpecialFolderPathA( 0, spacePtr, Integer(csidl), false ) then
Return GetFolderItem( space.CString(0) )
Else
Return nil
end if

End Function
[/code]

Then, I wrote an accessor to use that and pass the special ID for the Local App Data folder:

[code]Protected Property LocalApplicationData as FolderItem
Get
#If TargetWindows
Return GetFolder( CSIDL.CSIDL_LOCAL_APPDATA )
#Endif

Return SpecialFolder.ApplicationData

End Get

Set

End Set

End Property
[/code]

And I defined this constant so that it could map to the directory I wanted:

Private Enum CSIDL CSIDL_LOCAL_APPDATA = 28 End Enum

I did this on a module so that I could easily reuse it across projects. I thought I would share it with anyone else. When Apple introduced Desktop and Documents sync through iCloud, it really messed up many of my customers who were using more than one machine. This scenario of data being overwritten happened on multiple occasions, and it was not pleasant to tell the customers that their data was “lost”.

So I decided to start putting the data away in the SpecialFolder.ApplicationData location and I’ll simply provide an easy Import/Export feature when customers want to move their data. This is going to be received well by the customers, but because of the “Roaming” problem, I had to write this modified ApplicationData accessor to make sure it stores it only the local directory and doesn’t give a false impression that their data is going to sync across profiles in a reliable way.

I thought I’d share this with anyone else who it may help.

Thanks for sharing. Just to clarify, this will point to UserName\\AppData\\Local as opposed to UserName\\AppData\\Roaming? I’m a bit confused since the document you reference is for Windows 2000, and outlines a hierarchy that no longer exist in Win 7 or 10.

If you abuse a feature of the operating system you should not be surprised if it does not work out. As you have discovered the roaming feature has a specific purpose!

Rather than rolling your own, you should store your files in the documents folder. It is more portable and likely to be compatible with future system updates. It also means that your applications data will work in the same way the users other documents work. Furthermore it will handle situations where the documents folder is mapped to a network drive, which your solution will not.

Don’t roll your own where there is a perfectly good solution available, it could end up being a breaking feature the next time the OS is updated.

Yes, this will give you the local version as you mentioned. The important part is to rely on the OS to provide the final path, as Windows could have this stored in different locations between different versions.

The document does refer to older operating systems, and if there are more modern APIs to use, I wasn’t able to find them, outside of the .Net version. However, it is my understanding that Xojo is still generally using the Win32 APIs, so I believe that is what I’m sticking with for this too. I tested it on my latest Windows 10 version and it worked like a charm.

Ive been doing that for some years… but now and then, I come across a machine that doesnt HAVE a documents folder.
IT departments being draconian?

I totally agree about not rolling your own. That’s why I’m leveraging the Windows APIs rather than constructing the path myself or making assumptions about the underlying file system.

As far as I’ve checked, the local AppData is a suitable place to store application support data, which may also include user data.

In the case of the apps I’m working on, they do not operate on typical “documents” like word processor or similar. It is more like a set of database and support files, which it is not good for users to manually manipulate outside of the application. So, I have experienced many users who can’t seem to leave the data as it is. They seem to feel, and possibly rightly so, that if the data is in their documents folder, it is theirs to move, rename, copy, etc. In this case, the application needs to rely on the data being in a certain place, with certain names, etc. So, I’ve opted to put the data away, and provide the user with means to Export and Import the data as they would like. Some of my apps more similar to a mail application, where the interworking of the messages database, attachments folder, etc. are not something a user should manage. But if the application allows the user to import and export the data, the user can do what they want to the data as they need.

I think this model of application data is also made more common based on how many of the mobile devices manipulate application data. On the iOS devices especially, the data is not available to the user unless the application makes it available. In some ways this can be annoying for users, but as long as the application doesn’t hold their data hostage, I think it is a good tradeoff to traditional documents. However, if traditional documents are appropriate, I also encourage their use.