Consider a non-empty file (in f), and this code of mine in a larger app:
dim content as String = BinaryStream.Open(f).Read(f.Length)
vs. this code:
dim bs as BinaryStream = BinaryStream.Open(f)
dim content as String = bs.Read(f.Length)
The first should work and did work in older Xojo and RealStudio versions.
But since Xojo 2019r2 the first returns an empty string. No exception, no error set in f.LastErrorCode. The second just silently fails, which is terrible.
I eventually narrowed this down and found what’s causing this:
The FolderItem object f is created at a time when the file exists but is empty, so its Length property is zero.
Then the file is written to with non-Xojo functions (in my case, by using CFWriteStream). This won’t automatically update f.Length.
Now I try to read from the file, but Xojo’s code appears to take a shortcut and, instead of trying to read from the file (which would return the data because the macOS file system knows that the file is not empty any more), it assumes that the file is still empty and returns an empty string.
I don’t understand, however, why unchaining the Open and Read calls avoids this issue.
This is quite bad, IMO. And in Xojo 2019r1.1 and all earlier versions this was not causing any trouble. So, someone at Xojo made the concious decision to take this shortcut.
To work around the issue, I need to add a line that force-refreshes f, like this:
could it be that LLVM will evaluate f.length in the first case before calling Open(f) and in second case wait for it, so it happens a bit later and Open(f) may trigger the cache invalidation?
In no way. That’s a Xojo bug if confirmed. If we say A = B()+C(), C() should never be evaluated first and then A() as same precedence follows left to right. Even worse a chained operation, like A = A().FuncB() where funcB() needs the returned object from A() to call its FuncB() method.
Not sure but Xojo, erroneously, could be doing something like processing everything ok, but releasing the returned string right away, dismissed on-the-fly, not raising some error, but resulting in a nil/empty string, maybe?
Yes, I reproduced this on both High Sierra and Monterey, on different Macs. The only factor that changed the behavior was the version of Xojo I built it with.
And it’s not a timing issue - I had other code run in between, such as read other files, which made no difference. So it’s more likely a caching issue with the Length function. It should not cache the value at all but instead, whenever it’s called, fetch it again from the file system. And it probably did before the change in 2019r2.
Well, new folderItem(f) starts with no cached properties, so that, when calling Length, it’ll have to fetch it from the file system. But FolderItem has the tendency to cache the values once it has fetched them, for performance.
I had already advocated better control over caching many many years ago but that idea never caught on.
So, instead, one has to assume that whenever a file’s on-disk properties change, one needs to refresh a FolderItem by using the method f = New FolderItem(f), or one may keep getting out-of-date values. This has been like this even before 2019r2 (and is therefore predictable), but something else has changed that causes the weird (and unpredictable!) behavior in my initial post.
At this point, it’s basically a “good to know” thing. Xojo will probably not see this as a bug, so why bother. Just be aware that whenever you want to check if a file’s on-disk properties have changed, you need to create a new FolderItem in order to get those current values.
Agreed. And few others too. We could have a way to disable/enable such caching for this object and have an extra FolderItem.Refresh() method to manually control it when caching is on.
if there is an internal method to invalidate cache, we should know it and call it, e.g. when a plugin or declare does something to the folderitem. anyone knows the trick?
Rick, that feature request has been made more than once in the 20 years past, I’m sure. Good luck
Also, I may have been wrong about f = new FolderItem(f) being the work-around to refresh a folderitem object’s properties. I just did a Shell command that would change the contents of a dir:
dim src as FolderItem = GetFolderItem("traversables")
dim dst as FolderItem = GetFolderItem("traversables-test")
dim sh as new Shell
sh.Execute "/bin/rm -Rf "+dst.ShellPath
sh.Execute "/bin/cp -Rp "+src.ShellPath+" "+dst.ShellPath
After this, dst.Exists was false and dst.Count was zero. So I added this line:
dst = new FolderItem(f)
if dst.Count = 0 then break
