So I dutifully submit my crash feedbacks and a few days later get a note - per the title of this thread.
The trouble is, unless I’ve written down exactly what I was doing at the time, the specific cause is, by then, somewhat of a blur.
I don’t know how you all log your customer bugs, but I keep track of all of them, regardless of whether or not I can reproduce them at the time.
Does Xojo really bin them after 10 days? If so, seems like the resultant KPI might be a bit like Fake News.
[quote=377355:@James Pitchford]So I dutifully submit my crash feedbacks and a few days later get a note - per the title of this thread.
The trouble is, unless I’ve written down exactly what I was doing at the time, the specific cause is, by then, somewhat of a blur.
I don’t know how you all log your customer bugs, but I keep track of all of them, regardless of whether or not I can reproduce them at the time.
Does Xojo really bin them after 10 days? If so, seems like the resultant KPI might be a bit like Fake News.[/quote]
The problem is if you don’t know what you were doing, neither does Xojo. So what do you recommend happen instead?
My crash reports (in the software I write) come back to me with function names and call stacks, as well as at least a hint about what happened (index out of range, NOE, etc). At the very least I go look at the method where the error was reported and see if I can determine what may have happened. What could have possibly been throwing the index out of range? What am I referencing that could possibly be nil? I’m usually able to get at least a good idea of what may have happened, even without any extra information from my users.
More information is always good, but not always absolutely necessary. I would hope that Xojo at least spends 5 minutes looking at the area of code that generated the error to see if there is something obvious that caused it. If they do, and there is not, then requesting more information is perfectly reasonable.
However, I suspect that this is not par for the course - they will likely try to replicate the problem (if you report a button throwing a NOE when you press it, the’ll create a project, put a button on it, and if there is no NOE, back to you it goes for more info) but I don’t think they review code all that often at first report. I think you have to be more stubborn to get them to look under the hood.
Thom - a bug is a bug, regardless of how it occurred. Daft example would be 100 of us reporting the same bug, once every two weeks over the year, but without detail of what we were doing - would Xojo bin each one and hence ignore the problem?
But if they were kept - perhaps you might start to see a pattern. And if they were kept your internal KPIs would be more representative of the pain your customers feel. And you might start to see trends in your KPIs that are representative of the improvements (or otherwise) of your software.
I’ve had the same problem in the telecoms world, where operators try to discount faults due to stuff beyond their control (eg rain fall, storms etc), but the poor customer still feels it, and if their systems were better engineered they wouldn’t suffer those problems.
I would hope that some preliminary work is done on all faults reported, regardless of whether or not there is info about the environment that caused it, and that faults are not ignored just because they can’t be reproduced.
A loud problem that cannot be reproduced is still a problem that can’t be fixed. Even if an engineer looks at the code and sees a problem that might be similar, that engineer is not you and therefore cannot know for certain that it is the same problem. For all intents and purposes, that engineer begins working to solve a similar problem that may or may not be the same as the one being reported.
Keeping the case is limbo is not helpful either. They aren’t discarded, they can still be located if necessary.
The fact of the matter is Xojo’s exception object doesn’t have enough detail. It’ll give the engineers the location and type of exception, but not line, and since the exception messages are almost always blank, no context clues either. So the engineers in this case reach out for any details they can get, because the case will die otherwise, either explicitly or implicitly. Xojo chooses to take the explicit route rather than just ignore it.
At least some statistics would be good.
If 100 people report the same crash it may need attention. e.g. by constructing a test case which covers various branches of this method causing the problem.