No, if the bug is not going to be fixed for any reason, there’s no point in keeping it open. Sometimes it’s not going to be fixed because it’s not our bug. Perhaps it’s a bug in the underlying OS. Sometimes it’s a bug in a deprecated API that already has a replacement.
I get that but I think it’s important that the user make that choice. Because we have a single set of engineers. I’d like to know that this bug is more important than that feature. And if a new bug comes up, the user should be able to make that judgement. If I can only do one, which one is more important, the feature or the bug? Sometimes it will be the bug and sometimes it will be the feature.
Well, we don’t mark them as fixed. We mark them as no longer reproducible but the effect is the same.
Then I’m confused why not reproducible (or fixed) bugs would still be a problem when you’re looking at age as a criterion. You filter those out and move along.
It wouldn’t be for bugs that are no longer reproducible. I was talking about for bugs that still are.
This is where I got the idea you were saying otherwise:
I see. Yeah, that’s not what I mean. The age of as case is worth reverifying as it may not be reproducible any more. What I was saying as that given two verified bugs, one that is 5 years old and another that is just a year old, the age itself is not much of a criterion. The number of people experiencing the bug, if there’s no workaround, etc., are all more important reasons to chose one over the other.
IMHO, bugs should have three attributes that track the numbers of users who have experienced the bug along with users who tried and can’t verify the bug. Imagine:
- User Tested & Verified: 80
- User Tested & Negative: 49
- Xojo Verified: Not Checked
versus
- User Tested & Verified: 4
- User Tested & Negative: 12
- Xojo Verified: Not Checked
With these two metrics the bugs affecting your user base is obvious and will allow you to prioritize the bugs. Users who are willing to spend some time testing specific bugs can record their results, either way. It might allow you to glean whether a bug is it related to a specific hardware, system OS, or other attribute.
Personally, I would like to see feature requests and bugs totally separate from a ranking standpoint. Feature requests guide long term roadmap, bugs are the focus of rapid release updates.
Moving forward ignoring bugs is like a boxer who goes into a fight against young Mike Tyson but is totally neglecting his defense. (The defense in this analogy could be a loyal user base and a good reputation.)
Serious companies will decide to fix bugs first, then implement new features.
Why? Because the customer paid for that feature, that is not working correctly. If he gets angry when he has to wait too long for a fix, then rightfully so. And he could ruin your reputation by spreading bad things about you all over the internet.
Psychology tells us: People can deal better with waiting for new features (because they never had/used those features) than with features on which they have relied that suddenly do not work (because they had/used those features already and someone took them away). So if the users are complaining, it’s better not because of broken features.
Anyway (and back to my analogy), Xojo can’t beat the big boys feature wise so why not take a look what some other small but very smart guys do: beat the big boys by delivering more reliable products with fantastic support.
It’s rare that some users can reproduce the bug and others can’t. The common case is that the user reporting the issue either encounters the issue but can’t reproduce it or the steps they provide don’t result in our ability to reproduce it. If we can reproduce it, we can almost always fix it. If we can’t, the odds of a deliberate fix on our part drop off dramatically. We really need to be able to reproduce the issue here in order to fix it.
Providing a sample project and/or steps to reproduce the issue makes a huge difference in how fast we fix a bug. Now, I could see a way of marking a case as having a valid sample project and/or steps to reproduce and that resulting in the case getting more points.
Unfortunately, it’s not that simple. It’s difficult to know how much impact a bug is having unless a lot of users report it or a user indicates that it’s a showstopping bug. Often the user reports it but then just works around it. All things being equal, if a bug with 3 users reporting it has an easy workaround and another has just one with no workaround and is a showstopper for a user, we will likely fix the second one first.
Most bugs are about something subtle that is broken, not the entire feature. A bug that impacts user A may never impact user B. But user B might be wanting us to add a feature that enables him or her to do something new that his or her users really want. You can see that it’s all a balancing act. If it were as easy as don’t add new features until all bugs are fixed, we’d do that but it’s not that easy.
This is true of most software but especially development tools. It’s great to be the most reliable product imaginable but if other products do a whole lot more, most users will put up with some bugs to get the advantage of the additional features.
But what I’m really looking for here is the best way for users to help us prioritize bugs by letting us know which ones they care about most. That was the point of this conversation.
But there are a lot of bugs marked “can’t reproduce”, so how is that possible that the user can reproduce it, but Xojo can’t? The answer may lie in something the user has that is unique (a plugin, system setting, etc). If you had 100 users that reported they can reproduce it, but Xojo can’t it would dictate much more research and having the common data points between 100 users would allow you to pinpoint it much more quickly. Right now, you don’t know that.
Instead you may have lot of users searching and not finding the bug in feedback (already closed), then creating new feedback cases because it is not there (double work). Many users dread creating feedback cases and the consensus is that users are not doing it because it is too time consuming and too many are marked as can’t reproduce (why waste my time). Whereas if all they have to do click a button, that makes it easy.
It would also tell you how important if that bug is reported by numerous users.
This should be the byproduct of a well designed system that is easy to use and automatic. It should tell you team what to work on without a lot of work and wading through pages of reports. Bug reports don’t die unless they are fixed. But aggregate user reported confirmed cases of the bugs drive them automatically to the top. Imagine just half your users spending 15 minutes trying to reproduce just a couple of bugs per month. Big data - Big results.
In my experience @Thom_McGrath has this right. Case ageing (and ageing within case status), analogous to an aged debtors/creditors ledger has a profound effect on case issue visibility and resolution internally. I have implemented this on several occasions and it is a great point of focus.
If there is to be a voting system, for equality, please ensure each user has the same amount of points to distribute between cases important to them (rather than allowing users an unlimited number of Likes). To explain, Users Tom and Jerry each have 100 voting points. Tom allocates 90 of his points to a case he considers a “show stopper”, whilst Jerry allocates 20 points to each of five cases.
Whilst @Thom_McGrath is again right in saying Team Xojo will already be aware of major issues, users allocating points to their priority areas is at least affirming (if not influential).
Not an attempt to hijack the thread, but if Feedback becomes a Web App, I’d like to see the seamless ability to create feedback cases from forum posts as an alternate pathway of reporting issues (The Ability to Elevate a Forum Post to Feedback Case).
Thank you for asking, thank you for listening.
Add points if the person “pulls” a feedback bug forward and verifies it with the most recent version. This not only shows that it still exists but also shows that someone is monitoring that bug (it’s important) and is willing to put forth effort to get it some attention.
IMHO the current system is fundamental flawed. I do not know what the “best” solution is, but the problem that many of us face is that we run into issues that are so specific to our workflow or what we’re trying to do. Because these bugs or feature requests are not encountered by the general user (if you could generalize), they don’t get attention, even if they’re reproducible.
In some situations the issue can be worked around, in some it cannot and this causes users (for all pricing tiers) to face issues with Xojo that simply will not be resolved.
The solution I came up with is an allotted number of “points” per year, per tier. Say Lite has 1, desktop 2, Pro 3, Pro + 8. When a bug/request is assessed by Xojo, they state how many “points” are required to address that item. If a Xojo customer doesn’t have enough points that year, they can save them and combine them with next years points, or pay extra to add points. If multiple users encounter the same issue, they can combine their points.
There are things I simply do not bother adding to Feedback, because I know they simply will NOT get enough points for them to ever be considered, like smaller file sizes. Is everything in the Xojo Framework actually needed or or there things in there that could be broken off into dylibs, which are only included on demand.
I think <https://xojo.com/issue/61031> is demonstrative of what I think has gone wrong with Feedback. It’s not the feature set, it’s the human element. For those that cannot access Feedback right now, the case is about an IDE bug with file type UTI codes. The case was bounced back requesting a sample project, but a sample project doesn’t make any sense in this context. I’d basically be submitting a project with an empty file type. I included detailed steps. After I responded, the case is now in limbo. Though I’m betting something will happen to it now that I’ve complained publicly.
For those that can read the case, my attitude definitely falters at the end because this is not a one time thing. Cases being stuck like this is a very common occurrence. It makes me not want to bother submitting reports. It’s easier to just work around the problem than to spend the reporting effort just to be ignored. And this is coming from the person who wrote Feedback!
I am so torn because I get calls from Geoff when it’s important. William PM’d me the other day about my URLConnection (which we’re still working on) and I cannot appreciate the effort more. But then I see situations like the linked case and it literally makes me think “why do I bother?” My feelings are so conflicted on the matter.
That’s where I think Feedback’s effort needs to be. The human element. Not to say the technical element is perfect, of course.
That’s a really interesting idea, Sam. I’ll give that some thought.
Assigning more or less points doesn’t fix the broken system:
- I write “bug can be closed” because Apple fixed it on their side. Bug is still open.
- Compiler bugs need to get the highest priority. Instead the my bug just rots.
- Obsolete bugs MUST be closed. Instead they are kept open.
- The automatic bug reports are useless. They must contain enough information for Xojo to fix the problem. I stopped reporting those problems.
- I can understand the need for the 10 days. But there is a danger of closing important bugs.
- Feedback sends me an email for my own entries. But when bugs are closed - especially with “couldn’t reproduce” - I get nothing. So the emails needs to be improved, too.
Xojo’s stack information needs to be better. We’d all benefit from that.
OMG this is so obnoxious.