[quote=182589:@Norman Palardy]We DO however also nominate individuals for inclusion in the beta program based on their history reporting bugs.
If you write good bug reports that can lead to us including you whether or not you have a Xojo Pro license.[/quote]
I’ll bet a good number of people didn’t know that, Norman. I know I didn’t.
That’s an excellent example of “using the information you have available to you to make smart decisions” … my only point is that there are probably other actions out there like that which have been yet untapped that would yield additional incremental improvements. Over the years of my career, I lost count of the number of “facilitated team brainstorming sessions” I was involved in which were conducted for just such reasons … and yielded many a surprisingly profitable outcome (even when it looked ‘hopeless’ on the front end at times).
Here’s a wild idea … how about considering commissioning a joint Xojo/Customer Task Force to look into it and see if all the possibilities within the current constraints have been exhausted?
The Pro users are guaranteed access to betas but other users can request to be part of the beta. And if they test and provide coherent feedback they can remain on the beta. I was a tester for awhile and wasn’t ever a Pro user but life got in the way and eventually dropped off the program.
When you open a Beta up to more users that are not good testers or reporters, the level of useless communication goes up dramatically, and that is a huge time sync. Speaking from personal experience of 10 years in a test group. So many users say they want to beta test, but they really just want to kick the tires. Then others have meaningful intentions, but they just don’t have the testing skills, spare cycles or inclination to put together, concise, meaningful bug reports with short examples that demonstrate the problem to the engineers quickly. It’s a bad ROI just throwing beta code at more eyeballs. You will catch some new stuff, but a lot of it is just tons of crap you have to wade through.
Sorry, i don’t want to annoy anyone here. The folowing is just an idea:
Someone could intepret this like: The Xojo pro users are also bad testers.
That combined with:
can result in:
Why not exclude all Xojo Pro users from the beta program and only include the ones, who write good bug reports over the feedback system, independet from their Xojo-product.
Who’s going to review the bug reports of each Xojo user that applies to the new Beta program to determine if they would make good Beta testers? The engineers? Testers? Tech Support? Geoff? I hope they can find better things to do with their time. Seems like a pretty daunting task and a problem 10 times as bad as the problem it is trying to address. Sounds like a very bad ROI to me.
You have to draw the line somewhere. Setting it at people that are clearly buying into the product / system to the tune of $700 a year is a quick and easy divider. It may not be perfect, but it is better than flinging the door wide open or taking the enormous time it would take to have users apply and then review their previous bug reports.
That time is the cost of doing business. The alternatives are (a) just don’t fix bugs for a decade or so to save time, and (b) keep having buggy releases, as seems still to happen. Pay the dues, or at least more dues.
No, that’s called a bad use of time. I don’t believe for one second that redoing the list of eligible beta testers is going to produces hordes of new, quality beta testers that are going to improve the quality of the product. The issue to this point has been finding the resources to fix what has been already reported. Not trying to find things the engineers can work on.
[quote=182617:@Torben Vikow]
Why not exclude all Xojo Pro users from the beta program and only include the ones, who write good bug reports over the feedback system, independet from their Xojo-product.[/quote]
Because that’s not the commitment we made about Xojo Pro licenses
I think limiting the Beta to Pro users and those that have been additionally added by Xojo, probably does close to the same thing without all the overhead of managing an active Beta list membership based on merit. Besides, how many users would you tick off with, “Sorry, you can’t join the Beta program because you don’t give good bug reports”? EVERYONE thinks they write good bug reports.
I am not so sure the bad written reports as so numerous they clog the system. Actually I am not sure neither Xojo needs more reports not they have not enough. Perusing “Newest” in Feedback shows on average a dozen reports and feature requests a day. If you think about it, it is 360 a month, hardly insignificant.
The whole bug thing is seldom perfectly objective. On the spot, I find some bugs severe, then I move on other projects, and totally forget about them. That was the case recently for the BevelButton firing the window Exit event when hovered a certain way under Windows. And that was not for having left BevelButton on the way side. As a matter of fact, that is the ideal button for the Modern API in Windows apps. Simply, I did not need the window Exit event.
Sometimes, I see angry posts demanding an immediate solution to a sometimes old problem. I understand the despair of the author who sometimes did not find an appropriate workaround, or simply would not use one. Then his project is stuck and it becomes the center of his world. But for all other users, it may be just benign. All a question of perspective.
When I develop in Windows, I hate labels that are not transparent as default, since they are always transparent in Mac OS X and Linux. I call that a bug. But at Xojo, they are wary about breaking existing code.
Difficult for them to keep everybody happy, between those who want more features, and those who don’t like the Xojo IDE, or those who would rather see any new development stop to fix every bug. Personally, I rather have new features and the familiar bugs I can work around. But that’s just me
Well, I have been very happy and relieved since 2015r2.1 came out and am using it without any issues on OS X. I didn’t have a chance yet to use it on Windows.
[quote=182545:@Greg O’Lone]There were 8 or 9 bugs reported just as we released r2 which we felt needed to be resolved before XDC, and that was the reason for 2.1.
We do try to test on all platforms, but we also rely on our beta testers to help us in that endeavor. 2015r2 had 4 weeks of beta testing, if I recall correctly, and we do our best to address regressions when they are reported. That said, the sooner a regression is reported during a beta cycle, the more likely we’ll have time to fix it. If testers wait until we reach final candidate to start testing, that doesn’t help us very much.[/quote]
For the record, I reported <https://xojo.com/issue/38576> on 13th March, Joe marked it as reviewed on 20th March and Robin asked me for a sample project on 27th March. As I had still not figured out what was causing the issue, it was not until 2nd April that I was able to provide a sample project that reliably demonstrated the problem. On 16th April Norman found the cause and fixed it, but it was then too late for 2015r2.
In the final days of 2014r2 betas, I reported a bug with web dialogs <https://xojo.com/issue/34250>. I had tested most of the preceding betas and it was only by chance that I found this issue.
The problem is that even with quite systematic testing of betas, bugs can go undiscovered. Maybe it’s Murphy’s Law that only in the Final Candidate will some bugs rear their heads!
