The Feedback Points System

Google uses a tag system in GitHub with continuous evaluation of the triage team adjusting priorities (tags p1 to p5). No case is left behind, engineering and administration do cherry picking and set a lot of them to be started every month (some are not solved in that iteration and become “overdue”), publicly. When you look at a line of any case you have an idea about everything that case brings in looking at its tags, like priority if it was inspected, what it affects like: framework, desktop, MacOS… Many things, there is a standard, and the engineers reviewing the case sets them. It’s a continuous work scheduling things and evaluating priorities. Every case has a “conversation” about the case and history, like x engineer/adm removed p4 and assigned p3 (increased priority), assigned “Windows”, “Framework”. There’s a lot of “searching” possibilities. But… that position of triage, assigning engineers, closing cases, releasing new more stable versions, is a non-stop work. No bug is an insignificant bug that will be left behind (unless it’s not really a bug, and will be closed as duplicate, behavior works as designed, etc)
The equivalent from GitLab (that can be installed and hosted anywhere): https://docs.gitlab.com/ee/user/project/issues/

1 Like

First, let me say I appreciate the passion you exhibit regarding Xojo. You clearly care a lot about it and that doesn’t go unnoticed.

As to the number of open bugs, the actual numbers are far less than you’re imagining. There 7382 open bugs or about 14% of all cases. Some number of those will be duplicates, many will not be bugs at all (as the user sometimes misunderstands how a feature works) and some will not be reproducible. The automatic crash reports for example are almost never reproducible but what people tell us over time does sometimes reflect a pattern that helps us figure out what’s causing a particular crash.

If that number 7382 seems like a big number, you should know that Apple’s Xcode and Microsoft’s Visual Studio have many 10’s of thousands of open bug reports with teams that are several orders of magnitude larger than ours. The raw numbers truly are not relevant. It’s the impact surface that is relevant. A bug that impacts 100 users is a more efficient use of engineering time than 100 bugs that affect only a single user each. Chasing zero bugs in software is mind-bogglingly expensive. The software that ran the space shuttle was supposedly bug-free but at a cost of $25,000 per line of code. That would put the cost of Xojo at somewhere between $25B and $50B USD. Bug-free software, especially when it comes to development tools that have greater API surfaces than the typical application, is not a practical target.

I’d also like to point out that you’ve only ever reported one bug (a crash report) and it was closed 5 days later as a duplicate of another report.

Most Xojo users are satisfied with Xojo. We know that from our surveys. That doesn’t mean we can’t and shouldn’t do better. Of course we should but it’s easy to get skewed perspective if you’re not looking at the stats from all users. Remember that most users don’t participate in this forum for example.

I worked for a development tools company in Silicon Valley and I can tell you that when users evaluate a development tool, they don’t spend much time evaluating 3rd party products. They evaluate the features in the core product itself and many specifically will not use a 3rd party add-on at all because they want a single source to deal with. Some want to know that there IS a third party market but again, that’s an extra.

Also, there are many features that 3rd party developers could not add even if they wanted to do so. They can’t add support for entirely new platforms for example. They can’t add features to our compiler. They can’t add features to the IDE. Now, we could make all of that possible of course but then that would be a new feature. :slight_smile:

I do want to thank @Beatrix_Willius for her excellent bug reports. She writes concisely and usually provides a sample project or steps to reproduce. This makes it far easier for the engineers to fix the bugs she reports and as a result, her bugs tend to get fixed faster. She also as a result has a reputation amongst the engineers such that when they see a report from her, they know ahead of time it’s likely to be very well-written and complete which, as you can imagine, makes them only wish to tackle those reports more quickly.

4 Likes

My bug reports are nothing special.

1 Like

I realize you don’t think that but they are compared to many that we get.

First, let me say I appreciate the passion you exhibit regarding Xojo. You clearly care a lot about it and that doesn’t go unnoticed.

I appreciate that, we are here because, fundamentally, we love this product and want to see it continue, and grow if possible.

As to the number of open bugs, the actual numbers are far less than you’re imagining. There 7382 open bugs or about 14% of all cases.

Doesn’t matter, 7382, 700 or 70,000+. They continue to pile up, get in the way of production and need to be seriously dealt with on a wartime footing as they make the product get weaker with each unfinished and regressive release.

software that ran the space shuttle was supposedly bug-free

Nobody here is making the demand for certified error free code, aerospace grade quality control or defense-in-depth engineering practice. We just want the core functional or a 20 year old product to deliver as advertised. New features can be alpha for however long it takes to get it right, but the foundation must be solid… period.

Ironic that you mention the STS, shortcuts (i.e. bugs) were allowed to persist in both the Solid Rocket Booster and the Thermal Protection System… Rather than fix those level 1 criticality issues, NASA got a case of go-fever (feature creep, or in their case mission creep) and choose to chase congressional funding instead. It didn’t end well.

I’d also like to point out that you’ve only ever reported one bug (a crash report) and it was closed 5 days later as a duplicate of another report.

How long did that duplicate report languish and go unresolved (actually fixed)?

Are you sure that this is my only charge-card avatar? It it possible that I have been using Real Basic since the Power PC days?

Is my singular report you continue to cite actually a relevant counter argument against the possibility of a major quality control pass to fix Xojo Core API?

Most Xojo users are satisfied with Xojo

Not consistent with observation of the currently available public information sources, including this forum.

Remember that most users don’t participate in this forum

They are starting to now, and appear to be coming out of the woodwork and resurrecting accounts that have been inactive for years, just to have a shot at getting the ship pointed in the right direction.

There have been quite a few of posts telling you directly that feedback is not being used, not working or not able to perform its data gathering function due to people being ignored for years… Basing assumptions and data-driven decisions on what you believe using a system with known and reported issues, with plummeting user confidence and engagement is not going to arrive at a meaningful and effective solution.

If you want fresh and accurate data (and you need it), we are here now, if we are not enough of a representative sample, then set up a poll and mail, call, and solicit from the entire customer base by any and all means at your disposal. Please don’t continue to make false assumptions based on faulty and incomplete data, it is not moving the problem forward, only putting earplugs in and ignoring chest pains in the hope they will go away again.

they don’t spend much time evaluating 3rd party products.

Your third party marketplace and the small group of dedicated and talented engineers, working tirelessly to serve your (and their) customers are the only reason this language is still useful and relevant. Also, although they are single individuals, they still manage to produce solid code with prompt fixes at no charge. Can’t say I have ever had a showstopper or even a bug that was not ultimately my fault when using the marketplace providers.

They evaluate the features in the core product itself and many specifically will not use a 3rd party add-on at all because they want a single source to deal with.

Yes, if they were developing algorithms for a reactor control system, classified information processing or banking software sure… but then they would not consider Xojo at all. “Citizen developers” certainly want marketplace to source options from, as do pros and semi-pros. If rigid vendor control and 100% independence from external source was a critical consideration, github and the copy-paste culture for software development would cease to exist.

They can’t add features to our compiler. They can’t add features to the IDE. Now, we could make all of that possible of course but then that would be a new feature.

Now that’s a feature we can all get behind as it would open the floodgate and make the product’s future independent of the limited resources of a few overworked individuals… but AFTER the FOUNDATION WORKS.

4 Likes

Thanks for taking the time to answer. I will add a Feedback case for the double entry tomorrow. What about not being able to assign points to unreviewed cases - is this considered a bug or meant to be that way? In case of the latter it sure is a form of ‚double punishment‘ though.

Juyoun,

Again I ask which cases are interfering with your ability to build applications? I hear you talking about a problem that does not appear to be impacting you.

While there will always be more bugs to fix, the impact surface is what matters. We each need to focus on that and not on this nebulous idea that there’s a quality control problem. There is for you if there are specific bugs that are preventing you from using Xojo. But if there are not, then your time and my time are better spent on other things, right?

The bug you discovered of course could be taken advantage of. In theory, you could assign the same case to all 5 slots to add even more points to it. :slight_smile:

As for unreviewed cases, honestly, I think if a user puts a case in their top 5, that should alert us to immediately review that particular case because the user is indicating that it’s a very important one. Please add a feature request for that. Make sure of course to choose “Feedback” as the app rather than Xojo in this case.

We are working on the web-based version of Feedback so it would be great to address these issues in that version.

2 Likes

Geoff, I can multitask.

The future longevity and current stability of Xojo are of direct and paramount concern to every single individual who pays to use your product to derive an income.

The number of bugs we report or experience have no bearing on our need to seek and obtain assurances that the product is going to continue to be viable and not waste our time investment.

As I have said before, I spent months (LITERAL MONTHS) to build a ridiculously complex workaround to a major bug in the core API because fixes can not only be counted on, they are not even to be expected. I was successful in my workaround, and spend more months (MOAR MONTHS) performance tuning my product only to have it all transmute into excrement when imported into 2020… I did not get even a single debugger or depreciation warning when going from 2017r3 to 2020, that’s how vanilla and optimized my code was, yet I am left with no future upgrade path through no fault of my own.

Now, I am in no position to spend more months tracking down whatever was broken and outside the locus of my control… I have to ship what I have in hand that works.

My position, arguments and requests are not invalidated simply because I have not amassed a 500+ bug report list as others have. I am addressing issues that are relevant to all commercial users, those who are needing to rely on this tool in the future and who are grinding their teeth into dust at the seemingly ludicrous inability of Xojo to accept that there is a major, existential quality control problem and the product must be made to perform as advertised before any new feature creep is allowed to continue.

Is it going to take all the customers leaving en mass before egos give way to frank observation, mutual consultation, critical evaluation and reconciliation? That is in the near future if the status quo continues, like it or not.

or:

“We can ignore reality, but we cannot ignore the consequences of ignoring reality.”

― Ayn Rand

3 Likes

So create incentives that get you what you want. Bug reports with a sample that demonstrates an issue (reproducible) get more points. I mentioned earlier but I’ll say it again, add points for folks that verify an issue still exists in the most recent version. Having a list of items that are reproducible and exist in the current version is a much more interesting data point than 7k total that exist since 2013 many of which may or may not be legit in the current version because of x,y, and z reasons.

Why didn’t you contact us when you ran into this problem?

Why didn’t you contact us when you ran into this problem?

I apologize if this comes off as crude and I certainly desire to keep this mutually respectful, however…

Are the mountain of posts about ALL feedback being ignored, bugs languishing for literal years, APIs changing underneath you, customer projects being dead-ended overnight and NOBODY having confidence in anything ever getting fixed NOT BEING RECEIVED BY ANYBODY IN AUTHORITY?

There is a zero-confidence problem at play here, many (clearly not only me) feel and rightfully so from repeat observation and experience, that we are on our own. If its broken, too bad, try the third party devs, try a community suggested workaround and if that fails… Well that’s your problem, we are busy with the next gizmo.

I would have gladly paid cash money to Xojo Inc. if they could have saved me all that work… But even those who pay more dont get the Eye of Xojo to glance their way. Many feel, and have expressed that we dont seem to matter in the grand scheme, a user was even told so much in a telephone call as I paraphrase “The loss of the single licence will not effect Xojo Inc.”

With that sort of customer service, coupled with languishing bug lists and Feedback purgatory, how can relying on official support at the critical moment be taken seriously?

Look, I am not here to damage the reputation of the company or product, the actions of the company are doing that all by itself. I simply need Xojo to acknowledge its qualitative and process deficiencies, and take corrective action to insure that we are not all putting our time, money and future plans into a runaway train, on fire, without breaks or a conductor. Nobody is blaming anybody here, we all want the company and product to thrive. That’s why we are all being so persistent, we NEED XOJO to continue and improve to earn a living.

We need to have a serious public facing survey as to the desire and viability of a major bug overhaul to the core API.

If everybody responds that NO, ANDROID is our TOP priority and core api bugs be dammed or if you learn that a major bugsweep is the only thing between now and certain collapse, then YOU WILL KNOW THE UNMANAGED, UN-BIASED, UNCENSORED and absolute undeniable truth, in ANY EVENT. You will be able to cite a proper and recent survey to counter any discontent.

YOU WILL BE ABLE TO ACT ON GOOD DATA, empirical and reproducible data from a pure, recent and assured source.

Honestly, I cant see why this is so hard to accept, acknowledge or even discuss without the consideration of ulterior motives or just plain unrestrained pride and stubbornness for the sake of winning at all costs, even risking obliteration.

-What have you got to loose? Put it to a vote and show us that you really want to serve the customer interests, that you value the product longevity and that you are the leadership that we all hope you are.

PLEASE SIR

2 Likes

I’ll ask you again. If you had a show stopping bug that you were spending hours and hours trying to resolve, why didn’t you contact us?

What cases are stopping you from building apps?

It’s extremely difficult for me to take your quality control argument seriously when you reported a total of 1 bug and subscribed to only 5 others all of which are closed.

How about a case of mine from 2016: see Feedback case 45017.
Can you see there that after some time I got tired and gave up reporting about the problem? We even had mail exchange where I told you about the problem in case 45017.
I updated the case today, because it still exists. I do not need the fix any longer, in the meantime I switched to another programming language.

4 Likes

I’m sorry, but it’s not MY quality control argument at all, it is the position of the majority of active participants in the two most recent and highest active threads in recent memory that there are quality control issues that need to be acknowledged and addressed.

That said issues are causing customers to leave the ecosystem in disgust and that are causing serious concerns regarding the fitness, viability, serviceability and longevity of the product, and more importantly to us customers, the viability and retention of the enormous time investment, ongoing customer obligations, contractual commitments and revenues entrusted to Xojo Inc by us, the customer. Please be aware, this is not a game to us, for many it is a major deal-breaker on a extremely sensitive and financially significant issue.

Please consider it OUR position as there have been more than enough voices that are saying the exact same thing. This is not about me, and even if Xojo performed flawlessly for my personal needs (which it did not), it does not absolve Xojo Inc. from being accountable to it’s customers requirements and demands.

It’s extremely difficult for me to take your quality control argument seriously when you reported a total of 1 bug and subscribed to only 5 others all of which are closed.

It is extremely difficult for me to take your ignorance of the quality control issues with the product seriously, in the face of withering criticism (see: open revolt) from a large percentage of active users, absolute abysmal third party reviews, active censorship, prolific damage control and the exodus of so many in our community who should have been considered the real MVPs.

Lets move away from fingerprinting, and straw-men, and focus on a vote to for once and all, reveal the will of the customer and allow Xojo to correctly respond to market forces and customer needs.

All the valuable CEO time and effort you need to put out so many brush fires (hours being spent to avoid just polling the users, even if it may definitively and publicly confirm your position) could have be better spent simply getting the vote, and proceeding from universally accepted premise and solid data.

I’m not wrong here, and everybody can see it.

1 Like

Hi Christian,

Looking at the report, it’s not clear if the issue is the line endings, the offset or the length. If it’s line endings, we have added several new options for 2020r2. If it’s not that, could you give me some specifics? Neither I nor the engineer I reviewed the case with were certain exactly what the issue was.

Oh I agree with you. That’s why I have asked you repeatedly for the issues you are facing that prevent you from delivering your projects and yet you refuse to answer. If anyone else has issues, they can communicate with us about them. It is not your place to speak for them and it’s unhelpful to talk in generalities.

We are contacted regularly through customer service, tech support and Feedback about various issues users are facing and we do what we can to help solve them. You seem unwilling to do so. That’s your choice but please speak for yourself rather than attempting to speak for everyone.

If you can provide your specific cases, please private message me as I’m unwilling continue to participate in the highjacking of this thread.

3 Likes

Ok fair enough.

Anybody else in here named Spartacus? Or am I just a complainer without any support of the consensus?

See, I am willing to put my argument to the test, openly and in full view on principal.

Is it just me folks?

1 Like

I admire your persistence @Juyoun_Park , but I’m afraid you are talking to a wall.

Many ex-Xojo users (me included) are eagerly following this forum over the last few weeks again, because if there is even a little spark of hope Xojo would finally see that fixing the ever growing pile of bugs should be priority #1, our heart skips a beat. It is the main reason many pro users left (leave). Building ones business on sand is always dangerous, especially in uncertain times like we are in now. We need a strong fundament as a professional programmer and unfortunately, Xojo cannot deliver this. Many of us would love to come back, but only under that condition.

I will no further engage in this topic, as I’m not an active user anymore. But I will actively follow and continue reading, just in case… I just wanted to let you know: no, you definitely are not alone!

2 Likes